2023-11-24T07:46:05.093 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T07:46:05.093 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} --name rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580 -- /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/orig.config.yaml 2023-11-24T07:46:05.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T07:46:05.220 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} email: null first_in_suite: false job_id: '7465580' kernel: flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-vol-tests sha1: 73843baf45f8a92d8e3fcbea3f024c97e73036da owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9764 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: mgr-vol-tests suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 73843baf45f8a92d8e3fcbea3f024c97e73036da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF5omeBjoYC53ar9tv0sjHnuDu9+h9ZwYTKIpSdWxz/gGoVgOd+nmj3zVpebk7jcROuIaYj44iGeSBgZod3iONc= smithi091.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSQUKW2luMwj0cnVdRy1CbmC41uHhE7G7DW1zzQj+Jw8O8mNR32YMkj5ensrEk5RXvpqpGFi2y3frTvsZraysQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_07:31:48 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T07:46:05.220 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa; will attempt to use it 2023-11-24T07:46:05.221 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks 2023-11-24T07:46:05.221 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T07:46:05.223 INFO:teuthology.task.internal:Checking packages... 2023-11-24T07:46:05.245 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T07:46:05.245 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T07:46:05.245 INFO:teuthology.packaging:ref: None 2023-11-24T07:46:05.246 INFO:teuthology.packaging:tag: None 2023-11-24T07:46:05.246 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T07:46:05.246 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:46:05.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T07:46:05.395 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T07:46:05.395 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T07:46:05.402 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T07:46:05.402 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T07:46:05.444 INFO:teuthology.task.internal:Saving configuration 2023-11-24T07:46:05.462 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T07:46:05.468 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T07:46:05.493 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:41:06.516615', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF5omeBjoYC53ar9tv0sjHnuDu9+h9ZwYTKIpSdWxz/gGoVgOd+nmj3zVpebk7jcROuIaYj44iGeSBgZod3iONc='} 2023-11-24T07:46:05.519 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi091.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:41:06.515174', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSQUKW2luMwj0cnVdRy1CbmC41uHhE7G7DW1zzQj+Jw8O8mNR32YMkj5ensrEk5RXvpqpGFi2y3frTvsZraysQ='} 2023-11-24T07:46:05.519 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T07:46:05.525 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T07:46:05.525 INFO:teuthology.task.internal:roles: ubuntu@smithi091.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T07:46:05.525 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T07:46:05.610 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f882176db80>, signals=[15]) 2023-11-24T07:46:05.610 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T07:46:05.616 INFO:teuthology.task.internal:Opening connections... 2023-11-24T07:46:05.616 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T07:46:05.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:46:05.720 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T07:46:05.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:46:05.805 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T07:46:05.812 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2023-11-24T07:46:05.831 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-11-24T07:46:05.831 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2023-11-24T07:46:05.889 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2023-11-24T07:46:05.889 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2023-11-24T07:46:05.889 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2023-11-24T07:46:05.889 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T07:46:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T07:46:05.891 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-11-24T07:46:05.911 DEBUG:teuthology.orchestra.run.smithi091:> uname -m 2023-11-24T07:46:05.928 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2023-11-24T07:46:05.928 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release 2023-11-24T07:46:05.984 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="CentOS Stream" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="9" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:ID="centos" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="rhel fedora" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="9" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el9" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:LOGO="fedora-logo-icon" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://centos.org/" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T07:46:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T07:46:05.986 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T07:46:05.986 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2023-11-24T07:46:06.010 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T07:46:06.020 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T07:46:06.036 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T07:46:06.037 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2023-11-24T07:46:06.039 DEBUG:teuthology.orchestra.run.smithi091:> test '!' -e /home/ubuntu/cephtest 2023-11-24T07:46:06.055 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T07:46:06.060 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T07:46:06.061 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2023-11-24T07:46:06.097 DEBUG:teuthology.orchestra.run.smithi091:> test -z $(ls -A /var/lib/ceph) 2023-11-24T07:46:06.188 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T07:46:06.254 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T07:46:06.267 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T07:46:06.267 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, timeout 300 2023-11-24T07:46:06.268 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T07:46:06.268 DEBUG:teuthology.orchestra.run.smithi091:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T07:46:06.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:46:06.285 DEBUG:teuthology.task.kernel:sha1 for mon.b is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:46:06.285 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T07:46:06.285 DEBUG:teuthology.orchestra.run.smithi091:> uname -r 2023-11-24T07:46:06.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:46:06.286 DEBUG:teuthology.task.kernel:sha1 for mon.a is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:46:06.286 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T07:46:06.286 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-11-24T07:46:06.342 INFO:teuthology.orchestra.run.smithi091.stdout:5.14.0-361.el9.x86_64 2023-11-24T07:46:06.342 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:46:06.343 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T07:46:06.343 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:46:06.347 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-361.el9.x86_64 2023-11-24T07:46:06.347 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:46:06.347 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T07:46:06.347 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:46:06.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum upgrade -y linux-firmware 2023-11-24T07:46:06.583 DEBUG:teuthology.orchestra.run.smithi091:> sudo yum upgrade -y linux-firmware 2023-11-24T07:46:08.560 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2023-11-24T07:46:08.770 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - AppStream 11 MB/s | 6.4 MB 00:00 2023-11-24T07:46:10.044 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2023-11-24T07:46:10.145 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2023-11-24T07:46:11.418 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 7.9 MB 00:00 2023-11-24T07:46:13.102 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 7.9 MB 00:02 2023-11-24T07:46:15.886 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-11-24T07:46:16.719 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 18 MB 00:04 2023-11-24T07:46:21.295 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 5.8 MB 00:02 2023-11-24T07:46:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 5.8 MB 00:02 2023-11-24T07:46:22.779 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2023-11-24T07:46:23.828 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 15 kB 00:00 2023-11-24T07:46:24.674 INFO:teuthology.orchestra.run.smithi091.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2023-11-24T07:46:25.396 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2023-11-24T07:46:29.346 INFO:teuthology.orchestra.run.smithi091.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2023-11-24T07:46:30.396 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2023-11-24T07:46:31.807 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies resolved. 2023-11-24T07:46:31.817 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2023-11-24T07:46:31.817 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2023-11-24T07:46:31.817 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2023-11-24T07:46:31.817 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrading: 2023-11-24T07:46:31.818 INFO:teuthology.orchestra.run.smithi091.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T07:46:31.818 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T07:46:31.818 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2023-11-24T07:46:31.818 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2023-11-24T07:46:31.818 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrade 1 Package 2023-11-24T07:46:31.818 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T07:46:31.818 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 328 M 2023-11-24T07:46:31.818 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading Packages: 2023-11-24T07:46:32.914 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-11-24T07:46:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-11-24T07:46:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2023-11-24T07:46:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-11-24T07:46:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2023-11-24T07:46:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T07:46:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T07:46:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-11-24T07:46:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-11-24T07:46:32.930 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 1 Package 2023-11-24T07:46:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T07:46:32.930 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 328 M 2023-11-24T07:46:32.930 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2023-11-24T07:46:36.630 INFO:teuthology.orchestra.run.smithi091.stdout:linux-firmware-20231030-141.el9.noarch.rpm 68 MB/s | 328 MB 00:04 2023-11-24T07:46:36.631 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2023-11-24T07:46:36.632 INFO:teuthology.orchestra.run.smithi091.stdout:Total 68 MB/s | 328 MB 00:04 2023-11-24T07:46:36.632 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2023-11-24T07:46:36.724 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction check succeeded. 2023-11-24T07:46:36.724 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2023-11-24T07:46:37.797 INFO:teuthology.orchestra.run.smithi064.stdout:linux-firmware-20231030-141.el9.noarch.rpm 67 MB/s | 328 MB 00:04 2023-11-24T07:46:37.798 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-11-24T07:46:37.798 INFO:teuthology.orchestra.run.smithi064.stdout:Total 67 MB/s | 328 MB 00:04 2023-11-24T07:46:37.799 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2023-11-24T07:46:37.895 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2023-11-24T07:46:37.895 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2023-11-24T07:46:39.018 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded. 2023-11-24T07:46:39.018 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2023-11-24T07:46:40.086 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2023-11-24T07:46:40.087 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2023-11-24T07:46:44.835 INFO:teuthology.orchestra.run.smithi091.stdout: Preparing : 1/1 2023-11-24T07:46:45.246 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T07:46:45.318 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:46:46.005 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2023-11-24T07:46:46.337 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:46:46.337 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T07:46:46.344 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T07:46:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:46:46.721 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:46:46.721 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T07:46:46.721 INFO:teuthology.orchestra.run.smithi091.stdout:Upgraded: 2023-11-24T07:46:46.721 INFO:teuthology.orchestra.run.smithi091.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T07:46:46.721 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T07:46:46.722 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2023-11-24T07:46:47.010 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T07:46:47.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:46:47.234 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T07:46:47.234 DEBUG:teuthology.orchestra.run.smithi091:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T07:46:47.237 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi091.front.sepia.ceph.com to complete... 2023-11-24T07:46:47.410 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:46:47.410 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T07:46:47.814 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:46:47.814 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T07:46:47.815 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2023-11-24T07:46:47.815 INFO:teuthology.orchestra.run.smithi064.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T07:46:47.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T07:46:47.815 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-11-24T07:46:48.093 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T07:46:48.093 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:46:48.316 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T07:46:48.317 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T07:46:48.319 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi064.front.sepia.ceph.com to complete... 2023-11-24T07:46:58.464 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24 07:46:58 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T07:46:58.464 INFO:teuthology.orchestra.run.smithi091.stderr:FINISHED --2023-11-24 07:46:58-- 2023-11-24T07:46:58.464 INFO:teuthology.orchestra.run.smithi091.stderr:Total wall clock time: 11s 2023-11-24T07:46:58.464 INFO:teuthology.orchestra.run.smithi091.stderr:Downloaded: 1 files, 306M in 11s (28.0 MB/s) 2023-11-24T07:46:58.465 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T07:46:58.465 DEBUG:teuthology.orchestra.run.smithi091:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T07:47:00.313 INFO:teuthology.orchestra.run.smithi091.stdout:Verifying... ######################################## 2023-11-24T07:47:00.403 INFO:teuthology.orchestra.run.smithi091.stdout:Preparing... ######################################## 2023-11-24T07:47:00.483 INFO:teuthology.orchestra.run.smithi091.stdout:Updating / installing... 2023-11-24T07:47:07.569 INFO:teuthology.orchestra.run.smithi091.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T07:47:17.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24 07:47:17 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T07:47:17.479 INFO:teuthology.orchestra.run.smithi064.stderr:FINISHED --2023-11-24 07:47:17-- 2023-11-24T07:47:17.480 INFO:teuthology.orchestra.run.smithi064.stderr:Total wall clock time: 29s 2023-11-24T07:47:17.480 INFO:teuthology.orchestra.run.smithi064.stderr:Downloaded: 1 files, 306M in 29s (10.7 MB/s) 2023-11-24T07:47:17.482 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T07:47:17.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T07:47:19.417 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying... ######################################## 2023-11-24T07:47:19.477 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing... ######################################## 2023-11-24T07:47:19.561 INFO:teuthology.orchestra.run.smithi064.stdout:Updating / installing... 2023-11-24T07:47:22.745 INFO:teuthology.orchestra.run.smithi091.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T07:47:26.949 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T07:47:40.040 INFO:teuthology.orchestra.run.smithi064.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T07:47:43.885 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi091.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T07:47:43.885 DEBUG:teuthology.orchestra.run.smithi091:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T07:47:43.957 INFO:teuthology.orchestra.run.smithi091.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T07:47:43.958 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T07:47:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T07:47:43.960 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T07:47:43.961 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T07:47:43.962 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T07:47:43.963 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T07:47:43.964 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T07:47:43.965 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T07:47:43.966 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T07:47:43.967 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T07:47:43.968 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T07:47:43.969 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T07:47:43.970 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T07:47:43.971 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T07:47:43.972 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T07:47:43.973 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T07:47:43.974 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T07:47:43.975 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T07:47:43.976 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T07:47:43.977 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T07:47:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T07:47:43.979 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T07:47:43.980 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T07:47:43.981 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T07:47:43.982 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T07:47:43.983 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T07:47:43.984 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T07:47:43.985 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T07:47:43.986 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T07:47:43.987 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T07:47:43.988 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T07:47:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T07:47:43.990 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T07:47:43.991 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T07:47:43.992 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T07:47:43.993 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T07:47:43.994 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T07:47:43.995 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T07:47:43.996 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T07:47:43.997 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T07:47:43.998 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T07:47:43.999 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T07:47:44.000 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T07:47:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T07:47:44.002 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T07:47:44.003 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T07:47:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T07:47:44.005 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T07:47:44.006 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T07:47:44.007 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T07:47:44.008 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T07:47:44.009 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T07:47:44.010 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T07:47:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T07:47:44.012 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T07:47:44.013 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T07:47:44.014 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T07:47:44.015 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T07:47:44.016 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T07:47:44.017 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T07:47:44.018 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T07:47:44.019 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T07:47:44.020 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T07:47:44.021 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T07:47:44.022 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T07:47:44.023 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T07:47:44.024 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T07:47:44.025 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T07:47:44.026 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T07:47:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T07:47:44.028 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T07:47:44.029 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T07:47:44.030 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T07:47:44.031 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T07:47:44.032 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T07:47:44.033 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T07:47:44.034 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T07:47:44.035 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T07:47:44.036 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T07:47:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T07:47:44.038 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T07:47:44.039 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T07:47:44.040 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T07:47:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T07:47:44.042 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T07:47:44.043 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T07:47:44.044 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T07:47:44.045 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T07:47:44.046 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T07:47:44.047 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T07:47:44.048 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T07:47:44.049 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T07:47:44.050 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T07:47:44.051 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T07:47:44.052 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T07:47:44.053 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T07:47:44.054 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T07:47:44.055 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T07:47:44.056 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T07:47:44.057 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T07:47:44.058 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T07:47:44.059 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T07:47:44.060 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T07:47:44.061 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T07:47:44.062 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T07:47:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T07:47:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T07:47:44.065 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T07:47:44.066 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T07:47:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T07:47:44.069 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T07:47:44.069 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T07:47:44.069 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T07:47:44.069 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T07:47:44.069 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T07:47:44.069 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T07:47:44.069 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T07:47:44.069 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T07:47:44.070 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T07:47:44.071 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T07:47:44.072 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T07:47:44.073 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T07:47:44.074 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T07:47:44.075 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T07:47:44.076 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T07:47:44.077 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T07:47:44.104 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T07:47:44.104 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T07:47:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T07:47:44.106 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T07:47:44.107 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T07:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T07:47:44.109 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T07:47:44.110 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T07:47:44.111 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T07:47:44.112 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T07:47:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T07:47:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T07:47:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T07:47:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T07:47:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T07:47:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T07:47:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T07:47:44.113 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T07:47:44.114 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T07:47:44.115 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T07:47:44.116 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T07:47:44.117 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T07:47:44.118 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T07:47:44.119 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T07:47:44.119 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T07:47:44.119 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T07:47:44.119 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T07:47:44.119 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T07:47:44.120 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T07:47:44.121 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T07:47:44.122 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T07:47:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T07:47:44.124 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T07:47:44.125 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T07:47:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T07:47:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T07:47:44.128 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T07:47:44.128 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T07:47:44.128 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T07:47:44.128 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T07:47:44.128 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T07:47:44.128 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T07:47:44.129 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T07:47:44.129 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T07:47:44.129 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T07:47:44.129 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T07:47:44.129 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T07:47:44.129 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T07:47:44.129 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T07:47:44.129 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T07:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T07:47:44.131 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T07:47:44.132 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T07:47:44.132 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T07:47:44.132 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T07:47:44.132 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T07:47:44.132 INFO:teuthology.orchestra.run.smithi091.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T07:47:44.132 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T07:47:44.133 DEBUG:teuthology.orchestra.run.smithi091:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T07:47:44.155 INFO:teuthology.orchestra.run.smithi091.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T07:47:44.155 INFO:teuthology.orchestra.run.smithi091.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T07:47:44.155 INFO:teuthology.orchestra.run.smithi091.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:47:44.155 INFO:teuthology.orchestra.run.smithi091.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:47:44.155 INFO:teuthology.orchestra.run.smithi091.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T07:47:44.155 INFO:teuthology.orchestra.run.smithi091.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:fi 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:fi 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T07:47:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:fi 2023-11-24T07:47:44.158 DEBUG:teuthology.orchestra.run.smithi091:> sudo rpm -qi grub2-tools 2023-11-24T07:47:44.231 INFO:teuthology.orchestra.run.smithi091.stdout:Name : grub2-tools 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Epoch : 1 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Version : 2.06 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Release : 68.el9 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Architecture: x86_64 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Group : Unspecified 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Size : 8264254 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:License : GPLv3+ 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Packager : builder@centos.org 2023-11-24T07:47:44.232 INFO:teuthology.orchestra.run.smithi091.stdout:Vendor : CentOS 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout:Summary : Support tools for GRUB. 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout:Description : 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout:hardware devices. 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T07:47:44.233 INFO:teuthology.orchestra.run.smithi091.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T07:47:44.234 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T07:47:44.234 INFO:teuthology.task.kernel:Updating grub on smithi091 to boot 6.6.0-g8247bbb5b470 2023-11-24T07:47:44.234 DEBUG:teuthology.orchestra.run.smithi091:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T07:47:44.909 INFO:teuthology.orchestra.run.smithi091.stderr:Generating grub configuration file ... 2023-11-24T07:47:46.296 INFO:teuthology.orchestra.run.smithi091.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T07:47:46.332 INFO:teuthology.orchestra.run.smithi091.stderr:done 2023-11-24T07:47:46.335 DEBUG:teuthology.orchestra.run.smithi091:> mktemp 2023-11-24T07:47:46.350 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.jArx7JtvfI 2023-11-24T07:47:46.351 DEBUG:teuthology.orchestra.run.smithi091:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jArx7JtvfI 2023-11-24T07:47:46.424 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.jArx7JtvfI 2023-11-24T07:47:46.574 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.jArx7JtvfI is 6KB 2023-11-24T07:47:46.623 DEBUG:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.jArx7JtvfI 2023-11-24T07:47:46.638 DEBUG:teuthology.orchestra.run.smithi091:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T07:47:46.702 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T07:47:46.702 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T07:47:46.702 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T07:47:46.703 INFO:teuthology.orchestra.run.smithi091.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T07:47:46.703 DEBUG:teuthology.orchestra.run.smithi091:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T07:47:46.855 DEBUG:teuthology.orchestra.run.smithi091:> sudo shutdown -r now 2023-11-24T07:48:01.849 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T07:48:01.849 DEBUG:teuthology.orchestra.run.smithi064:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T07:48:01.923 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T07:48:01.923 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T07:48:01.923 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T07:48:01.923 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T07:48:01.923 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T07:48:01.923 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T07:48:01.923 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T07:48:01.924 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T07:48:01.925 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T07:48:01.926 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T07:48:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T07:48:01.928 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T07:48:01.929 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T07:48:01.930 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T07:48:01.931 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T07:48:01.932 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T07:48:01.933 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T07:48:01.934 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T07:48:01.935 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T07:48:01.936 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T07:48:01.937 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T07:48:01.938 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T07:48:01.939 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T07:48:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T07:48:01.941 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T07:48:01.942 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T07:48:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T07:48:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T07:48:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T07:48:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T07:48:01.947 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T07:48:01.948 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T07:48:01.949 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T07:48:01.950 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T07:48:01.951 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T07:48:01.952 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T07:48:01.953 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T07:48:01.954 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T07:48:01.955 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T07:48:01.956 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T07:48:01.957 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T07:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T07:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T07:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T07:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T07:48:01.962 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T07:48:01.963 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T07:48:01.964 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T07:48:01.965 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T07:48:01.966 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T07:48:01.967 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T07:48:01.968 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T07:48:01.969 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T07:48:01.970 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T07:48:01.971 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T07:48:01.972 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T07:48:01.973 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T07:48:01.974 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T07:48:01.975 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T07:48:01.976 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T07:48:01.977 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T07:48:01.978 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T07:48:01.979 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T07:48:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T07:48:01.981 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T07:48:01.982 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T07:48:01.983 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T07:48:01.984 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T07:48:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T07:48:01.986 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T07:48:01.987 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T07:48:01.988 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T07:48:01.989 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T07:48:01.990 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T07:48:01.991 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T07:48:01.992 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T07:48:01.993 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T07:48:01.994 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T07:48:01.995 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T07:48:01.996 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T07:48:01.997 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T07:48:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T07:48:01.999 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T07:48:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T07:48:02.001 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T07:48:02.002 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T07:48:02.003 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T07:48:02.004 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T07:48:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T07:48:02.006 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T07:48:02.007 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T07:48:02.008 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T07:48:02.009 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T07:48:02.010 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T07:48:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T07:48:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T07:48:02.013 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T07:48:02.014 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T07:48:02.015 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T07:48:02.016 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T07:48:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T07:48:02.018 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T07:48:02.019 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T07:48:02.020 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T07:48:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T07:48:02.022 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T07:48:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T07:48:02.024 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T07:48:02.025 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T07:48:02.026 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T07:48:02.027 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T07:48:02.028 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T07:48:02.029 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T07:48:02.030 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T07:48:02.031 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T07:48:02.032 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T07:48:02.033 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T07:48:02.034 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T07:48:02.035 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T07:48:02.036 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T07:48:02.037 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T07:48:02.038 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T07:48:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T07:48:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T07:48:02.041 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T07:48:02.042 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T07:48:02.043 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T07:48:02.044 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T07:48:02.045 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T07:48:02.046 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T07:48:02.047 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T07:48:02.048 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T07:48:02.049 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T07:48:02.050 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T07:48:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T07:48:02.052 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T07:48:02.053 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T07:48:02.054 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T07:48:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T07:48:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T07:48:02.057 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T07:48:02.058 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T07:48:02.059 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T07:48:02.060 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T07:48:02.061 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T07:48:02.062 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T07:48:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T07:48:02.064 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T07:48:02.065 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T07:48:02.066 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T07:48:02.067 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T07:48:02.068 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T07:48:02.069 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T07:48:02.070 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T07:48:02.071 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T07:48:02.072 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T07:48:02.073 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T07:48:02.074 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T07:48:02.075 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T07:48:02.076 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T07:48:02.077 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T07:48:02.078 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T07:48:02.079 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T07:48:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T07:48:02.081 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T07:48:02.082 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T07:48:02.083 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T07:48:02.084 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T07:48:02.085 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T07:48:02.086 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T07:48:02.087 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T07:48:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T07:48:02.089 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T07:48:02.089 DEBUG:teuthology.orchestra.run.smithi064:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T07:48:02.112 INFO:teuthology.orchestra.run.smithi064.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T07:48:02.112 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T07:48:02.112 INFO:teuthology.orchestra.run.smithi064.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:48:02.112 INFO:teuthology.orchestra.run.smithi064.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-11-24T07:48:02.113 INFO:teuthology.orchestra.run.smithi064.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T07:48:02.114 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T07:48:02.114 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T07:48:02.114 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-11-24T07:48:02.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2023-11-24T07:48:02.193 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2023-11-24T07:48:02.193 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 68.el9 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8264254 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T07:48:02.194 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T07:48:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2023-11-24T07:48:02.196 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T07:48:02.196 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T07:48:02.196 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T07:48:02.197 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 6.6.0-g8247bbb5b470 2023-11-24T07:48:02.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T07:48:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2023-11-24T07:48:04.338 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T07:48:04.377 INFO:teuthology.orchestra.run.smithi064.stderr:done 2023-11-24T07:48:04.379 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-11-24T07:48:04.434 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.eF75NnX48q 2023-11-24T07:48:04.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eF75NnX48q 2023-11-24T07:48:04.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.eF75NnX48q 2023-11-24T07:48:04.654 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.eF75NnX48q is 6KB 2023-11-24T07:48:04.674 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.eF75NnX48q 2023-11-24T07:48:04.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T07:48:04.758 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T07:48:04.758 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T07:48:04.758 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T07:48:04.758 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T07:48:04.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T07:48:04.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2023-11-24T07:48:16.870 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-24T07:48:16.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2023-11-24T07:48:16.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:48:34.938 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-24T07:48:34.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T07:48:34.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:48:35.298 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.91 2023-11-24T07:48:44.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2023-11-24T07:48:44.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:48:47.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.91 2023-11-24T07:48:53.474 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-11-24T07:48:59.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2023-11-24T07:48:59.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:49:02.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T07:49:02.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:49:05.538 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-11-24T07:49:17.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T07:49:17.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:49:49.355 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-11-24T07:49:50.054 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T07:49:50.055 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-g8247bbb5b470"... 2023-11-24T07:49:50.055 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-11-24T07:49:50.095 INFO:teuthology.orchestra.run.smithi064.stdout:6.6.0-g8247bbb5b470 2023-11-24T07:49:50.095 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T07:49:50.096 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T07:49:50.096 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T07:49:51.097 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-24T07:49:51.098 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T07:49:51.222 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2023-11-24T07:49:51.264 DEBUG:teuthology.parallel:result is None 2023-11-24T07:49:59.389 DEBUG:teuthology.orchestra.remote:timed out 2023-11-24T07:50:14.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2023-11-24T07:50:14.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:50:14.839 DEBUG:teuthology.orchestra.run.smithi091:> true 2023-11-24T07:50:15.535 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi091.front.sepia.ceph.com' 2023-11-24T07:50:15.535 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-g8247bbb5b470"... 2023-11-24T07:50:15.536 DEBUG:teuthology.orchestra.run.smithi091:> uname -r 2023-11-24T07:50:15.590 INFO:teuthology.orchestra.run.smithi091.stdout:6.6.0-g8247bbb5b470 2023-11-24T07:50:15.590 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T07:50:15.590 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T07:50:15.590 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T07:50:16.591 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-24T07:50:16.592 DEBUG:teuthology.orchestra.run.smithi091:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T07:50:16.751 INFO:teuthology.orchestra.run.smithi091.stdout:ttyS1 2023-11-24T07:50:16.789 DEBUG:teuthology.parallel:result is None 2023-11-24T07:50:16.790 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T07:50:16.797 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T07:50:16.797 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T07:50:16.807 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T07:50:16.865 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T07:50:16.877 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T07:50:16.892 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T07:50:16.892 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T07:50:16.901 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T07:50:16.973 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T07:50:16.980 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T07:50:16.980 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T07:50:16.987 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T07:50:17.119 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:50:17.122 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:50:17.164 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:50:17.170 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:50:17.174 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T07:50:17.181 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T07:50:17.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T07:50:17.187 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T07:50:17.282 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T07:50:17.296 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T07:50:17.297 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T07:50:17.305 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T07:50:17.362 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:50:17.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:50:17.560 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:50:17.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:50:17.755 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T07:50:17.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T07:50:17.856 DEBUG:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:50:17.943 DEBUG:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:50:18.052 DEBUG:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:50:18.141 DEBUG:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:50:18.235 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T07:50:18.236 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T07:50:18.328 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2023-11-24T07:50:18.337 DEBUG:teuthology.orchestra.run.smithi091:> sudo service rsyslog restart 2023-11-24T07:50:18.480 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T07:50:18.493 INFO:teuthology.orchestra.run.smithi091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T07:50:18.952 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T07:50:18.961 INFO:teuthology.task.internal:Starting timer... 2023-11-24T07:50:18.961 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T07:50:18.995 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T07:50:19.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2023-11-24T07:50:19.157 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2023-11-24T07:50:19.162 DEBUG:teuthology.orchestra.run.smithi091:> sudo service auditd rotate 2023-11-24T07:50:19.297 INFO:teuthology.orchestra.run.smithi091.stdout:Rotating logs: 2023-11-24T07:50:19.301 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T07:50:19.301 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2023-11-24T07:50:19.376 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2023-11-24T07:50:19.377 DEBUG:teuthology.orchestra.run.smithi091:> /usr/sbin/getenforce 2023-11-24T07:50:19.447 INFO:teuthology.orchestra.run.smithi091.stdout:Permissive 2023-11-24T07:50:19.447 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi091.front.sepia.ceph.com': 'permissive'} 2023-11-24T07:50:19.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T07:50:19.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:50:19.548 DEBUG:teuthology.orchestra.run.smithi091:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T07:50:19.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:50:19.647 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T07:50:19.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2023-11-24T07:50:19.755 DEBUG:teuthology.orchestra.run.smithi091:> sudo /usr/sbin/setenforce permissive 2023-11-24T07:50:19.851 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T07:50:19.863 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-24T07:50:19.863 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T07:50:19.879 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T07:50:19.880 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi091.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T08:00:43.350 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi091.front.sepia.ceph.com')] 2023-11-24T08:00:43.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T08:00:43.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T08:00:43.491 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-11-24T08:00:43.648 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-11-24T08:00:43.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com' 2023-11-24T08:00:43.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T08:00:43.790 DEBUG:teuthology.orchestra.run.smithi091:> true 2023-11-24T08:00:43.947 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi091.front.sepia.ceph.com' 2023-11-24T08:00:43.947 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T08:00:43.959 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T08:00:43.959 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T08:00:43.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:00:43.969 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T08:00:43.969 DEBUG:teuthology.orchestra.run.smithi091:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:00:44.072 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T08:00:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T08:00:44.125 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T08:00:44.135 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T08:00:44.211 INFO:teuthology.orchestra.run.smithi091.stderr:sudo: ntpd: command not found 2023-11-24T08:00:44.238 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2023-11-24T08:00:44.250 INFO:teuthology.orchestra.run.smithi091.stdout:506 Cannot talk to daemon 2023-11-24T08:00:44.283 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2023-11-24T08:00:44.303 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T08:00:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T08:00:44.354 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T08:00:44.397 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T08:00:44.483 INFO:teuthology.orchestra.run.smithi091.stderr:bash: line 1: ntpq: command not found 2023-11-24T08:00:44.490 INFO:teuthology.orchestra.run.smithi091.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:00:44.491 INFO:teuthology.orchestra.run.smithi091.stdout:=============================================================================== 2023-11-24T08:00:44.491 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:44.491 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:44.491 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:44.491 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:44.536 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2023-11-24T08:00:44.544 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:00:44.544 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2023-11-24T08:00:44.544 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:44.544 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:44.545 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:44.545 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:44.546 INFO:teuthology.run_tasks:Running task install... 2023-11-24T08:00:44.555 DEBUG:teuthology.task.install:project ceph 2023-11-24T08:00:44.555 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T08:00:44.555 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T08:00:44.556 INFO:teuthology.task.install:Using flavor: default 2023-11-24T08:00:44.562 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T08:00:44.562 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T08:00:44.562 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T08:00:44.563 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T08:00:44.564 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T08:00:44.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T08:00:44.784 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T08:00:44.784 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T08:00:44.789 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T08:00:44.789 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T08:00:44.900 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T08:00:44.900 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:00:44.901 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T08:00:44.952 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T08:00:44.952 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:00:44.952 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T08:00:45.003 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T08:00:45.003 DEBUG:teuthology.orchestra.run.smithi091:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T08:00:45.072 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T08:00:45.072 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T08:00:45.143 DEBUG:teuthology.orchestra.run.smithi091:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T08:00:45.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T08:00:45.269 DEBUG:teuthology.orchestra.run.smithi091:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T08:00:45.355 INFO:teuthology.orchestra.run.smithi091.stdout:check_obsoletes = 1 2023-11-24T08:00:45.359 DEBUG:teuthology.orchestra.run.smithi091:> sudo yum clean all 2023-11-24T08:00:45.368 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T08:00:45.467 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2023-11-24T08:00:45.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2023-11-24T08:00:45.705 INFO:teuthology.orchestra.run.smithi091.stdout:73 files removed 2023-11-24T08:00:45.744 DEBUG:teuthology.orchestra.run.smithi091:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T08:00:45.841 INFO:teuthology.orchestra.run.smithi064.stdout:73 files removed 2023-11-24T08:00:45.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T08:00:46.701 INFO:teuthology.orchestra.run.smithi091.stdout:ceph packages for x86_64 172 kB/s | 78 kB 00:00 2023-11-24T08:00:46.898 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 160 kB/s | 78 kB 00:00 2023-11-24T08:00:47.195 INFO:teuthology.orchestra.run.smithi091.stdout:ceph noarch packages 24 kB/s | 9.5 kB 00:00 2023-11-24T08:00:47.465 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 20 kB/s | 9.5 kB 00:00 2023-11-24T08:00:47.647 INFO:teuthology.orchestra.run.smithi091.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2023-11-24T08:00:47.966 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2023-11-24T08:00:48.350 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - AppStream 10 MB/s | 6.4 MB 00:00 2023-11-24T08:00:48.703 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.4 MB 00:00 2023-11-24T08:00:50.151 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2023-11-24T08:00:50.495 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2023-11-24T08:00:51.708 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 7.9 MB 00:00 2023-11-24T08:00:54.521 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 7.9 MB 00:03 2023-11-24T08:00:57.359 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 18 MB 00:04 2023-11-24T08:00:58.144 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 18 MB 00:02 2023-11-24T08:01:01.761 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 5.8 MB 00:00 2023-11-24T08:01:03.572 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 5.8 MB 00:01 2023-11-24T08:01:03.645 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2023-11-24T08:01:05.615 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 15 kB 00:00 2023-11-24T08:01:08.360 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 20 MB 00:04 2023-11-24T08:01:09.038 INFO:teuthology.orchestra.run.smithi091.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 20 MB 00:03 2023-11-24T08:01:14.129 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2023-11-24T08:01:14.574 INFO:teuthology.orchestra.run.smithi091.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2023-11-24T08:01:17.153 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T08:01:17.155 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T08:01:17.177 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T08:01:17.178 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T08:01:17.180 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T08:01:17.181 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T08:01:17.184 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T08:01:17.188 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T08:01:17.189 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T08:01:17.275 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-11-24T08:01:17.304 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2023-11-24T08:01:17.304 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2023-11-24T08:01:17.304 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T08:01:17.305 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T08:01:17.306 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T08:01:17.307 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T08:01:17.308 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T08:01:17.308 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T08:01:17.308 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T08:01:17.308 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T08:01:17.308 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T08:01:17.308 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T08:01:17.308 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T08:01:17.309 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T08:01:17.310 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T08:01:17.311 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T08:01:17.312 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T08:01:17.313 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T08:01:17.314 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T08:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T08:01:17.316 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T08:01:17.317 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T08:01:17.317 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T08:01:17.317 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T08:01:17.317 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T08:01:17.317 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T08:01:17.317 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T08:01:17.318 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:01:17.319 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-11-24T08:01:17.320 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2023-11-24T08:01:17.320 INFO:teuthology.orchestra.run.smithi064.stdout:Install 132 Packages 2023-11-24T08:01:17.320 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 15 Packages 2023-11-24T08:01:17.320 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:01:17.321 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 204 M 2023-11-24T08:01:17.321 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2023-11-24T08:01:17.415 INFO:teuthology.orchestra.run.smithi091.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T08:01:17.417 INFO:teuthology.orchestra.run.smithi091.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T08:01:17.438 INFO:teuthology.orchestra.run.smithi091.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T08:01:17.439 INFO:teuthology.orchestra.run.smithi091.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T08:01:17.440 INFO:teuthology.orchestra.run.smithi091.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T08:01:17.442 INFO:teuthology.orchestra.run.smithi091.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T08:01:17.444 INFO:teuthology.orchestra.run.smithi091.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T08:01:17.448 INFO:teuthology.orchestra.run.smithi091.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T08:01:17.449 INFO:teuthology.orchestra.run.smithi091.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T08:01:17.528 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies resolved. 2023-11-24T08:01:17.555 INFO:teuthology.orchestra.run.smithi091.stdout:============================================================================================ 2023-11-24T08:01:17.555 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2023-11-24T08:01:17.555 INFO:teuthology.orchestra.run.smithi091.stdout:============================================================================================ 2023-11-24T08:01:17.556 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2023-11-24T08:01:17.556 INFO:teuthology.orchestra.run.smithi091.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T08:01:17.556 INFO:teuthology.orchestra.run.smithi091.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T08:01:17.556 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T08:01:17.556 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T08:01:17.556 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T08:01:17.557 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T08:01:17.557 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T08:01:17.557 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T08:01:17.557 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T08:01:17.557 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T08:01:17.558 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T08:01:17.558 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T08:01:17.558 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T08:01:17.558 INFO:teuthology.orchestra.run.smithi091.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T08:01:17.558 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T08:01:17.559 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T08:01:17.559 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T08:01:17.559 INFO:teuthology.orchestra.run.smithi091.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T08:01:17.559 INFO:teuthology.orchestra.run.smithi091.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T08:01:17.559 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T08:01:17.559 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T08:01:17.560 INFO:teuthology.orchestra.run.smithi091.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T08:01:17.560 INFO:teuthology.orchestra.run.smithi091.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T08:01:17.560 INFO:teuthology.orchestra.run.smithi091.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T08:01:17.560 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T08:01:17.560 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T08:01:17.560 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T08:01:17.561 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T08:01:17.561 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T08:01:17.561 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T08:01:17.561 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T08:01:17.561 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T08:01:17.561 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T08:01:17.562 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T08:01:17.562 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T08:01:17.562 INFO:teuthology.orchestra.run.smithi091.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T08:01:17.562 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrading: 2023-11-24T08:01:17.562 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T08:01:17.562 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T08:01:17.563 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T08:01:17.563 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T08:01:17.563 INFO:teuthology.orchestra.run.smithi091.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T08:01:17.563 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T08:01:17.563 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T08:01:17.563 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T08:01:17.564 INFO:teuthology.orchestra.run.smithi091.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T08:01:17.564 INFO:teuthology.orchestra.run.smithi091.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T08:01:17.564 INFO:teuthology.orchestra.run.smithi091.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T08:01:17.564 INFO:teuthology.orchestra.run.smithi091.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T08:01:17.564 INFO:teuthology.orchestra.run.smithi091.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T08:01:17.564 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T08:01:17.565 INFO:teuthology.orchestra.run.smithi091.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T08:01:17.565 INFO:teuthology.orchestra.run.smithi091.stdout:Installing dependencies: 2023-11-24T08:01:17.565 INFO:teuthology.orchestra.run.smithi091.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T08:01:17.565 INFO:teuthology.orchestra.run.smithi091.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T08:01:17.565 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T08:01:17.565 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T08:01:17.566 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T08:01:17.566 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T08:01:17.566 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T08:01:17.566 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T08:01:17.566 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T08:01:17.567 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T08:01:17.567 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T08:01:17.567 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T08:01:17.567 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T08:01:17.567 INFO:teuthology.orchestra.run.smithi091.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T08:01:17.567 INFO:teuthology.orchestra.run.smithi091.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T08:01:17.568 INFO:teuthology.orchestra.run.smithi091.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T08:01:17.568 INFO:teuthology.orchestra.run.smithi091.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T08:01:17.568 INFO:teuthology.orchestra.run.smithi091.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T08:01:17.568 INFO:teuthology.orchestra.run.smithi091.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T08:01:17.568 INFO:teuthology.orchestra.run.smithi091.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T08:01:17.568 INFO:teuthology.orchestra.run.smithi091.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T08:01:17.569 INFO:teuthology.orchestra.run.smithi091.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T08:01:17.569 INFO:teuthology.orchestra.run.smithi091.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T08:01:17.569 INFO:teuthology.orchestra.run.smithi091.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T08:01:17.569 INFO:teuthology.orchestra.run.smithi091.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T08:01:17.569 INFO:teuthology.orchestra.run.smithi091.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T08:01:17.569 INFO:teuthology.orchestra.run.smithi091.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T08:01:17.570 INFO:teuthology.orchestra.run.smithi091.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T08:01:17.570 INFO:teuthology.orchestra.run.smithi091.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T08:01:17.570 INFO:teuthology.orchestra.run.smithi091.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T08:01:17.570 INFO:teuthology.orchestra.run.smithi091.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T08:01:17.570 INFO:teuthology.orchestra.run.smithi091.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T08:01:17.570 INFO:teuthology.orchestra.run.smithi091.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T08:01:17.571 INFO:teuthology.orchestra.run.smithi091.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T08:01:17.571 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T08:01:17.571 INFO:teuthology.orchestra.run.smithi091.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T08:01:17.571 INFO:teuthology.orchestra.run.smithi091.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T08:01:17.571 INFO:teuthology.orchestra.run.smithi091.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T08:01:17.572 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T08:01:17.572 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T08:01:17.572 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T08:01:17.572 INFO:teuthology.orchestra.run.smithi091.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T08:01:17.572 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T08:01:17.572 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T08:01:17.573 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T08:01:17.573 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T08:01:17.573 INFO:teuthology.orchestra.run.smithi091.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T08:01:17.573 INFO:teuthology.orchestra.run.smithi091.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T08:01:17.573 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T08:01:17.573 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T08:01:17.574 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T08:01:17.574 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T08:01:17.574 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T08:01:17.574 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T08:01:17.574 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T08:01:17.574 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T08:01:17.575 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T08:01:17.575 INFO:teuthology.orchestra.run.smithi091.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T08:01:17.575 INFO:teuthology.orchestra.run.smithi091.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T08:01:17.575 INFO:teuthology.orchestra.run.smithi091.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T08:01:17.575 INFO:teuthology.orchestra.run.smithi091.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T08:01:17.575 INFO:teuthology.orchestra.run.smithi091.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T08:01:17.576 INFO:teuthology.orchestra.run.smithi091.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T08:01:17.576 INFO:teuthology.orchestra.run.smithi091.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T08:01:17.576 INFO:teuthology.orchestra.run.smithi091.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T08:01:17.576 INFO:teuthology.orchestra.run.smithi091.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T08:01:17.576 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T08:01:17.576 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T08:01:17.577 INFO:teuthology.orchestra.run.smithi091.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T08:01:17.577 INFO:teuthology.orchestra.run.smithi091.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T08:01:17.577 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T08:01:17.577 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T08:01:17.577 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T08:01:17.578 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T08:01:17.578 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T08:01:17.578 INFO:teuthology.orchestra.run.smithi091.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T08:01:17.578 INFO:teuthology.orchestra.run.smithi091.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T08:01:17.578 INFO:teuthology.orchestra.run.smithi091.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T08:01:17.578 INFO:teuthology.orchestra.run.smithi091.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T08:01:17.579 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T08:01:17.579 INFO:teuthology.orchestra.run.smithi091.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T08:01:17.579 INFO:teuthology.orchestra.run.smithi091.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T08:01:17.579 INFO:teuthology.orchestra.run.smithi091.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T08:01:17.579 INFO:teuthology.orchestra.run.smithi091.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T08:01:17.579 INFO:teuthology.orchestra.run.smithi091.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T08:01:17.580 INFO:teuthology.orchestra.run.smithi091.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T08:01:17.580 INFO:teuthology.orchestra.run.smithi091.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T08:01:17.580 INFO:teuthology.orchestra.run.smithi091.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T08:01:17.580 INFO:teuthology.orchestra.run.smithi091.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T08:01:17.580 INFO:teuthology.orchestra.run.smithi091.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T08:01:17.580 INFO:teuthology.orchestra.run.smithi091.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T08:01:17.581 INFO:teuthology.orchestra.run.smithi091.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T08:01:17.581 INFO:teuthology.orchestra.run.smithi091.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T08:01:17.581 INFO:teuthology.orchestra.run.smithi091.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T08:01:17.581 INFO:teuthology.orchestra.run.smithi091.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T08:01:17.581 INFO:teuthology.orchestra.run.smithi091.stdout:Installing weak dependencies: 2023-11-24T08:01:17.581 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T08:01:17.582 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:01:17.582 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2023-11-24T08:01:17.582 INFO:teuthology.orchestra.run.smithi091.stdout:============================================================================================ 2023-11-24T08:01:17.582 INFO:teuthology.orchestra.run.smithi091.stdout:Install 132 Packages 2023-11-24T08:01:17.582 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrade 15 Packages 2023-11-24T08:01:17.582 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:01:17.583 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 204 M 2023-11-24T08:01:17.583 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading Packages: 2023-11-24T08:01:17.989 INFO:teuthology.orchestra.run.smithi064.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 25 kB/s | 6.5 kB 00:00 2023-11-24T08:01:18.222 INFO:teuthology.orchestra.run.smithi064.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 3.6 MB/s | 859 kB 00:00 2023-11-24T08:01:18.306 INFO:teuthology.orchestra.run.smithi064.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 1.7 MB/s | 141 kB 00:00 2023-11-24T08:01:18.570 INFO:teuthology.orchestra.run.smithi091.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 16 kB/s | 6.5 kB 00:00 2023-11-24T08:01:18.590 INFO:teuthology.orchestra.run.smithi064.stdout:(4/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 6.1 MB/s | 5.2 MB 00:00 2023-11-24T08:01:18.690 INFO:teuthology.orchestra.run.smithi064.stdout:(5/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 5.5 MB/s | 2.1 MB 00:00 2023-11-24T08:01:18.782 INFO:teuthology.orchestra.run.smithi064.stdout:(6/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 7.7 MB/s | 1.5 MB 00:00 2023-11-24T08:01:19.340 INFO:teuthology.orchestra.run.smithi064.stdout:(7/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 6.8 MB/s | 4.4 MB 00:00 2023-11-24T08:01:19.608 INFO:teuthology.orchestra.run.smithi091.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 828 kB/s | 859 kB 00:01 2023-11-24T08:01:19.724 INFO:teuthology.orchestra.run.smithi064.stdout:(8/147): ceph-common-18.0.0-7484.ga13ea5ff.el9. 9.5 MB/s | 19 MB 00:01 2023-11-24T08:01:19.725 INFO:teuthology.orchestra.run.smithi091.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 1.2 MB/s | 141 kB 00:00 2023-11-24T08:01:19.824 INFO:teuthology.orchestra.run.smithi064.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 243 kB/s | 24 kB 00:00 2023-11-24T08:01:20.500 INFO:teuthology.orchestra.run.smithi064.stdout:(10/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 7.5 MB/s | 8.6 MB 00:01 2023-11-24T08:01:20.575 INFO:teuthology.orchestra.run.smithi064.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.9 MB/s | 217 kB 00:00 2023-11-24T08:01:20.676 INFO:teuthology.orchestra.run.smithi091.stdout:(4/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 2.1 MB/s | 5.2 MB 00:02 2023-11-24T08:01:20.689 INFO:teuthology.orchestra.run.smithi064.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 275 kB/s | 31 kB 00:00 2023-11-24T08:01:20.875 INFO:teuthology.orchestra.run.smithi064.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 3.8 MB/s | 719 kB 00:00 2023-11-24T08:01:20.976 INFO:teuthology.orchestra.run.smithi064.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.6 MB/s | 163 kB 00:00 2023-11-24T08:01:21.051 INFO:teuthology.orchestra.run.smithi064.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.7 MB/s | 126 kB 00:00 2023-11-24T08:01:21.080 INFO:teuthology.orchestra.run.smithi091.stdout:(5/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 3.6 MB/s | 1.5 MB 00:00 2023-11-24T08:01:21.202 INFO:teuthology.orchestra.run.smithi064.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 3.1 MB/s | 476 kB 00:00 2023-11-24T08:01:21.452 INFO:teuthology.orchestra.run.smithi064.stdout:(17/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 6.6 MB/s | 18 MB 00:02 2023-11-24T08:01:21.544 INFO:teuthology.orchestra.run.smithi064.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 498 kB/s | 45 kB 00:00 2023-11-24T08:01:21.618 INFO:teuthology.orchestra.run.smithi091.stdout:(6/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 1.1 MB/s | 2.1 MB 00:01 2023-11-24T08:01:21.752 INFO:teuthology.orchestra.run.smithi064.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 620 kB/s | 129 kB 00:00 2023-11-24T08:01:21.886 INFO:teuthology.orchestra.run.smithi064.stdout:(20/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 7.0 MB/s | 4.8 MB 00:00 2023-11-24T08:01:21.901 INFO:teuthology.orchestra.run.smithi091.stdout:(7/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 5.4 MB/s | 4.4 MB 00:00 2023-11-24T08:01:21.920 INFO:teuthology.orchestra.run.smithi064.stdout:(21/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 977 kB/s | 162 kB 00:00 2023-11-24T08:01:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:(22/147): python3-rados-18.0.0-7484.ga13ea5ff.e 4.2 MB/s | 319 kB 00:00 2023-11-24T08:01:22.020 INFO:teuthology.orchestra.run.smithi064.stdout:(23/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 2.9 MB/s | 298 kB 00:00 2023-11-24T08:01:22.054 INFO:teuthology.orchestra.run.smithi064.stdout:(24/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.1 MB/s | 99 kB 00:00 2023-11-24T08:01:22.113 INFO:teuthology.orchestra.run.smithi064.stdout:(25/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 936 kB/s | 85 kB 00:00 2023-11-24T08:01:22.596 INFO:teuthology.orchestra.run.smithi064.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 352 kB/s | 170 kB 00:00 2023-11-24T08:01:22.730 INFO:teuthology.orchestra.run.smithi064.stdout:(27/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 4.5 MB/s | 3.0 MB 00:00 2023-11-24T08:01:22.755 INFO:teuthology.orchestra.run.smithi064.stdout:(28/147): ceph-grafana-dashboards-18.0.0-7484.g 178 kB/s | 28 kB 00:00 2023-11-24T08:01:22.814 INFO:teuthology.orchestra.run.smithi064.stdout:(29/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.6 MB/s | 133 kB 00:00 2023-11-24T08:01:23.272 INFO:teuthology.orchestra.run.smithi064.stdout:(30/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 3.4 MB/s | 1.8 MB 00:00 2023-11-24T08:01:23.310 INFO:teuthology.orchestra.run.smithi091.stdout:(8/147): ceph-common-18.0.0-7484.ga13ea5ff.el9. 3.7 MB/s | 19 MB 00:05 2023-11-24T08:01:23.356 INFO:teuthology.orchestra.run.smithi064.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 2.9 MB/s | 243 kB 00:00 2023-11-24T08:01:23.414 INFO:teuthology.orchestra.run.smithi064.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 873 kB/s | 50 kB 00:00 2023-11-24T08:01:23.473 INFO:teuthology.orchestra.run.smithi064.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7484.ga 264 kB/s | 15 kB 00:00 2023-11-24T08:01:23.493 INFO:teuthology.orchestra.run.smithi091.stdout:(9/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el9 5.4 MB/s | 8.6 MB 00:01 2023-11-24T08:01:23.519 INFO:teuthology.orchestra.run.smithi091.stdout:(10/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el 116 kB/s | 24 kB 00:00 2023-11-24T08:01:23.590 INFO:teuthology.orchestra.run.smithi064.stdout:(34/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 2.2 MB/s | 262 kB 00:00 2023-11-24T08:01:23.619 INFO:teuthology.orchestra.run.smithi091.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.1 MB/s | 217 kB 00:00 2023-11-24T08:01:23.711 INFO:teuthology.orchestra.run.smithi091.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 343 kB/s | 31 kB 00:00 2023-11-24T08:01:23.807 INFO:teuthology.orchestra.run.smithi064.stdout:(35/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 3.5 MB/s | 772 kB 00:00 2023-11-24T08:01:23.878 INFO:teuthology.orchestra.run.smithi091.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 4.2 MB/s | 719 kB 00:00 2023-11-24T08:01:23.890 INFO:teuthology.orchestra.run.smithi064.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 221 kB/s | 18 kB 00:00 2023-11-24T08:01:23.953 INFO:teuthology.orchestra.run.smithi091.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 2.2 MB/s | 163 kB 00:00 2023-11-24T08:01:24.037 INFO:teuthology.orchestra.run.smithi091.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.5 MB/s | 126 kB 00:00 2023-11-24T08:01:24.041 INFO:teuthology.orchestra.run.smithi064.stdout:(37/147): ceph-mgr-diskprediction-local-18.0.0- 6.0 MB/s | 7.4 MB 00:01 2023-11-24T08:01:24.120 INFO:teuthology.orchestra.run.smithi091.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 5.6 MB/s | 476 kB 00:00 2023-11-24T08:01:24.333 INFO:teuthology.orchestra.run.smithi064.stdout:(38/147): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2023-11-24T08:01:24.375 INFO:teuthology.orchestra.run.smithi064.stdout:(39/147): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 920 kB 00:00 2023-11-24T08:01:24.400 INFO:teuthology.orchestra.run.smithi064.stdout:(40/147): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 106 kB 00:00 2023-11-24T08:01:24.417 INFO:teuthology.orchestra.run.smithi064.stdout:(41/147): elfutils-libelf-devel-0.190-1.el9.x86 540 kB/s | 21 kB 00:00 2023-11-24T08:01:24.459 INFO:teuthology.orchestra.run.smithi064.stdout:(42/147): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2023-11-24T08:01:24.476 INFO:teuthology.orchestra.run.smithi064.stdout:(43/147): flexiblas-3.0.4-8.el9.x86_64.rpm 547 kB/s | 32 kB 00:00 2023-11-24T08:01:24.526 INFO:teuthology.orchestra.run.smithi064.stdout:(44/147): flexiblas-openblas-openmp-3.0.4-8.el9 348 kB/s | 17 kB 00:00 2023-11-24T08:01:24.618 INFO:teuthology.orchestra.run.smithi064.stdout:(45/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2023-11-24T08:01:24.637 INFO:teuthology.orchestra.run.smithi091.stdout:(17/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 9.3 MB/s | 4.8 MB 00:00 2023-11-24T08:01:24.652 INFO:teuthology.orchestra.run.smithi064.stdout:(46/147): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 186 kB 00:00 2023-11-24T08:01:24.704 INFO:teuthology.orchestra.run.smithi091.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 688 kB/s | 45 kB 00:00 2023-11-24T08:01:24.708 INFO:teuthology.orchestra.run.smithi064.stdout:(47/147): libaio-devel-0.3.111-13.el9.x86_64.rp 115 kB/s | 10 kB 00:00 2023-11-24T08:01:24.745 INFO:teuthology.orchestra.run.smithi064.stdout:(48/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2023-11-24T08:01:24.769 INFO:teuthology.orchestra.run.smithi064.stdout:(49/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 757 kB/s | 44 kB 00:00 2023-11-24T08:01:24.771 INFO:teuthology.orchestra.run.smithi091.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 1.9 MB/s | 129 kB 00:00 2023-11-24T08:01:24.811 INFO:teuthology.orchestra.run.smithi064.stdout:(50/147): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 661 kB 00:00 2023-11-24T08:01:24.853 INFO:teuthology.orchestra.run.smithi064.stdout:(51/147): libtool-2.4.6-45.el9.x86_64.rpm 7.4 MB/s | 578 kB 00:00 2023-11-24T08:01:24.854 INFO:teuthology.orchestra.run.smithi091.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.9 MB/s | 162 kB 00:00 2023-11-24T08:01:24.879 INFO:teuthology.orchestra.run.smithi064.stdout:(52/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 405 kB/s | 24 kB 00:00 2023-11-24T08:01:24.904 INFO:teuthology.orchestra.run.smithi064.stdout:(53/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 46 kB 00:00 2023-11-24T08:01:24.937 INFO:teuthology.orchestra.run.smithi064.stdout:(54/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-11-24T08:01:24.938 INFO:teuthology.orchestra.run.smithi091.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 3.8 MB/s | 319 kB 00:00 2023-11-24T08:01:24.971 INFO:teuthology.orchestra.run.smithi064.stdout:(55/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-11-24T08:01:24.996 INFO:teuthology.orchestra.run.smithi064.stdout:(56/147): openblas-0.3.21-2.el9.x86_64.rpm 614 kB/s | 34 kB 00:00 2023-11-24T08:01:25.030 INFO:teuthology.orchestra.run.smithi091.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 3.2 MB/s | 298 kB 00:00 2023-11-24T08:01:25.097 INFO:teuthology.orchestra.run.smithi091.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.5 MB/s | 99 kB 00:00 2023-11-24T08:01:25.138 INFO:teuthology.orchestra.run.smithi064.stdout:(57/147): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2023-11-24T08:01:25.164 INFO:teuthology.orchestra.run.smithi091.stdout:(24/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.3 MB/s | 85 kB 00:00 2023-11-24T08:01:25.197 INFO:teuthology.orchestra.run.smithi064.stdout:(58/147): postgresql-13.11-1.el9.x86_64.rpm 8.3 MB/s | 1.5 MB 00:00 2023-11-24T08:01:25.239 INFO:teuthology.orchestra.run.smithi064.stdout:(59/147): postgresql-contrib-13.11-1.el9.x86_64 8.0 MB/s | 811 kB 00:00 2023-11-24T08:01:25.294 INFO:teuthology.orchestra.run.smithi064.stdout:(60/147): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2023-11-24T08:01:25.472 INFO:teuthology.orchestra.run.smithi091.stdout:(25/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 9.8 MB/s | 3.0 MB 00:00 2023-11-24T08:01:25.539 INFO:teuthology.orchestra.run.smithi091.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 2.5 MB/s | 170 kB 00:00 2023-11-24T08:01:25.615 INFO:teuthology.orchestra.run.smithi091.stdout:(27/147): ceph-grafana-dashboards-18.0.0-7484.g 379 kB/s | 28 kB 00:00 2023-11-24T08:01:25.690 INFO:teuthology.orchestra.run.smithi091.stdout:(28/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.8 MB/s | 133 kB 00:00 2023-11-24T08:01:25.765 INFO:teuthology.orchestra.run.smithi064.stdout:(61/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 6.9 MB/s | 41 MB 00:05 2023-11-24T08:01:25.898 INFO:teuthology.orchestra.run.smithi091.stdout:(29/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 8.5 MB/s | 1.8 MB 00:00 2023-11-24T08:01:25.906 INFO:teuthology.orchestra.run.smithi064.stdout:(62/147): postgresql-server-13.11-1.el9.x86_64. 8.5 MB/s | 5.7 MB 00:00 2023-11-24T08:01:26.049 INFO:teuthology.orchestra.run.smithi064.stdout:(63/147): python3-babel-2.9.1-2.el9.noarch.rpm 7.8 MB/s | 5.8 MB 00:00 2023-11-24T08:01:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:(64/147): python3-jinja2-2.11.3-4.el9.noarch.rp 769 kB/s | 228 kB 00:00 2023-11-24T08:01:26.099 INFO:teuthology.orchestra.run.smithi064.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 137 kB/s | 24 kB 00:00 2023-11-24T08:01:26.149 INFO:teuthology.orchestra.run.smithi064.stdout:(66/147): python3-jsonpointer-2.0-4.el9.noarch. 372 kB/s | 18 kB 00:00 2023-11-24T08:01:26.175 INFO:teuthology.orchestra.run.smithi064.stdout:(67/147): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-11-24T08:01:26.200 INFO:teuthology.orchestra.run.smithi064.stdout:(68/147): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 32 kB 00:00 2023-11-24T08:01:26.342 INFO:teuthology.orchestra.run.smithi064.stdout:(69/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-11-24T08:01:26.367 INFO:teuthology.orchestra.run.smithi064.stdout:(70/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2023-11-24T08:01:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 970 kB/s | 174 kB 00:00 2023-11-24T08:01:26.451 INFO:teuthology.orchestra.run.smithi064.stdout:(72/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2023-11-24T08:01:26.484 INFO:teuthology.orchestra.run.smithi064.stdout:(73/147): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2023-11-24T08:01:26.510 INFO:teuthology.orchestra.run.smithi064.stdout:(74/147): python3-prettytable-0.7.2-27.el9.noar 356 kB/s | 41 kB 00:00 2023-11-24T08:01:26.532 INFO:teuthology.orchestra.run.smithi091.stdout:(30/147): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-11-24T08:01:26.535 INFO:teuthology.orchestra.run.smithi064.stdout:(75/147): python3-pytz-2021.1-5.el9.noarch.rpm 579 kB/s | 47 kB 00:00 2023-11-24T08:01:26.560 INFO:teuthology.orchestra.run.smithi064.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 581 kB/s | 43 kB 00:00 2023-11-24T08:01:26.594 INFO:teuthology.orchestra.run.smithi064.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 681 kB/s | 39 kB 00:00 2023-11-24T08:01:26.599 INFO:teuthology.orchestra.run.smithi091.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 3.6 MB/s | 243 kB 00:00 2023-11-24T08:01:26.691 INFO:teuthology.orchestra.run.smithi091.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 554 kB/s | 50 kB 00:00 2023-11-24T08:01:26.766 INFO:teuthology.orchestra.run.smithi091.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7484.ga 204 kB/s | 15 kB 00:00 2023-11-24T08:01:26.842 INFO:teuthology.orchestra.run.smithi091.stdout:(34/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 3.5 MB/s | 262 kB 00:00 2023-11-24T08:01:26.919 INFO:teuthology.orchestra.run.smithi064.stdout:(78/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2023-11-24T08:01:26.950 INFO:teuthology.orchestra.run.smithi091.stdout:(35/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 7.0 MB/s | 772 kB 00:00 2023-11-24T08:01:26.952 INFO:teuthology.orchestra.run.smithi064.stdout:(79/147): socat-1.7.4.1-5.el9.x86_64.rpm 778 kB/s | 300 kB 00:00 2023-11-24T08:01:26.969 INFO:teuthology.orchestra.run.smithi064.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 157 kB/s | 56 kB 00:00 2023-11-24T08:01:27.025 INFO:teuthology.orchestra.run.smithi091.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 246 kB/s | 18 kB 00:00 2023-11-24T08:01:27.028 INFO:teuthology.orchestra.run.smithi064.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 803 kB/s | 85 kB 00:00 2023-11-24T08:01:27.045 INFO:teuthology.orchestra.run.smithi064.stdout:(82/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 602 kB/s | 44 kB 00:00 2023-11-24T08:01:27.070 INFO:teuthology.orchestra.run.smithi064.stdout:(83/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 634 kB/s | 63 kB 00:00 2023-11-24T08:01:27.104 INFO:teuthology.orchestra.run.smithi064.stdout:(84/147): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2023-11-24T08:01:27.137 INFO:teuthology.orchestra.run.smithi064.stdout:(85/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2023-11-24T08:01:27.171 INFO:teuthology.orchestra.run.smithi064.stdout:(86/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2023-11-24T08:01:27.213 INFO:teuthology.orchestra.run.smithi064.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 241 kB 00:00 2023-11-24T08:01:27.255 INFO:teuthology.orchestra.run.smithi064.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.1 MB 00:00 2023-11-24T08:01:27.280 INFO:teuthology.orchestra.run.smithi064.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2023-11-24T08:01:27.314 INFO:teuthology.orchestra.run.smithi064.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2023-11-24T08:01:27.339 INFO:teuthology.orchestra.run.smithi064.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.3 MB/s | 113 kB 00:00 2023-11-24T08:01:27.381 INFO:teuthology.orchestra.run.smithi064.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 1.9 MB/s | 187 kB 00:00 2023-11-24T08:01:27.417 INFO:teuthology.orchestra.run.smithi091.stdout:(37/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 3.1 MB/s | 18 MB 00:05 2023-11-24T08:01:27.476 INFO:teuthology.orchestra.run.smithi091.stdout:(38/147): automake-1.16.2-8.el9.noarch.rpm 1.4 MB/s | 662 kB 00:00 2023-11-24T08:01:27.576 INFO:teuthology.orchestra.run.smithi091.stdout:(39/147): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 920 kB 00:00 2023-11-24T08:01:27.598 INFO:teuthology.orchestra.run.smithi064.stdout:(93/147): python3-devel-3.9.17-2.el9.x86_64.rpm 961 kB/s | 245 kB 00:00 2023-11-24T08:01:27.601 INFO:teuthology.orchestra.run.smithi091.stdout:(40/147): boost-program-options-1.75.0-8.el9.x8 868 kB/s | 106 kB 00:00 2023-11-24T08:01:27.618 INFO:teuthology.orchestra.run.smithi091.stdout:(41/147): elfutils-libelf-devel-0.190-1.el9.x86 532 kB/s | 21 kB 00:00 2023-11-24T08:01:27.685 INFO:teuthology.orchestra.run.smithi091.stdout:(42/147): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2023-11-24T08:01:27.702 INFO:teuthology.orchestra.run.smithi091.stdout:(43/147): flexiblas-3.0.4-8.el9.x86_64.rpm 382 kB/s | 32 kB 00:00 2023-11-24T08:01:27.836 INFO:teuthology.orchestra.run.smithi091.stdout:(44/147): flexiblas-openblas-openmp-3.0.4-8.el9 129 kB/s | 17 kB 00:00 2023-11-24T08:01:27.919 INFO:teuthology.orchestra.run.smithi091.stdout:(45/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2023-11-24T08:01:27.953 INFO:teuthology.orchestra.run.smithi091.stdout:(46/147): jq-1.6-16.el9.x86_64.rpm 1.6 MB/s | 186 kB 00:00 2023-11-24T08:01:27.978 INFO:teuthology.orchestra.run.smithi091.stdout:(47/147): libaio-devel-0.3.111-13.el9.x86_64.rp 180 kB/s | 10 kB 00:00 2023-11-24T08:01:28.003 INFO:teuthology.orchestra.run.smithi091.stdout:(48/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2023-11-24T08:01:28.054 INFO:teuthology.orchestra.run.smithi091.stdout:(49/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 605 kB/s | 44 kB 00:00 2023-11-24T08:01:28.096 INFO:teuthology.orchestra.run.smithi091.stdout:(50/147): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2023-11-24T08:01:28.138 INFO:teuthology.orchestra.run.smithi091.stdout:(51/147): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2023-11-24T08:01:28.163 INFO:teuthology.orchestra.run.smithi091.stdout:(52/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 400 kB/s | 24 kB 00:00 2023-11-24T08:01:28.206 INFO:teuthology.orchestra.run.smithi064.stdout:(94/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 4.8 MB/s | 4.1 MB 00:00 2023-11-24T08:01:28.226 INFO:teuthology.orchestra.run.smithi091.stdout:(53/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 533 kB/s | 46 kB 00:00 2023-11-24T08:01:28.240 INFO:teuthology.orchestra.run.smithi064.stdout:(95/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 360 kB/s | 309 kB 00:00 2023-11-24T08:01:28.389 INFO:teuthology.orchestra.run.smithi091.stdout:(54/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 282 kB 00:00 2023-11-24T08:01:28.407 INFO:teuthology.orchestra.run.smithi064.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 293 kB/s | 49 kB 00:00 2023-11-24T08:01:28.439 INFO:teuthology.orchestra.run.smithi091.stdout:(55/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.0 MB/s | 217 kB 00:00 2023-11-24T08:01:28.464 INFO:teuthology.orchestra.run.smithi091.stdout:(56/147): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 34 kB 00:00 2023-11-24T08:01:28.541 INFO:teuthology.orchestra.run.smithi064.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 508 kB/s | 67 kB 00:00 2023-11-24T08:01:28.744 INFO:teuthology.orchestra.run.smithi064.stdout:(98/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 45 kB/s | 24 kB 00:00 2023-11-24T08:01:28.900 INFO:teuthology.orchestra.run.smithi064.stdout:(99/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.3 MB/s | 837 kB 00:00 2023-11-24T08:01:28.973 INFO:teuthology.orchestra.run.smithi091.stdout:(57/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 7.5 MB/s | 41 MB 00:05 2023-11-24T08:01:28.999 INFO:teuthology.orchestra.run.smithi064.stdout:(100/147): libarrow-9.0.0-9.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2023-11-24T08:01:29.025 INFO:teuthology.orchestra.run.smithi064.stdout:(101/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 350 kB/s | 43 kB 00:00 2023-11-24T08:01:29.090 INFO:teuthology.orchestra.run.smithi091.stdout:(58/147): openblas-openmp-0.3.21-2.el9.x86_64.r 7.3 MB/s | 4.7 MB 00:00 2023-11-24T08:01:29.125 INFO:teuthology.orchestra.run.smithi064.stdout:(102/147): python3-cachetools-4.2.4-1.el9.noarc 258 kB/s | 32 kB 00:00 2023-11-24T08:01:29.151 INFO:teuthology.orchestra.run.smithi064.stdout:(103/147): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2023-11-24T08:01:29.170 INFO:teuthology.orchestra.run.smithi091.stdout:(59/147): postgresql-13.11-1.el9.x86_64.rpm 2.2 MB/s | 1.5 MB 00:00 2023-11-24T08:01:29.207 INFO:teuthology.orchestra.run.smithi091.stdout:(60/147): postgresql-contrib-13.11-1.el9.x86_64 3.6 MB/s | 811 kB 00:00 2023-11-24T08:01:29.234 INFO:teuthology.orchestra.run.smithi091.stdout:(61/147): postgresql-private-libs-13.11-1.el9.x 982 kB/s | 134 kB 00:00 2023-11-24T08:01:29.268 INFO:teuthology.orchestra.run.smithi064.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2023-11-24T08:01:29.334 INFO:teuthology.orchestra.run.smithi064.stdout:(105/147): python3-asyncssh-2.13.2-3.el9.noarch 930 kB/s | 548 kB 00:00 2023-11-24T08:01:29.368 INFO:teuthology.orchestra.run.smithi064.stdout:(106/147): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2023-11-24T08:01:29.418 INFO:teuthology.orchestra.run.smithi064.stdout:(107/147): python3-google-auth-2.23.4-1.el9.noa 1.3 MB/s | 197 kB 00:00 2023-11-24T08:01:29.443 INFO:teuthology.orchestra.run.smithi091.stdout:(62/147): python3-jinja2-2.11.3-4.el9.noarch.rp 1.1 MB/s | 228 kB 00:00 2023-11-24T08:01:29.452 INFO:teuthology.orchestra.run.smithi064.stdout:(108/147): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2023-11-24T08:01:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:(109/147): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2023-11-24T08:01:29.544 INFO:teuthology.orchestra.run.smithi064.stdout:(110/147): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2023-11-24T08:01:29.568 INFO:teuthology.orchestra.run.smithi091.stdout:(63/147): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2023-11-24T08:01:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:(111/147): python3-jaraco-functools-3.5.0-2.el9 156 kB/s | 19 kB 00:00 2023-11-24T08:01:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:(112/147): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2023-11-24T08:01:29.670 INFO:teuthology.orchestra.run.smithi064.stdout:(113/147): python3-jwt+crypto-2.4.0-1.el9.noarc 72 kB/s | 9.0 kB 00:00 2023-11-24T08:01:29.685 INFO:teuthology.orchestra.run.smithi091.stdout:(64/147): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2023-11-24T08:01:29.703 INFO:teuthology.orchestra.run.smithi064.stdout:(114/147): python3-jwt-2.4.0-1.el9.noarch.rpm 327 kB/s | 41 kB 00:00 2023-11-24T08:01:29.710 INFO:teuthology.orchestra.run.smithi091.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 92 kB/s | 24 kB 00:00 2023-11-24T08:01:29.735 INFO:teuthology.orchestra.run.smithi091.stdout:(66/147): python3-jsonpointer-2.0-4.el9.noarch. 118 kB/s | 18 kB 00:00 2023-11-24T08:01:29.761 INFO:teuthology.orchestra.run.smithi091.stdout:(67/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-24T08:01:29.777 INFO:teuthology.orchestra.run.smithi091.stdout:(68/147): python3-markupsafe-1.1.1-12.el9.x86_6 483 kB/s | 32 kB 00:00 2023-11-24T08:01:29.779 INFO:teuthology.orchestra.run.smithi064.stdout:(115/147): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2023-11-24T08:01:29.804 INFO:teuthology.orchestra.run.smithi064.stdout:(116/147): python3-logutils-0.3.5-21.el9.noarch 348 kB/s | 46 kB 00:00 2023-11-24T08:01:29.838 INFO:teuthology.orchestra.run.smithi064.stdout:(117/147): python3-more-itertools-8.12.0-2.el9. 591 kB/s | 79 kB 00:00 2023-11-24T08:01:29.913 INFO:teuthology.orchestra.run.smithi064.stdout:(118/147): python3-natsort-7.1.1-5.el9.noarch.r 434 kB/s | 58 kB 00:00 2023-11-24T08:01:29.953 INFO:teuthology.orchestra.run.smithi091.stdout:(69/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2023-11-24T08:01:29.954 INFO:teuthology.orchestra.run.smithi064.stdout:(119/147): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2023-11-24T08:01:29.980 INFO:teuthology.orchestra.run.smithi064.stdout:(120/147): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2023-11-24T08:01:29.986 INFO:teuthology.orchestra.run.smithi091.stdout:(70/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-11-24T08:01:30.011 INFO:teuthology.orchestra.run.smithi091.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 795 kB/s | 174 kB 00:00 2023-11-24T08:01:30.037 INFO:teuthology.orchestra.run.smithi091.stdout:(72/147): python3-prettytable-0.7.2-27.el9.noar 517 kB/s | 41 kB 00:00 2023-11-24T08:01:30.047 INFO:teuthology.orchestra.run.smithi064.stdout:(121/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 676 kB/s | 90 kB 00:00 2023-11-24T08:01:30.070 INFO:teuthology.orchestra.run.smithi091.stdout:(73/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2023-11-24T08:01:30.080 INFO:teuthology.orchestra.run.smithi064.stdout:(122/147): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2023-11-24T08:01:30.095 INFO:teuthology.orchestra.run.smithi091.stdout:(74/147): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2023-11-24T08:01:30.120 INFO:teuthology.orchestra.run.smithi091.stdout:(75/147): python3-pytz-2021.1-5.el9.noarch.rpm 592 kB/s | 47 kB 00:00 2023-11-24T08:01:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:(123/147): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2023-11-24T08:01:30.154 INFO:teuthology.orchestra.run.smithi091.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 751 kB/s | 43 kB 00:00 2023-11-24T08:01:30.173 INFO:teuthology.orchestra.run.smithi064.stdout:(124/147): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2023-11-24T08:01:30.206 INFO:teuthology.orchestra.run.smithi064.stdout:(125/147): python3-tempora-5.0.0-2.el9.noarch.r 287 kB/s | 36 kB 00:00 2023-11-24T08:01:30.257 INFO:teuthology.orchestra.run.smithi064.stdout:(126/147): python3-typing-extensions-4.4.0-2.el 408 kB/s | 51 kB 00:00 2023-11-24T08:01:30.299 INFO:teuthology.orchestra.run.smithi064.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2023-11-24T08:01:30.315 INFO:teuthology.orchestra.run.smithi091.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 211 kB/s | 39 kB 00:00 2023-11-24T08:01:30.341 INFO:teuthology.orchestra.run.smithi064.stdout:(128/147): python3-websocket-client-1.2.3-2.el9 673 kB/s | 90 kB 00:00 2023-11-24T08:01:30.424 INFO:teuthology.orchestra.run.smithi064.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2023-11-24T08:01:30.476 INFO:teuthology.orchestra.run.smithi064.stdout:(130/147): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2023-11-24T08:01:30.510 INFO:teuthology.orchestra.run.smithi064.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2023-11-24T08:01:30.530 INFO:teuthology.orchestra.run.smithi091.stdout:(78/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2023-11-24T08:01:30.555 INFO:teuthology.orchestra.run.smithi091.stdout:(79/147): socat-1.7.4.1-5.el9.x86_64.rpm 777 kB/s | 300 kB 00:00 2023-11-24T08:01:30.580 INFO:teuthology.orchestra.run.smithi091.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 214 kB/s | 56 kB 00:00 2023-11-24T08:01:30.602 INFO:teuthology.orchestra.run.smithi064.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 8.9 MB/s | 1.6 MB 00:00 2023-11-24T08:01:30.630 INFO:teuthology.orchestra.run.smithi091.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 85 kB 00:00 2023-11-24T08:01:30.652 INFO:teuthology.orchestra.run.smithi064.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2023-11-24T08:01:30.655 INFO:teuthology.orchestra.run.smithi091.stdout:(82/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 852 kB/s | 63 kB 00:00 2023-11-24T08:01:30.714 INFO:teuthology.orchestra.run.smithi091.stdout:(83/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 334 kB/s | 44 kB 00:00 2023-11-24T08:01:30.761 INFO:teuthology.orchestra.run.smithi064.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 21 MB/s | 2.2 MB 00:00 2023-11-24T08:01:30.764 INFO:teuthology.orchestra.run.smithi091.stdout:(84/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2023-11-24T08:01:30.798 INFO:teuthology.orchestra.run.smithi091.stdout:(85/147): fuse-2.9.9-15.el9.x86_64.rpm 955 kB/s | 79 kB 00:00 2023-11-24T08:01:30.831 INFO:teuthology.orchestra.run.smithi091.stdout:(86/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 6.8 MB/s | 803 kB 00:00 2023-11-24T08:01:30.845 INFO:teuthology.orchestra.run.smithi064.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 21 MB/s | 1.8 MB 00:00 2023-11-24T08:01:30.865 INFO:teuthology.orchestra.run.smithi091.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 241 kB 00:00 2023-11-24T08:01:30.895 INFO:teuthology.orchestra.run.smithi064.stdout:(136/147): NetworkManager-team-1.45.6-1.el9.x86 860 kB/s | 42 kB 00:00 2023-11-24T08:01:30.899 INFO:teuthology.orchestra.run.smithi091.stdout:(88/147): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2023-11-24T08:01:30.945 INFO:teuthology.orchestra.run.smithi064.stdout:(137/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.9 MB/s | 245 kB 00:00 2023-11-24T08:01:30.948 INFO:teuthology.orchestra.run.smithi091.stdout:(89/147): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.1 MB 00:00 2023-11-24T08:01:30.974 INFO:teuthology.orchestra.run.smithi091.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2023-11-24T08:01:30.995 INFO:teuthology.orchestra.run.smithi064.stdout:(138/147): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2023-11-24T08:01:30.999 INFO:teuthology.orchestra.run.smithi091.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.2 MB/s | 113 kB 00:00 2023-11-24T08:01:31.033 INFO:teuthology.orchestra.run.smithi091.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-24T08:01:31.079 INFO:teuthology.orchestra.run.smithi064.stdout:(139/147): librados2-18.0.0-7484.ga13ea5ff.el9. 5.5 MB/s | 3.3 MB 00:00 2023-11-24T08:01:31.096 INFO:teuthology.orchestra.run.smithi064.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 367 kB/s | 36 kB 00:00 2023-11-24T08:01:31.130 INFO:teuthology.orchestra.run.smithi064.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2023-11-24T08:01:31.164 INFO:teuthology.orchestra.run.smithi064.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.8 MB/s | 255 kB 00:00 2023-11-24T08:01:31.189 INFO:teuthology.orchestra.run.smithi064.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 397 kB/s | 22 kB 00:00 2023-11-24T08:01:31.231 INFO:teuthology.orchestra.run.smithi064.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2023-11-24T08:01:31.258 INFO:teuthology.orchestra.run.smithi091.stdout:(93/147): python3-devel-3.9.17-2.el9.x86_64.rpm 957 kB/s | 245 kB 00:00 2023-11-24T08:01:31.314 INFO:teuthology.orchestra.run.smithi064.stdout:(145/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 3.8 MB/s | 3.0 MB 00:00 2023-11-24T08:01:31.666 INFO:teuthology.orchestra.run.smithi091.stdout:(94/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 6.0 MB/s | 4.1 MB 00:00 2023-11-24T08:01:31.731 INFO:teuthology.orchestra.run.smithi064.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 2.3 MB/s | 1.2 MB 00:00 2023-11-24T08:01:31.876 INFO:teuthology.orchestra.run.smithi091.stdout:(95/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 367 kB/s | 309 kB 00:00 2023-11-24T08:01:31.898 INFO:teuthology.orchestra.run.smithi064.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 3.2 MB/s | 2.2 MB 00:00 2023-11-24T08:01:31.913 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-11-24T08:01:31.913 INFO:teuthology.orchestra.run.smithi064.stdout:Total 14 MB/s | 204 MB 00:14 2023-11-24T08:01:32.017 INFO:teuthology.orchestra.run.smithi091.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 345 kB/s | 49 kB 00:00 2023-11-24T08:01:32.142 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2023-11-24T08:01:32.159 INFO:teuthology.orchestra.run.smithi091.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 478 kB/s | 67 kB 00:00 2023-11-24T08:01:32.226 INFO:teuthology.orchestra.run.smithi091.stdout:(98/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 44 kB/s | 24 kB 00:00 2023-11-24T08:01:32.436 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2023-11-24T08:01:32.437 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2023-11-24T08:01:32.575 INFO:teuthology.orchestra.run.smithi091.stdout:(99/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.0 MB/s | 837 kB 00:00 2023-11-24T08:01:32.668 INFO:teuthology.orchestra.run.smithi091.stdout:(100/147): libarrow-9.0.0-9.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2023-11-24T08:01:32.702 INFO:teuthology.orchestra.run.smithi091.stdout:(101/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2023-11-24T08:01:32.760 INFO:teuthology.orchestra.run.smithi091.stdout:(102/147): python3-asyncssh-2.13.2-3.el9.noarch 1.0 MB/s | 548 kB 00:00 2023-11-24T08:01:32.794 INFO:teuthology.orchestra.run.smithi091.stdout:(103/147): python3-cachetools-4.2.4-1.el9.noarc 258 kB/s | 32 kB 00:00 2023-11-24T08:01:32.828 INFO:teuthology.orchestra.run.smithi091.stdout:(104/147): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2023-11-24T08:01:32.903 INFO:teuthology.orchestra.run.smithi091.stdout:(105/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2023-11-24T08:01:32.970 INFO:teuthology.orchestra.run.smithi091.stdout:(106/147): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2023-11-24T08:01:33.003 INFO:teuthology.orchestra.run.smithi091.stdout:(107/147): python3-google-auth-2.23.4-1.el9.noa 1.1 MB/s | 197 kB 00:00 2023-11-24T08:01:33.029 INFO:teuthology.orchestra.run.smithi091.stdout:(108/147): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2023-11-24T08:01:33.087 INFO:teuthology.orchestra.run.smithi091.stdout:(109/147): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2023-11-24T08:01:33.129 INFO:teuthology.orchestra.run.smithi091.stdout:(110/147): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2023-11-24T08:01:33.154 INFO:teuthology.orchestra.run.smithi091.stdout:(111/147): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2023-11-24T08:01:33.205 INFO:teuthology.orchestra.run.smithi091.stdout:(112/147): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2023-11-24T08:01:33.246 INFO:teuthology.orchestra.run.smithi091.stdout:(113/147): python3-jwt+crypto-2.4.0-1.el9.noarc 78 kB/s | 9.0 kB 00:00 2023-11-24T08:01:33.280 INFO:teuthology.orchestra.run.smithi091.stdout:(114/147): python3-jwt-2.4.0-1.el9.noarch.rpm 327 kB/s | 41 kB 00:00 2023-11-24T08:01:33.372 INFO:teuthology.orchestra.run.smithi091.stdout:(115/147): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2023-11-24T08:01:33.397 INFO:teuthology.orchestra.run.smithi091.stdout:(116/147): python3-logutils-0.3.5-21.el9.noarch 308 kB/s | 46 kB 00:00 2023-11-24T08:01:33.422 INFO:teuthology.orchestra.run.smithi091.stdout:(117/147): python3-more-itertools-8.12.0-2.el9. 557 kB/s | 79 kB 00:00 2023-11-24T08:01:33.498 INFO:teuthology.orchestra.run.smithi091.stdout:(118/147): python3-natsort-7.1.1-5.el9.noarch.r 463 kB/s | 58 kB 00:00 2023-11-24T08:01:33.540 INFO:teuthology.orchestra.run.smithi091.stdout:(119/147): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2023-11-24T08:01:33.565 INFO:teuthology.orchestra.run.smithi091.stdout:(120/147): python3-portend-3.1.0-2.el9.noarch.r 115 kB/s | 16 kB 00:00 2023-11-24T08:01:33.624 INFO:teuthology.orchestra.run.smithi091.stdout:(121/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 719 kB/s | 90 kB 00:00 2023-11-24T08:01:33.666 INFO:teuthology.orchestra.run.smithi091.stdout:(122/147): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2023-11-24T08:01:33.733 INFO:teuthology.orchestra.run.smithi091.stdout:(123/147): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2023-11-24T08:01:33.758 INFO:teuthology.orchestra.run.smithi091.stdout:(124/147): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2023-11-24T08:01:33.791 INFO:teuthology.orchestra.run.smithi091.stdout:(125/147): python3-tempora-5.0.0-2.el9.noarch.r 287 kB/s | 36 kB 00:00 2023-11-24T08:01:33.859 INFO:teuthology.orchestra.run.smithi091.stdout:(126/147): python3-typing-extensions-4.4.0-2.el 408 kB/s | 51 kB 00:00 2023-11-24T08:01:33.892 INFO:teuthology.orchestra.run.smithi091.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2023-11-24T08:01:33.917 INFO:teuthology.orchestra.run.smithi091.stdout:(128/147): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2023-11-24T08:01:34.026 INFO:teuthology.orchestra.run.smithi091.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2023-11-24T08:01:34.093 INFO:teuthology.orchestra.run.smithi091.stdout:(130/147): python3-werkzeug-2.0.3-3.el9.1.noarc 1.8 MB/s | 427 kB 00:00 2023-11-24T08:01:34.118 INFO:teuthology.orchestra.run.smithi091.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 959 kB/s | 192 kB 00:00 2023-11-24T08:01:34.218 INFO:teuthology.orchestra.run.smithi091.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2023-11-24T08:01:34.269 INFO:teuthology.orchestra.run.smithi091.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2023-11-24T08:01:34.369 INFO:teuthology.orchestra.run.smithi091.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 23 MB/s | 2.2 MB 00:00 2023-11-24T08:01:34.486 INFO:teuthology.orchestra.run.smithi091.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 15 MB/s | 1.8 MB 00:00 2023-11-24T08:01:34.520 INFO:teuthology.orchestra.run.smithi091.stdout:(136/147): NetworkManager-team-1.45.6-1.el9.x86 1.3 MB/s | 42 kB 00:00 2023-11-24T08:01:34.569 INFO:teuthology.orchestra.run.smithi091.stdout:(137/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.9 MB/s | 245 kB 00:00 2023-11-24T08:01:34.620 INFO:teuthology.orchestra.run.smithi091.stdout:(138/147): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2023-11-24T08:01:34.695 INFO:teuthology.orchestra.run.smithi091.stdout:(139/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 5.2 MB/s | 3.0 MB 00:00 2023-11-24T08:01:34.697 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2023-11-24T08:01:34.698 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2023-11-24T08:01:34.721 INFO:teuthology.orchestra.run.smithi091.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 367 kB/s | 36 kB 00:00 2023-11-24T08:01:34.754 INFO:teuthology.orchestra.run.smithi091.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2023-11-24T08:01:34.788 INFO:teuthology.orchestra.run.smithi091.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.9 MB/s | 255 kB 00:00 2023-11-24T08:01:34.844 INFO:teuthology.orchestra.run.smithi091.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 253 kB/s | 22 kB 00:00 2023-11-24T08:01:34.880 INFO:teuthology.orchestra.run.smithi091.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 3.5 MB/s | 321 kB 00:00 2023-11-24T08:01:34.964 INFO:teuthology.orchestra.run.smithi091.stdout:(145/147): librados2-18.0.0-7484.ga13ea5ff.el9. 3.8 MB/s | 3.3 MB 00:00 2023-11-24T08:01:35.739 INFO:teuthology.orchestra.run.smithi091.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 1.4 MB/s | 1.2 MB 00:00 2023-11-24T08:01:35.972 INFO:teuthology.orchestra.run.smithi091.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.0 MB/s | 2.2 MB 00:01 2023-11-24T08:01:35.990 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2023-11-24T08:01:35.990 INFO:teuthology.orchestra.run.smithi091.stdout:Total 11 MB/s | 204 MB 00:18 2023-11-24T08:01:36.214 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2023-11-24T08:01:36.493 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction check succeeded. 2023-11-24T08:01:36.493 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2023-11-24T08:01:38.624 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded. 2023-11-24T08:01:38.624 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2023-11-24T08:01:41.059 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2023-11-24T08:01:41.211 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T08:01:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T08:01:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T08:01:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T08:01:41.440 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T08:01:41.546 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T08:01:41.980 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T08:01:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:01:42.209 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:01:42.238 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:01:42.357 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:01:42.398 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T08:01:42.474 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T08:01:42.796 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T08:01:42.853 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T08:01:42.869 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T08:01:43.028 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T08:01:43.053 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T08:01:43.161 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T08:01:43.483 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T08:01:43.650 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T08:01:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T08:01:43.946 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:01:43.995 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:01:44.382 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:01:44.415 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T08:01:44.504 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T08:01:44.570 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T08:01:44.603 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T08:01:44.649 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T08:01:44.695 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T08:01:44.801 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T08:01:44.850 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T08:01:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T08:01:44.899 INFO:teuthology.orchestra.run.smithi091.stdout: Preparing : 1/1 2023-11-24T08:01:45.022 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T08:01:45.063 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T08:01:45.066 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T08:01:45.117 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T08:01:45.129 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T08:01:45.209 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T08:01:45.234 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T08:01:45.266 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T08:01:45.296 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T08:01:45.399 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T08:01:45.417 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T08:01:45.497 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T08:01:45.830 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T08:01:45.902 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:01:46.062 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:01:46.095 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:01:46.209 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:01:46.222 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T08:01:46.264 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T08:01:46.315 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T08:01:46.340 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T08:01:46.354 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T08:01:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T08:01:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T08:01:46.502 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T08:01:46.620 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T08:01:46.691 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T08:01:46.697 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T08:01:46.717 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T08:01:46.746 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T08:01:46.856 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T08:01:46.884 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T08:01:46.890 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T08:01:46.933 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T08:01:46.968 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T08:01:46.998 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T08:01:47.086 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T08:01:47.125 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T08:01:47.182 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T08:01:47.299 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T08:01:47.305 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T08:01:47.481 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T08:01:47.528 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T08:01:47.577 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T08:01:47.605 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T08:01:47.774 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:01:47.821 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T08:01:47.850 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:01:47.874 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T08:01:47.923 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T08:01:48.053 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T08:01:48.091 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T08:01:48.144 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T08:01:48.181 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T08:01:48.271 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:01:48.313 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T08:01:48.343 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T08:01:48.397 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T08:01:48.410 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T08:01:48.467 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T08:01:48.476 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T08:01:48.508 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T08:01:48.555 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T08:01:48.561 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T08:01:48.629 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T08:01:48.752 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T08:01:48.795 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T08:01:48.829 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T08:01:48.829 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T08:01:48.910 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T08:01:48.940 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T08:01:48.994 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T08:01:49.051 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T08:01:49.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T08:01:49.157 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T08:01:49.176 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T08:01:49.200 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T08:01:49.241 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T08:01:49.301 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T08:01:49.336 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T08:01:49.346 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T08:01:49.398 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T08:01:49.427 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T08:01:49.463 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T08:01:49.564 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T08:01:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T08:01:49.770 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T08:01:49.817 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T08:01:49.860 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T08:01:49.925 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T08:01:49.973 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T08:01:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T08:01:50.125 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T08:01:50.177 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T08:01:50.231 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T08:01:50.323 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T08:01:50.391 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T08:01:50.442 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T08:01:50.547 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T08:01:50.607 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T08:01:50.698 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T08:01:50.903 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T08:01:50.959 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T08:01:51.105 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T08:01:51.146 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T08:01:51.214 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T08:01:51.323 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T08:01:51.363 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T08:01:51.393 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T08:01:51.401 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T08:01:51.448 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T08:01:51.470 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T08:01:51.481 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T08:01:51.514 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T08:01:51.527 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T08:01:51.588 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T08:01:51.772 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T08:01:51.811 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T08:01:51.813 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T08:01:51.853 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T08:01:51.898 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T08:01:51.916 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T08:01:52.024 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T08:01:52.086 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T08:01:52.138 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T08:01:52.169 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T08:01:52.324 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T08:01:52.373 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T08:01:52.467 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T08:01:52.550 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T08:01:52.772 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T08:01:52.813 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T08:01:52.877 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T08:01:52.927 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T08:01:53.047 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T08:01:53.096 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T08:01:53.121 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T08:01:53.184 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T08:01:53.229 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T08:01:53.265 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T08:01:53.307 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T08:01:53.365 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T08:01:53.433 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T08:01:53.483 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T08:01:53.654 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T08:01:53.710 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T08:01:53.745 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T08:01:53.810 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T08:01:53.849 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T08:01:53.991 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T08:01:54.038 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T08:01:54.178 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T08:01:54.243 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T08:01:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T08:01:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T08:01:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T08:01:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T08:01:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T08:01:55.039 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T08:01:55.200 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T08:01:55.251 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T08:01:55.285 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T08:01:55.318 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T08:01:55.439 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T08:01:55.473 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T08:01:55.561 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T08:01:55.594 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T08:01:55.597 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T08:01:55.629 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T08:01:55.685 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T08:01:55.696 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T08:01:56.201 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T08:01:56.234 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T08:01:56.325 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T08:01:56.353 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T08:01:56.485 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T08:01:56.633 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T08:01:58.206 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T08:01:58.293 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T08:01:58.330 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T08:01:58.361 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T08:01:58.416 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T08:01:58.902 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T08:01:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:01:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:01:59.298 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T08:01:59.326 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T08:01:59.474 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T08:01:59.506 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T08:01:59.563 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T08:01:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:01:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T08:02:00.079 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T08:02:00.112 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T08:02:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T08:02:00.143 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T08:02:00.143 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T08:02:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T08:02:00.212 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T08:02:00.239 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T08:02:02.694 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:02:02.751 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:02:03.379 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:02:03.438 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T08:02:03.656 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T08:02:03.656 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T08:02:03.656 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T08:02:03.656 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:03.697 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T08:02:13.541 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T08:02:13.542 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2023-11-24T08:02:13.542 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2023-11-24T08:02:13.542 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2023-11-24T08:02:13.542 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2023-11-24T08:02:13.542 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2023-11-24T08:02:13.542 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2023-11-24T08:02:13.542 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2023-11-24T08:02:13.542 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:13.884 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T08:02:13.939 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T08:02:13.940 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:13.940 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T08:02:13.940 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:15.727 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T08:02:15.813 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T08:02:15.813 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:15.813 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T08:02:15.814 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T08:02:15.814 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T08:02:15.814 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:16.052 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T08:02:16.131 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T08:02:16.131 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:16.131 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T08:02:16.131 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T08:02:16.132 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T08:02:16.132 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:16.256 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T08:02:16.256 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /sys 2023-11-24T08:02:16.256 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /proc 2023-11-24T08:02:16.257 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /mnt 2023-11-24T08:02:16.257 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /var/tmp 2023-11-24T08:02:16.257 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /home 2023-11-24T08:02:16.257 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /root 2023-11-24T08:02:16.257 INFO:teuthology.orchestra.run.smithi091.stdout:skipping the directory /tmp 2023-11-24T08:02:16.257 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:16.516 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T08:02:16.566 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T08:02:16.566 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:16.566 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T08:02:16.567 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:16.653 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T08:02:16.726 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T08:02:16.726 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:16.726 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T08:02:16.726 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T08:02:16.727 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T08:02:16.727 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:16.838 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T08:02:17.509 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T08:02:17.539 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T08:02:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T08:02:17.673 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T08:02:17.710 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T08:02:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T08:02:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T08:02:18.297 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T08:02:18.386 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T08:02:18.386 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:18.386 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T08:02:18.387 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T08:02:18.387 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T08:02:18.387 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:18.611 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T08:02:18.686 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T08:02:18.686 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:18.686 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T08:02:18.687 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T08:02:18.687 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T08:02:18.687 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:19.179 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T08:02:19.248 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T08:02:19.249 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:19.249 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T08:02:19.249 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T08:02:19.249 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T08:02:19.249 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:19.265 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T08:02:19.294 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T08:02:19.356 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T08:02:19.445 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T08:02:19.628 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T08:02:19.665 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T08:02:19.753 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T08:02:19.753 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:19.754 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T08:02:19.754 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T08:02:19.754 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T08:02:19.754 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:19.814 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T08:02:19.856 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T08:02:19.915 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T08:02:20.152 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T08:02:20.189 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T08:02:20.234 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T08:02:20.344 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T08:02:20.377 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T08:02:20.697 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T08:02:20.736 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T08:02:20.854 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T08:02:20.882 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T08:02:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T08:02:20.971 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:20.971 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T08:02:20.971 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T08:02:20.972 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T08:02:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:21.033 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T08:02:21.142 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T08:02:21.142 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:21.142 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T08:02:21.143 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:21.226 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T08:02:21.343 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T08:02:21.343 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:21.343 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T08:02:21.343 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T08:02:21.343 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T08:02:21.343 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:21.743 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T08:02:21.843 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T08:02:21.844 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:21.844 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T08:02:21.844 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T08:02:21.844 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T08:02:21.844 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:21.851 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T08:02:21.886 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T08:02:22.029 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T08:02:22.195 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T08:02:22.247 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T08:02:22.327 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T08:02:22.327 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:22.327 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T08:02:22.327 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T08:02:22.328 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T08:02:22.328 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:22.389 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T08:02:22.429 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T08:02:22.506 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T08:02:23.364 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T08:02:23.400 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T08:02:23.478 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T08:02:23.478 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:23.478 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T08:02:23.478 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T08:02:23.478 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T08:02:23.479 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:23.525 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T08:02:23.592 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T08:02:23.592 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:23.593 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T08:02:23.593 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:23.668 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T08:02:23.765 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T08:02:23.765 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:23.765 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T08:02:23.766 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T08:02:23.766 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T08:02:23.766 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:24.118 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T08:02:24.193 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T08:02:24.193 INFO:teuthology.orchestra.run.smithi091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:02:24.193 INFO:teuthology.orchestra.run.smithi091.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T08:02:24.194 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T08:02:24.194 INFO:teuthology.orchestra.run.smithi091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T08:02:24.194 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:26.232 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T08:02:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T08:02:26.377 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T08:02:26.423 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T08:02:26.730 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T08:02:26.861 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T08:02:26.914 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T08:02:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T08:02:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T08:02:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T08:02:27.126 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T08:02:27.198 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T08:02:27.237 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T08:02:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:02:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:02:28.298 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T08:02:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:02:28.342 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T08:02:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T08:02:28.429 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T08:02:28.694 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T08:02:28.814 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T08:02:28.850 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T08:02:28.886 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T08:02:28.943 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T08:02:28.980 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T08:02:29.088 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T08:02:29.158 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T08:02:29.196 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T08:02:29.764 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T08:02:29.906 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T08:02:29.958 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T08:02:30.037 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T08:02:30.128 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:02:30.170 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:02:30.215 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T08:02:30.236 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:02:30.253 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T08:02:30.287 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T08:02:30.321 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T08:02:30.364 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T08:02:30.364 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T08:02:30.479 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T08:02:30.531 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T08:02:30.576 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T08:02:30.630 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T08:02:30.630 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T08:02:30.787 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T08:02:30.826 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T08:02:30.868 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:02:30.868 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:02:31.234 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:02:31.234 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T08:02:31.376 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T08:02:31.412 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T08:02:31.446 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T08:02:31.482 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T08:02:31.557 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T08:02:31.611 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T08:02:31.680 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T08:02:31.740 INFO:teuthology.orchestra.run.smithi091.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T08:02:31.816 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T08:02:31.888 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T08:02:31.889 INFO:teuthology.orchestra.run.smithi064.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T08:02:31.889 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:31.978 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T08:02:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T08:02:32.053 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T08:02:32.096 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T08:02:32.138 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T08:02:32.138 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T08:02:32.269 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T08:02:32.312 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T08:02:32.366 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T08:02:32.399 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T08:02:32.399 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T08:02:32.536 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T08:02:32.567 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T08:02:32.606 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:02:32.607 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:02:32.941 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:02:32.941 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T08:02:33.057 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T08:02:33.111 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T08:02:33.144 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T08:02:33.182 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T08:02:33.315 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T08:02:33.572 INFO:teuthology.orchestra.run.smithi091.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T08:02:33.572 INFO:teuthology.orchestra.run.smithi091.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T08:02:33.572 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:36.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T08:02:36.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T08:02:36.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T08:02:36.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T08:02:36.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T08:02:36.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T08:02:36.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T08:02:36.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:02:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:02:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T08:02:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T08:02:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T08:02:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T08:02:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T08:02:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T08:02:36.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T08:02:36.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T08:02:36.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T08:02:36.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T08:02:36.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T08:02:36.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T08:02:36.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T08:02:36.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T08:02:36.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T08:02:36.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T08:02:36.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T08:02:36.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T08:02:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T08:02:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T08:02:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T08:02:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T08:02:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T08:02:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T08:02:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T08:02:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T08:02:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T08:02:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T08:02:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T08:02:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T08:02:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T08:02:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T08:02:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T08:02:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T08:02:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T08:02:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T08:02:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T08:02:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T08:02:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T08:02:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T08:02:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T08:02:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T08:02:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T08:02:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T08:02:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T08:02:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T08:02:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T08:02:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T08:02:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T08:02:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T08:02:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T08:02:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T08:02:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T08:02:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T08:02:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T08:02:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T08:02:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T08:02:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T08:02:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T08:02:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T08:02:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T08:02:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T08:02:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T08:02:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T08:02:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T08:02:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T08:02:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T08:02:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T08:02:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T08:02:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T08:02:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T08:02:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T08:02:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T08:02:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T08:02:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T08:02:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T08:02:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T08:02:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T08:02:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T08:02:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T08:02:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T08:02:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T08:02:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T08:02:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T08:02:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T08:02:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T08:02:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T08:02:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T08:02:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T08:02:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T08:02:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T08:02:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T08:02:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T08:02:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T08:02:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T08:02:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T08:02:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T08:02:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T08:02:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T08:02:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T08:02:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T08:02:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T08:02:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T08:02:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T08:02:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T08:02:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T08:02:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T08:02:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T08:02:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T08:02:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T08:02:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T08:02:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T08:02:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T08:02:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T08:02:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T08:02:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T08:02:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T08:02:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T08:02:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T08:02:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T08:02:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T08:02:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T08:02:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T08:02:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T08:02:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T08:02:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T08:02:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T08:02:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T08:02:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T08:02:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T08:02:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T08:02:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T08:02:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T08:02:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T08:02:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T08:02:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T08:02:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T08:02:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T08:02:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T08:02:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T08:02:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T08:02:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T08:02:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T08:02:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T08:02:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T08:02:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T08:02:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T08:02:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T08:02:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T08:02:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T08:02:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T08:02:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T08:02:38.424 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T08:02:38.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T08:02:38.426 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T08:02:38.426 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T08:02:38.426 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T08:02:38.426 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T08:02:38.426 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T08:02:38.426 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T08:02:38.426 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T08:02:38.427 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T08:02:38.427 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T08:02:38.427 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T08:02:38.427 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T08:02:38.427 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T08:02:38.428 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T08:02:38.428 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T08:02:38.428 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T08:02:38.428 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T08:02:38.429 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T08:02:38.429 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T08:02:38.442 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T08:02:38.442 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T08:02:38.442 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T08:02:38.442 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T08:02:38.443 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T08:02:38.443 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T08:02:38.443 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T08:02:38.443 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T08:02:38.443 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T08:02:38.444 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T08:02:38.444 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T08:02:38.444 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T08:02:38.444 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T08:02:38.444 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T08:02:38.444 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T08:02:38.444 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T08:02:38.445 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T08:02:38.445 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T08:02:38.445 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T08:02:38.445 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T08:02:38.445 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T08:02:38.445 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T08:02:38.446 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T08:02:38.446 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T08:02:38.446 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T08:02:38.446 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T08:02:38.447 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T08:02:38.448 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T08:02:38.448 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T08:02:38.448 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T08:02:38.448 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T08:02:38.448 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T08:02:38.448 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T08:02:38.448 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T08:02:38.448 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T08:02:38.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T08:02:38.450 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T08:02:38.450 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T08:02:38.450 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T08:02:38.450 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T08:02:38.450 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T08:02:38.450 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T08:02:38.450 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T08:02:38.451 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T08:02:38.452 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T08:02:38.452 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T08:02:38.452 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T08:02:38.452 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T08:02:38.452 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T08:02:38.452 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T08:02:38.452 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T08:02:38.452 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T08:02:38.453 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T08:02:38.454 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T08:02:38.454 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T08:02:38.454 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T08:02:38.454 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T08:02:38.454 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T08:02:38.454 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T08:02:38.454 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T08:02:38.455 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T08:02:38.456 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T08:02:38.456 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T08:02:38.456 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T08:02:38.456 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T08:02:38.456 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T08:02:38.456 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T08:02:38.456 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T08:02:38.457 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T08:02:38.458 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T08:02:38.458 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T08:02:38.458 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T08:02:38.458 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T08:02:38.458 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T08:02:38.458 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T08:02:38.458 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T08:02:38.458 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T08:02:38.459 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T08:02:38.459 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T08:02:38.459 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T08:02:38.459 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T08:02:38.459 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T08:02:38.459 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T08:02:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T08:02:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:38.625 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2023-11-24T08:02:38.625 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T08:02:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T08:02:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T08:02:38.626 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T08:02:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T08:02:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T08:02:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T08:02:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T08:02:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T08:02:38.627 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T08:02:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T08:02:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T08:02:38.628 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T08:02:38.629 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2023-11-24T08:02:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T08:02:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T08:02:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T08:02:38.629 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.630 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.631 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.632 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.633 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T08:02:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T08:02:38.634 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-16.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T08:02:38.635 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T08:02:38.636 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T08:02:38.637 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T08:02:38.638 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T08:02:38.639 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T08:02:38.640 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T08:02:38.641 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.642 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T08:02:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T08:02:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T08:02:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T08:02:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T08:02:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T08:02:38.643 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T08:02:38.644 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:38.644 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-11-24T08:02:39.222 DEBUG:teuthology.parallel:result is None 2023-11-24T08:02:40.629 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T08:02:40.629 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:40.629 INFO:teuthology.orchestra.run.smithi091.stdout:Upgraded: 2023-11-24T08:02:40.629 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T08:02:40.629 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T08:02:40.629 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.630 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.631 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.632 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T08:02:40.633 INFO:teuthology.orchestra.run.smithi091.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: jq-1.6-16.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T08:02:40.634 INFO:teuthology.orchestra.run.smithi091.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T08:02:40.635 INFO:teuthology.orchestra.run.smithi091.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T08:02:40.636 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T08:02:40.637 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T08:02:40.638 INFO:teuthology.orchestra.run.smithi091.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T08:02:40.639 INFO:teuthology.orchestra.run.smithi091.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T08:02:40.640 INFO:teuthology.orchestra.run.smithi091.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T08:02:40.641 INFO:teuthology.orchestra.run.smithi091.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T08:02:40.642 INFO:teuthology.orchestra.run.smithi091.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T08:02:40.642 INFO:teuthology.orchestra.run.smithi091.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T08:02:40.642 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:40.642 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2023-11-24T08:02:41.228 DEBUG:teuthology.parallel:result is None 2023-11-24T08:02:41.229 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T08:02:41.445 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T08:02:41.505 INFO:teuthology.orchestra.run.smithi064.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T08:02:41.505 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T08:02:41.506 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T08:02:41.506 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T08:02:41.722 DEBUG:teuthology.orchestra.run.smithi091:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T08:02:41.779 INFO:teuthology.orchestra.run.smithi091.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T08:02:41.779 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T08:02:41.779 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T08:02:41.779 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T08:02:41.780 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:41.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T08:02:41.862 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:02:41.862 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T08:02:41.936 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T08:02:41.937 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:41.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T08:02:42.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T08:02:42.105 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:02:42.105 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T08:02:42.180 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T08:02:42.296 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T08:02:42.296 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:42.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T08:02:42.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T08:02:42.503 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:02:42.503 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T08:02:42.584 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T08:02:42.698 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T08:02:42.698 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:42.698 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T08:02:42.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T08:02:42.870 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:02:42.870 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T08:02:42.945 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T08:02:43.064 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T08:02:43.132 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T08:02:43.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2023-11-24T08:02:43.138 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 777 /var/log/ceph 2023-11-24T08:02:43.216 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T08:02:43.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T08:02:43.221 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T08:02:43.300 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T08:02:43.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T08:02:43.305 DEBUG:teuthology.orchestra.run.smithi091:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T08:02:43.383 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T08:02:43.383 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T08:02:43.383 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T08:02:43.383 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:43.384 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T08:02:43.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T08:02:43.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T08:02:43.718 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:02:43.725 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T08:02:43.777 DEBUG:teuthology.orchestra.run.smithi091:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T08:02:44.017 DEBUG:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T08:02:44.107 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T08:02:44.107 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T08:02:44.107 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}', 'email': None, 'first_in_suite': False, 'job_id': '7465580', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-vol-tests', 'sha1': '73843baf45f8a92d8e3fcbea3f024c97e73036da'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9764, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'mgr-vol-tests', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '73843baf45f8a92d8e3fcbea3f024c97e73036da', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF5omeBjoYC53ar9tv0sjHnuDu9+h9ZwYTKIpSdWxz/gGoVgOd+nmj3zVpebk7jcROuIaYj44iGeSBgZod3iONc=', 'smithi091.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSQUKW2luMwj0cnVdRy1CbmC41uHhE7G7DW1zzQj+Jw8O8mNR32YMkj5ensrEk5RXvpqpGFi2y3frTvsZraysQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_07:31:48', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T08:02:44.108 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T08:02:44.115 DEBUG:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T08:02:44.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T08:02:44.196 DEBUG:teuthology.orchestra.run.smithi091:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T08:02:44.275 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:44.276 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T08:02:44.313 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T08:02:44.314 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2023-11-24T08:02:44.393 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T08:02:44.393 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:02:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-11-24T08:02:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:02:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:02:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-11-24 08:02:32.357790092 +0000 2023-11-24T08:02:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-11-24 07:59:51.769006090 +0000 2023-11-24T08:02:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-11-24 07:59:51.769006090 +0000 2023-11-24T08:02:44.394 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-11-24 07:59:51.769006090 +0000 2023-11-24T08:02:44.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T08:02:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-11-24T08:02:44.499 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-11-24T08:02:44.500 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000377152 s, 1.4 MB/s 2023-11-24T08:02:44.502 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T08:02:44.547 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2023-11-24T08:02:44.629 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T08:02:44.629 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:02:44.630 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-11-24T08:02:44.630 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:02:44.630 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:02:44.630 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-11-24 08:02:32.357790092 +0000 2023-11-24T08:02:44.630 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-11-24 07:59:52.826985092 +0000 2023-11-24T08:02:44.630 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-11-24 07:59:52.826985092 +0000 2023-11-24T08:02:44.630 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-11-24 07:59:52.826985092 +0000 2023-11-24T08:02:44.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T08:02:44.743 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-11-24T08:02:44.743 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-11-24T08:02:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000305295 s, 1.7 MB/s 2023-11-24T08:02:44.746 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T08:02:44.787 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2023-11-24T08:02:44.869 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T08:02:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:02:44.870 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 671 Links: 1 2023-11-24T08:02:44.870 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:02:44.870 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:02:44.870 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-11-24 08:02:32.358790072 +0000 2023-11-24T08:02:44.870 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-11-24 07:59:53.871964352 +0000 2023-11-24T08:02:44.870 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-11-24 07:59:53.871964352 +0000 2023-11-24T08:02:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-11-24 07:59:53.871964352 +0000 2023-11-24T08:02:44.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T08:02:44.976 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-11-24T08:02:44.976 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-11-24T08:02:44.976 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000287842 s, 1.8 MB/s 2023-11-24T08:02:44.978 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T08:02:45.018 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-11-24 08:02:32.359790053 +0000 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-11-24 07:59:54.929943354 +0000 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-11-24 07:59:54.929943354 +0000 2023-11-24T08:02:45.114 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-11-24 07:59:54.929943354 +0000 2023-11-24T08:02:45.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T08:02:45.203 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-11-24T08:02:45.203 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-11-24T08:02:45.203 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000485611 s, 1.1 MB/s 2023-11-24T08:02:45.208 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T08:02:45.269 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:02:45.269 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:02:45.269 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T08:02:45.316 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T08:02:45.316 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_1 2023-11-24T08:02:45.398 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T08:02:45.398 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:02:45.399 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-11-24T08:02:45.399 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:02:45.399 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:02:45.399 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2023-11-24 08:02:34.014649759 +0000 2023-11-24T08:02:45.399 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2023-11-24 07:59:29.024223874 +0000 2023-11-24T08:02:45.399 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2023-11-24 07:59:29.024223874 +0000 2023-11-24T08:02:45.399 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2023-11-24 07:59:29.024223874 +0000 2023-11-24T08:02:45.399 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T08:02:45.510 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2023-11-24T08:02:45.510 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2023-11-24T08:02:45.510 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000352929 s, 1.5 MB/s 2023-11-24T08:02:45.513 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T08:02:45.559 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_2 2023-11-24T08:02:45.644 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T08:02:45.644 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:02:45.644 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-11-24T08:02:45.645 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:02:45.645 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:02:45.645 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2023-11-24 08:02:34.015649733 +0000 2023-11-24T08:02:45.645 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2023-11-24 07:59:29.999199173 +0000 2023-11-24T08:02:45.645 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2023-11-24 07:59:29.999199173 +0000 2023-11-24T08:02:45.645 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2023-11-24 07:59:29.999199173 +0000 2023-11-24T08:02:45.645 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T08:02:45.753 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2023-11-24T08:02:45.753 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2023-11-24T08:02:45.753 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000304223 s, 1.7 MB/s 2023-11-24T08:02:45.756 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T08:02:45.797 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_3 2023-11-24T08:02:45.882 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T08:02:45.882 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:02:45.882 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-11-24T08:02:45.882 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:02:45.882 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:02:45.882 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2023-11-24 08:02:34.016649708 +0000 2023-11-24T08:02:45.882 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2023-11-24 07:59:30.995173939 +0000 2023-11-24T08:02:45.882 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2023-11-24 07:59:30.995173939 +0000 2023-11-24T08:02:45.883 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2023-11-24 07:59:30.995173939 +0000 2023-11-24T08:02:45.883 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T08:02:45.992 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2023-11-24T08:02:45.992 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2023-11-24T08:02:45.992 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000301989 s, 1.7 MB/s 2023-11-24T08:02:45.995 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T08:02:46.039 DEBUG:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_4 2023-11-24T08:02:46.120 INFO:teuthology.orchestra.run.smithi091.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T08:02:46.121 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:02:46.121 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-11-24T08:02:46.121 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:02:46.121 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:02:46.121 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2023-11-24 08:02:34.016649708 +0000 2023-11-24T08:02:46.122 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2023-11-24 07:59:31.994148630 +0000 2023-11-24T08:02:46.122 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2023-11-24 07:59:31.994148630 +0000 2023-11-24T08:02:46.122 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: 2023-11-24 07:59:31.994148630 +0000 2023-11-24T08:02:46.122 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T08:02:46.230 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2023-11-24T08:02:46.230 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2023-11-24T08:02:46.230 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes copied, 0.000281351 s, 1.8 MB/s 2023-11-24T08:02:46.233 DEBUG:teuthology.orchestra.run.smithi091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T08:02:46.275 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:02:46.275 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi091.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T08:02:46.275 INFO:tasks.ceph:Generating config... 2023-11-24T08:02:46.276 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T08:02:46.276 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T08:02:46.277 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T08:02:46.277 INFO:tasks.ceph:[client] pid file = 2023-11-24T08:02:46.277 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T08:02:46.277 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T08:02:46.277 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T08:02:46.277 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T08:02:46.277 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T08:02:46.277 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T08:02:46.277 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T08:02:46.277 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T08:02:46.277 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T08:02:46.278 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T08:02:46.278 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T08:02:46.278 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T08:02:46.279 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T08:02:46.280 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T08:02:46.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T08:02:46.440 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T08:02:46.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T08:02:46.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T08:02:46.696 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.64'), ('mon.b', '172.21.15.91'), ('mon.c', '[v2:172.21.15.91:3301,v1:172.21.15.91:6790]')] 2023-11-24T08:02:46.697 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.64,172.21.15.91,[v2:172.21.15.91:3301,v1:172.21.15.91:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T08:02:46.698 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:46.698 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T08:02:46.753 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.64 --add b 172.21.15.91 --addv c '[v2:172.21.15.91:3301,v1:172.21.15.91:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:02:46.916 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:02:46.916 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: generated fsid 2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918 2023-11-24T08:02:46.916 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = pacific 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 0 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:fsid 2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:last_changed 2023-11-24T08:02:46.915895+0000 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:created 2023-11-24T08:02:46.915895+0000 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:min_mon_release 16 (pacific) 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:election_strategy: 1 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:1: [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] mon.b 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:2: [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] mon.c 2023-11-24T08:02:46.917 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T08:02:46.920 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T08:02:46.974 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918... 2023-11-24T08:02:46.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T08:02:46.981 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T08:02:47.131 INFO:teuthology.orchestra.run.smithi091.stdout:[global] 2023-11-24T08:02:47.131 INFO:teuthology.orchestra.run.smithi091.stdout: chdir = "" 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: auth supported = cephx 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: filestore xattr use omap = true 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: mon clock drift allowed = 1.000 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: osd crush chooseleaf type = 0 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: auth debug = true 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: ms die on old message = true 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: ms die on bug = true 2023-11-24T08:02:47.132 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: mon pg warn max object skew = 0 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: osd pool default size = 2 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd allow primary affinity = true 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd allow pg remap = true 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on legacy crush tunables = false 2023-11-24T08:02:47.133 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on crush straw calc version zero = false 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on no sortbitwise = false 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on osd down out interval zero = false 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: mon warn on too few osds = false 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: mon_allow_pool_size_one = true 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: osd default data pool replay window = 5 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.134 INFO:teuthology.orchestra.run.smithi091.stdout: mon allow pool delete = true 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: mon cluster log file level = debug 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: debug asserts on shutdown = true 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: mon health detail to clog = false 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: mon host = "172.21.15.64,172.21.15.91,[v2:172.21.15.91:3301,v1:172.21.15.91:6790]" 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: fsid = 2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout:[osd] 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: osd journal size = 100 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: osd scrub load threshold = 5.0 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: osd scrub max interval = 600 2023-11-24T08:02:47.135 INFO:teuthology.orchestra.run.smithi091.stdout: osd mclock profile = high_recovery_ops 2023-11-24T08:02:47.136 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.136 INFO:teuthology.orchestra.run.smithi091.stdout: osd recover clone overlap = true 2023-11-24T08:02:47.136 INFO:teuthology.orchestra.run.smithi091.stdout: osd recovery max chunk = 1048576 2023-11-24T08:02:47.136 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug shutdown = true 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug op order = true 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug verify stray on activate = true 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug trim objects = true 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: osd open classes on start = true 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug pg log writeout = true 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.137 INFO:teuthology.orchestra.run.smithi091.stdout: osd deep scrub update digest min age = 30 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: osd map max advance = 10 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: journal zero on create = true 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: filestore ondisk finisher threads = 3 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: filestore apply finisher threads = 3 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: bdev debug aio = true 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: osd debug misdirected ops = true 2023-11-24T08:02:47.138 INFO:teuthology.orchestra.run.smithi091.stdout: bdev async discard = True 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: bdev enable discard = True 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: bluestore allocator = bitmap 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: bluestore block size = 96636764160 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: bluestore fsck on mount = True 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: debug bluefs = 1/20 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: debug bluestore = 1/20 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: debug osd = 20 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: debug rocksdb = 4/10 2023-11-24T08:02:47.139 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd full ratio = 0.9 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd nearfull ratio = 0.8 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: osd failsafe full ratio = 0.95 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: osd objectstore = bluestore 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: osd op complaint time = 180 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr] 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: debug mgr = 20 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: debug mon = 20 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: debug auth = 20 2023-11-24T08:02:47.140 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min pgs per osd = 4 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min bytes per osd = 10 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: mgr/telemetry/nag = false 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: debug client = 20 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: debug finisher = 20 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout:[mon] 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: debug mon = 20 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: debug paxos = 20 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: debug auth = 20 2023-11-24T08:02:47.141 INFO:teuthology.orchestra.run.smithi091.stdout: mon data avail warn = 5 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: mon mgr mkfs grace = 240 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min pgs per osd = 4 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd reporter subtree level = osd 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: mon osd prime pg temp = true 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: mon reweight min bytes per osd = 10 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T08:02:47.142 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: # 1m isn't quite enough 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: mon_down_mkfs_grace = 2m 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: mon_warn_on_filestore_osds = false 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: mon op complaint time = 120 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout:[client] 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: rgw cache enabled = true 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: rgw enable ops log = true 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: rgw enable usage log = true 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T08:02:47.143 INFO:teuthology.orchestra.run.smithi091.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout: client mount timeout = 600 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout: debug client = 20 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout: pid file = "" 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout: rados mon op timeout = 900 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout: rados osd op timeout = 900 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.a] 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.b] 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout:[mon.c] 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.a] 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.c] 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.b] 2023-11-24T08:02:47.144 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.d] 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout:[mds] 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: debug mds = 20 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: debug mds balancer = 20 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: debug ms = 1 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: mds debug frag = True 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: mds debug scatterstat = True 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: mds op complaint time = 180 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: mds verify scatter = True 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: osd op complaint time = 180 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: rados mon op timeout = 900 2023-11-24T08:02:47.145 INFO:teuthology.orchestra.run.smithi091.stdout: rados osd op timeout = 900 2023-11-24T08:02:47.158 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2023-11-24T08:02:47.159 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.160 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T08:02:47.161 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.162 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2023-11-24T08:02:47.162 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.162 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2023-11-24T08:02:47.162 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.162 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2023-11-24T08:02:47.162 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.64,172.21.15.91,[v2:172.21.15.91:3301,v1:172.21.15.91:6790]" 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2023-11-24T08:02:47.163 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug trim objects = true 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.164 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: bdev async discard = True 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: bdev enable discard = True 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore allocator = bitmap 2023-11-24T08:02:47.165 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 1/20 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 1/20 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 4/10 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.166 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: debug finisher = 20 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T08:02:47.167 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T08:02:47.168 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2023-11-24T08:02:47.169 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = "" 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2023-11-24T08:02:47.170 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds balancer = 20 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-11-24T08:02:47.171 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-11-24T08:02:47.172 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-11-24T08:02:47.205 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T08:02:47.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T08:02:47.359 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T08:02:47.359 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:47.359 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T08:02:47.414 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:47.414 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T08:02:47.515 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:02:47.515 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:47.515 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T08:02:47.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T08:02:47.668 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:02:47.669 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:02:47.771 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:02:47.771 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:02:47.771 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T08:02:47.771 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T08:02:47.931 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:02:47.932 DEBUG:teuthology.orchestra.run.smithi091:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:02:48.027 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T08:02:48.027 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T08:02:48.027 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T08:02:48.235 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T08:02:48.246 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T08:02:48.447 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T08:02:48.459 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T08:02:48.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T08:02:48.667 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T08:02:48.678 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T08:02:48.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T08:02:48.969 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T08:02:48.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T08:02:49.084 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T08:02:49.275 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T08:02:49.285 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T08:02:49.380 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T08:02:49.565 INFO:teuthology.orchestra.run.smithi091.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T08:02:49.575 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T08:02:49.675 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T08:02:49.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T08:02:49.818 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T08:02:49.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T08:02:50.015 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T08:02:50.084 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T08:02:50.218 INFO:teuthology.orchestra.run.smithi091.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T08:02:50.269 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T08:02:50.395 INFO:teuthology.orchestra.run.smithi091.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T08:02:50.453 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T08:02:50.453 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi091.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T08:02:50.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T08:02:50.568 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:02:50.568 INFO:tasks.ceph:role: osd.0 2023-11-24T08:02:50.568 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:02:50.568 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T08:02:50.680 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:02:50.681 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:02:51.186 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2023-11-24T08:02:51.193 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-11-24T08:02:51.194 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T08:02:51.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T08:02:51.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T08:02:51.594 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:02:51.595 INFO:tasks.ceph:role: osd.1 2023-11-24T08:02:51.595 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:02:51.595 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T08:02:51.694 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:02:51.695 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:02:52.176 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2023-11-24T08:02:52.184 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-11-24T08:02:52.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T08:02:52.343 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T08:02:52.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T08:02:52.540 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:02:52.540 INFO:tasks.ceph:role: osd.2 2023-11-24T08:02:52.540 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:02:52.540 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T08:02:52.646 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:02:52.647 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:02:52.647 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:02:52.647 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:02:52.647 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:02:52.647 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:02:52.647 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:02:52.647 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:02:52.647 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:02:52.648 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:02:53.151 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2023-11-24T08:02:53.157 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-11-24T08:02:53.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T08:02:53.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T08:02:53.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T08:02:53.569 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:02:53.569 INFO:tasks.ceph:role: osd.3 2023-11-24T08:02:53.569 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:02:53.569 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T08:02:53.677 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:02:53.677 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:02:53.677 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:02:53.677 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:02:53.678 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:02:53.678 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:02:53.678 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:02:53.678 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:02:53.678 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:02:53.678 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:02:54.132 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2023-11-24T08:02:54.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-11-24T08:02:54.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T08:02:54.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T08:02:54.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:02:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:54.546+0000 7ffba3a10500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T08:02:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:54.548+0000 7ffba3a10500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T08:02:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:54.548+0000 7ffba3a10500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T08:02:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:54.548+0000 7ffba3a10500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T08:02:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:54.548+0000 7ffba3a10500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T08:02:54.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:54.552+0000 7ffba3a10500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T08:02:57.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T08:02:57.601 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:02:57.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:57.752+0000 7f1b7ec13500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T08:02:57.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:57.753+0000 7f1b7ec13500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T08:02:57.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:57.753+0000 7f1b7ec13500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T08:02:57.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:57.753+0000 7f1b7ec13500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T08:02:57.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:57.753+0000 7f1b7ec13500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T08:02:57.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:02:57.755+0000 7f1b7ec13500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T08:03:00.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T08:03:01.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:03:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:01.192+0000 7fb47a219500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T08:03:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:01.193+0000 7fb47a219500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T08:03:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:01.193+0000 7fb47a219500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T08:03:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:01.193+0000 7fb47a219500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T08:03:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:01.193+0000 7fb47a219500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T08:03:01.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:01.195+0000 7fb47a219500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T08:03:04.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T08:03:04.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:03:04.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:04.623+0000 7ff99b0d6500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T08:03:04.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:04.623+0000 7ff99b0d6500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T08:03:04.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:04.623+0000 7ff99b0d6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T08:03:04.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:04.623+0000 7ff99b0d6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T08:03:04.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:04.623+0000 7ff99b0d6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T08:03:04.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:04.626+0000 7ff99b0d6500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T08:03:07.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T08:03:07.914 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T08:03:08.018 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:03:08.018 INFO:tasks.ceph:role: osd.4 2023-11-24T08:03:08.018 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:03:08.018 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T08:03:08.116 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:03:08.117 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:03:08.638 INFO:teuthology.orchestra.run.smithi091.stdout:Discarding blocks...Done. 2023-11-24T08:03:08.644 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2023-11-24T08:03:08.644 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T08:03:08.808 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T08:03:08.910 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T08:03:08.993 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:03:08.993 INFO:tasks.ceph:role: osd.5 2023-11-24T08:03:08.993 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:03:08.993 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T08:03:09.078 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:03:09.079 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:03:09.534 INFO:teuthology.orchestra.run.smithi091.stdout:Discarding blocks...Done. 2023-11-24T08:03:09.538 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2023-11-24T08:03:09.538 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T08:03:09.674 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T08:03:09.760 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T08:03:09.836 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:03:09.836 INFO:tasks.ceph:role: osd.6 2023-11-24T08:03:09.836 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:03:09.836 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T08:03:09.918 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:03:09.918 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:03:09.918 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:03:09.918 INFO:teuthology.orchestra.run.smithi091.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:03:09.918 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:03:09.918 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:03:09.919 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:03:09.919 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:03:09.919 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:03:09.919 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:03:10.443 INFO:teuthology.orchestra.run.smithi091.stdout:Discarding blocks...Done. 2023-11-24T08:03:10.448 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2023-11-24T08:03:10.448 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T08:03:10.628 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T08:03:10.727 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T08:03:10.818 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:03:10.818 INFO:tasks.ceph:role: osd.7 2023-11-24T08:03:10.818 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:03:10.818 DEBUG:teuthology.orchestra.run.smithi091:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T08:03:10.920 INFO:teuthology.orchestra.run.smithi091.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:03:10.920 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:03:10.920 INFO:teuthology.orchestra.run.smithi091.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:03:10.921 INFO:teuthology.orchestra.run.smithi091.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:03:10.921 INFO:teuthology.orchestra.run.smithi091.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:03:10.921 INFO:teuthology.orchestra.run.smithi091.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:03:10.921 INFO:teuthology.orchestra.run.smithi091.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:03:10.921 INFO:teuthology.orchestra.run.smithi091.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:03:10.921 INFO:teuthology.orchestra.run.smithi091.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:03:10.922 INFO:teuthology.orchestra.run.smithi091.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:03:11.366 INFO:teuthology.orchestra.run.smithi091.stdout:Discarding blocks...Done. 2023-11-24T08:03:11.369 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi091.front.sepia.ceph.com -o noatime 2023-11-24T08:03:11.369 DEBUG:teuthology.orchestra.run.smithi091:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T08:03:11.489 DEBUG:teuthology.orchestra.run.smithi091:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T08:03:11.569 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:03:11.697 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:11.695+0000 7fa217c12500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T08:03:11.697 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:11.696+0000 7fa217c12500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T08:03:11.698 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:11.696+0000 7fa217c12500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T08:03:11.698 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:11.696+0000 7fa217c12500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T08:03:11.698 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:11.696+0000 7fa217c12500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T08:03:11.700 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:11.698+0000 7fa217c12500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T08:03:14.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:03:14.119 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:03:14.878 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T08:03:14.982 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:03:15.129 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:15.127+0000 7f8f7901d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T08:03:15.129 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:15.127+0000 7f8f7901d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T08:03:15.129 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:15.127+0000 7f8f7901d500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T08:03:15.130 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:15.127+0000 7f8f7901d500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T08:03:15.130 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:15.127+0000 7f8f7901d500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T08:03:15.131 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:15.130+0000 7f8f7901d500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T08:03:18.308 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T08:03:18.409 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:03:18.547 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:18.545+0000 7fee45ed8500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T08:03:18.548 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:18.546+0000 7fee45ed8500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T08:03:18.548 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:18.546+0000 7fee45ed8500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T08:03:18.548 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:18.546+0000 7fee45ed8500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T08:03:18.548 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:18.546+0000 7fee45ed8500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T08:03:18.550 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:18.548+0000 7fee45ed8500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T08:03:21.491 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T08:03:21.587 DEBUG:teuthology.orchestra.run.smithi091:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:03:21.727 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:21.725+0000 7fc9e134a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T08:03:21.728 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:21.726+0000 7fc9e134a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T08:03:21.728 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:21.726+0000 7fc9e134a500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T08:03:21.728 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:21.726+0000 7fc9e134a500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T08:03:21.728 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:21.726+0000 7fc9e134a500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T08:03:21.729 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:21.728+0000 7fc9e134a500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T08:03:24.903 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T08:03:24.994 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T08:03:24.994 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:24.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T08:03:25.107 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:25.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T08:03:25.207 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:25.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T08:03:25.310 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:25.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T08:03:25.417 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:25.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T08:03:25.518 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:25.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T08:03:25.618 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:25.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T08:03:25.720 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:25.720 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T08:03:25.825 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:25.825 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T08:03:25.923 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:25.924 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T08:03:26.021 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:26.021 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T08:03:26.114 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:26.115 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T08:03:26.210 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:26.210 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T08:03:26.355 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:26.355 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T08:03:26.451 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:26.451 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T08:03:26.513 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:26.514 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T08:03:26.575 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:26.575 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T08:03:26.635 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:26.635 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T08:03:26.688 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T08:03:26.688 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T08:03:26.697 DEBUG:teuthology.orchestra.run.smithi091:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T08:03:26.788 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr.y] 2023-11-24T08:03:26.788 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCoWGBloKAEDhAAFSBzeCcgrTtmsfXrxKqCRQ== 2023-11-24T08:03:26.788 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.a] 2023-11-24T08:03:26.788 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCoWGBllV/IJxAAMEWwBM8H9uSLw5Bd93bScA== 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.c] 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCoWGBlZ1rGORAAjjOrPcL663guE4Q9MKA+6g== 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.0] 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCuWGBlsrGpIBAAVTJlhEtf7rPd1J4wUZRWTg== 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.1] 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCxWGBl+u/pLBAAwh85K8evI69aY9Nij5dBRQ== 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.2] 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQC1WGBl1kGECxAAtoQ9UI319UpAU7OwZehzew== 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.3] 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQC4WGBlSlE2JRAAjPT1020nD4Dnrw1hrB2bKQ== 2023-11-24T08:03:26.789 INFO:teuthology.orchestra.run.smithi091.stdout:[mgr.x] 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCoWGBllqymGhAA+wKOSJeuBn60R4gZJ3edoA== 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.b] 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCpWGBl35hoEBAADvW80b1cRhhgBRkYgNaftg== 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout:[mds.d] 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCpWGBlqV+1IRAAaLczF6+eDIuMM6X5Jgni/A== 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.4] 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQC/WGBlrRSGKRAAaiWZZvvd7O8PuipXF1oAUQ== 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.5] 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDDWGBls+mkBxAAJZXdXFq1+8QC5dBE14+YJA== 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.6] 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDGWGBl71+WIBAA/NNCMwpi0ycNJvXZFHKuIA== 2023-11-24T08:03:26.790 INFO:teuthology.orchestra.run.smithi091.stdout:[osd.7] 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQDJWGBl11BPKxAAnXhK4EqlIyvbHCpHzPCDUg== 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout:[client.0] 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA== 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout:[client.2] 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCqWGBlhsnfABAA5sZEDm1Uyin3dIOEoMX+gg== 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout:[client.1] 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCqWGBlsyr3DBAASDVvIWUkg06i7YARod5spw== 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout:[client.3] 2023-11-24T08:03:26.791 INFO:teuthology.orchestra.run.smithi091.stdout: key = AQCqWGBlT9iOFxAAiupeG8QnYzYA8pJYftfxNA== 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCoWGBloKAEDhAAFSBzeCcgrTtmsfXrxKqCRQ== 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCoWGBllV/IJxAAMEWwBM8H9uSLw5Bd93bScA== 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCoWGBlZ1rGORAAjjOrPcL663guE4Q9MKA+6g== 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCuWGBlsrGpIBAAVTJlhEtf7rPd1J4wUZRWTg== 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2023-11-24T08:03:26.792 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCxWGBl+u/pLBAAwh85K8evI69aY9Nij5dBRQ== 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC1WGBl1kGECxAAtoQ9UI319UpAU7OwZehzew== 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC4WGBlSlE2JRAAjPT1020nD4Dnrw1hrB2bKQ== 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCoWGBllqymGhAA+wKOSJeuBn60R4gZJ3edoA== 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpWGBl35hoEBAADvW80b1cRhhgBRkYgNaftg== 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpWGBlqV+1IRAAaLczF6+eDIuMM6X5Jgni/A== 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2023-11-24T08:03:26.793 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC/WGBlrRSGKRAAaiWZZvvd7O8PuipXF1oAUQ== 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDDWGBls+mkBxAAJZXdXFq1+8QC5dBE14+YJA== 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDGWGBl71+WIBAA/NNCMwpi0ycNJvXZFHKuIA== 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDJWGBl11BPKxAAnXhK4EqlIyvbHCpHzPCDUg== 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA== 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout:[client.2] 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCqWGBlhsnfABAA5sZEDm1Uyin3dIOEoMX+gg== 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2023-11-24T08:03:26.794 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCqWGBlsyr3DBAASDVvIWUkg06i7YARod5spw== 2023-11-24T08:03:26.795 INFO:teuthology.orchestra.run.smithi064.stdout:[client.3] 2023-11-24T08:03:26.795 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCqWGBlT9iOFxAAiupeG8QnYzYA8pJYftfxNA== 2023-11-24T08:03:26.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T08:03:26.804 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T08:03:26.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:03:26.956 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:03:27.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:03:27.114 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:03:27.258 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:27.263 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:27.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:27.409 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:27.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:27.564 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:27.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:27.719 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:27.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T08:03:27.873 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T08:03:28.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:03:28.025 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:03:28.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:03:28.184 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:03:28.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:28.341 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:28.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:28.492 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:28.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:28.644 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:28.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:28.801 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:03:28.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:03:28.953 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:03:29.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:03:29.111 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:03:29.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:03:29.267 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:03:29.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:03:29.418 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:03:29.569 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T08:03:29.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T08:03:29.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T08:03:29.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T08:03:29.950 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T08:03:30.055 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T08:03:30.211 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T08:03:30.301 DEBUG:teuthology.orchestra.run.smithi091:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T08:03:30.398 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T08:03:30.554 DEBUG:teuthology.orchestra.run.smithi091:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T08:03:30.644 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:03:30.653 DEBUG:teuthology.orchestra.run.smithi091:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:03:30.708 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T08:03:30.709 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T08:03:30.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T08:03:30.716 INFO:tasks.ceph.mon.a:Started 2023-11-24T08:03:30.717 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T08:03:30.717 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T08:03:30.724 INFO:tasks.ceph.mon.b:Started 2023-11-24T08:03:30.724 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T08:03:30.725 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T08:03:30.732 INFO:tasks.ceph.mon.c:Started 2023-11-24T08:03:30.733 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T08:03:30.733 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T08:03:30.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T08:03:30.740 INFO:tasks.ceph.mgr.y:Started 2023-11-24T08:03:30.740 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T08:03:30.740 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T08:03:30.745 INFO:tasks.ceph.mgr.x:Started 2023-11-24T08:03:30.745 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T08:03:30.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T08:03:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.936+0000 7f7888b11640 1 Processor -- start 2023-11-24T08:03:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.936+0000 7f7888b11640 1 -- start start 2023-11-24T08:03:30.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.937+0000 7f7888b11640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f788415d3b0 0x7f78840ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:30.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.937+0000 7f7888b11640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840c6bf0 con 0x7f78840d1d90 2023-11-24T08:03:30.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.937+0000 7f7888b11640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840d1430 con 0x7f78840d0fa0 2023-11-24T08:03:30.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.937+0000 7f7888b11640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78840cedc0 con 0x7f788415d3b0 2023-11-24T08:03:30.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.938+0000 7f787bfff640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f78840d0fa0 0x7f78840d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:34154/0 (socket says 172.21.15.64:34154) 2023-11-24T08:03:30.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.938+0000 7f787bfff640 1 -- 172.21.15.64:0/425280764 learned_addr learned my addr 172.21.15.64:0/425280764 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:30.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.938+0000 7f78837fe640 1 --2- 172.21.15.64:0/425280764 >> v2:172.21.15.64:3300/0 conn(0x7f788415d3b0 0x7f78840ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:30.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.938+0000 7f78837fe640 1 -- 172.21.15.64:0/425280764 >> v1:172.21.15.91:6789/0 conn(0x7f78840d0fa0 legacy=0x7f78840d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:30.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.939+0000 7f78837fe640 1 -- 172.21.15.64:0/425280764 >> v1:172.21.15.64:6789/0 conn(0x7f78840d1d90 legacy=0x7f78840b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:30.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.939+0000 7f7882ffd640 1 -- 172.21.15.64:0/425280764 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70361126 0 0) 0x7f78840c6bf0 con 0x7f78840d1d90 2023-11-24T08:03:30.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.939+0000 7f78837fe640 1 -- 172.21.15.64:0/425280764 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78840c6e40 con 0x7f788415d3b0 2023-11-24T08:03:30.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.940+0000 7f78837fe640 1 --2- 172.21.15.64:0/425280764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788415d3b0 0x7f78840ce6c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f78740097a0 tx=0x7f787402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a26ba0920656232 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:30.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.941+0000 7f7882ffd640 1 -- 172.21.15.64:0/425280764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874031830 con 0x7f788415d3b0 2023-11-24T08:03:30.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.941+0000 7f7882ffd640 1 -- 172.21.15.64:0/425280764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7874031990 con 0x7f788415d3b0 2023-11-24T08:03:30.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.941+0000 7f7882ffd640 1 -- 172.21.15.64:0/425280764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874031d20 con 0x7f788415d3b0 2023-11-24T08:03:30.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.942+0000 7f7888b11640 1 -- 172.21.15.64:0/425280764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788415d3b0 msgr2=0x7f78840ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:30.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.942+0000 7f7888b11640 1 --2- 172.21.15.64:0/425280764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788415d3b0 0x7f78840ce6c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f78740097a0 tx=0x7f787402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:03:30.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.942+0000 7f7888b11640 1 -- 172.21.15.64:0/425280764 shutdown_connections 2023-11-24T08:03:30.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.942+0000 7f7888b11640 1 --2- 172.21.15.64:0/425280764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788415d3b0 0x7f78840ce6c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:30.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.942+0000 7f7888b11640 1 -- 172.21.15.64:0/425280764 >> 172.21.15.64:0/425280764 conn(0x7f78840d30a0 msgr2=0x7f78840d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:30.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.942+0000 7f7888b11640 1 -- 172.21.15.64:0/425280764 shutdown_connections 2023-11-24T08:03:30.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.942+0000 7f7888b11640 1 -- 172.21.15.64:0/425280764 wait complete. 2023-11-24T08:03:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.944+0000 7f7888b11640 1 Processor -- start 2023-11-24T08:03:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.944+0000 7f7888b11640 1 -- start start 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f7888b11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78840d0fa0 0x7f788414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f7888b11640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78840d1d90 0x7f78840b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f7888b11640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f788415d3b0 0x7f78841efec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f7888b11640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840b8e10 con 0x7f78840d0fa0 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f7888b11640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840cb340 con 0x7f788415d3b0 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f7888b11640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840cb620 con 0x7f78840d1d90 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f787bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78840d1d90 0x7f78840b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f787bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78840d1d90 0x7f78840b88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:34288/0 (socket says 172.21.15.64:34288) 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f787bfff640 1 -- 172.21.15.64:0/369233952 learned_addr learned my addr 172.21.15.64:0/369233952 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.945+0000 7f78837fe640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78840d0fa0 0x7f788414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.946+0000 7f7883fff640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f788415d3b0 0x7f78841efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.946+0000 7f787bfff640 1 -- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f788415d3b0 msgr2=0x7f78841efec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.946+0000 7f787bfff640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f788415d3b0 0x7f78841efec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.946+0000 7f787bfff640 1 -- 172.21.15.64:0/369233952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78840d0fa0 msgr2=0x7f788414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.946+0000 7f787bfff640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78840d0fa0 0x7f788414df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.946+0000 7f787bfff640 1 -- 172.21.15.64:0/369233952 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7874009400 con 0x7f78840d1d90 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.946+0000 7f7883fff640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f788415d3b0 0x7f78841efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.946+0000 7f787bfff640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78840d1d90 0x7f78840b88d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f786c00c4a0 tx=0x7f786c00c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.947+0000 7f7880ff9640 1 -- 172.21.15.64:0/369233952 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c005ce0 con 0x7f78840d1d90 2023-11-24T08:03:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.947+0000 7f7880ff9640 1 -- 172.21.15.64:0/369233952 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c005e40 con 0x7f78840d1d90 2023-11-24T08:03:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.947+0000 7f7880ff9640 1 -- 172.21.15.64:0/369233952 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c007270 con 0x7f78840d1d90 2023-11-24T08:03:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.947+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78840cb900 con 0x7f78840d1d90 2023-11-24T08:03:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.947+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78841f0400 con 0x7f78840d1d90 2023-11-24T08:03:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.947+0000 7f7880ff9640 1 -- 172.21.15.64:0/369233952 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f786c01c440 con 0x7f78840d1d90 2023-11-24T08:03:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.947+0000 7f7880ff9640 1 -- 172.21.15.64:0/369233952 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f786c012790 con 0x7f78840d1d90 2023-11-24T08:03:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.949+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78840cbe20 con 0x7f78840d1d90 2023-11-24T08:03:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:30.952+0000 7f7880ff9640 1 -- 172.21.15.64:0/369233952 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f78840cbe20 con 0x7f78840d1d90 2023-11-24T08:03:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.025+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f78840ce4c0 con 0x7f78840d1d90 2023-11-24T08:03:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.036+0000 7f7880ff9640 1 -- 172.21.15.64:0/369233952 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f786c021060 con 0x7f78840d1d90 2023-11-24T08:03:31.038 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2023-11-24T08:03:31.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.037+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78840d1d90 msgr2=0x7f78840b88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.037+0000 7f7888b11640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78840d1d90 0x7f78840b88d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f786c00c4a0 tx=0x7f786c00c960 comp rx=0 tx=0).stop 2023-11-24T08:03:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.037+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 shutdown_connections 2023-11-24T08:03:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.037+0000 7f7888b11640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f788415d3b0 0x7f78841efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.037+0000 7f7888b11640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78840d1d90 0x7f78840b88d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.037+0000 7f7888b11640 1 --2- 172.21.15.64:0/369233952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78840d0fa0 0x7f788414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.037+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 >> 172.21.15.64:0/369233952 conn(0x7f78840d30a0 msgr2=0x7f788406c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.038+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 shutdown_connections 2023-11-24T08:03:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:31.038+0000 7f7888b11640 1 -- 172.21.15.64:0/369233952 wait complete. 2023-11-24T08:03:31.058 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T08:03:31.058 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T08:03:31.058 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:31.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T08:03:31.136 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:31.136 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T08:03:31.265 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:31.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T08:03:31.356 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:03:31.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T08:03:31.444 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:31.445 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T08:03:31.508 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:31.507+0000 7fe80b03b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T08:03:31.533 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:31.533 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T08:03:31.538 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:31.536+0000 7f922b213040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T08:03:31.572 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:31.570+0000 7fe80b03b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T08:03:31.605 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:31.603+0000 7f922b213040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T08:03:31.613 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:31.614 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T08:03:31.739 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:03:31.739 DEBUG:teuthology.orchestra.run.smithi091:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T08:03:31.828 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 75bdc55f-4ba8-4fce-8181-d53970482ee4 0 2023-11-24T08:03:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.063+0000 7f7a9bd4c640 1 Processor -- start 2023-11-24T08:03:32.065 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.063+0000 7f7a9bd4c640 1 -- start start 2023-11-24T08:03:32.067 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.065+0000 7f7a9bd4c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7a9415d350 0x7f7a940d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.067 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.065+0000 7f7a9bd4c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a940d1840 0x7f7a940cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.067 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.065+0000 7f7a9bd4c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940cf240 con 0x7f7a9415d730 2023-11-24T08:03:32.067 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.065+0000 7f7a9bd4c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940d1fb0 con 0x7f7a940d1840 2023-11-24T08:03:32.067 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.065+0000 7f7a9bd4c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940ce050 con 0x7f7a9415d350 2023-11-24T08:03:32.067 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.065+0000 7f7a99ae8640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7a9415d350 0x7f7a940d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.067 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.065+0000 7f7a992e7640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f7a9415d730 0x7f7a940ceb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39246/0 (socket says 172.21.15.91:39246) 2023-11-24T08:03:32.067 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.065+0000 7f7a992e7640 1 -- 172.21.15.91:0/2932635205 learned_addr learned my addr 172.21.15.91:0/2932635205 (peer_addr_for_me v1:172.21.15.91:0/0) 2023-11-24T08:03:32.068 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.066+0000 7f7a9a2e9640 1 --2- 172.21.15.91:0/2932635205 >> v2:172.21.15.64:3300/0 conn(0x7f7a940d1840 0x7f7a940cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.069 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.066+0000 7f7a99ae8640 1 -- 172.21.15.91:0/2932635205 >> v2:172.21.15.64:3300/0 conn(0x7f7a940d1840 msgr2=0x7f7a940cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:32.069 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.066+0000 7f7a99ae8640 1 --2- 172.21.15.91:0/2932635205 >> v2:172.21.15.64:3300/0 conn(0x7f7a940d1840 0x7f7a940cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.069 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.066+0000 7f7a99ae8640 1 -- 172.21.15.91:0/2932635205 >> v1:172.21.15.91:6789/0 conn(0x7f7a9415d730 legacy=0x7f7a940ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.069 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.066+0000 7f7a99ae8640 1 -- 172.21.15.91:0/2932635205 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a940ce330 con 0x7f7a9415d350 2023-11-24T08:03:32.069 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.066+0000 7f7a98ae6640 1 -- 172.21.15.91:0/2932635205 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901906058 0 0) 0x7f7a940cf240 con 0x7f7a9415d730 2023-11-24T08:03:32.069 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.066+0000 7f7a9a2e9640 1 --2- 172.21.15.91:0/2932635205 >> v2:172.21.15.64:3300/0 conn(0x7f7a940d1840 0x7f7a940cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:32.069 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.067+0000 7f7a99ae8640 1 --2- 172.21.15.91:0/2932635205 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d350 0x7f7a940d3d30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7a9408bab0 tx=0x7f7a9002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae99ba9ea2f1cd9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.067+0000 7f7a98ae6640 1 -- 172.21.15.91:0/2932635205 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90031830 con 0x7f7a9415d350 2023-11-24T08:03:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.067+0000 7f7a98ae6640 1 -- 172.21.15.91:0/2932635205 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a90031990 con 0x7f7a9415d350 2023-11-24T08:03:32.070 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.067+0000 7f7a98ae6640 1 -- 172.21.15.91:0/2932635205 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90031d20 con 0x7f7a9415d350 2023-11-24T08:03:32.071 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.068+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/2932635205 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d350 msgr2=0x7f7a940d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.071 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.068+0000 7f7a9bd4c640 1 --2- 172.21.15.91:0/2932635205 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d350 0x7f7a940d3d30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7a9408bab0 tx=0x7f7a9002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.071 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.068+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/2932635205 shutdown_connections 2023-11-24T08:03:32.071 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.068+0000 7f7a9bd4c640 1 --2- 172.21.15.91:0/2932635205 >> v2:172.21.15.64:3300/0 conn(0x7f7a940d1840 0x7f7a940cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.071 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.068+0000 7f7a9bd4c640 1 --2- 172.21.15.91:0/2932635205 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d350 0x7f7a940d3d30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.071 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.068+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/2932635205 >> 172.21.15.91:0/2932635205 conn(0x7f7a9409c5c0 msgr2=0x7f7a940d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:32.071 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.068+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/2932635205 shutdown_connections 2023-11-24T08:03:32.071 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.068+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/2932635205 wait complete. 2023-11-24T08:03:32.072 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.070+0000 7f7a9bd4c640 1 Processor -- start 2023-11-24T08:03:32.072 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.070+0000 7f7a9bd4c640 1 -- start start 2023-11-24T08:03:32.074 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9bd4c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a940d1840 0x7f7a940cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.074 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9bd4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a9415d350 0x7f7a941f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.074 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9bd4c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d730 0x7f7a94157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.074 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9bd4c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9415db40 con 0x7f7a9415d350 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9bd4c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9415b340 con 0x7f7a9415d730 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9bd4c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940cc660 con 0x7f7a940d1840 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9a2e9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d730 0x7f7a94157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9a2e9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d730 0x7f7a94157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59146/0 (socket says 172.21.15.91:59146) 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9a2e9640 1 -- 172.21.15.91:0/903855234 learned_addr learned my addr 172.21.15.91:0/903855234 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a99ae8640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a940d1840 0x7f7a940cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a992e7640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a9415d350 0x7f7a941f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9a2e9640 1 -- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a940d1840 msgr2=0x7f7a940cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9a2e9640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a940d1840 0x7f7a940cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9a2e9640 1 -- 172.21.15.91:0/903855234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a9415d350 msgr2=0x7f7a941f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.071+0000 7f7a9a2e9640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a9415d350 0x7f7a941f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.075 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a9a2e9640 1 -- 172.21.15.91:0/903855234 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90009400 con 0x7f7a9415d730 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a99ae8640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a940d1840 0x7f7a940cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a9a2e9640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d730 0x7f7a94157610 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00cef0 tx=0x7f7a8c00a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a867fc640 1 -- 172.21.15.91:0/903855234 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00d840 con 0x7f7a9415d730 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a867fc640 1 -- 172.21.15.91:0/903855234 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c00d9c0 con 0x7f7a9415d730 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a867fc640 1 -- 172.21.15.91:0/903855234 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00dd70 con 0x7f7a9415d730 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a940cc940 con 0x7f7a9415d730 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a940ccf00 con 0x7f7a9415d730 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.072+0000 7f7a867fc640 1 -- 172.21.15.91:0/903855234 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7a8c01f740 con 0x7f7a9415d730 2023-11-24T08:03:32.076 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.073+0000 7f7a867fc640 1 -- 172.21.15.91:0/903855234 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7a8c015b30 con 0x7f7a9415d730 2023-11-24T08:03:32.077 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.074+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9415dcb0 con 0x7f7a9415d730 2023-11-24T08:03:32.078 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.077+0000 7f7a867fc640 1 -- 172.21.15.91:0/903855234 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f7a9415dcb0 con 0x7f7a9415d730 2023-11-24T08:03:32.157 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.156+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "75bdc55f-4ba8-4fce-8181-d53970482ee4", "id": 0} v 0) v1 -- 0x7f7a940a1230 con 0x7f7a9415d730 2023-11-24T08:03:32.166 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.164+0000 7f7a867fc640 1 -- 172.21.15.91:0/903855234 <== mon.1 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "75bdc55f-4ba8-4fce-8181-d53970482ee4", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7a8c01a070 con 0x7f7a9415d730 2023-11-24T08:03:32.166 INFO:teuthology.orchestra.run.smithi091.stdout:0 2023-11-24T08:03:32.168 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d730 msgr2=0x7f7a94157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.168 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d730 0x7f7a94157610 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00cef0 tx=0x7f7a8c00a710 comp rx=0 tx=0).stop 2023-11-24T08:03:32.168 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 shutdown_connections 2023-11-24T08:03:32.168 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a9415d730 0x7f7a94157610 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.168 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a9415d350 0x7f7a941f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.169 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 --2- 172.21.15.91:0/903855234 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a940d1840 0x7f7a940cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.169 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 >> 172.21.15.91:0/903855234 conn(0x7f7a9409c5c0 msgr2=0x7f7a9406c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:32.169 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 shutdown_connections 2023-11-24T08:03:32.169 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.166+0000 7f7a9bd4c640 1 -- 172.21.15.91:0/903855234 wait complete. 2023-11-24T08:03:32.186 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 5288e629-bdb6-4be4-9165-11a1d6d7d39d 1 2023-11-24T08:03:32.193 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.188+0000 7fe80b03b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T08:03:32.240 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.238+0000 7f922b213040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T08:03:32.254 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.252+0000 7fe80b03b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T08:03:32.303 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.301+0000 7f922b213040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T08:03:32.308 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.307+0000 7fe80b03b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T08:03:32.357 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.355+0000 7f922b213040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T08:03:32.374 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.372+0000 7f28d20ff640 1 Processor -- start 2023-11-24T08:03:32.374 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.372+0000 7f28d20ff640 1 -- start start 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.373+0000 7f28d20ff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f28cc0cfcb0 0x7f28cc0d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.373+0000 7f28d20ff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28cc0ce6f0 0x7f28cc0b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.373+0000 7f28d20ff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc0cf190 con 0x7f28cc160ab0 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.373+0000 7f28d20ff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc0b2e60 con 0x7f28cc0ce6f0 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.373+0000 7f28d20ff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc0cebd0 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.374+0000 7f28d08fc640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f28cc0cfcb0 0x7f28cc0d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.374+0000 7f28d08fc640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f28cc0cfcb0 0x7f28cc0d3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59158/0 (socket says 172.21.15.91:59158) 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.374+0000 7f28d08fc640 1 -- 172.21.15.91:0/1069232684 learned_addr learned my addr 172.21.15.91:0/1069232684 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.374+0000 7f28d08fc640 1 -- 172.21.15.91:0/1069232684 >> v2:172.21.15.64:3300/0 conn(0x7f28cc0ce6f0 msgr2=0x7f28cc0b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.374+0000 7f28d08fc640 1 --2- 172.21.15.91:0/1069232684 >> v2:172.21.15.64:3300/0 conn(0x7f28cc0ce6f0 0x7f28cc0b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.374+0000 7f28d08fc640 1 -- 172.21.15.91:0/1069232684 >> v1:172.21.15.64:6789/0 conn(0x7f28cc160ab0 legacy=0x7f28cc0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:32.377 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.374+0000 7f28d08fc640 1 -- 172.21.15.91:0/1069232684 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0ceeb0 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.377 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.375+0000 7f28d08fc640 1 --2- 172.21.15.91:0/1069232684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0cfcb0 0x7f28cc0d3b80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f28cc08c9a0 tx=0x7f28c002b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a01cc077018c54 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:32.377 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.375+0000 7f28cbfff640 1 -- 172.21.15.91:0/1069232684 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00317e0 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.377 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.375+0000 7f28cbfff640 1 -- 172.21.15.91:0/1069232684 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0031940 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.377 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.375+0000 7f28cbfff640 1 -- 172.21.15.91:0/1069232684 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0031cd0 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.378 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.376+0000 7f28d20ff640 1 -- 172.21.15.91:0/1069232684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0cfcb0 msgr2=0x7f28cc0d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.379 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.376+0000 7f28d20ff640 1 --2- 172.21.15.91:0/1069232684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0cfcb0 0x7f28cc0d3b80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f28cc08c9a0 tx=0x7f28c002b880 comp rx=0 tx=0).stop 2023-11-24T08:03:32.379 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.376+0000 7f28d20ff640 1 -- 172.21.15.91:0/1069232684 shutdown_connections 2023-11-24T08:03:32.379 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.376+0000 7f28d20ff640 1 --2- 172.21.15.91:0/1069232684 >> v2:172.21.15.64:3300/0 conn(0x7f28cc0ce6f0 0x7f28cc0b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.379 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.376+0000 7f28d20ff640 1 --2- 172.21.15.91:0/1069232684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0cfcb0 0x7f28cc0d3b80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.379 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.376+0000 7f28d20ff640 1 -- 172.21.15.91:0/1069232684 >> 172.21.15.91:0/1069232684 conn(0x7f28cc0d3060 msgr2=0x7f28cc0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:32.379 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.377+0000 7f28d20ff640 1 -- 172.21.15.91:0/1069232684 shutdown_connections 2023-11-24T08:03:32.379 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.377+0000 7f28d20ff640 1 -- 172.21.15.91:0/1069232684 wait complete. 2023-11-24T08:03:32.380 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.378+0000 7f28d20ff640 1 Processor -- start 2023-11-24T08:03:32.380 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.378+0000 7f28d20ff640 1 -- start start 2023-11-24T08:03:32.382 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.379+0000 7f28d20ff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0ce6f0 0x7f28cc1585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.382 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.379+0000 7f28d20ff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28cc0cfcb0 0x7f28cc15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.382 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d20ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28cc160ab0 0x7f28cc0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.382 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d20ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15c3f0 con 0x7f28cc160ab0 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d20ff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc158af0 con 0x7f28cc0ce6f0 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d20ff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15b770 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d08fc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28cc0cfcb0 0x7f28cc15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d10fd640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0ce6f0 0x7f28cc1585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d08fc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28cc0cfcb0 0x7f28cc15ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:50410/0 (socket says 172.21.15.91:50410) 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d08fc640 1 -- 172.21.15.91:0/2494620550 learned_addr learned my addr 172.21.15.91:0/2494620550 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d18fe640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28cc160ab0 0x7f28cc0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d08fc640 1 -- 172.21.15.91:0/2494620550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0ce6f0 msgr2=0x7f28cc1585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d08fc640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0ce6f0 0x7f28cc1585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d08fc640 1 -- 172.21.15.91:0/2494620550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28cc160ab0 msgr2=0x7f28cc0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d08fc640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28cc160ab0 0x7f28cc0c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d08fc640 1 -- 172.21.15.91:0/2494620550 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c00094d0 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.383 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.380+0000 7f28d10fd640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0ce6f0 0x7f28cc1585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:03:32.384 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.381+0000 7f28d08fc640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28cc0cfcb0 0x7f28cc15ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28c002b8c0 tx=0x7f28c00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:32.384 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.381+0000 7f28c9ffb640 1 -- 172.21.15.91:0/2494620550 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0032940 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.385 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.381+0000 7f28c9ffb640 1 -- 172.21.15.91:0/2494620550 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0032aa0 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.385 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.381+0000 7f28c9ffb640 1 -- 172.21.15.91:0/2494620550 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0032e50 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.385 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.381+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc15ba50 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.385 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.381+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc15c010 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.385 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.382+0000 7f28c9ffb640 1 -- 172.21.15.91:0/2494620550 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f28c004e750 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.385 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.382+0000 7f28c9ffb640 1 -- 172.21.15.91:0/2494620550 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f28c0045af0 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.385 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.382+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc158c60 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.386 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.385+0000 7f28c9ffb640 1 -- 172.21.15.91:0/2494620550 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f28cc158c60 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.387 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.385+0000 7fe80b03b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T08:03:32.422 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.420+0000 7f922b213040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T08:03:32.439 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.437+0000 7fe80b03b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T08:03:32.464 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.462+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5288e629-bdb6-4be4-9165-11a1d6d7d39d", "id": 1} v 0) v1 -- 0x7f28cc15c530 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.470 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.468+0000 7f28c9ffb640 1 -- 172.21.15.91:0/2494620550 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5288e629-bdb6-4be4-9165-11a1d6d7d39d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f28cc15c530 con 0x7f28cc0cfcb0 2023-11-24T08:03:32.470 INFO:teuthology.orchestra.run.smithi091.stdout:1 2023-11-24T08:03:32.473 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28cc0cfcb0 msgr2=0x7f28cc15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.474 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28cc0cfcb0 0x7f28cc15ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28c002b8c0 tx=0x7f28c00046c0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.474 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 shutdown_connections 2023-11-24T08:03:32.474 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28cc160ab0 0x7f28cc0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.474 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f28cc0cfcb0 0x7f28cc15ad20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.474 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 --2- 172.21.15.91:0/2494620550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f28cc0ce6f0 0x7f28cc1585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.475 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 >> 172.21.15.91:0/2494620550 conn(0x7f28cc0d3060 msgr2=0x7f28cc15e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:32.475 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 shutdown_connections 2023-11-24T08:03:32.475 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.470+0000 7f28d20ff640 1 -- 172.21.15.91:0/2494620550 wait complete. 2023-11-24T08:03:32.476 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.473+0000 7f922b213040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T08:03:32.489 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 5ab7bfce-76ed-4cea-892b-fd522573d0b0 2 2023-11-24T08:03:32.560 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.558+0000 7fe80b03b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T08:03:32.593 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.591+0000 7f922b213040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T08:03:32.645 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.643+0000 7fe80b03b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T08:03:32.677 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.675+0000 7f922b213040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T08:03:32.678 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.676+0000 7f95df61e640 1 Processor -- start 2023-11-24T08:03:32.678 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.676+0000 7f95df61e640 1 -- start start 2023-11-24T08:03:32.679 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.677+0000 7f95df61e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d8160bd0 0x7f95d80b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.680 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.677+0000 7f95df61e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f95d80b3210 0x7f95d8167290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.680 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.677+0000 7f95df61e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d80b23e0 con 0x7f95d80b48d0 2023-11-24T08:03:32.680 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.677+0000 7f95df61e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d80cdc70 con 0x7f95d80b3210 2023-11-24T08:03:32.680 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.677+0000 7f95df61e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80b2870 con 0x7f95d8160bd0 2023-11-24T08:03:32.681 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.678+0000 7f95dd3ba640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d8160bd0 0x7f95d80b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.681 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.678+0000 7f95dd3ba640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d8160bd0 0x7f95d80b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:50426/0 (socket says 172.21.15.91:50426) 2023-11-24T08:03:32.681 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.678+0000 7f95dd3ba640 1 -- 172.21.15.91:0/2800466599 learned_addr learned my addr 172.21.15.91:0/2800466599 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:32.681 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.678+0000 7f95dd3ba640 1 -- 172.21.15.91:0/2800466599 >> v2:172.21.15.64:3300/0 conn(0x7f95d80b3210 msgr2=0x7f95d8167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:32.682 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.678+0000 7f95dd3ba640 1 --2- 172.21.15.91:0/2800466599 >> v2:172.21.15.64:3300/0 conn(0x7f95d80b3210 0x7f95d8167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.682 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.678+0000 7f95dd3ba640 1 -- 172.21.15.91:0/2800466599 >> v1:172.21.15.64:6789/0 conn(0x7f95d80b48d0 legacy=0x7f95d80b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:32.682 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.678+0000 7f95dd3ba640 1 -- 172.21.15.91:0/2800466599 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80d41b0 con 0x7f95d8160bd0 2023-11-24T08:03:32.683 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.679+0000 7f95dd3ba640 1 --2- 172.21.15.91:0/2800466599 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d8160bd0 0x7f95d80b1ea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009870 tx=0x7f95d402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bfc55463694183d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:32.684 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.679+0000 7f95cffff640 1 -- 172.21.15.91:0/2800466599 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d40317c0 con 0x7f95d8160bd0 2023-11-24T08:03:32.684 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.679+0000 7f95cffff640 1 -- 172.21.15.91:0/2800466599 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4031920 con 0x7f95d8160bd0 2023-11-24T08:03:32.684 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.679+0000 7f95cffff640 1 -- 172.21.15.91:0/2800466599 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4031cb0 con 0x7f95d8160bd0 2023-11-24T08:03:32.684 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.680+0000 7f95df61e640 1 -- 172.21.15.91:0/2800466599 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d8160bd0 msgr2=0x7f95d80b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.680+0000 7f95df61e640 1 --2- 172.21.15.91:0/2800466599 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d8160bd0 0x7f95d80b1ea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009870 tx=0x7f95d402b860 comp rx=0 tx=0).stop 2023-11-24T08:03:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.680+0000 7f95df61e640 1 -- 172.21.15.91:0/2800466599 shutdown_connections 2023-11-24T08:03:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.680+0000 7f95df61e640 1 --2- 172.21.15.91:0/2800466599 >> v2:172.21.15.64:3300/0 conn(0x7f95d80b3210 0x7f95d8167290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.680+0000 7f95df61e640 1 --2- 172.21.15.91:0/2800466599 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d8160bd0 0x7f95d80b1ea0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.680+0000 7f95df61e640 1 -- 172.21.15.91:0/2800466599 >> 172.21.15.91:0/2800466599 conn(0x7f95d809c5a0 msgr2=0x7f95d80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.680+0000 7f95df61e640 1 -- 172.21.15.91:0/2800466599 shutdown_connections 2023-11-24T08:03:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.680+0000 7f95df61e640 1 -- 172.21.15.91:0/2800466599 wait complete. 2023-11-24T08:03:32.686 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.682+0000 7f95df61e640 1 Processor -- start 2023-11-24T08:03:32.686 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.682+0000 7f95df61e640 1 -- start start 2023-11-24T08:03:32.688 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95df61e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80b3210 0x7f95d8150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.688 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95df61e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d80b48d0 0x7f95d80cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.688 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95df61e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d8160bd0 0x7f95d80cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.689 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95df61e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80b7910 con 0x7f95d80b3210 2023-11-24T08:03:32.689 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95df61e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80b6a80 con 0x7f95d8160bd0 2023-11-24T08:03:32.689 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95df61e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80b6c90 con 0x7f95d80b48d0 2023-11-24T08:03:32.689 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95dcbb9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d80b48d0 0x7f95d80cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.690 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95dcbb9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d80b48d0 0x7f95d80cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:50434/0 (socket says 172.21.15.91:50434) 2023-11-24T08:03:32.690 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95dcbb9640 1 -- 172.21.15.91:0/1911276128 learned_addr learned my addr 172.21.15.91:0/1911276128 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:32.690 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.683+0000 7f95ddbbb640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d8160bd0 0x7f95d80cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.690 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95dd3ba640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80b3210 0x7f95d8150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.690 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95dcbb9640 1 -- 172.21.15.91:0/1911276128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d8160bd0 msgr2=0x7f95d80cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.690 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95dcbb9640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d8160bd0 0x7f95d80cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.691 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95dcbb9640 1 -- 172.21.15.91:0/1911276128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80b3210 msgr2=0x7f95d8150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.691 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95dcbb9640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80b3210 0x7f95d8150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.691 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95dcbb9640 1 -- 172.21.15.91:0/1911276128 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40094d0 con 0x7f95d80b48d0 2023-11-24T08:03:32.691 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95dcbb9640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d80b48d0 0x7f95d80cab20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c400 tx=0x7f95c800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:32.691 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95cdffb640 1 -- 172.21.15.91:0/1911276128 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80035f0 con 0x7f95d80b48d0 2023-11-24T08:03:32.691 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95cdffb640 1 -- 172.21.15.91:0/1911276128 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8003750 con 0x7f95d80b48d0 2023-11-24T08:03:32.691 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95cdffb640 1 -- 172.21.15.91:0/1911276128 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8003b40 con 0x7f95d80b48d0 2023-11-24T08:03:32.692 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.684+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d80b6f70 con 0x7f95d80b48d0 2023-11-24T08:03:32.692 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.685+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d80b7530 con 0x7f95d80b48d0 2023-11-24T08:03:32.692 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.685+0000 7f95cdffb640 1 -- 172.21.15.91:0/1911276128 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f95c8007020 con 0x7f95d80b48d0 2023-11-24T08:03:32.692 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.685+0000 7f95cdffb640 1 -- 172.21.15.91:0/1911276128 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f95c80072d0 con 0x7f95d80b48d0 2023-11-24T08:03:32.692 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.686+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d80b7a50 con 0x7f95d80b48d0 2023-11-24T08:03:32.693 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.688+0000 7f95cdffb640 1 -- 172.21.15.91:0/1911276128 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f95d80b7a50 con 0x7f95d80b48d0 2023-11-24T08:03:32.702 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.700+0000 7fe80b03b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T08:03:32.733 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.731+0000 7f922b213040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T08:03:32.764 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.762+0000 7fe80b03b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T08:03:32.767 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.766+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5ab7bfce-76ed-4cea-892b-fd522573d0b0", "id": 2} v 0) v1 -- 0x7f95d80a1210 con 0x7f95d80b48d0 2023-11-24T08:03:32.776 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.774+0000 7f95cdffb640 1 -- 172.21.15.91:0/1911276128 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ab7bfce-76ed-4cea-892b-fd522573d0b0", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f95c8020060 con 0x7f95d80b48d0 2023-11-24T08:03:32.776 INFO:teuthology.orchestra.run.smithi091.stdout:2 2023-11-24T08:03:32.777 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.775+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d80b48d0 msgr2=0x7f95d80cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.777 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.775+0000 7f95df61e640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d80b48d0 0x7f95d80cab20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c400 tx=0x7f95c800c8c0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.778 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.775+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 shutdown_connections 2023-11-24T08:03:32.778 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.775+0000 7f95df61e640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95d8160bd0 0x7f95d80cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.778 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.775+0000 7f95df61e640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95d80b48d0 0x7f95d80cab20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.778 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.775+0000 7f95df61e640 1 --2- 172.21.15.91:0/1911276128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80b3210 0x7f95d8150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.778 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.775+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 >> 172.21.15.91:0/1911276128 conn(0x7f95d809c5a0 msgr2=0x7f95d80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:32.778 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.776+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 shutdown_connections 2023-11-24T08:03:32.778 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.776+0000 7f95df61e640 1 -- 172.21.15.91:0/1911276128 wait complete. 2023-11-24T08:03:32.793 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 814b901b-da27-4b2b-95a8-19ddad51b97e 3 2023-11-24T08:03:32.796 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.795+0000 7f922b213040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T08:03:32.824 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:32.823+0000 7fe80b03b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T08:03:32.853 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:32.852+0000 7f922b213040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T08:03:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.984+0000 7f986ffff640 1 Processor -- start 2023-11-24T08:03:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.984+0000 7f986ffff640 1 -- start start 2023-11-24T08:03:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.986+0000 7f986ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98700cfcb0 0x7f98700d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.986+0000 7f986ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98700cf190 con 0x7f98700ce6f0 2023-11-24T08:03:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.986+0000 7f986ffff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98700b2e60 con 0x7f9870160ab0 2023-11-24T08:03:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.986+0000 7f986ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98700cebd0 con 0x7f98700cfcb0 2023-11-24T08:03:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.986+0000 7f986effd640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9870160ab0 0x7f98700cf5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39262/0 (socket says 172.21.15.91:39262) 2023-11-24T08:03:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.986+0000 7f986effd640 1 -- 172.21.15.91:0/2207197439 learned_addr learned my addr 172.21.15.91:0/2207197439 (peer_addr_for_me v1:172.21.15.91:0/0) 2023-11-24T08:03:32.990 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178132742 0 0) 0x7f98700b2e60 con 0x7f9870160ab0 2023-11-24T08:03:32.990 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f985c003520 con 0x7f9870160ab0 2023-11-24T08:03:32.991 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f9867fff640 1 --2- 172.21.15.91:0/2207197439 >> v2:172.21.15.64:3300/0 conn(0x7f98700cfcb0 0x7f98700d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.991 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 <== mon.1 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 485838139 0 0) 0x7f985c003520 con 0x7f9870160ab0 2023-11-24T08:03:32.991 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 >> v2:172.21.15.64:3300/0 conn(0x7f98700cfcb0 msgr2=0x7f98700d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:32.991 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 --2- 172.21.15.91:0/2207197439 >> v2:172.21.15.64:3300/0 conn(0x7f98700cfcb0 0x7f98700d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.991 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 >> v1:172.21.15.64:6789/0 conn(0x7f98700ce6f0 legacy=0x7f98700b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.991 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98700ceeb0 con 0x7f9870160ab0 2023-11-24T08:03:32.991 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 <== mon.1 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f9860001670 con 0x7f9870160ab0 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f9867fff640 1 --2- 172.21.15.91:0/2207197439 >> v2:172.21.15.64:3300/0 conn(0x7f98700cfcb0 0x7f98700d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 >> v1:172.21.15.91:6789/0 conn(0x7f9870160ab0 legacy=0x7f98700cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f985c0039e0 0x7f985c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f985c0047c0 0x7f985c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0052a0 0x7f985c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98700cebd0 con 0x7f985c0052a0 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c00c1f0 con 0x7f985c0039e0 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.987+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c00c3b0 con 0x7f985c0047c0 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f986effd640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f985c0039e0 0x7f985c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f9867fff640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f985c0047c0 0x7f985c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f986f7fe640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0052a0 0x7f985c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.992 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f9867fff640 1 -- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f985c0039e0 msgr2=0x7f985c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:32.993 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f9867fff640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f985c0039e0 0x7f985c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.993 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f9867fff640 1 -- 172.21.15.91:0/2207197439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0052a0 msgr2=0x7f985c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:32.993 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f9867fff640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0052a0 0x7f985c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.993 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f9867fff640 1 -- 172.21.15.91:0/2207197439 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c003520 con 0x7f985c0047c0 2023-11-24T08:03:32.993 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.988+0000 7f986f7fe640 1 -- 172.21.15.91:0/2207197439 reap_dead start 2023-11-24T08:03:32.995 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.989+0000 7f9867fff640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f985c0047c0 0x7f985c004c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9858006fd0 tx=0x7f985c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7849c68e861bac9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:32.995 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.989+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858004030 con 0x7f985c0047c0 2023-11-24T08:03:32.995 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.989+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9858004190 con 0x7f985c0047c0 2023-11-24T08:03:32.995 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.989+0000 7f986e7fc640 1 -- 172.21.15.91:0/2207197439 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858003610 con 0x7f985c0047c0 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.990+0000 7f986ffff640 1 -- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f985c0047c0 msgr2=0x7f985c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.990+0000 7f986ffff640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f985c0047c0 0x7f985c004c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9858006fd0 tx=0x7f985c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.990+0000 7f986ffff640 1 -- 172.21.15.91:0/2207197439 shutdown_connections 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.990+0000 7f986ffff640 1 --2- 172.21.15.91:0/2207197439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f985c0047c0 0x7f985c004c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.990+0000 7f986ffff640 1 -- 172.21.15.91:0/2207197439 >> 172.21.15.91:0/2207197439 conn(0x7f98700d3060 msgr2=0x7f98700d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.990+0000 7f986ffff640 1 -- 172.21.15.91:0/2207197439 shutdown_connections 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.990+0000 7f986ffff640 1 -- 172.21.15.91:0/2207197439 wait complete. 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.992+0000 7f986ffff640 1 Processor -- start 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.992+0000 7f986ffff640 1 -- start start 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0047c0 0x7f98700c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98700c8770 0x7f987015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9870151000 0x7f9870151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:32.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98701f3e10 con 0x7f985c0047c0 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98701519d0 con 0x7f9870151000 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98701f3850 con 0x7f98700c8770 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986f7fe640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9870151000 0x7f9870151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f9867fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98700c8770 0x7f987015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986f7fe640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9870151000 0x7f9870151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.91:59210/0 (socket says 172.21.15.91:59210) 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.993+0000 7f986f7fe640 1 -- 172.21.15.91:0/3290882440 learned_addr learned my addr 172.21.15.91:0/3290882440 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986effd640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0047c0 0x7f98700c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986f7fe640 1 -- 172.21.15.91:0/3290882440 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98700c8770 msgr2=0x7f987015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986f7fe640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98700c8770 0x7f987015e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986f7fe640 1 -- 172.21.15.91:0/3290882440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0047c0 msgr2=0x7f98700c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986f7fe640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0047c0 0x7f98700c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:32.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986f7fe640 1 -- 172.21.15.91:0/3290882440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98701f3b30 con 0x7f9870151000 2023-11-24T08:03:32.998 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986effd640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0047c0 0x7f98700c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986f7fe640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9870151000 0x7f9870151490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9868007fd0 tx=0x7f986800eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f98677fe640 1 -- 172.21.15.91:0/3290882440 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986800eed0 con 0x7f9870151000 2023-11-24T08:03:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f98677fe640 1 -- 172.21.15.91:0/3290882440 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9868006b50 con 0x7f9870151000 2023-11-24T08:03:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98700c6d30 con 0x7f9870151000 2023-11-24T08:03:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f98677fe640 1 -- 172.21.15.91:0/3290882440 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868003970 con 0x7f9870151000 2023-11-24T08:03:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.994+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98700c7a50 con 0x7f9870151000 2023-11-24T08:03:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.995+0000 7f98677fe640 1 -- 172.21.15.91:0/3290882440 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9868005110 con 0x7f9870151000 2023-11-24T08:03:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.995+0000 7f98677fe640 1 -- 172.21.15.91:0/3290882440 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f986800c7d0 con 0x7f9870151000 2023-11-24T08:03:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.995+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987004e840 con 0x7f9870151000 2023-11-24T08:03:33.001 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:32.998+0000 7f98677fe640 1 -- 172.21.15.91:0/3290882440 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9868019070 con 0x7f9870151000 2023-11-24T08:03:33.077 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.076+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "814b901b-da27-4b2b-95a8-19ddad51b97e", "id": 3} v 0) v1 -- 0x7f98700c9ef0 con 0x7f9870151000 2023-11-24T08:03:33.085 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.083+0000 7f98677fe640 1 -- 172.21.15.91:0/3290882440 <== mon.1 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "814b901b-da27-4b2b-95a8-19ddad51b97e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f986800cab0 con 0x7f9870151000 2023-11-24T08:03:33.086 INFO:teuthology.orchestra.run.smithi091.stdout:3 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9870151000 msgr2=0x7f9870151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9870151000 0x7f9870151490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9868007fd0 tx=0x7f986800eb90 comp rx=0 tx=0).stop 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 shutdown_connections 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9870151000 0x7f9870151490 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f98700c8770 0x7f987015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 --2- 172.21.15.91:0/3290882440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f985c0047c0 0x7f98700c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 >> 172.21.15.91:0/3290882440 conn(0x7f98700d3060 msgr2=0x7f987015d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 shutdown_connections 2023-11-24T08:03:33.087 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.085+0000 7f986ffff640 1 -- 172.21.15.91:0/3290882440 wait complete. 2023-11-24T08:03:33.102 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 7833546d-f061-414c-a385-3875be27241e 4 2023-11-24T08:03:33.301 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:33.299+0000 7fe80b03b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T08:03:33.328 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:33.326+0000 7f922b213040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T08:03:33.335 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.333+0000 7f9a71675640 1 Processor -- start 2023-11-24T08:03:33.335 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.334+0000 7f9a71675640 1 -- start start 2023-11-24T08:03:33.337 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.335+0000 7f9a71675640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b3210 0x7f9a6c0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.337 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.335+0000 7f9a71675640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6c0b23e0 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.337 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.335+0000 7f9a71675640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6c0d41b0 con 0x7f9a6c160bd0 2023-11-24T08:03:33.337 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.335+0000 7f9a71675640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c0cdc70 con 0x7f9a6c0b3210 2023-11-24T08:03:33.338 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.336+0000 7f9a6affd640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f9a6c160bd0 0x7f9a6c0b2b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39278/0 (socket says 172.21.15.91:39278) 2023-11-24T08:03:33.338 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.336+0000 7f9a6affd640 1 -- 172.21.15.91:0/3351152946 learned_addr learned my addr 172.21.15.91:0/3351152946 (peer_addr_for_me v1:172.21.15.91:0/0) 2023-11-24T08:03:33.338 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.336+0000 7f9a6b7fe640 1 --2- 172.21.15.91:0/3351152946 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b3210 0x7f9a6c0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.338 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.336+0000 7f9a6a7fc640 1 -- 172.21.15.91:0/3351152946 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550605557 0 0) 0x7f9a6c0d41b0 con 0x7f9a6c160bd0 2023-11-24T08:03:33.338 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.336+0000 7f9a6a7fc640 1 -- 172.21.15.91:0/3351152946 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a58003520 con 0x7f9a6c160bd0 2023-11-24T08:03:33.339 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.336+0000 7f9a6b7fe640 1 -- 172.21.15.91:0/3351152946 >> v1:172.21.15.91:6789/0 conn(0x7f9a6c160bd0 legacy=0x7f9a6c0b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.339 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.336+0000 7f9a6b7fe640 1 -- 172.21.15.91:0/3351152946 >> v1:172.21.15.64:6789/0 conn(0x7f9a6c0b48d0 legacy=0x7f9a6c165280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.339 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.336+0000 7f9a6b7fe640 1 -- 172.21.15.91:0/3351152946 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c160fb0 con 0x7f9a6c0b3210 2023-11-24T08:03:33.339 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.338+0000 7f9a6b7fe640 1 --2- 172.21.15.91:0/3351152946 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b3210 0x7f9a6c0b1ea0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a640097d0 tx=0x7f9a6402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b786ab28f6863e4b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:33.340 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.338+0000 7f9a6a7fc640 1 -- 172.21.15.91:0/3351152946 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64031800 con 0x7f9a6c0b3210 2023-11-24T08:03:33.340 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.338+0000 7f9a6a7fc640 1 -- 172.21.15.91:0/3351152946 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a64031960 con 0x7f9a6c0b3210 2023-11-24T08:03:33.340 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.338+0000 7f9a6a7fc640 1 -- 172.21.15.91:0/3351152946 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64031cf0 con 0x7f9a6c0b3210 2023-11-24T08:03:33.341 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.339+0000 7f9a71675640 1 -- 172.21.15.91:0/3351152946 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b3210 msgr2=0x7f9a6c0b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.341 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.339+0000 7f9a71675640 1 --2- 172.21.15.91:0/3351152946 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b3210 0x7f9a6c0b1ea0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a640097d0 tx=0x7f9a6402b8a0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.341 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.339+0000 7f9a71675640 1 -- 172.21.15.91:0/3351152946 shutdown_connections 2023-11-24T08:03:33.341 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.339+0000 7f9a71675640 1 --2- 172.21.15.91:0/3351152946 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b3210 0x7f9a6c0b1ea0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.341 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.339+0000 7f9a71675640 1 -- 172.21.15.91:0/3351152946 >> 172.21.15.91:0/3351152946 conn(0x7f9a6c09c5a0 msgr2=0x7f9a6c0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:33.341 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.339+0000 7f9a71675640 1 -- 172.21.15.91:0/3351152946 shutdown_connections 2023-11-24T08:03:33.341 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.339+0000 7f9a71675640 1 -- 172.21.15.91:0/3351152946 wait complete. 2023-11-24T08:03:33.342 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.341+0000 7f9a71675640 1 Processor -- start 2023-11-24T08:03:33.342 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.341+0000 7f9a71675640 1 -- start start 2023-11-24T08:03:33.345 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a71675640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0b3210 0x7f9a6c158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.345 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a71675640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b48d0 0x7f9a6c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.345 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a71675640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a6c160bd0 0x7f9a6c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.345 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a71675640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c0b7c50 con 0x7f9a6c0b3210 2023-11-24T08:03:33.345 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a71675640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c15a870 con 0x7f9a6c160bd0 2023-11-24T08:03:33.345 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a71675640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c0b6d70 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.345 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a627fc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b48d0 0x7f9a6c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a627fc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b48d0 0x7f9a6c15a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:50472/0 (socket says 172.21.15.91:50472) 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a627fc640 1 -- 172.21.15.91:0/2572529797 learned_addr learned my addr 172.21.15.91:0/2572529797 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a6b7fe640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a6c160bd0 0x7f9a6c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a6affd640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0b3210 0x7f9a6c158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a627fc640 1 -- 172.21.15.91:0/2572529797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a6c160bd0 msgr2=0x7f9a6c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a627fc640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a6c160bd0 0x7f9a6c0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a627fc640 1 -- 172.21.15.91:0/2572529797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0b3210 msgr2=0x7f9a6c158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.342+0000 7f9a627fc640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0b3210 0x7f9a6c158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a627fc640 1 -- 172.21.15.91:0/2572529797 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64009400 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a6affd640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0b3210 0x7f9a6c158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a627fc640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b48d0 0x7f9a6c15a330 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c3f0 tx=0x7f9a5400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:33.346 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a63fff640 1 -- 172.21.15.91:0/2572529797 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540052d0 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.347 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a63fff640 1 -- 172.21.15.91:0/2572529797 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a54005430 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.347 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a63fff640 1 -- 172.21.15.91:0/2572529797 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54003830 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.347 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c0b7050 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.347 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c0b7870 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.347 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.343+0000 7f9a63fff640 1 -- 172.21.15.91:0/2572529797 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9a54003c90 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.348 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.344+0000 7f9a63fff640 1 -- 172.21.15.91:0/2572529797 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9a54012420 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.348 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.344+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c0b7a30 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.348 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.347+0000 7f9a63fff640 1 -- 172.21.15.91:0/2572529797 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9a54017070 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.427 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.425+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7833546d-f061-414c-a385-3875be27241e", "id": 4} v 0) v1 -- 0x7f9a6c0b7d90 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.435 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.433+0000 7f9a63fff640 1 -- 172.21.15.91:0/2572529797 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7833546d-f061-414c-a385-3875be27241e", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9a6c0b7d90 con 0x7f9a6c0b48d0 2023-11-24T08:03:33.435 INFO:teuthology.orchestra.run.smithi091.stdout:4 2023-11-24T08:03:33.436 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.434+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b48d0 msgr2=0x7f9a6c15a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.436 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.434+0000 7f9a71675640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b48d0 0x7f9a6c15a330 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c3f0 tx=0x7f9a5400c8b0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.436 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.434+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 shutdown_connections 2023-11-24T08:03:33.436 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.435+0000 7f9a71675640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a6c160bd0 0x7f9a6c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.437 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.435+0000 7f9a71675640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a6c0b48d0 0x7f9a6c15a330 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.437 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.435+0000 7f9a71675640 1 --2- 172.21.15.91:0/2572529797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6c0b3210 0x7f9a6c158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.437 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.435+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 >> 172.21.15.91:0/2572529797 conn(0x7f9a6c09c5a0 msgr2=0x7f9a6c06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:33.437 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.435+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 shutdown_connections 2023-11-24T08:03:33.437 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.435+0000 7f9a71675640 1 -- 172.21.15.91:0/2572529797 wait complete. 2023-11-24T08:03:33.454 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new b45f456d-e76a-47b6-a1ee-d7e726c68288 5 2023-11-24T08:03:33.472 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:33.470+0000 7fe80b03b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T08:03:33.497 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:33.494+0000 7f922b213040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T08:03:33.619 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:33.617+0000 7fe80b03b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T08:03:33.639 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:33.637+0000 7f922b213040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T08:03:33.643 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.641+0000 7f50f8b06640 1 Processor -- start 2023-11-24T08:03:33.643 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.641+0000 7f50f8b06640 1 -- start start 2023-11-24T08:03:33.645 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.642+0000 7f50f8b06640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f415d3b0 0x7f50f40ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.645 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f8b06640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f50f40d0fa0 0x7f50f40d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.645 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f8b06640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f40d2010 con 0x7f50f40d1bc0 2023-11-24T08:03:33.645 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f8b06640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f40c6bf0 con 0x7f50f40d0fa0 2023-11-24T08:03:33.645 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f8b06640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40d1400 con 0x7f50f415d3b0 2023-11-24T08:03:33.645 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f415d3b0 0x7f50f40ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.645 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f3fff640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f50f40d1bc0 0x7f50f40b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39294/0 (socket says 172.21.15.91:39294) 2023-11-24T08:03:33.645 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f3fff640 1 -- 172.21.15.91:0/3960640955 learned_addr learned my addr 172.21.15.91:0/3960640955 (peer_addr_for_me v1:172.21.15.91:0/0) 2023-11-24T08:03:33.647 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50ebfff640 1 --2- 172.21.15.91:0/3960640955 >> v2:172.21.15.64:3300/0 conn(0x7f50f40d0fa0 0x7f50f40d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.647 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f37fe640 1 -- 172.21.15.91:0/3960640955 >> v2:172.21.15.64:3300/0 conn(0x7f50f40d0fa0 msgr2=0x7f50f40d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:33.647 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f37fe640 1 --2- 172.21.15.91:0/3960640955 >> v2:172.21.15.64:3300/0 conn(0x7f50f40d0fa0 0x7f50f40d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.647 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f37fe640 1 -- 172.21.15.91:0/3960640955 >> v1:172.21.15.91:6789/0 conn(0x7f50f40d1bc0 legacy=0x7f50f40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.647 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.643+0000 7f50f37fe640 1 -- 172.21.15.91:0/3960640955 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f40cedc0 con 0x7f50f415d3b0 2023-11-24T08:03:33.648 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.644+0000 7f50f37fe640 1 --2- 172.21.15.91:0/3960640955 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f415d3b0 0x7f50f40ce6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50e40097a0 tx=0x7f50e402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc96d1afb0faf24 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:33.648 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.645+0000 7f50f2ffd640 1 -- 172.21.15.91:0/3960640955 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4031780 con 0x7f50f415d3b0 2023-11-24T08:03:33.648 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.645+0000 7f50f2ffd640 1 -- 172.21.15.91:0/3960640955 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e40318e0 con 0x7f50f415d3b0 2023-11-24T08:03:33.648 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.645+0000 7f50f2ffd640 1 -- 172.21.15.91:0/3960640955 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4031c70 con 0x7f50f415d3b0 2023-11-24T08:03:33.648 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.645+0000 7f50f8b06640 1 -- 172.21.15.91:0/3960640955 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f415d3b0 msgr2=0x7f50f40ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.648 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.645+0000 7f50f8b06640 1 --2- 172.21.15.91:0/3960640955 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f415d3b0 0x7f50f40ce6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50e40097a0 tx=0x7f50e402b820 comp rx=0 tx=0).stop 2023-11-24T08:03:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.646+0000 7f50f8b06640 1 -- 172.21.15.91:0/3960640955 shutdown_connections 2023-11-24T08:03:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.646+0000 7f50f8b06640 1 --2- 172.21.15.91:0/3960640955 >> v2:172.21.15.64:3300/0 conn(0x7f50f40d0fa0 0x7f50f40d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.646+0000 7f50f8b06640 1 --2- 172.21.15.91:0/3960640955 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f415d3b0 0x7f50f40ce6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.646+0000 7f50f8b06640 1 -- 172.21.15.91:0/3960640955 >> 172.21.15.91:0/3960640955 conn(0x7f50f40d30a0 msgr2=0x7f50f40d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.646+0000 7f50f8b06640 1 -- 172.21.15.91:0/3960640955 shutdown_connections 2023-11-24T08:03:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.646+0000 7f50f8b06640 1 -- 172.21.15.91:0/3960640955 wait complete. 2023-11-24T08:03:33.649 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.647+0000 7f50f8b06640 1 Processor -- start 2023-11-24T08:03:33.650 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.647+0000 7f50f8b06640 1 -- start start 2023-11-24T08:03:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.648+0000 7f50f8b06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f40d0fa0 0x7f50f409d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.648+0000 7f50f8b06640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f40d1bc0 0x7f50f415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.648+0000 7f50f8b06640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50f415d3b0 0x7f50f415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.652 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.648+0000 7f50f8b06640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f409efa0 con 0x7f50f40d0fa0 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.648+0000 7f50f8b06640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f409fb20 con 0x7f50f415d3b0 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.648+0000 7f50f8b06640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f409f1f0 con 0x7f50f40d1bc0 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f40d1bc0 0x7f50f415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f40d1bc0 0x7f50f415b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:50490/0 (socket says 172.21.15.91:50490) 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50f415d3b0 0x7f50f415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 -- 172.21.15.91:0/3366264943 learned_addr learned my addr 172.21.15.91:0/3366264943 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50f37fe640 1 --2- 172.21.15.91:0/3366264943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f40d0fa0 0x7f50f409d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 -- 172.21.15.91:0/3366264943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50f415d3b0 msgr2=0x7f50f415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 --2- 172.21.15.91:0/3366264943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50f415d3b0 0x7f50f415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 -- 172.21.15.91:0/3366264943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f40d0fa0 msgr2=0x7f50f409d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 --2- 172.21.15.91:0/3366264943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f40d0fa0 0x7f50f409d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.653 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 -- 172.21.15.91:0/3366264943 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e4009400 con 0x7f50f40d1bc0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.649+0000 7f50ebfff640 1 --2- 172.21.15.91:0/3366264943 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f40d1bc0 0x7f50f415b9a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00d620 tx=0x7f50dc00dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.650+0000 7f50f0ff9640 1 -- 172.21.15.91:0/3366264943 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc00b620 con 0x7f50f40d1bc0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.650+0000 7f50f0ff9640 1 -- 172.21.15.91:0/3366264943 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc00b780 con 0x7f50f40d1bc0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.650+0000 7f50f0ff9640 1 -- 172.21.15.91:0/3366264943 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc00bb30 con 0x7f50f40d1bc0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.650+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f409f4d0 con 0x7f50f40d1bc0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.650+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f409dae0 con 0x7f50f40d1bc0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.650+0000 7f50f0ff9640 1 -- 172.21.15.91:0/3366264943 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f50dc01e460 con 0x7f50f40d1bc0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.650+0000 7f50f0ff9640 1 -- 172.21.15.91:0/3366264943 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f50dc01ec90 con 0x7f50f40d1bc0 2023-11-24T08:03:33.654 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.651+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f40cd4d0 con 0x7f50f40d1bc0 2023-11-24T08:03:33.655 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.653+0000 7f50f0ff9640 1 -- 172.21.15.91:0/3366264943 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f50dc01e710 con 0x7f50f40d1bc0 2023-11-24T08:03:33.677 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:33.675+0000 7fe80b03b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T08:03:33.694 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:33.692+0000 7f922b213040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T08:03:33.733 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.731+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b45f456d-e76a-47b6-a1ee-d7e726c68288", "id": 5} v 0) v1 -- 0x7f50f409dca0 con 0x7f50f40d1bc0 2023-11-24T08:03:33.739 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.738+0000 7f50f0ff9640 1 -- 172.21.15.91:0/3366264943 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b45f456d-e76a-47b6-a1ee-d7e726c68288", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f50dc01e9e0 con 0x7f50f40d1bc0 2023-11-24T08:03:33.740 INFO:teuthology.orchestra.run.smithi091.stdout:5 2023-11-24T08:03:33.741 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f40d1bc0 msgr2=0x7f50f415b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:33.741 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 --2- 172.21.15.91:0/3366264943 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f40d1bc0 0x7f50f415b9a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00d620 tx=0x7f50dc00dae0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.741 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 shutdown_connections 2023-11-24T08:03:33.741 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 --2- 172.21.15.91:0/3366264943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f50f415d3b0 0x7f50f415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.741 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 --2- 172.21.15.91:0/3366264943 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f50f40d1bc0 0x7f50f415b9a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.741 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 --2- 172.21.15.91:0/3366264943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f40d0fa0 0x7f50f409d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:33.741 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 >> 172.21.15.91:0/3366264943 conn(0x7f50f40d30a0 msgr2=0x7f50f40d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:33.742 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 shutdown_connections 2023-11-24T08:03:33.742 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.739+0000 7f50f8b06640 1 -- 172.21.15.91:0/3366264943 wait complete. 2023-11-24T08:03:33.757 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new cb020221-0c2e-4aef-9cea-26ea555234a4 6 2023-11-24T08:03:33.794 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:33.792+0000 7fe80b03b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T08:03:33.809 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:33.807+0000 7f922b213040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T08:03:33.912 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:33.911+0000 7fe80b03b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T08:03:33.925 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:33.923+0000 7f922b213040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T08:03:33.994 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.993+0000 7f316ce48640 1 Processor -- start 2023-11-24T08:03:33.995 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.993+0000 7f316ce48640 1 -- start start 2023-11-24T08:03:33.996 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.994+0000 7f316ce48640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31680b48d0 0x7f31680b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:33.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.994+0000 7f316ce48640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31680b23e0 con 0x7f3168160bd0 2023-11-24T08:03:33.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.994+0000 7f316ce48640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31680d41b0 con 0x7f31680b3210 2023-11-24T08:03:33.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.994+0000 7f316ce48640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31680cdc70 con 0x7f31680b48d0 2023-11-24T08:03:33.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.994+0000 7f3166d9d640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f31680b3210 0x7f3168167290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39296/0 (socket says 172.21.15.91:39296) 2023-11-24T08:03:33.997 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.994+0000 7f3166d9d640 1 -- 172.21.15.91:0/1837173591 learned_addr learned my addr 172.21.15.91:0/1837173591 (peer_addr_for_me v1:172.21.15.91:0/0) 2023-11-24T08:03:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205292128 0 0) 0x7f31680d41b0 con 0x7f31680b3210 2023-11-24T08:03:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f315ffff640 1 --2- 172.21.15.91:0/1837173591 >> v2:172.21.15.64:3300/0 conn(0x7f31680b48d0 0x7f31680b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3154003520 con 0x7f31680b3210 2023-11-24T08:03:33.999 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 <== mon.1 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2051408605 0 0) 0x7f3154003520 con 0x7f31680b3210 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 >> v2:172.21.15.64:3300/0 conn(0x7f31680b48d0 msgr2=0x7f31680b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 --2- 172.21.15.91:0/1837173591 >> v2:172.21.15.64:3300/0 conn(0x7f31680b48d0 0x7f31680b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 >> v1:172.21.15.64:6789/0 conn(0x7f3168160bd0 legacy=0x7f31680b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3168160fb0 con 0x7f31680b3210 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 <== mon.1 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f31600019f0 con 0x7f31680b3210 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 >> v1:172.21.15.91:6789/0 conn(0x7f31680b3210 legacy=0x7f3168167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31540039c0 0x7f3154003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31540047c0 0x7f3154004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31540051f0 0x7f315400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680cdc70 con 0x7f31540039c0 2023-11-24T08:03:34.000 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315400c150 con 0x7f31540051f0 2023-11-24T08:03:34.001 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.995+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315400c310 con 0x7f31540047c0 2023-11-24T08:03:34.001 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.996+0000 7f315ffff640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31540047c0 0x7f3154004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.001 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.996+0000 7f3166d9d640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31540051f0 0x7f315400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.001 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.996+0000 7f315ffff640 1 -- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31540051f0 msgr2=0x7f315400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:34.001 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.996+0000 7f315ffff640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31540051f0 0x7f315400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.001 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.996+0000 7f315ffff640 1 -- 172.21.15.91:0/1837173591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31540039c0 msgr2=0x7f3154003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:34.001 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.996+0000 7f315ffff640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31540039c0 0x7f3154003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.996+0000 7f315ffff640 1 -- 172.21.15.91:0/1837173591 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154003520 con 0x7f31540047c0 2023-11-24T08:03:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.996+0000 7f3166d9d640 1 -- 172.21.15.91:0/1837173591 reap_dead start 2023-11-24T08:03:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.997+0000 7f315ffff640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31540047c0 0x7f3154004c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f31680b2a90 tx=0x7f31540057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a249e8c2bc51a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.997+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315002c630 con 0x7f31540047c0 2023-11-24T08:03:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315002c790 con 0x7f31540047c0 2023-11-24T08:03:34.003 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f3165d9b640 1 -- 172.21.15.91:0/1837173591 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315002cb40 con 0x7f31540047c0 2023-11-24T08:03:34.005 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f316ce48640 1 -- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31540047c0 msgr2=0x7f3154004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.005 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f316ce48640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31540047c0 0x7f3154004c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f31680b2a90 tx=0x7f31540057f0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f316ce48640 1 -- 172.21.15.91:0/1837173591 shutdown_connections 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f316ce48640 1 --2- 172.21.15.91:0/1837173591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31540047c0 0x7f3154004c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f316ce48640 1 -- 172.21.15.91:0/1837173591 >> 172.21.15.91:0/1837173591 conn(0x7f316809c5a0 msgr2=0x7f31680d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f316ce48640 1 -- 172.21.15.91:0/1837173591 shutdown_connections 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:33.998+0000 7f316ce48640 1 -- 172.21.15.91:0/1837173591 wait complete. 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.000+0000 7f316ce48640 1 Processor -- start 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.000+0000 7f316ce48640 1 -- start start 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f316ce48640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31540047c0 0x7f3168151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f316ce48640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f316815a940 0x7f31680c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f316ce48640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31680c73e0 0x7f31680c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.006 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f316ce48640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680c6f50 con 0x7f31540047c0 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f316ce48640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680c8c00 con 0x7f316815a940 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f316ce48640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31680c8ee0 con 0x7f31680c73e0 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f3166d9d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31680c73e0 0x7f31680c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f315ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f316815a940 0x7f31680c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f3166d9d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31680c73e0 0x7f31680c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:50514/0 (socket says 172.21.15.91:50514) 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f3166d9d640 1 -- 172.21.15.91:0/1853761290 learned_addr learned my addr 172.21.15.91:0/1853761290 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.001+0000 7f316659c640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31540047c0 0x7f3168151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315ffff640 1 -- 172.21.15.91:0/1853761290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31680c73e0 msgr2=0x7f31680c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315ffff640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31680c73e0 0x7f31680c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315ffff640 1 -- 172.21.15.91:0/1853761290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31540047c0 msgr2=0x7f3168151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.007 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315ffff640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31540047c0 0x7f3168151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315ffff640 1 -- 172.21.15.91:0/1853761290 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680c6be0 con 0x7f316815a940 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f3166d9d640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31680c73e0 0x7f31680c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315ffff640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f316815a940 0x7f31680c8290 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3150033000 tx=0x7f3150004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315effd640 1 -- 172.21.15.91:0/1853761290 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150034a10 con 0x7f316815a940 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315effd640 1 -- 172.21.15.91:0/1853761290 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3150034b90 con 0x7f316815a940 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f315effd640 1 -- 172.21.15.91:0/1853761290 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315003e400 con 0x7f316815a940 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3168151ad0 con 0x7f316815a940 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.002+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31680c87d0 con 0x7f316815a940 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.003+0000 7f315effd640 1 -- 172.21.15.91:0/1853761290 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f315003e860 con 0x7f316815a940 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.003+0000 7f315effd640 1 -- 172.21.15.91:0/1853761290 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f315003d3a0 con 0x7f316815a940 2023-11-24T08:03:34.008 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.004+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31680c70c0 con 0x7f316815a940 2023-11-24T08:03:34.009 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.006+0000 7f315effd640 1 -- 172.21.15.91:0/1853761290 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f315004c060 con 0x7f316815a940 2023-11-24T08:03:34.084 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:34.081+0000 7fe80b03b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T08:03:34.084 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.083+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cb020221-0c2e-4aef-9cea-26ea555234a4", "id": 6} v 0) v1 -- 0x7f31680a1210 con 0x7f316815a940 2023-11-24T08:03:34.091 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.089+0000 7f315effd640 1 -- 172.21.15.91:0/1853761290 <== mon.1 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb020221-0c2e-4aef-9cea-26ea555234a4", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3150042070 con 0x7f316815a940 2023-11-24T08:03:34.091 INFO:teuthology.orchestra.run.smithi091.stdout:6 2023-11-24T08:03:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.090+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f316815a940 msgr2=0x7f31680c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.090+0000 7f316ce48640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f316815a940 0x7f31680c8290 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3150033000 tx=0x7f3150004730 comp rx=0 tx=0).stop 2023-11-24T08:03:34.092 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.090+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 shutdown_connections 2023-11-24T08:03:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.090+0000 7f316ce48640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31680c73e0 0x7f31680c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.090+0000 7f316ce48640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f316815a940 0x7f31680c8290 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.091+0000 7f316ce48640 1 --2- 172.21.15.91:0/1853761290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31540047c0 0x7f3168151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.091+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 >> 172.21.15.91:0/1853761290 conn(0x7f316809c5a0 msgr2=0x7f31680afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.091+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 shutdown_connections 2023-11-24T08:03:34.093 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.091+0000 7f316ce48640 1 -- 172.21.15.91:0/1853761290 wait complete. 2023-11-24T08:03:34.094 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:34.091+0000 7f922b213040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T08:03:34.110 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --cluster ceph osd new 5a7445aa-1172-4b2e-9d8e-7a2dd6a95208 7 2023-11-24T08:03:34.137 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:34.135+0000 7fe80b03b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T08:03:34.145 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:34.143+0000 7f922b213040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T08:03:34.302 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.300+0000 7fdace059640 1 Processor -- start 2023-11-24T08:03:34.302 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.300+0000 7fdace059640 1 -- start start 2023-11-24T08:03:34.303 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:34.301+0000 7fe80b03b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T08:03:34.303 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.302+0000 7fdace059640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80cfcb0 0x7fdac80d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.304 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.302+0000 7fdace059640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80cf190 con 0x7fdac8160ab0 2023-11-24T08:03:34.304 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.302+0000 7fdace059640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80b2e60 con 0x7fdac80ce6f0 2023-11-24T08:03:34.304 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.302+0000 7fdace059640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80cebd0 con 0x7fdac80cfcb0 2023-11-24T08:03:34.304 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.302+0000 7fdacd858640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdac80ce6f0 0x7fdac80b32f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.91:39310/0 (socket says 172.21.15.91:39310) 2023-11-24T08:03:34.304 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.302+0000 7fdacd858640 1 -- 172.21.15.91:0/2476124996 learned_addr learned my addr 172.21.15.91:0/2476124996 (peer_addr_for_me v1:172.21.15.91:0/0) 2023-11-24T08:03:34.304 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.302+0000 7fdacc856640 1 --2- 172.21.15.91:0/2476124996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80cfcb0 0x7fdac80d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.305 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.303+0000 7fdacc856640 1 -- 172.21.15.91:0/2476124996 >> v1:172.21.15.91:6789/0 conn(0x7fdac80ce6f0 legacy=0x7fdac80b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.305 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.303+0000 7fdabffff640 1 -- 172.21.15.91:0/2476124996 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426324908 0 0) 0x7fdac80b2e60 con 0x7fdac80ce6f0 2023-11-24T08:03:34.305 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.303+0000 7fdacc856640 1 -- 172.21.15.91:0/2476124996 >> v1:172.21.15.64:6789/0 conn(0x7fdac8160ab0 legacy=0x7fdac80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.305 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.303+0000 7fdacc856640 1 -- 172.21.15.91:0/2476124996 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80ceeb0 con 0x7fdac80cfcb0 2023-11-24T08:03:34.305 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.304+0000 7fdacc856640 1 --2- 172.21.15.91:0/2476124996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80cfcb0 0x7fdac80d3b80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdab80097d0 tx=0x7fdab802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9db257cbd958f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:34.306 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.304+0000 7fdabffff640 1 -- 172.21.15.91:0/2476124996 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8031830 con 0x7fdac80cfcb0 2023-11-24T08:03:34.306 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.304+0000 7fdabffff640 1 -- 172.21.15.91:0/2476124996 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab8031990 con 0x7fdac80cfcb0 2023-11-24T08:03:34.306 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.304+0000 7fdabffff640 1 -- 172.21.15.91:0/2476124996 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8031d20 con 0x7fdac80cfcb0 2023-11-24T08:03:34.306 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.305+0000 7fdace059640 1 -- 172.21.15.91:0/2476124996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80cfcb0 msgr2=0x7fdac80d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.307 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.305+0000 7fdace059640 1 --2- 172.21.15.91:0/2476124996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80cfcb0 0x7fdac80d3b80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdab80097d0 tx=0x7fdab802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.307 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.305+0000 7fdace059640 1 -- 172.21.15.91:0/2476124996 shutdown_connections 2023-11-24T08:03:34.307 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.305+0000 7fdace059640 1 --2- 172.21.15.91:0/2476124996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80cfcb0 0x7fdac80d3b80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.307 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.305+0000 7fdace059640 1 -- 172.21.15.91:0/2476124996 >> 172.21.15.91:0/2476124996 conn(0x7fdac80d3060 msgr2=0x7fdac80d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:34.307 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.305+0000 7fdace059640 1 -- 172.21.15.91:0/2476124996 shutdown_connections 2023-11-24T08:03:34.307 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.305+0000 7fdace059640 1 -- 172.21.15.91:0/2476124996 wait complete. 2023-11-24T08:03:34.308 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.307+0000 7fdace059640 1 Processor -- start 2023-11-24T08:03:34.308 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.307+0000 7fdace059640 1 -- start start 2023-11-24T08:03:34.310 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdace059640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80ce6f0 0x7fdac814ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.310 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdace059640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac80cfcb0 0x7fdac815e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.310 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdace059640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdac8160ab0 0x7fdac814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdace059640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac814f490 con 0x7fdac80cfcb0 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdace059640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac814c850 con 0x7fdac8160ab0 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdace059640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac814cb30 con 0x7fdac80ce6f0 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdacd057640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80ce6f0 0x7fdac814ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdacd057640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80ce6f0 0x7fdac814ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.91:50540/0 (socket says 172.21.15.91:50540) 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdacd057640 1 -- 172.21.15.91:0/4051323931 learned_addr learned my addr 172.21.15.91:0/4051323931 (peer_addr_for_me v2:172.21.15.91:0/0) 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdacd858640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdac8160ab0 0x7fdac814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.308+0000 7fdacc856640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac80cfcb0 0x7fdac815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdacd858640 1 -- 172.21.15.91:0/4051323931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80ce6f0 msgr2=0x7fdac814ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdacd858640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80ce6f0 0x7fdac814ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdacd858640 1 -- 172.21.15.91:0/4051323931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac80cfcb0 msgr2=0x7fdac815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdacd858640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac80cfcb0 0x7fdac815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.312 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdacd858640 1 -- 172.21.15.91:0/4051323931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8009400 con 0x7fdac8160ab0 2023-11-24T08:03:34.312 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdacd057640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80ce6f0 0x7fdac814ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:03:34.312 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdacd858640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdac8160ab0 0x7fdac814f9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdac0001560 tx=0x7fdac000cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:34.312 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:34.308+0000 7f922b213040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T08:03:34.313 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdabdffb640 1 -- 172.21.15.91:0/4051323931 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac000ce30 con 0x7fdac8160ab0 2023-11-24T08:03:34.313 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdabdffb640 1 -- 172.21.15.91:0/4051323931 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac000a610 con 0x7fdac8160ab0 2023-11-24T08:03:34.313 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac814ce10 con 0x7fdac8160ab0 2023-11-24T08:03:34.313 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdabdffb640 1 -- 172.21.15.91:0/4051323931 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac000a9e0 con 0x7fdac8160ab0 2023-11-24T08:03:34.313 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac814d3d0 con 0x7fdac8160ab0 2023-11-24T08:03:34.313 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.309+0000 7fdabdffb640 1 -- 172.21.15.91:0/4051323931 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdac00070d0 con 0x7fdac8160ab0 2023-11-24T08:03:34.314 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.310+0000 7fdabdffb640 1 -- 172.21.15.91:0/4051323931 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fdac001b360 con 0x7fdac8160ab0 2023-11-24T08:03:34.314 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.311+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac814d5c0 con 0x7fdac8160ab0 2023-11-24T08:03:34.314 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.313+0000 7fdabdffb640 1 -- 172.21.15.91:0/4051323931 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fdac0018070 con 0x7fdac8160ab0 2023-11-24T08:03:34.402 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.400+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5a7445aa-1172-4b2e-9d8e-7a2dd6a95208", "id": 7} v 0) v1 -- 0x7fdac804e840 con 0x7fdac8160ab0 2023-11-24T08:03:34.409 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.407+0000 7fdabdffb640 1 -- 172.21.15.91:0/4051323931 <== mon.1 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a7445aa-1172-4b2e-9d8e-7a2dd6a95208", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdac0007450 con 0x7fdac8160ab0 2023-11-24T08:03:34.409 INFO:teuthology.orchestra.run.smithi091.stdout:7 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.408+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdac8160ab0 msgr2=0x7fdac814f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.408+0000 7fdace059640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdac8160ab0 0x7fdac814f9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdac0001560 tx=0x7fdac000cbe0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.408+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 shutdown_connections 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.408+0000 7fdace059640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdac8160ab0 0x7fdac814f9a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.408+0000 7fdace059640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac80cfcb0 0x7fdac815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.408+0000 7fdace059640 1 --2- 172.21.15.91:0/4051323931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdac80ce6f0 0x7fdac814ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.408+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 >> 172.21.15.91:0/4051323931 conn(0x7fdac80d3060 msgr2=0x7fdac815f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.409+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 shutdown_connections 2023-11-24T08:03:34.411 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:03:34.409+0000 7fdace059640 1 -- 172.21.15.91:0/4051323931 wait complete. 2023-11-24T08:03:34.425 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T08:03:34.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T08:03:34.434 INFO:tasks.ceph.osd.0:Started 2023-11-24T08:03:34.435 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T08:03:34.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T08:03:34.443 INFO:tasks.ceph.osd.1:Started 2023-11-24T08:03:34.444 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T08:03:34.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T08:03:34.449 INFO:tasks.ceph.osd.2:Started 2023-11-24T08:03:34.450 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T08:03:34.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T08:03:34.457 INFO:tasks.ceph.osd.3:Started 2023-11-24T08:03:34.457 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T08:03:34.458 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T08:03:34.474 INFO:tasks.ceph.osd.4:Started 2023-11-24T08:03:34.474 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T08:03:34.474 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T08:03:34.482 INFO:tasks.ceph.osd.5:Started 2023-11-24T08:03:34.482 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T08:03:34.483 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T08:03:34.489 INFO:tasks.ceph.osd.6:Started 2023-11-24T08:03:34.489 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T08:03:34.489 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T08:03:34.495 INFO:tasks.ceph.osd.7:Started 2023-11-24T08:03:34.496 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:03:34.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.720+0000 7f9a9f3db640 1 Processor -- start 2023-11-24T08:03:34.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.720+0000 7f9a9f3db640 1 -- start start 2023-11-24T08:03:34.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9f3db640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980d14a0 0x7f9a980d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9f3db640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9a980b0640 0x7f9a980b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9f3db640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a980cf0c0 con 0x7f9a9815d5b0 2023-11-24T08:03:34.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9f3db640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a980b01b0 con 0x7f9a980b0640 2023-11-24T08:03:34.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9f3db640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9815d990 con 0x7f9a980d14a0 2023-11-24T08:03:34.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9e3d9640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9a9815d5b0 0x7f9a980d3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34320/0 (socket says 172.21.15.64:34320) 2023-11-24T08:03:34.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9e3d9640 1 -- 172.21.15.64:0/1359853980 learned_addr learned my addr 172.21.15.64:0/1359853980 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:34.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9ebda640 1 --2- 172.21.15.64:0/1359853980 >> v2:172.21.15.91:3300/0 conn(0x7f9a980b0640 0x7f9a980b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.721+0000 7f9a9dbd8640 1 --2- 172.21.15.64:0/1359853980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980d14a0 0x7f9a980d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.722+0000 7f9a9ebda640 1 -- 172.21.15.64:0/1359853980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980d14a0 msgr2=0x7f9a980d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:34.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.722+0000 7f9a9ebda640 1 --2- 172.21.15.64:0/1359853980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980d14a0 0x7f9a980d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.722+0000 7f9a9ebda640 1 -- 172.21.15.64:0/1359853980 >> v1:172.21.15.64:6789/0 conn(0x7f9a9815d5b0 legacy=0x7f9a980d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.722+0000 7f9a9d3d7640 1 -- 172.21.15.64:0/1359853980 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680906210 0 0) 0x7f9a980cf0c0 con 0x7f9a9815d5b0 2023-11-24T08:03:34.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.722+0000 7f9a9ebda640 1 -- 172.21.15.64:0/1359853980 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a980d1e40 con 0x7f9a980b0640 2023-11-24T08:03:34.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.723+0000 7f9a9ebda640 1 --2- 172.21.15.64:0/1359853980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980b0640 0x7f9a980b0aa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009d90 tx=0x7f9a9002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dcc3febe1c100e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:34.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9d3d7640 1 -- 172.21.15.64:0/1359853980 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9002cc60 con 0x7f9a980b0640 2023-11-24T08:03:34.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9d3d7640 1 -- 172.21.15.64:0/1359853980 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9002cdc0 con 0x7f9a980b0640 2023-11-24T08:03:34.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9d3d7640 1 -- 172.21.15.64:0/1359853980 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90034660 con 0x7f9a980b0640 2023-11-24T08:03:34.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9f3db640 1 -- 172.21.15.64:0/1359853980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980b0640 msgr2=0x7f9a980b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9f3db640 1 --2- 172.21.15.64:0/1359853980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980b0640 0x7f9a980b0aa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009d90 tx=0x7f9a9002beb0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9f3db640 1 -- 172.21.15.64:0/1359853980 shutdown_connections 2023-11-24T08:03:34.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9f3db640 1 --2- 172.21.15.64:0/1359853980 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980b0640 0x7f9a980b0aa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9f3db640 1 --2- 172.21.15.64:0/1359853980 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980d14a0 0x7f9a980d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9f3db640 1 -- 172.21.15.64:0/1359853980 >> 172.21.15.64:0/1359853980 conn(0x7f9a9809c3b0 msgr2=0x7f9a980d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:34.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9f3db640 1 -- 172.21.15.64:0/1359853980 shutdown_connections 2023-11-24T08:03:34.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.724+0000 7f9a9f3db640 1 -- 172.21.15.64:0/1359853980 wait complete. 2023-11-24T08:03:34.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.726+0000 7f9a9f3db640 1 Processor -- start 2023-11-24T08:03:34.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.726+0000 7f9a9f3db640 1 -- start start 2023-11-24T08:03:34.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9f3db640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980b0640 0x7f9a980ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9f3db640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980d14a0 0x7f9a981f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9f3db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9815d5b0 0x7f9a981f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9f3db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a981f6910 con 0x7f9a9815d5b0 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9f3db640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a981f37d0 con 0x7f9a980d14a0 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9f3db640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a981f3ab0 con 0x7f9a980b0640 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9ebda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9815d5b0 0x7f9a981f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9dbd8640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980d14a0 0x7f9a981f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9ebda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9815d5b0 0x7f9a981f2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45990/0 (socket says 172.21.15.64:45990) 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9ebda640 1 -- 172.21.15.64:0/252595105 learned_addr learned my addr 172.21.15.64:0/252595105 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:34.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.727+0000 7f9a9e3d9640 1 --2- 172.21.15.64:0/252595105 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980b0640 0x7f9a980ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a9dbd8640 1 -- 172.21.15.64:0/252595105 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980b0640 msgr2=0x7f9a980ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a9dbd8640 1 --2- 172.21.15.64:0/252595105 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980b0640 0x7f9a980ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a9dbd8640 1 -- 172.21.15.64:0/252595105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9815d5b0 msgr2=0x7f9a981f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a9dbd8640 1 --2- 172.21.15.64:0/252595105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9815d5b0 0x7f9a981f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a9dbd8640 1 -- 172.21.15.64:0/252595105 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a90009a70 con 0x7f9a980d14a0 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a9dbd8640 1 --2- 172.21.15.64:0/252595105 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980d14a0 0x7f9a981f6da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e870 tx=0x7f9a8800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a8effd640 1 -- 172.21.15.64:0/252595105 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800ee60 con 0x7f9a980d14a0 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a8effd640 1 -- 172.21.15.64:0/252595105 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8800c940 con 0x7f9a980d14a0 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a8effd640 1 -- 172.21.15.64:0/252595105 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800ccd0 con 0x7f9a980d14a0 2023-11-24T08:03:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.728+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a981f65a0 con 0x7f9a980d14a0 2023-11-24T08:03:34.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.729+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a981f72e0 con 0x7f9a980d14a0 2023-11-24T08:03:34.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.729+0000 7f9a8effd640 1 -- 172.21.15.64:0/252595105 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9a88005c90 con 0x7f9a980d14a0 2023-11-24T08:03:34.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.729+0000 7f9a8effd640 1 -- 172.21.15.64:0/252595105 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9a8800ce30 con 0x7f9a980d14a0 2023-11-24T08:03:34.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.730+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9804e840 con 0x7f9a980d14a0 2023-11-24T08:03:34.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.732+0000 7f9a8effd640 1 -- 172.21.15.64:0/252595105 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9a88021060 con 0x7f9a980d14a0 2023-11-24T08:03:34.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.797+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a980678d0 con 0x7f9a980d14a0 2023-11-24T08:03:34.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.798+0000 7f9a8effd640 1 -- 172.21.15.64:0/252595105 <== mon.1 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f9a88029020 con 0x7f9a980d14a0 2023-11-24T08:03:34.801 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:03:34.801 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:34.403042+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:03:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980d14a0 msgr2=0x7f9a981f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 --2- 172.21.15.64:0/252595105 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980d14a0 0x7f9a981f6da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e870 tx=0x7f9a8800ed30 comp rx=0 tx=0).stop 2023-11-24T08:03:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 shutdown_connections 2023-11-24T08:03:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 --2- 172.21.15.64:0/252595105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9815d5b0 0x7f9a981f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 --2- 172.21.15.64:0/252595105 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a980d14a0 0x7f9a981f6da0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 --2- 172.21.15.64:0/252595105 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a980b0640 0x7f9a980ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 >> 172.21.15.64:0/252595105 conn(0x7f9a9809c3b0 msgr2=0x7f9a98160400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 shutdown_connections 2023-11-24T08:03:34.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:34.800+0000 7f9a9f3db640 1 -- 172.21.15.64:0/252595105 wait complete. 2023-11-24T08:03:34.819 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T08:03:34.819 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T08:03:35.143 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-11-24T08:03:35.141+0000 7f3978915500 -1 Falling back to public interface 2023-11-24T08:03:35.161 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-11-24T08:03:35.160+0000 7efd40c1d500 -1 Falling back to public interface 2023-11-24T08:03:35.173 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-11-24T08:03:35.171+0000 7f9f156cf500 -1 Falling back to public interface 2023-11-24T08:03:35.179 INFO:tasks.ceph.osd.6.smithi091.stderr:2023-11-24T08:03:35.177+0000 7f4ded115500 -1 Falling back to public interface 2023-11-24T08:03:35.187 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-11-24T08:03:35.185+0000 7f9c49da9500 -1 Falling back to public interface 2023-11-24T08:03:35.191 INFO:tasks.ceph.osd.7.smithi091.stderr:2023-11-24T08:03:35.189+0000 7f3d8dca5500 -1 Falling back to public interface 2023-11-24T08:03:35.204 INFO:tasks.ceph.osd.5.smithi091.stderr:2023-11-24T08:03:35.202+0000 7f2fc5e1f500 -1 Falling back to public interface 2023-11-24T08:03:35.208 INFO:tasks.ceph.osd.4.smithi091.stderr:2023-11-24T08:03:35.206+0000 7f28013e2500 -1 Falling back to public interface 2023-11-24T08:03:35.476 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:35.475+0000 7fe80b03b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T08:03:35.491 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:35.489+0000 7f922b213040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T08:03:35.625 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T08:03:35.796 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.795+0000 7fc080ee7640 1 Processor -- start 2023-11-24T08:03:35.797 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.795+0000 7fc080ee7640 1 -- start start 2023-11-24T08:03:35.799 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.797+0000 7fc080ee7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc07c15d570 0x7fc07c161b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:35.799 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.797+0000 7fc080ee7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07c15d950 con 0x7fc07c0d1830 2023-11-24T08:03:35.799 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.797+0000 7fc080ee7640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07c0d03c0 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.799 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.797+0000 7fc080ee7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c0d06a0 con 0x7fc07c15d570 2023-11-24T08:03:35.799 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.797+0000 7fc07a59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc07c15d570 0x7fc07c161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:35.799 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.797+0000 7fc07ad9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc07c0d1830 0x7fc07c0d1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34324/0 (socket says 172.21.15.64:34324) 2023-11-24T08:03:35.799 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.797+0000 7fc07ad9d640 1 -- 172.21.15.64:0/3885281807 learned_addr learned my addr 172.21.15.64:0/3885281807 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:35.800 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.798+0000 7fc07a59c640 1 -- 172.21.15.64:0/3885281807 >> v1:172.21.15.91:6789/0 conn(0x7fc07c0cfbe0 legacy=0x7fc07c0cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:35.800 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.798+0000 7fc07a59c640 1 -- 172.21.15.64:0/3885281807 >> v1:172.21.15.64:6789/0 conn(0x7fc07c0d1830 legacy=0x7fc07c0d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:35.800 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.798+0000 7fc07a59c640 1 -- 172.21.15.64:0/3885281807 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0d0980 con 0x7fc07c15d570 2023-11-24T08:03:35.801 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.799+0000 7fc07a59c640 1 --2- 172.21.15.64:0/3885281807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c15d570 0x7fc07c161b80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0097a0 tx=0x7fc06c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131b0a9bdc3f3a58 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:35.801 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.799+0000 7fc079d9b640 1 -- 172.21.15.64:0/3885281807 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c031830 con 0x7fc07c15d570 2023-11-24T08:03:35.801 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.799+0000 7fc079d9b640 1 -- 172.21.15.64:0/3885281807 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c031990 con 0x7fc07c15d570 2023-11-24T08:03:35.801 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.799+0000 7fc079d9b640 1 -- 172.21.15.64:0/3885281807 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c031d20 con 0x7fc07c15d570 2023-11-24T08:03:35.802 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.800+0000 7fc080ee7640 1 -- 172.21.15.64:0/3885281807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c15d570 msgr2=0x7fc07c161b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:35.802 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.800+0000 7fc080ee7640 1 --2- 172.21.15.64:0/3885281807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c15d570 0x7fc07c161b80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0097a0 tx=0x7fc06c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:03:35.802 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.800+0000 7fc080ee7640 1 -- 172.21.15.64:0/3885281807 shutdown_connections 2023-11-24T08:03:35.802 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.800+0000 7fc080ee7640 1 --2- 172.21.15.64:0/3885281807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c15d570 0x7fc07c161b80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:35.802 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.800+0000 7fc080ee7640 1 -- 172.21.15.64:0/3885281807 >> 172.21.15.64:0/3885281807 conn(0x7fc07c08cc30 msgr2=0x7fc07c09c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:35.802 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.800+0000 7fc080ee7640 1 -- 172.21.15.64:0/3885281807 shutdown_connections 2023-11-24T08:03:35.802 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.800+0000 7fc080ee7640 1 -- 172.21.15.64:0/3885281807 wait complete. 2023-11-24T08:03:35.804 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.803+0000 7fc080ee7640 1 Processor -- start 2023-11-24T08:03:35.804 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.803+0000 7fc080ee7640 1 -- start start 2023-11-24T08:03:35.807 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.804+0000 7fc080ee7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c0cfbe0 0x7fc07c0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:35.807 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.804+0000 7fc080ee7640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c0d1830 0x7fc07c14e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:35.807 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.804+0000 7fc080ee7640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc07c15d570 0x7fc07c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:35.807 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc080ee7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0cb8c0 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.807 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc080ee7640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0cbba0 con 0x7fc07c15d570 2023-11-24T08:03:35.807 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc080ee7640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0cbe80 con 0x7fc07c0d1830 2023-11-24T08:03:35.808 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c0cfbe0 0x7fc07c0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:35.808 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c0cfbe0 0x7fc07c0c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46012/0 (socket says 172.21.15.64:46012) 2023-11-24T08:03:35.808 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 -- 172.21.15.64:0/1233331246 learned_addr learned my addr 172.21.15.64:0/1233331246 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:35.808 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc073fff640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c0d1830 0x7fc07c14e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:35.808 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07ad9d640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc07c15d570 0x7fc07c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:35.808 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 -- 172.21.15.64:0/1233331246 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c0d1830 msgr2=0x7fc07c14e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:35.809 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c0d1830 0x7fc07c14e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:35.809 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 -- 172.21.15.64:0/1233331246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc07c15d570 msgr2=0x7fc07c15db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:35.809 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc07c15d570 0x7fc07c15db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:35.809 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 -- 172.21.15.64:0/1233331246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c009400 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.809 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.805+0000 7fc07a59c640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c0cfbe0 0x7fc07c0c80b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc06c034040 tx=0x7fc06c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:35.810 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.806+0000 7fc072ffd640 1 -- 172.21.15.64:0/1233331246 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c03aa60 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.810 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.806+0000 7fc072ffd640 1 -- 172.21.15.64:0/1233331246 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c03abe0 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.810 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.806+0000 7fc072ffd640 1 -- 172.21.15.64:0/1233331246 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c044400 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.810 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.806+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c0cc160 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.810 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.806+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c14ebf0 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.811 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.806+0000 7fc072ffd640 1 -- 172.21.15.64:0/1233331246 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc06c044860 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.811 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.806+0000 7fc072ffd640 1 -- 172.21.15.64:0/1233331246 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc06c04d440 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.811 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.807+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c067960 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.811 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.810+0000 7fc072ffd640 1 -- 172.21.15.64:0/1233331246 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc06c035020 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.812 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:35.810+0000 7fe80b03b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T08:03:35.817 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:35.815+0000 7f922b213040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T08:03:35.873 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:35.871+0000 7fe80b03b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T08:03:35.878 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.877+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc07c04e840 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.879 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.877+0000 7fc072ffd640 1 -- 172.21.15.64:0/1233331246 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc06c04d8f0 con 0x7fc07c0cfbe0 2023-11-24T08:03:35.879 INFO:teuthology.misc.health.smithi064.stdout: 2023-11-24T08:03:35.880 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":10,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:34.403042+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:03:35.880 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:35.878+0000 7f922b213040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.878+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c0cfbe0 msgr2=0x7fc07c0c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.878+0000 7fc080ee7640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c0cfbe0 0x7fc07c0c80b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc06c034040 tx=0x7fc06c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.879+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 shutdown_connections 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.879+0000 7fc080ee7640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc07c15d570 0x7fc07c15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.879+0000 7fc080ee7640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c0d1830 0x7fc07c14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.879+0000 7fc080ee7640 1 --2- 172.21.15.64:0/1233331246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c0cfbe0 0x7fc07c0c80b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.879+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 >> 172.21.15.64:0/1233331246 conn(0x7fc07c08cc30 msgr2=0x7fc07c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.879+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 shutdown_connections 2023-11-24T08:03:35.881 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:35.879+0000 7fc080ee7640 1 -- 172.21.15.64:0/1233331246 wait complete. 2023-11-24T08:03:35.893 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T08:03:35.988 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:35.986+0000 7fe80b03b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T08:03:36.002 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-11-24T08:03:36.001+0000 7f922b213040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T08:03:37.032 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:37.031+0000 7fe7ac411640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T08:03:37.032 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:03:37.031+0000 7fe7ac411640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T08:03:38.735 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-11-24T08:03:38.733+0000 7efd40c1d500 -1 osd.1 0 log_to_monitors true 2023-11-24T08:03:38.751 INFO:tasks.ceph.osd.7.smithi091.stderr:2023-11-24T08:03:38.750+0000 7f3d8dca5500 -1 osd.7 0 log_to_monitors true 2023-11-24T08:03:38.756 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-11-24T08:03:38.754+0000 7f9f156cf500 -1 osd.2 0 log_to_monitors true 2023-11-24T08:03:38.766 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-11-24T08:03:38.765+0000 7f9c49da9500 -1 osd.3 0 log_to_monitors true 2023-11-24T08:03:38.782 INFO:tasks.ceph.osd.5.smithi091.stderr:2023-11-24T08:03:38.781+0000 7f2fc5e1f500 -1 osd.5 0 log_to_monitors true 2023-11-24T08:03:38.806 INFO:tasks.ceph.osd.4.smithi091.stderr:2023-11-24T08:03:38.805+0000 7f28013e2500 -1 osd.4 0 log_to_monitors true 2023-11-24T08:03:38.938 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-11-24T08:03:38.936+0000 7f3978915500 -1 osd.0 0 log_to_monitors true 2023-11-24T08:03:38.947 INFO:tasks.ceph.osd.6.smithi091.stderr:2023-11-24T08:03:38.945+0000 7f4ded115500 -1 osd.6 0 log_to_monitors true 2023-11-24T08:03:42.461 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-11-24T08:03:42.459+0000 7f9c45514640 -1 osd.3 0 waiting for initial osdmap 2023-11-24T08:03:42.466 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-11-24T08:03:42.465+0000 7f3974080640 -1 osd.0 0 waiting for initial osdmap 2023-11-24T08:03:42.467 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-11-24T08:03:42.465+0000 7f9f12672640 -1 osd.2 0 waiting for initial osdmap 2023-11-24T08:03:42.473 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-11-24T08:03:42.472+0000 7efd3bd3e640 -1 osd.1 0 waiting for initial osdmap 2023-11-24T08:03:42.475 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-11-24T08:03:42.473+0000 7f9c3db4d640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:03:42.478 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-11-24T08:03:42.477+0000 7f396c6b9640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:03:42.483 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-11-24T08:03:42.481+0000 7f9f09473640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:03:42.483 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-11-24T08:03:42.481+0000 7efd34377640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:03:42.505 INFO:tasks.ceph.osd.6.smithi091.stderr:2023-11-24T08:03:42.503+0000 7f4dea0b8640 -1 osd.6 0 waiting for initial osdmap 2023-11-24T08:03:42.512 INFO:tasks.ceph.osd.6.smithi091.stderr:2023-11-24T08:03:42.511+0000 7f4de0eb9640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:03:42.543 INFO:tasks.ceph.osd.7.smithi091.stderr:2023-11-24T08:03:42.541+0000 7f3d8ac48640 -1 osd.7 0 waiting for initial osdmap 2023-11-24T08:03:42.557 INFO:tasks.ceph.osd.7.smithi091.stderr:2023-11-24T08:03:42.556+0000 7f3d81a49640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:03:42.573 INFO:tasks.ceph.osd.4.smithi091.stderr:2023-11-24T08:03:42.571+0000 7f27fcb4f640 -1 osd.4 0 waiting for initial osdmap 2023-11-24T08:03:42.591 INFO:tasks.ceph.osd.4.smithi091.stderr:2023-11-24T08:03:42.589+0000 7f27f5188640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:03:42.617 INFO:tasks.ceph.osd.5.smithi091.stderr:2023-11-24T08:03:42.615+0000 7f2fc0eca640 -1 osd.5 0 waiting for initial osdmap 2023-11-24T08:03:42.625 INFO:tasks.ceph.osd.5.smithi091.stderr:2023-11-24T08:03:42.624+0000 7f2fb9503640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:03:42.699 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T08:03:42.894 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.892+0000 7f70c52ea640 1 Processor -- start 2023-11-24T08:03:42.894 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.892+0000 7f70c52ea640 1 -- start start 2023-11-24T08:03:42.896 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.894+0000 7f70c52ea640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70c015b010 0x7f70c00abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:42.896 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.894+0000 7f70c52ea640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f70c006a860 0x7f70c006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:42.896 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.894+0000 7f70c52ea640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c00ae6c0 con 0x7f70c00ad890 2023-11-24T08:03:42.896 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.894+0000 7f70c52ea640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c006c670 con 0x7f70c015b010 2023-11-24T08:03:42.896 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.894+0000 7f70c52ea640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c006d1a0 con 0x7f70c006a860 2023-11-24T08:03:42.896 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.894+0000 7f70bffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70c015b010 0x7f70c00abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:42.896 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.894+0000 7f70bf7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f70c00ad890 0x7f70c006ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57042/0 (socket says 172.21.15.64:57042) 2023-11-24T08:03:42.896 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.894+0000 7f70bf7fe640 1 -- 172.21.15.64:0/835378009 learned_addr learned my addr 172.21.15.64:0/835378009 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:42.897 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.895+0000 7f70c4ae9640 1 --2- 172.21.15.64:0/835378009 >> v2:172.21.15.91:3300/0 conn(0x7f70c006a860 0x7f70c006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:42.897 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.895+0000 7f70beffd640 1 -- 172.21.15.64:0/835378009 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429143928 0 0) 0x7f70c00ae6c0 con 0x7f70c00ad890 2023-11-24T08:03:42.897 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.895+0000 7f70beffd640 1 -- 172.21.15.64:0/835378009 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a8003520 con 0x7f70c00ad890 2023-11-24T08:03:42.897 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.895+0000 7f70bffff640 1 -- 172.21.15.64:0/835378009 >> v2:172.21.15.91:3300/0 conn(0x7f70c006a860 msgr2=0x7f70c006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:42.897 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.895+0000 7f70bffff640 1 --2- 172.21.15.64:0/835378009 >> v2:172.21.15.91:3300/0 conn(0x7f70c006a860 0x7f70c006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:42.897 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.895+0000 7f70bffff640 1 -- 172.21.15.64:0/835378009 >> v1:172.21.15.64:6789/0 conn(0x7f70c00ad890 legacy=0x7f70c006ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:42.897 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.895+0000 7f70bffff640 1 -- 172.21.15.64:0/835378009 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c006b1d0 con 0x7f70c015b010 2023-11-24T08:03:42.897 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.895+0000 7f70c4ae9640 1 --2- 172.21.15.64:0/835378009 >> v2:172.21.15.91:3300/0 conn(0x7f70c006a860 0x7f70c006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:42.898 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.896+0000 7f70bffff640 1 --2- 172.21.15.64:0/835378009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c015b010 0x7f70c00abf50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f70c015fc80 tx=0x7f70b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4453b68096c2a2a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.896+0000 7f70beffd640 1 -- 172.21.15.64:0/835378009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0031830 con 0x7f70c015b010 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.896+0000 7f70beffd640 1 -- 172.21.15.64:0/835378009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b0031990 con 0x7f70c015b010 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.896+0000 7f70beffd640 1 -- 172.21.15.64:0/835378009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0031d20 con 0x7f70c015b010 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.897+0000 7f70c52ea640 1 -- 172.21.15.64:0/835378009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c015b010 msgr2=0x7f70c00abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.897+0000 7f70c52ea640 1 --2- 172.21.15.64:0/835378009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c015b010 0x7f70c00abf50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f70c015fc80 tx=0x7f70b002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.897+0000 7f70c52ea640 1 -- 172.21.15.64:0/835378009 shutdown_connections 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.897+0000 7f70c52ea640 1 --2- 172.21.15.64:0/835378009 >> v2:172.21.15.91:3300/0 conn(0x7f70c006a860 0x7f70c006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.897+0000 7f70c52ea640 1 --2- 172.21.15.64:0/835378009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c015b010 0x7f70c00abf50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.897+0000 7f70c52ea640 1 -- 172.21.15.64:0/835378009 >> 172.21.15.64:0/835378009 conn(0x7f70c00b0d00 msgr2=0x7f70c00b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.897+0000 7f70c52ea640 1 -- 172.21.15.64:0/835378009 shutdown_connections 2023-11-24T08:03:42.899 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.897+0000 7f70c52ea640 1 -- 172.21.15.64:0/835378009 wait complete. 2023-11-24T08:03:42.901 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.899+0000 7f70c52ea640 1 Processor -- start 2023-11-24T08:03:42.901 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.899+0000 7f70c52ea640 1 -- start start 2023-11-24T08:03:42.902 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70c52ea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70c006a860 0x7f70c015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:42.902 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70c52ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c00ad890 0x7f70c01527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:42.902 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70c52ea640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70c015b010 0x7f70c015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:42.902 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70c52ea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c015e550 con 0x7f70c00ad890 2023-11-24T08:03:42.902 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70c52ea640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c00cc3c0 con 0x7f70c015b010 2023-11-24T08:03:42.902 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70c52ea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c0152cf0 con 0x7f70c006a860 2023-11-24T08:03:42.902 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c00ad890 0x7f70c01527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:42.903 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c00ad890 0x7f70c01527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40936/0 (socket says 172.21.15.64:40936) 2023-11-24T08:03:42.903 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70bf7fe640 1 -- 172.21.15.64:0/2290540113 learned_addr learned my addr 172.21.15.64:0/2290540113 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70bffff640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70c006a860 0x7f70c015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.900+0000 7f70c4ae9640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70c015b010 0x7f70c015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bf7fe640 1 -- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70c006a860 msgr2=0x7f70c015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70c006a860 0x7f70c015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bf7fe640 1 -- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70c015b010 msgr2=0x7f70c015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70c015b010 0x7f70c015b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bf7fe640 1 -- 172.21.15.64:0/2290540113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0009400 con 0x7f70c00ad890 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c00ad890 0x7f70c01527b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c500 tx=0x7f70b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bcff9640 1 -- 172.21.15.64:0/2290540113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4005ce0 con 0x7f70c00ad890 2023-11-24T08:03:42.904 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bcff9640 1 -- 172.21.15.64:0/2290540113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b4005e40 con 0x7f70c00ad890 2023-11-24T08:03:42.905 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70bcff9640 1 -- 172.21.15.64:0/2290540113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4007330 con 0x7f70c00ad890 2023-11-24T08:03:42.905 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c015e270 con 0x7f70c00ad890 2023-11-24T08:03:42.905 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.901+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c00cc050 con 0x7f70c00ad890 2023-11-24T08:03:42.905 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.902+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c00cbbd0 con 0x7f70c00ad890 2023-11-24T08:03:42.905 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.903+0000 7f70bcff9640 1 -- 172.21.15.64:0/2290540113 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f70b4007490 con 0x7f70c00ad890 2023-11-24T08:03:42.906 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.905+0000 7f70bcff9640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70ac06adb0 0x7f70ac06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:42.906 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.905+0000 7f70bcff9640 1 -- 172.21.15.64:0/2290540113 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f70b4087df0 con 0x7f70c00ad890 2023-11-24T08:03:42.907 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.905+0000 7f70bffff640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70ac06adb0 0x7f70ac06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:42.908 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.906+0000 7f70bffff640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70ac06adb0 0x7f70ac06d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f70b0036000 tx=0x7f70b003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:42.911 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:42.910+0000 7f70bcff9640 1 -- 172.21.15.64:0/2290540113 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70c00cbbd0 con 0x7f70c00ad890 2023-11-24T08:03:43.102 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.100+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70c00cbbd0 con 0x7f70c00ad890 2023-11-24T08:03:43.103 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.101+0000 7f70bcff9640 1 -- 172.21.15.64:0/2290540113 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f70c00cbbd0 con 0x7f70c00ad890 2023-11-24T08:03:43.103 INFO:teuthology.misc.health.smithi064.stdout: 2023-11-24T08:03:43.103 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":12,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:40.021974+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:03:43.106 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70ac06adb0 msgr2=0x7f70ac06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:43.106 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70ac06adb0 0x7f70ac06d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f70b0036000 tx=0x7f70b003c040 comp rx=0 tx=0).stop 2023-11-24T08:03:43.106 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c00ad890 msgr2=0x7f70c01527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:43.106 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c00ad890 0x7f70c01527b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c500 tx=0x7f70b400c9c0 comp rx=0 tx=0).stop 2023-11-24T08:03:43.106 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 shutdown_connections 2023-11-24T08:03:43.106 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70ac06adb0 0x7f70ac06d270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:43.106 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70c015b010 0x7f70c015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:43.107 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c00ad890 0x7f70c01527b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:43.107 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 --2- 172.21.15.64:0/2290540113 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70c006a860 0x7f70c015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:43.107 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 >> 172.21.15.64:0/2290540113 conn(0x7f70c00b0d00 msgr2=0x7f70c0158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:43.107 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 shutdown_connections 2023-11-24T08:03:43.107 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:43.104+0000 7f70c52ea640 1 -- 172.21.15.64:0/2290540113 wait complete. 2023-11-24T08:03:43.120 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T08:03:44.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:03:44.326 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:03:49.927 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T08:03:50.126 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.124+0000 7fc233875640 1 Processor -- start 2023-11-24T08:03:50.126 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.124+0000 7fc233875640 1 -- start start 2023-11-24T08:03:50.128 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc233875640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc22c0ca910 0x7fc22c0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.128 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc233875640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc22c0c96f0 0x7fc22c0ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.128 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc233875640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c15a520 con 0x7fc22c15a140 2023-11-24T08:03:50.128 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc233875640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c0cca60 con 0x7fc22c0ca910 2023-11-24T08:03:50.128 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc233875640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c071a80 con 0x7fc22c0c96f0 2023-11-24T08:03:50.128 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc231611640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc22c15a140 0x7fc22c0ca200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58650/0 (socket says 172.21.15.64:58650) 2023-11-24T08:03:50.128 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc231611640 1 -- 172.21.15.64:0/229261031 learned_addr learned my addr 172.21.15.64:0/229261031 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:50.130 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc230e10640 1 --2- 172.21.15.64:0/229261031 >> v2:172.21.15.91:3300/0 conn(0x7fc22c0ca910 0x7fc22c0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.130 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc231e12640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc22c0c96f0 0x7fc22c0ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.130 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.126+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146856924 0 0) 0x7fc22c15a520 con 0x7fc22c15a140 2023-11-24T08:03:50.130 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc214003520 con 0x7fc22c15a140 2023-11-24T08:03:50.130 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1860879597 0 0) 0x7fc214003520 con 0x7fc22c15a140 2023-11-24T08:03:50.130 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc22c0c96f0 msgr2=0x7fc22c0ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:50.130 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc22c0c96f0 0x7fc22c0ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 >> v2:172.21.15.91:3300/0 conn(0x7fc22c0ca910 msgr2=0x7fc22c0c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 --2- 172.21.15.64:0/229261031 >> v2:172.21.15.91:3300/0 conn(0x7fc22c0ca910 0x7fc22c0c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c071d60 con 0x7fc22c15a140 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fc21c001a10 con 0x7fc22c15a140 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 >> v1:172.21.15.64:6789/0 conn(0x7fc22c15a140 legacy=0x7fc22c0ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc214003970 0x7fc214003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc22c0c96f0 0x7fc214005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.131 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0cca60 con 0x7fc2140047c0 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c071a80 con 0x7fc214003970 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21400b980 con 0x7fc22c0c96f0 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.127+0000 7fc230e10640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.128+0000 7fc231611640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc214003970 0x7fc214003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.128+0000 7fc230e10640 1 -- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc22c0c96f0 msgr2=0x7fc214005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.128+0000 7fc230e10640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc22c0c96f0 0x7fc214005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.128+0000 7fc230e10640 1 -- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc214003970 msgr2=0x7fc214003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.128+0000 7fc230e10640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc214003970 0x7fc214003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.128+0000 7fc230e10640 1 -- 172.21.15.64:0/229261031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214003520 con 0x7fc2140047c0 2023-11-24T08:03:50.132 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.128+0000 7fc231611640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc214003970 0x7fc214003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:50.134 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.129+0000 7fc230e10640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc220009650 tx=0x7fc2140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d45f762e3d2e3f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:50.134 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.129+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22002e8c0 con 0x7fc2140047c0 2023-11-24T08:03:50.134 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.129+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22002ea20 con 0x7fc2140047c0 2023-11-24T08:03:50.134 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.129+0000 7fc22bfff640 1 -- 172.21.15.64:0/229261031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22002edb0 con 0x7fc2140047c0 2023-11-24T08:03:50.134 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.130+0000 7fc233875640 1 -- 172.21.15.64:0/229261031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2140047c0 msgr2=0x7fc214004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.134 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.130+0000 7fc233875640 1 --2- 172.21.15.64:0/229261031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc220009650 tx=0x7fc2140057f0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.134 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.130+0000 7fc231e12640 1 -- 172.21.15.64:0/229261031 reap_dead start 2023-11-24T08:03:50.135 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.130+0000 7fc233875640 1 -- 172.21.15.64:0/229261031 shutdown_connections 2023-11-24T08:03:50.135 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.130+0000 7fc233875640 1 -- 172.21.15.64:0/229261031 >> 172.21.15.64:0/229261031 conn(0x7fc22c0cee90 msgr2=0x7fc22c0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:50.135 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.130+0000 7fc233875640 1 -- 172.21.15.64:0/229261031 shutdown_connections 2023-11-24T08:03:50.135 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.130+0000 7fc233875640 1 -- 172.21.15.64:0/229261031 wait complete. 2023-11-24T08:03:50.135 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.132+0000 7fc233875640 1 Processor -- start 2023-11-24T08:03:50.135 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.132+0000 7fc233875640 1 -- start start 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.133+0000 7fc233875640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2140047c0 0x7fc22c15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.133+0000 7fc233875640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc22c141950 0x7fc22c15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.133+0000 7fc233875640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c157670 0x7fc22c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.133+0000 7fc233875640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c15c0e0 con 0x7fc22c157670 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.133+0000 7fc233875640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c141dd0 con 0x7fc22c141950 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.133+0000 7fc233875640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c15ab00 con 0x7fc2140047c0 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c157670 0x7fc22c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c157670 0x7fc22c15cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50994/0 (socket says 172.21.15.64:50994) 2023-11-24T08:03:50.137 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 -- 172.21.15.64:0/2947746253 learned_addr learned my addr 172.21.15.64:0/2947746253 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231611640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2140047c0 0x7fc22c15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc230e10640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc22c141950 0x7fc22c15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 -- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2140047c0 msgr2=0x7fc22c15a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2140047c0 0x7fc22c15a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 -- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc22c141950 msgr2=0x7fc22c15bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc22c141950 0x7fc22c15bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 -- 172.21.15.64:0/2947746253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c15ade0 con 0x7fc22c157670 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.134+0000 7fc231e12640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c157670 0x7fc22c15cc80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0b1b80 tx=0x7fc22400e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.135+0000 7fc229ffb640 1 -- 172.21.15.64:0/2947746253 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224013680 con 0x7fc22c157670 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.135+0000 7fc229ffb640 1 -- 172.21.15.64:0/2947746253 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2240137e0 con 0x7fc22c157670 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.135+0000 7fc229ffb640 1 -- 172.21.15.64:0/2947746253 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224013b90 con 0x7fc22c157670 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.135+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c15b0c0 con 0x7fc22c157670 2023-11-24T08:03:50.138 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.135+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c15b680 con 0x7fc22c157670 2023-11-24T08:03:50.139 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.136+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c15b8a0 con 0x7fc22c157670 2023-11-24T08:03:50.139 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.137+0000 7fc229ffb640 1 -- 172.21.15.64:0/2947746253 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc224013cf0 con 0x7fc22c157670 2023-11-24T08:03:50.139 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.138+0000 7fc229ffb640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc21806ae40 0x7fc21806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.140 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.138+0000 7fc229ffb640 1 -- 172.21.15.64:0/2947746253 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc224088f60 con 0x7fc22c157670 2023-11-24T08:03:50.140 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.138+0000 7fc231611640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc21806ae40 0x7fc21806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.141 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.140+0000 7fc231611640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc21806ae40 0x7fc21806d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc21c001c80 tx=0x7fc21c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:50.146 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.143+0000 7fc229ffb640 1 -- 172.21.15.64:0/2947746253 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc224054350 con 0x7fc22c157670 2023-11-24T08:03:50.340 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.338+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc22c04e840 con 0x7fc22c157670 2023-11-24T08:03:50.341 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.339+0000 7fc229ffb640 1 -- 172.21.15.64:0/2947746253 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7fc224057970 con 0x7fc22c157670 2023-11-24T08:03:50.341 INFO:teuthology.misc.health.smithi064.stdout: 2023-11-24T08:03:50.341 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":16,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:46.510360+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc21806ae40 msgr2=0x7fc21806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc21806ae40 0x7fc21806d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc21c001c80 tx=0x7fc21c009040 comp rx=0 tx=0).stop 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c157670 msgr2=0x7fc22c15cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c157670 0x7fc22c15cc80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0b1b80 tx=0x7fc22400e9f0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 shutdown_connections 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc21806ae40 0x7fc21806d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c157670 0x7fc22c15cc80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc22c141950 0x7fc22c15bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 --2- 172.21.15.64:0/2947746253 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2140047c0 0x7fc22c15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 >> 172.21.15.64:0/2947746253 conn(0x7fc22c0cee90 msgr2=0x7fc22c0b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:50.344 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 shutdown_connections 2023-11-24T08:03:50.345 INFO:teuthology.misc.health.smithi064.stderr:2023-11-24T08:03:50.342+0000 7fc233875640 1 -- 172.21.15.64:0/2947746253 wait complete. 2023-11-24T08:03:50.359 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T08:03:50.359 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T08:03:50.359 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:03:50.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:03:50.364 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:03:50.365 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:03:50.365 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:03:50.369 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:03:50.370 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:03:50.370 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:03:50.378 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:03:50.379 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:03:50.379 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:03:50.388 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:03:50.388 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T08:03:50.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:03:50.522 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T08:03:50.526 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:03:50.545 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2023-11-24T08:03:50.545 INFO:tasks.ceph.mds.d.smithi091.stdout:starting mds.d at 2023-11-24T08:03:50.548 INFO:tasks.ceph.mds.b.smithi091.stderr:2023-11-24T08:03:50.545+0000 7fb6b342f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-24T08:03:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.574+0000 7f2a596bb640 1 Processor -- start 2023-11-24T08:03:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.574+0000 7f2a596bb640 1 -- start start 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a596bb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a5406d520 0x7f2a5406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a596bb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2a540ad720 0x7f2a5415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a596bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540abff0 0x7f2a5415f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a596bb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a540b41a0 con 0x7f2a5406d520 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a596bb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5415c360 con 0x7f2a540ad720 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a596bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5406de40 con 0x7f2a540abff0 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a52ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a5406d520 0x7f2a5406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a52ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a5406d520 0x7f2a5406d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51054/0 (socket says 172.21.15.64:51054) 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a52ffd640 1 -- 172.21.15.64:0/3199501871 learned_addr learned my addr 172.21.15.64:0/3199501871 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a537fe640 1 --2- 172.21.15.64:0/3199501871 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540abff0 0x7f2a5415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.576+0000 7f2a4bfff640 1 --2- 172.21.15.64:0/3199501871 >> v2:172.21.15.91:3300/0 conn(0x7f2a540ad720 0x7f2a5415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.577+0000 7f2a52ffd640 1 -- 172.21.15.64:0/3199501871 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540abff0 msgr2=0x7f2a5415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.577+0000 7f2a52ffd640 1 --2- 172.21.15.64:0/3199501871 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540abff0 0x7f2a5415f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.577+0000 7f2a52ffd640 1 -- 172.21.15.64:0/3199501871 >> v2:172.21.15.91:3300/0 conn(0x7f2a540ad720 msgr2=0x7f2a5415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.577+0000 7f2a52ffd640 1 --2- 172.21.15.64:0/3199501871 >> v2:172.21.15.91:3300/0 conn(0x7f2a540ad720 0x7f2a5415ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.577+0000 7f2a52ffd640 1 -- 172.21.15.64:0/3199501871 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5415ba30 con 0x7f2a5406d520 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.577+0000 7f2a537fe640 1 --2- 172.21.15.64:0/3199501871 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540abff0 0x7f2a5415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.577+0000 7f2a4bfff640 1 --2- 172.21.15.64:0/3199501871 >> v2:172.21.15.91:3300/0 conn(0x7f2a540ad720 0x7f2a5415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.578+0000 7f2a52ffd640 1 --2- 172.21.15.64:0/3199501871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a5406d900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a540adb00 tx=0x7f2a4402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d30f0cd7b81807 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.578+0000 7f2a527fc640 1 -- 172.21.15.64:0/3199501871 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44031860 con 0x7f2a5406d520 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.578+0000 7f2a527fc640 1 -- 172.21.15.64:0/3199501871 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a440319c0 con 0x7f2a5406d520 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.578+0000 7f2a527fc640 1 -- 172.21.15.64:0/3199501871 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44031d50 con 0x7f2a5406d520 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.578+0000 7f2a596bb640 1 -- 172.21.15.64:0/3199501871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 msgr2=0x7f2a5406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.578+0000 7f2a596bb640 1 --2- 172.21.15.64:0/3199501871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a5406d900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a540adb00 tx=0x7f2a4402b900 comp rx=0 tx=0).stop 2023-11-24T08:03:50.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.579+0000 7f2a596bb640 1 -- 172.21.15.64:0/3199501871 shutdown_connections 2023-11-24T08:03:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.579+0000 7f2a596bb640 1 --2- 172.21.15.64:0/3199501871 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540abff0 0x7f2a5415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.579+0000 7f2a596bb640 1 --2- 172.21.15.64:0/3199501871 >> v2:172.21.15.91:3300/0 conn(0x7f2a540ad720 0x7f2a5415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.579+0000 7f2a596bb640 1 --2- 172.21.15.64:0/3199501871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a5406d900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.579+0000 7f2a596bb640 1 -- 172.21.15.64:0/3199501871 >> 172.21.15.64:0/3199501871 conn(0x7f2a540b0d80 msgr2=0x7f2a540b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.579+0000 7f2a596bb640 1 -- 172.21.15.64:0/3199501871 shutdown_connections 2023-11-24T08:03:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.579+0000 7f2a596bb640 1 -- 172.21.15.64:0/3199501871 wait complete. 2023-11-24T08:03:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.580+0000 7f2a596bb640 1 Processor -- start 2023-11-24T08:03:50.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.581+0000 7f2a596bb640 1 -- start start 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.581+0000 7f2a596bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a541463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.581+0000 7f2a596bb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a540abff0 0x7f2a540cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.581+0000 7f2a596bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540ad720 0x7f2a540c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a596bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540c8400 con 0x7f2a5406d520 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a596bb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540c8f80 con 0x7f2a540abff0 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a596bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540cdb20 con 0x7f2a540ad720 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a541463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a541463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51070/0 (socket says 172.21.15.64:51070) 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 -- 172.21.15.64:0/1169333656 learned_addr learned my addr 172.21.15.64:0/1169333656 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a4bfff640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a540abff0 0x7f2a540cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a537fe640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540ad720 0x7f2a540c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 -- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540ad720 msgr2=0x7f2a540c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540ad720 0x7f2a540c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 -- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a540abff0 msgr2=0x7f2a540cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a540abff0 0x7f2a540cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 -- 172.21.15.64:0/1169333656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440094d0 con 0x7f2a5406d520 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a537fe640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540ad720 0x7f2a540c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.582+0000 7f2a52ffd640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a541463a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a44000c00 tx=0x7f2a440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.583+0000 7f2a4b7fe640 1 -- 172.21.15.64:0/1169333656 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44031850 con 0x7f2a5406d520 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.583+0000 7f2a4b7fe640 1 -- 172.21.15.64:0/1169333656 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a44044400 con 0x7f2a5406d520 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.583+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a540c8650 con 0x7f2a5406d520 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.583+0000 7f2a4b7fe640 1 -- 172.21.15.64:0/1169333656 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440447b0 con 0x7f2a5406d520 2023-11-24T08:03:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.583+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a540c8c10 con 0x7f2a5406d520 2023-11-24T08:03:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.584+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a540cdda0 con 0x7f2a5406d520 2023-11-24T08:03:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.585+0000 7f2a4b7fe640 1 -- 172.21.15.64:0/1169333656 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a4404d440 con 0x7f2a5406d520 2023-11-24T08:03:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.586+0000 7f2a4b7fe640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a3406ae40 0x7f2a3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.586+0000 7f2a4b7fe640 1 -- 172.21.15.64:0/1169333656 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a440abad0 con 0x7f2a5406d520 2023-11-24T08:03:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.586+0000 7f2a4bfff640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a3406ae40 0x7f2a3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.587+0000 7f2a4bfff640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a3406ae40 0x7f2a3406d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2a540c9dd0 tx=0x7f2a3c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.590+0000 7f2a4b7fe640 1 -- 172.21.15.64:0/1169333656 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a44076da0 con 0x7f2a5406d520 2023-11-24T08:03:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.785+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a54074770 con 0x7f2a5406d520 2023-11-24T08:03:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.788+0000 7f2a4b7fe640 1 -- 172.21.15.64:0/1169333656 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7f2a44035090 con 0x7f2a5406d520 2023-11-24T08:03:50.790 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:03:50.790 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:46.510360+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:03:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a3406ae40 msgr2=0x7f2a3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a3406ae40 0x7f2a3406d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2a540c9dd0 tx=0x7f2a3c008040 comp rx=0 tx=0).stop 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 msgr2=0x7f2a541463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a541463a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a44000c00 tx=0x7f2a440046c0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 shutdown_connections 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a3406ae40 0x7f2a3406d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a540ad720 0x7f2a540c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a540abff0 0x7f2a540cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 --2- 172.21.15.64:0/1169333656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a5406d520 0x7f2a541463a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.790+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 >> 172.21.15.64:0/1169333656 conn(0x7f2a540b0d80 msgr2=0x7f2a5415ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.791+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 shutdown_connections 2023-11-24T08:03:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:50.791+0000 7f2a596bb640 1 -- 172.21.15.64:0/1169333656 wait complete. 2023-11-24T08:03:50.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:03:50.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.035+0000 7fdc27fff640 1 Processor -- start 2023-11-24T08:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.035+0000 7fdc27fff640 1 -- start start 2023-11-24T08:03:51.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.036+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc280d1d90 0x7fdc280ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.036+0000 7fdc27fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc280c6bf0 con 0x7fdc2815d3b0 2023-11-24T08:03:51.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.036+0000 7fdc27fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc280d13d0 con 0x7fdc280d0fa0 2023-11-24T08:03:51.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.036+0000 7fdc27fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280cedc0 con 0x7fdc280d1d90 2023-11-24T08:03:51.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.036+0000 7fdc26ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdc2815d3b0 0x7fdc280d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58664/0 (socket says 172.21.15.64:58664) 2023-11-24T08:03:51.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.036+0000 7fdc26ffd640 1 -- 172.21.15.64:0/444702559 learned_addr learned my addr 172.21.15.64:0/444702559 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429685949 0 0) 0x7fdc280c6bf0 con 0x7fdc2815d3b0 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc14003520 con 0x7fdc2815d3b0 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc267fc640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc280d1d90 0x7fdc280ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86859614 0 0) 0x7fdc14003520 con 0x7fdc2815d3b0 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc280d1d90 msgr2=0x7fdc280ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc280d1d90 0x7fdc280ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 >> v1:172.21.15.91:6789/0 conn(0x7fdc280d0fa0 legacy=0x7fdc280b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280c6e40 con 0x7fdc2815d3b0 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fdc180019f0 con 0x7fdc2815d3b0 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 >> v1:172.21.15.64:6789/0 conn(0x7fdc2815d3b0 legacy=0x7fdc280d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc14003a10 0x7fdc14003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc140047c0 0x7fdc14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc280d1d90 0x7fdc1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280cedc0 con 0x7fdc14003a10 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc140055f0 con 0x7fdc140047c0 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1400bec0 con 0x7fdc280d1d90 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.037+0000 7fdc26ffd640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc14003a10 0x7fdc14003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.038+0000 7fdc267fc640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc140047c0 0x7fdc14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.038+0000 7fdc26ffd640 1 -- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc280d1d90 msgr2=0x7fdc1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.038+0000 7fdc26ffd640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc280d1d90 0x7fdc1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.038+0000 7fdc26ffd640 1 -- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc140047c0 msgr2=0x7fdc14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.038+0000 7fdc26ffd640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc140047c0 0x7fdc14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.038+0000 7fdc26ffd640 1 -- 172.21.15.64:0/444702559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14003520 con 0x7fdc14003a10 2023-11-24T08:03:51.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.038+0000 7fdc267fc640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc140047c0 0x7fdc14004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:51.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.039+0000 7fdc26ffd640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc14003a10 0x7fdc14003e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800a760 tx=0x7fdc140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed8cb2d9aa4ce1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.039+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18034630 con 0x7fdc14003a10 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.039+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1802f070 con 0x7fdc14003a10 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.039+0000 7fdc25ffb640 1 -- 172.21.15.64:0/444702559 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180349d0 con 0x7fdc14003a10 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.040+0000 7fdc27fff640 1 -- 172.21.15.64:0/444702559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc14003a10 msgr2=0x7fdc14003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.040+0000 7fdc27fff640 1 --2- 172.21.15.64:0/444702559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc14003a10 0x7fdc14003e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800a760 tx=0x7fdc140057f0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.040+0000 7fdc277fe640 1 -- 172.21.15.64:0/444702559 reap_dead start 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.040+0000 7fdc27fff640 1 -- 172.21.15.64:0/444702559 shutdown_connections 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.040+0000 7fdc27fff640 1 -- 172.21.15.64:0/444702559 >> 172.21.15.64:0/444702559 conn(0x7fdc280d30a0 msgr2=0x7fdc280d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.040+0000 7fdc27fff640 1 -- 172.21.15.64:0/444702559 shutdown_connections 2023-11-24T08:03:51.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.040+0000 7fdc27fff640 1 -- 172.21.15.64:0/444702559 wait complete. 2023-11-24T08:03:51.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.041+0000 7fdc27fff640 1 Processor -- start 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.042+0000 7fdc27fff640 1 -- start start 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.042+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc14003a10 0x7fdc280b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.042+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc280b86e0 0x7fdc2815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc280cc340 0x7fdc280cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc27fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280a0ae0 con 0x7fdc280cc340 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc27fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280ccd00 con 0x7fdc280b86e0 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc27fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280a0160 con 0x7fdc14003a10 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc280cc340 0x7fdc280cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc280cc340 0x7fdc280cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51092/0 (socket says 172.21.15.64:51092) 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc277fe640 1 -- 172.21.15.64:0/1328316255 learned_addr learned my addr 172.21.15.64:0/1328316255 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc26ffd640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc14003a10 0x7fdc280b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc267fc640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc280b86e0 0x7fdc2815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc277fe640 1 -- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc14003a10 msgr2=0x7fdc280b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc277fe640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc14003a10 0x7fdc280b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc277fe640 1 -- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc280b86e0 msgr2=0x7fdc2815b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc277fe640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc280b86e0 0x7fdc2815b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.043+0000 7fdc277fe640 1 -- 172.21.15.64:0/1328316255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280b7b90 con 0x7fdc280cc340 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.044+0000 7fdc277fe640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc280cc340 0x7fdc280cc7c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdc2808c9e0 tx=0x7fdc2000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.044+0000 7fdc1f7fe640 1 -- 172.21.15.64:0/1328316255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20004220 con 0x7fdc280cc340 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.044+0000 7fdc1f7fe640 1 -- 172.21.15.64:0/1328316255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc200043a0 con 0x7fdc280cc340 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.044+0000 7fdc1f7fe640 1 -- 172.21.15.64:0/1328316255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20007310 con 0x7fdc280cc340 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.044+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc280a0420 con 0x7fdc280cc340 2023-11-24T08:03:51.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.044+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281f2fd0 con 0x7fdc280cc340 2023-11-24T08:03:51.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.045+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc280b8ac0 con 0x7fdc280cc340 2023-11-24T08:03:51.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.046+0000 7fdc1f7fe640 1 -- 172.21.15.64:0/1328316255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc20007470 con 0x7fdc280cc340 2023-11-24T08:03:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.047+0000 7fdc1f7fe640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdc0806ae70 0x7fdc0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.048+0000 7fdc1f7fe640 1 -- 172.21.15.64:0/1328316255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc20088290 con 0x7fdc280cc340 2023-11-24T08:03:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.048+0000 7fdc26ffd640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdc0806ae70 0x7fdc0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.049+0000 7fdc26ffd640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdc0806ae70 0x7fdc0806d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc18034000 tx=0x7fdc1802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.052+0000 7fdc1f7fe640 1 -- 172.21.15.64:0/1328316255 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc20053680 con 0x7fdc280cc340 2023-11-24T08:03:51.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.244+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc2804e840 con 0x7fdc280cc340 2023-11-24T08:03:51.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.245+0000 7fdc1f7fe640 1 -- 172.21.15.64:0/1328316255 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc20056ca0 con 0x7fdc280cc340 2023-11-24T08:03:51.247 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:03:51.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdc0806ae70 msgr2=0x7fdc0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdc0806ae70 0x7fdc0806d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc18034000 tx=0x7fdc1802f000 comp rx=0 tx=0).stop 2023-11-24T08:03:51.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc280cc340 msgr2=0x7fdc280cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc280cc340 0x7fdc280cc7c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdc2808c9e0 tx=0x7fdc2000c820 comp rx=0 tx=0).stop 2023-11-24T08:03:51.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 shutdown_connections 2023-11-24T08:03:51.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdc0806ae70 0x7fdc0806d330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc280cc340 0x7fdc280cc7c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdc280b86e0 0x7fdc2815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 --2- 172.21.15.64:0/1328316255 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdc14003a10 0x7fdc280b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 >> 172.21.15.64:0/1328316255 conn(0x7fdc280d30a0 msgr2=0x7fdc280d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:51.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 shutdown_connections 2023-11-24T08:03:51.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.248+0000 7fdc27fff640 1 -- 172.21.15.64:0/1328316255 wait complete. 2023-11-24T08:03:51.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:03:51.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.488+0000 7f25903c9640 1 Processor -- start 2023-11-24T08:03:51.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.488+0000 7f25903c9640 1 -- start start 2023-11-24T08:03:51.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.490+0000 7f25903c9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258806d520 0x7f258806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.490+0000 7f25903c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258815c360 con 0x7f25880abff0 2023-11-24T08:03:51.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.490+0000 7f25903c9640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258815faf0 con 0x7f25880ad720 2023-11-24T08:03:51.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.490+0000 7f25903c9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25880b41a0 con 0x7f258806d520 2023-11-24T08:03:51.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.490+0000 7f258e966640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f25880abff0 0x7f258815f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58670/0 (socket says 172.21.15.64:58670) 2023-11-24T08:03:51.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.490+0000 7f258e966640 1 -- 172.21.15.64:0/2444475315 learned_addr learned my addr 172.21.15.64:0/2444475315 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:51.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.490+0000 7f258e165640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258806d520 0x7f258806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334395573 0 0) 0x7f258815c360 con 0x7f25880abff0 2023-11-24T08:03:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2574003520 con 0x7f25880abff0 2023-11-24T08:03:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198991282 0 0) 0x7f2574003520 con 0x7f25880abff0 2023-11-24T08:03:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258806d520 msgr2=0x7f258806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258806d520 0x7f258806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 >> v1:172.21.15.91:6789/0 conn(0x7f25880ad720 legacy=0x7f258815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258806de40 con 0x7f25880abff0 2023-11-24T08:03:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f25800019f0 con 0x7f25880abff0 2023-11-24T08:03:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 >> v1:172.21.15.64:6789/0 conn(0x7f25880abff0 legacy=0x7f258815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258806d520 0x7f25740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2574003f20 0x7f25740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574004d00 0x7f2574005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257400ba30 con 0x7f2574003f20 2023-11-24T08:03:51.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257400bba0 con 0x7f2574004d00 2023-11-24T08:03:51.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257400bd60 con 0x7f258806d520 2023-11-24T08:03:51.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.491+0000 7f258d964640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2574003f20 0x7f25740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.492+0000 7f258e966640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574004d00 0x7f2574005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.492+0000 7f258e165640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258806d520 0x7f25740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.492+0000 7f258d964640 1 -- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258806d520 msgr2=0x7f25740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.492+0000 7f258d964640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258806d520 0x7f25740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.492+0000 7f258d964640 1 -- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574004d00 msgr2=0x7f2574005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.492+0000 7f258d964640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574004d00 0x7f2574005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.492+0000 7f258d964640 1 -- 172.21.15.64:0/2444475315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574003520 con 0x7f2574003f20 2023-11-24T08:03:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.493+0000 7f258d964640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2574003f20 0x7f25740047c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2578001720 tx=0x7f25740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29aa053c0c11f45 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.493+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780052d0 con 0x7f2574003f20 2023-11-24T08:03:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.493+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257802e050 con 0x7f2574003f20 2023-11-24T08:03:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.493+0000 7f258d163640 1 -- 172.21.15.64:0/2444475315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578003830 con 0x7f2574003f20 2023-11-24T08:03:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.494+0000 7f25903c9640 1 -- 172.21.15.64:0/2444475315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2574003f20 msgr2=0x7f25740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.494+0000 7f25903c9640 1 --2- 172.21.15.64:0/2444475315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2574003f20 0x7f25740047c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2578001720 tx=0x7f25740057f0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.494+0000 7f258e966640 1 -- 172.21.15.64:0/2444475315 reap_dead start 2023-11-24T08:03:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.494+0000 7f25903c9640 1 -- 172.21.15.64:0/2444475315 shutdown_connections 2023-11-24T08:03:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.494+0000 7f25903c9640 1 -- 172.21.15.64:0/2444475315 >> 172.21.15.64:0/2444475315 conn(0x7f25880b0d80 msgr2=0x7f25880b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.494+0000 7f25903c9640 1 -- 172.21.15.64:0/2444475315 shutdown_connections 2023-11-24T08:03:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.494+0000 7f25903c9640 1 -- 172.21.15.64:0/2444475315 wait complete. 2023-11-24T08:03:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.496+0000 7f25903c9640 1 Processor -- start 2023-11-24T08:03:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.496+0000 7f25903c9640 1 -- start start 2023-11-24T08:03:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f25903c9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574003f20 0x7f25880cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f25903c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588147480 0x7f25880cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f25903c9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25880cab30 0x7f2588159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f25903c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588158f80 con 0x7f2588147480 2023-11-24T08:03:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f25903c9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25881597e0 con 0x7f2574003f20 2023-11-24T08:03:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f25903c9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2588155830 con 0x7f25880cab30 2023-11-24T08:03:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f258d964640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588147480 0x7f25880cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f258d964640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588147480 0x7f25880cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51110/0 (socket says 172.21.15.64:51110) 2023-11-24T08:03:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.497+0000 7f258d964640 1 -- 172.21.15.64:0/1501398354 learned_addr learned my addr 172.21.15.64:0/1501398354 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f258e165640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574003f20 0x7f25880cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f258d964640 1 -- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25880cab30 msgr2=0x7f2588159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f258d964640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25880cab30 0x7f2588159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f258d964640 1 -- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574003f20 msgr2=0x7f25880cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f258d964640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574003f20 0x7f25880cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f258d964640 1 -- 172.21.15.64:0/1501398354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25880cb0a0 con 0x7f2588147480 2023-11-24T08:03:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f258d964640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588147480 0x7f25880cdbe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2578005040 tx=0x7f2578001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f257effd640 1 -- 172.21.15.64:0/1501398354 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780076c0 con 0x7f2588147480 2023-11-24T08:03:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f257effd640 1 -- 172.21.15.64:0/1501398354 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578007a30 con 0x7f2588147480 2023-11-24T08:03:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f257effd640 1 -- 172.21.15.64:0/1501398354 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578006be0 con 0x7f2588147480 2023-11-24T08:03:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.498+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2588159190 con 0x7f2588147480 2023-11-24T08:03:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.499+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25880ce120 con 0x7f2588147480 2023-11-24T08:03:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.500+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2588074770 con 0x7f2588147480 2023-11-24T08:03:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.500+0000 7f257effd640 1 -- 172.21.15.64:0/1501398354 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2578006d40 con 0x7f2588147480 2023-11-24T08:03:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.501+0000 7f257effd640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06ae40 0x7f256c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.501+0000 7f257effd640 1 -- 172.21.15.64:0/1501398354 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25780ab060 con 0x7f2588147480 2023-11-24T08:03:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.502+0000 7f258e165640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06ae40 0x7f256c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.503+0000 7f258e165640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06ae40 0x7f256c06d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2584009a10 tx=0x7f2584008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.506+0000 7f257effd640 1 -- 172.21.15.64:0/1501398354 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2578076580 con 0x7f2588147480 2023-11-24T08:03:51.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.730+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2588158f80 con 0x7f2588147480 2023-11-24T08:03:51.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.731+0000 7f257effd640 1 -- 172.21.15.64:0/1501398354 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f2578079ba0 con 0x7f2588147480 2023-11-24T08:03:51.733 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:03:51.733 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T08:03:51.734 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06ae40 msgr2=0x7f256c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06ae40 0x7f256c06d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2584009a10 tx=0x7f2584008040 comp rx=0 tx=0).stop 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588147480 msgr2=0x7f25880cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588147480 0x7f25880cdbe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2578005040 tx=0x7f2578001120 comp rx=0 tx=0).stop 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 shutdown_connections 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06ae40 0x7f256c06d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25880cab30 0x7f2588159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2588147480 0x7f25880cdbe0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 --2- 172.21.15.64:0/1501398354 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2574003f20 0x7f25880cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 >> 172.21.15.64:0/1501398354 conn(0x7f25880b0d80 msgr2=0x7f25880b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 shutdown_connections 2023-11-24T08:03:51.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.734+0000 7f25903c9640 1 -- 172.21.15.64:0/1501398354 wait complete. 2023-11-24T08:03:51.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:03:51.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.971+0000 7fd0ffb68640 1 Processor -- start 2023-11-24T08:03:51.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.971+0000 7fd0ffb68640 1 -- start start 2023-11-24T08:03:51.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.972+0000 7fd0ffb68640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f806d520 0x7fd0f806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.973+0000 7fd0ffb68640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd0f80ad720 0x7fd0f815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.973+0000 7fd0ffb68640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f815c360 con 0x7fd0f80abff0 2023-11-24T08:03:51.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.973+0000 7fd0ffb68640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f80b41a0 con 0x7fd0f80ad720 2023-11-24T08:03:51.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.973+0000 7fd0ffb68640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f806de40 con 0x7fd0f806d520 2023-11-24T08:03:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.973+0000 7fd0fd103640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd0f80ad720 0x7fd0f815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.973+0000 7fd0fd904640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f806d520 0x7fd0f806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.973+0000 7fd0fd103640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd0f80ad720 0x7fd0f815ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:52914/0 (socket says 172.21.15.64:52914) 2023-11-24T08:03:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.973+0000 7fd0fd103640 1 -- 172.21.15.64:0/1479976682 learned_addr learned my addr 172.21.15.64:0/1479976682 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.974+0000 7fd0fd103640 1 -- 172.21.15.64:0/1479976682 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f806d520 msgr2=0x7fd0f806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.974+0000 7fd0fd103640 1 --2- 172.21.15.64:0/1479976682 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f806d520 0x7fd0f806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.974+0000 7fd0fd103640 1 -- 172.21.15.64:0/1479976682 >> v1:172.21.15.91:6789/0 conn(0x7fd0f80abff0 legacy=0x7fd0f815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.974+0000 7fd0fd103640 1 -- 172.21.15.64:0/1479976682 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f815ba30 con 0x7fd0f80ad720 2023-11-24T08:03:51.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.975+0000 7fd0fd103640 1 --2- 172.21.15.64:0/1479976682 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80ad720 0x7fd0f815ecd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009870 tx=0x7fd0e802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e36d3627a04b9956 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0fc902640 1 -- 172.21.15.64:0/1479976682 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8030840 con 0x7fd0f80ad720 2023-11-24T08:03:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0fc902640 1 -- 172.21.15.64:0/1479976682 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e80309a0 con 0x7fd0f80ad720 2023-11-24T08:03:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0fc902640 1 -- 172.21.15.64:0/1479976682 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8030d30 con 0x7fd0f80ad720 2023-11-24T08:03:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0ffb68640 1 -- 172.21.15.64:0/1479976682 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80ad720 msgr2=0x7fd0f815ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/1479976682 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80ad720 0x7fd0f815ecd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009870 tx=0x7fd0e802b980 comp rx=0 tx=0).stop 2023-11-24T08:03:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0ffb68640 1 -- 172.21.15.64:0/1479976682 shutdown_connections 2023-11-24T08:03:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/1479976682 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80ad720 0x7fd0f815ecd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/1479976682 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f806d520 0x7fd0f806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.976+0000 7fd0ffb68640 1 -- 172.21.15.64:0/1479976682 >> 172.21.15.64:0/1479976682 conn(0x7fd0f80b0d80 msgr2=0x7fd0f80b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.977+0000 7fd0ffb68640 1 -- 172.21.15.64:0/1479976682 shutdown_connections 2023-11-24T08:03:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.977+0000 7fd0ffb68640 1 -- 172.21.15.64:0/1479976682 wait complete. 2023-11-24T08:03:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.978+0000 7fd0ffb68640 1 Processor -- start 2023-11-24T08:03:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.979+0000 7fd0ffb68640 1 -- start start 2023-11-24T08:03:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.979+0000 7fd0ffb68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f806d520 0x7fd0f81f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.979+0000 7fd0ffb68640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80abff0 0x7fd0f81f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0ffb68640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f80ad720 0x7fd0f80c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0ffb68640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f81f7030 con 0x7fd0f806d520 2023-11-24T08:03:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0ffb68640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80c99f0 con 0x7fd0f80abff0 2023-11-24T08:03:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0ffb68640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80c9cd0 con 0x7fd0f80ad720 2023-11-24T08:03:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd904640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f806d520 0x7fd0f81f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd904640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f806d520 0x7fd0f81f61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51126/0 (socket says 172.21.15.64:51126) 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd904640 1 -- 172.21.15.64:0/4054751743 learned_addr learned my addr 172.21.15.64:0/4054751743 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd103640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80abff0 0x7fd0f81f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fe105640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f80ad720 0x7fd0f80c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd904640 1 -- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f80ad720 msgr2=0x7fd0f80c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd904640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f80ad720 0x7fd0f80c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd904640 1 -- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80abff0 msgr2=0x7fd0f81f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd904640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80abff0 0x7fd0f81f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fd904640 1 -- 172.21.15.64:0/4054751743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e80094d0 con 0x7fd0f806d520 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.980+0000 7fd0fe105640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f80ad720 0x7fd0f80c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:03:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.981+0000 7fd0fd904640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f806d520 0x7fd0f81f61b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400e9b0 tx=0x7fd0f400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.981+0000 7fd0ee7fc640 1 -- 172.21.15.64:0/4054751743 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4009670 con 0x7fd0f806d520 2023-11-24T08:03:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.981+0000 7fd0ee7fc640 1 -- 172.21.15.64:0/4054751743 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f400c9a0 con 0x7fd0f806d520 2023-11-24T08:03:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.981+0000 7fd0ee7fc640 1 -- 172.21.15.64:0/4054751743 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400cd50 con 0x7fd0f806d520 2023-11-24T08:03:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.981+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f80c9fb0 con 0x7fd0f806d520 2023-11-24T08:03:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.981+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f80ca570 con 0x7fd0f806d520 2023-11-24T08:03:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.982+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8074770 con 0x7fd0f806d520 2023-11-24T08:03:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.983+0000 7fd0ee7fc640 1 -- 172.21.15.64:0/4054751743 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0f4013470 con 0x7fd0f806d520 2023-11-24T08:03:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.984+0000 7fd0ee7fc640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0dc06ae40 0x7fd0dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:51.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.984+0000 7fd0ee7fc640 1 -- 172.21.15.64:0/4054751743 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0f4089500 con 0x7fd0f806d520 2023-11-24T08:03:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.985+0000 7fd0fd103640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0dc06ae40 0x7fd0dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.986+0000 7fd0fd103640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0dc06ae40 0x7fd0dc06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8036000 tx=0x7fd0e8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:51.989+0000 7fd0ee7fc640 1 -- 172.21.15.64:0/4054751743 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0f4054910 con 0x7fd0f806d520 2023-11-24T08:03:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.184+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0f80cea40 con 0x7fd0f806d520 2023-11-24T08:03:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.186+0000 7fd0ee7fc640 1 -- 172.21.15.64:0/4054751743 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7fd0f4057f30 con 0x7fd0f806d520 2023-11-24T08:03:52.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:03:52.188 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:46.510360+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0dc06ae40 msgr2=0x7fd0dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0dc06ae40 0x7fd0dc06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8036000 tx=0x7fd0e8008030 comp rx=0 tx=0).stop 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f806d520 msgr2=0x7fd0f81f61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f806d520 0x7fd0f81f61b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400e9b0 tx=0x7fd0f400ee70 comp rx=0 tx=0).stop 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 shutdown_connections 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0dc06ae40 0x7fd0dc06d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0f80ad720 0x7fd0f80c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0f80abff0 0x7fd0f81f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 --2- 172.21.15.64:0/4054751743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f806d520 0x7fd0f81f61b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 >> 172.21.15.64:0/4054751743 conn(0x7fd0f80b0d80 msgr2=0x7fd0f80acc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:52.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 shutdown_connections 2023-11-24T08:03:52.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.189+0000 7fd0ffb68640 1 -- 172.21.15.64:0/4054751743 wait complete. 2023-11-24T08:03:52.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:03:52.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:03:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.437+0000 7f0a85c6d640 1 Processor -- start 2023-11-24T08:03:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.437+0000 7f0a85c6d640 1 -- start start 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.439+0000 7f0a85c6d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8006d9e0 0x7f0a80165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.439+0000 7f0a85c6d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a8015f930 con 0x7f0a80144770 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.439+0000 7f0a85c6d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a800b3b10 con 0x7f0a8015f270 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.439+0000 7f0a85c6d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800b43b0 con 0x7f0a8006d9e0 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.439+0000 7f0a84c6b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0a80144770 0x7f0a8015eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58684/0 (socket says 172.21.15.64:58684) 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.439+0000 7f0a84c6b640 1 -- 172.21.15.64:0/3981233449 learned_addr learned my addr 172.21.15.64:0/3981233449 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.439+0000 7f0a8546c640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8006d9e0 0x7f0a80165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593115730 0 0) 0x7f0a8015f930 con 0x7f0a80144770 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a68003520 con 0x7f0a80144770 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556190586 0 0) 0x7f0a68003520 con 0x7f0a80144770 2023-11-24T08:03:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8006d9e0 msgr2=0x7f0a80165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8006d9e0 0x7f0a80165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 >> v1:172.21.15.91:6789/0 conn(0x7f0a8015f270 legacy=0x7f0a800cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800b3df0 con 0x7f0a80144770 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0a7c0019f0 con 0x7f0a80144770 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 >> v1:172.21.15.64:6789/0 conn(0x7f0a80144770 legacy=0x7f0a8015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68003a10 0x7f0a68003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8006d9e0 0x7f0a680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68004d00 0x7f0a68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800b43b0 con 0x7f0a68004d00 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6800ba30 con 0x7f0a68003a10 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6800bbf0 con 0x7f0a8006d9e0 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.440+0000 7f0a8546c640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68004d00 0x7f0a68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.441+0000 7f0a77fff640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8006d9e0 0x7f0a680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.441+0000 7f0a84c6b640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68003a10 0x7f0a68003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.441+0000 7f0a8546c640 1 -- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8006d9e0 msgr2=0x7f0a680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:52.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.441+0000 7f0a8546c640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8006d9e0 0x7f0a680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.441+0000 7f0a8546c640 1 -- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68003a10 msgr2=0x7f0a68003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:52.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.441+0000 7f0a8546c640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68003a10 0x7f0a68003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.441+0000 7f0a8546c640 1 -- 172.21.15.64:0/3981233449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a68003520 con 0x7f0a68004d00 2023-11-24T08:03:52.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.441+0000 7f0a84c6b640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68003a10 0x7f0a68003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:52.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.442+0000 7f0a8546c640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68004d00 0x7f0a68005180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009ac0 tx=0x7f0a680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a823beece50fcee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:52.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.442+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7802c6b0 con 0x7f0a68004d00 2023-11-24T08:03:52.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.442+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7802c810 con 0x7f0a68004d00 2023-11-24T08:03:52.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.442+0000 7f0a777fe640 1 -- 172.21.15.64:0/3981233449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7802cba0 con 0x7f0a68004d00 2023-11-24T08:03:52.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.443+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3981233449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68004d00 msgr2=0x7f0a68005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.443+0000 7f0a85c6d640 1 --2- 172.21.15.64:0/3981233449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68004d00 0x7f0a68005180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009ac0 tx=0x7f0a680057f0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.443+0000 7f0a8546c640 1 -- 172.21.15.64:0/3981233449 reap_dead start 2023-11-24T08:03:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.443+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3981233449 shutdown_connections 2023-11-24T08:03:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.443+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3981233449 >> 172.21.15.64:0/3981233449 conn(0x7f0a800cec10 msgr2=0x7f0a800cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.443+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3981233449 shutdown_connections 2023-11-24T08:03:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.443+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3981233449 wait complete. 2023-11-24T08:03:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.445+0000 7f0a85c6d640 1 Processor -- start 2023-11-24T08:03:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.445+0000 7f0a85c6d640 1 -- start start 2023-11-24T08:03:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a85c6d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68004d00 0x7f0a801f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a85c6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8013a680 0x7f0a8014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a85c6d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8013f6e0 0x7f0a8013fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a85c6d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800ab190 con 0x7f0a8013a680 2023-11-24T08:03:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a85c6d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8014bb20 con 0x7f0a68004d00 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a85c6d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801400a0 con 0x7f0a8013f6e0 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a77fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8013a680 0x7f0a8014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a77fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8013a680 0x7f0a8014e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51144/0 (socket says 172.21.15.64:51144) 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.446+0000 7f0a77fff640 1 -- 172.21.15.64:0/3502993707 learned_addr learned my addr 172.21.15.64:0/3502993707 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a77fff640 1 -- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8013f6e0 msgr2=0x7f0a8013fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a77fff640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8013f6e0 0x7f0a8013fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a77fff640 1 -- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68004d00 msgr2=0x7f0a801f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a84c6b640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68004d00 0x7f0a801f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a77fff640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68004d00 0x7f0a801f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a77fff640 1 -- 172.21.15.64:0/3502993707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8013ab00 con 0x7f0a8013a680 2023-11-24T08:03:52.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a77fff640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8013a680 0x7f0a8014e500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000e750 tx=0x7f0a7000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a757fa640 1 -- 172.21.15.64:0/3502993707 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a700050d0 con 0x7f0a8013a680 2023-11-24T08:03:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a757fa640 1 -- 172.21.15.64:0/3502993707 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a70005230 con 0x7f0a8013a680 2023-11-24T08:03:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a757fa640 1 -- 172.21.15.64:0/3502993707 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000c6c0 con 0x7f0a8013a680 2023-11-24T08:03:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8014ea40 con 0x7f0a8013a680 2023-11-24T08:03:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.447+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a801f33d0 con 0x7f0a8013a680 2023-11-24T08:03:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.449+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8004e840 con 0x7f0a8013a680 2023-11-24T08:03:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.449+0000 7f0a757fa640 1 -- 172.21.15.64:0/3502993707 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a7000c820 con 0x7f0a8013a680 2023-11-24T08:03:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.450+0000 7f0a757fa640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a6c06ae70 0x7f0a6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.450+0000 7f0a757fa640 1 -- 172.21.15.64:0/3502993707 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0a70089190 con 0x7f0a8013a680 2023-11-24T08:03:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.451+0000 7f0a84c6b640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a6c06ae70 0x7f0a6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.452+0000 7f0a84c6b640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a6c06ae70 0x7f0a6c06d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c001b20 tx=0x7f0a7c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.455+0000 7f0a757fa640 1 -- 172.21.15.64:0/3502993707 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a700544f0 con 0x7f0a8013a680 2023-11-24T08:03:52.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.646+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a800ce8f0 con 0x7f0a8013a680 2023-11-24T08:03:52.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.647+0000 7f0a757fa640 1 -- 172.21.15.64:0/3502993707 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a700012d0 con 0x7f0a8013a680 2023-11-24T08:03:52.648 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:03:52.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a6c06ae70 msgr2=0x7f0a6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a6c06ae70 0x7f0a6c06d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c001b20 tx=0x7f0a7c009040 comp rx=0 tx=0).stop 2023-11-24T08:03:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8013a680 msgr2=0x7f0a8014e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8013a680 0x7f0a8014e500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000e750 tx=0x7f0a7000ec10 comp rx=0 tx=0).stop 2023-11-24T08:03:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 shutdown_connections 2023-11-24T08:03:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a6c06ae70 0x7f0a6c06d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a8013f6e0 0x7f0a8013fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8013a680 0x7f0a8014e500 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 --2- 172.21.15.64:0/3502993707 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a68004d00 0x7f0a801f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 >> 172.21.15.64:0/3502993707 conn(0x7f0a800cec10 msgr2=0x7f0a800b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:52.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 shutdown_connections 2023-11-24T08:03:52.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.649+0000 7f0a85c6d640 1 -- 172.21.15.64:0/3502993707 wait complete. 2023-11-24T08:03:52.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:03:52.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.892+0000 7f07865ef640 1 Processor -- start 2023-11-24T08:03:52.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.892+0000 7f07865ef640 1 -- start start 2023-11-24T08:03:52.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.893+0000 7f07865ef640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f078015d5b0 0x7f07800d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.893+0000 7f07865ef640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f07800b1650 0x7f07800d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.893+0000 7f07865ef640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07800cf0c0 con 0x7f07800d14a0 2023-11-24T08:03:52.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.893+0000 7f07865ef640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07800b01b0 con 0x7f078015d5b0 2023-11-24T08:03:52.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.893+0000 7f07865ef640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078015d990 con 0x7f07800b1650 2023-11-24T08:03:52.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.894+0000 7f07855ed640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f078015d5b0 0x7f07800d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.894+0000 7f0784dec640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f07800d14a0 0x7f07800cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58698/0 (socket says 172.21.15.64:58698) 2023-11-24T08:03:52.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.894+0000 7f0784dec640 1 -- 172.21.15.64:0/2738154874 learned_addr learned my addr 172.21.15.64:0/2738154874 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:52.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.894+0000 7f0785dee640 1 --2- 172.21.15.64:0/2738154874 >> v2:172.21.15.91:3300/0 conn(0x7f07800b1650 0x7f07800d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.894+0000 7f07855ed640 1 -- 172.21.15.64:0/2738154874 >> v2:172.21.15.91:3300/0 conn(0x7f07800b1650 msgr2=0x7f07800d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:52.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.894+0000 7f07855ed640 1 --2- 172.21.15.64:0/2738154874 >> v2:172.21.15.91:3300/0 conn(0x7f07800b1650 0x7f07800d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.894+0000 7f07855ed640 1 -- 172.21.15.64:0/2738154874 >> v1:172.21.15.64:6789/0 conn(0x7f07800d14a0 legacy=0x7f07800cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.894+0000 7f07855ed640 1 -- 172.21.15.64:0/2738154874 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07800d1e40 con 0x7f078015d5b0 2023-11-24T08:03:52.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.895+0000 7f07855ed640 1 --2- 172.21.15.64:0/2738154874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800d3990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f078008bc30 tx=0x7f077c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf1cdb9902c24e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:52.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.896+0000 7f0777fff640 1 -- 172.21.15.64:0/2738154874 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c031830 con 0x7f078015d5b0 2023-11-24T08:03:52.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.896+0000 7f0777fff640 1 -- 172.21.15.64:0/2738154874 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c031990 con 0x7f078015d5b0 2023-11-24T08:03:52.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.896+0000 7f0777fff640 1 -- 172.21.15.64:0/2738154874 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c031d20 con 0x7f078015d5b0 2023-11-24T08:03:52.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.896+0000 7f07865ef640 1 -- 172.21.15.64:0/2738154874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 msgr2=0x7f07800d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.896+0000 7f07865ef640 1 --2- 172.21.15.64:0/2738154874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800d3990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f078008bc30 tx=0x7f077c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.896+0000 7f07865ef640 1 -- 172.21.15.64:0/2738154874 shutdown_connections 2023-11-24T08:03:52.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.896+0000 7f07865ef640 1 --2- 172.21.15.64:0/2738154874 >> v2:172.21.15.91:3300/0 conn(0x7f07800b1650 0x7f07800d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.896+0000 7f07865ef640 1 --2- 172.21.15.64:0/2738154874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800d3990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.897+0000 7f07865ef640 1 -- 172.21.15.64:0/2738154874 >> 172.21.15.64:0/2738154874 conn(0x7f078009c3b0 msgr2=0x7f07800d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:52.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.897+0000 7f07865ef640 1 -- 172.21.15.64:0/2738154874 shutdown_connections 2023-11-24T08:03:52.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.897+0000 7f07865ef640 1 -- 172.21.15.64:0/2738154874 wait complete. 2023-11-24T08:03:52.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.898+0000 7f07865ef640 1 Processor -- start 2023-11-24T08:03:52.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.899+0000 7f07865ef640 1 -- start start 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.899+0000 7f07865ef640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07800b1650 0x7f07800c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f07865ef640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07800d14a0 0x7f078015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f07865ef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f07865ef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800c8000 con 0x7f078015d5b0 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f07865ef640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800c73f0 con 0x7f07800d14a0 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f07865ef640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07800c76d0 con 0x7f07800b1650 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0785dee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0785dee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51164/0 (socket says 172.21.15.64:51164) 2023-11-24T08:03:52.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0785dee640 1 -- 172.21.15.64:0/3268157970 learned_addr learned my addr 172.21.15.64:0/3268157970 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0784dec640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07800d14a0 0x7f078015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f07855ed640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07800b1650 0x7f07800c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0785dee640 1 -- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07800b1650 msgr2=0x7f07800c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0785dee640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07800b1650 0x7f07800c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0785dee640 1 -- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07800d14a0 msgr2=0x7f078015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0785dee640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07800d14a0 0x7f078015a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0785dee640 1 -- 172.21.15.64:0/3268157970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c009400 con 0x7f078015d5b0 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.900+0000 7f0784dec640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07800d14a0 0x7f078015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.901+0000 7f0785dee640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800c8490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0778009fd0 tx=0x7f077800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.901+0000 7f0775ffb640 1 -- 172.21.15.64:0/3268157970 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778014420 con 0x7f078015d5b0 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.901+0000 7f0775ffb640 1 -- 172.21.15.64:0/3268157970 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778014580 con 0x7f078015d5b0 2023-11-24T08:03:52.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.901+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07800c79b0 con 0x7f078015d5b0 2023-11-24T08:03:52.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.901+0000 7f0775ffb640 1 -- 172.21.15.64:0/3268157970 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778014930 con 0x7f078015d5b0 2023-11-24T08:03:52.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.901+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07800c70e0 con 0x7f078015d5b0 2023-11-24T08:03:52.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.902+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07800c81b0 con 0x7f078015d5b0 2023-11-24T08:03:52.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.903+0000 7f0775ffb640 1 -- 172.21.15.64:0/3268157970 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0778014a90 con 0x7f078015d5b0 2023-11-24T08:03:52.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.904+0000 7f0775ffb640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f076406ae40 0x7f076406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:52.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.904+0000 7f0775ffb640 1 -- 172.21.15.64:0/3268157970 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0778089c60 con 0x7f078015d5b0 2023-11-24T08:03:52.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.905+0000 7f07855ed640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f076406ae40 0x7f076406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:52.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.906+0000 7f07855ed640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f076406ae40 0x7f076406d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f077c036000 tx=0x7f077c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:52.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:52.908+0000 7f0775ffb640 1 -- 172.21.15.64:0/3268157970 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07800c81b0 con 0x7f078015d5b0 2023-11-24T08:03:53.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.103+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f078004e840 con 0x7f078015d5b0 2023-11-24T08:03:53.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.104+0000 7f0775ffb640 1 -- 172.21.15.64:0/3268157970 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0778054fc0 con 0x7f078015d5b0 2023-11-24T08:03:53.106 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:03:53.106 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:03:53.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:03:53.106 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-11-24T08:03:53.106 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-11-24T08:03:53.106 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:03:53.106 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:03:53.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.106+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f076406ae40 msgr2=0x7f076406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.106+0000 7f07865ef640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f076406ae40 0x7f076406d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f077c036000 tx=0x7f077c03c040 comp rx=0 tx=0).stop 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.106+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 msgr2=0x7f07800c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.106+0000 7f07865ef640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800c8490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0778009fd0 tx=0x7f077800e4b0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.107+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 shutdown_connections 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.107+0000 7f07865ef640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f076406ae40 0x7f076406d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.107+0000 7f07865ef640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078015d5b0 0x7f07800c8490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.107+0000 7f07865ef640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f07800d14a0 0x7f078015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.107+0000 7f07865ef640 1 --2- 172.21.15.64:0/3268157970 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07800b1650 0x7f07800c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.107+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 >> 172.21.15.64:0/3268157970 conn(0x7f078009c3b0 msgr2=0x7f078015dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.107+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 shutdown_connections 2023-11-24T08:03:53.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.107+0000 7f07865ef640 1 -- 172.21.15.64:0/3268157970 wait complete. 2023-11-24T08:03:53.126 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:03:53.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:03:53.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.355+0000 7f8f80b6c640 1 Processor -- start 2023-11-24T08:03:53.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.355+0000 7f8f80b6c640 1 -- start start 2023-11-24T08:03:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.357+0000 7f8f80b6c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f7c144770 0x7f8f7c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.357+0000 7f8f80b6c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c15f0a0 0x7f8f7c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.357+0000 7f8f80b6c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7c15f930 con 0x7f8f7c0b35a0 2023-11-24T08:03:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.357+0000 7f8f80b6c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7c0b43b0 con 0x7f8f7c144770 2023-11-24T08:03:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.357+0000 7f8f80b6c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c15f530 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7b7fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f7c144770 0x7f8f7c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c15f0a0 0x7f8f7c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7b7fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8f7c144770 0x7f8f7c15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:52964/0 (socket says 172.21.15.64:52964) 2023-11-24T08:03:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c15f0a0 0x7f8f7c0ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:48162/0 (socket says 172.21.15.64:48162) 2023-11-24T08:03:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7b7fe640 1 -- 172.21.15.64:0/3655150045 learned_addr learned my addr 172.21.15.64:0/3655150045 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7bfff640 1 -- 172.21.15.64:0/3655150045 >> v2:172.21.15.91:3300/0 conn(0x7f8f7c144770 msgr2=0x7f8f7c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:53.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7bfff640 1 --2- 172.21.15.64:0/3655150045 >> v2:172.21.15.91:3300/0 conn(0x7f8f7c144770 0x7f8f7c15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7bfff640 1 -- 172.21.15.64:0/3655150045 >> v1:172.21.15.91:6789/0 conn(0x7f8f7c0b35a0 legacy=0x7f8f7c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.358+0000 7f8f7bfff640 1 -- 172.21.15.64:0/3655150045 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c0ce9f0 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.360+0000 7f8f7bfff640 1 --2- 172.21.15.64:0/3655150045 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c15f0a0 0x7f8f7c0ce4b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8f740097a0 tx=0x7f8f7402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d3fbd53324ee4b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.360+0000 7f8f7affd640 1 -- 172.21.15.64:0/3655150045 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74031830 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.361+0000 7f8f7affd640 1 -- 172.21.15.64:0/3655150045 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f74031990 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.361+0000 7f8f7affd640 1 -- 172.21.15.64:0/3655150045 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74031d20 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.361+0000 7f8f80b6c640 1 -- 172.21.15.64:0/3655150045 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c15f0a0 msgr2=0x7f8f7c0ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:53.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.361+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/3655150045 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c15f0a0 0x7f8f7c0ce4b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8f740097a0 tx=0x7f8f7402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.361+0000 7f8f80b6c640 1 -- 172.21.15.64:0/3655150045 shutdown_connections 2023-11-24T08:03:53.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.361+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/3655150045 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c15f0a0 0x7f8f7c0ce4b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.361+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/3655150045 >> v2:172.21.15.91:3300/0 conn(0x7f8f7c144770 0x7f8f7c15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.361+0000 7f8f80b6c640 1 -- 172.21.15.64:0/3655150045 >> 172.21.15.64:0/3655150045 conn(0x7f8f7c0cec10 msgr2=0x7f8f7c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:53.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.362+0000 7f8f80b6c640 1 -- 172.21.15.64:0/3655150045 shutdown_connections 2023-11-24T08:03:53.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.362+0000 7f8f80b6c640 1 -- 172.21.15.64:0/3655150045 wait complete. 2023-11-24T08:03:53.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.363+0000 7f8f80b6c640 1 Processor -- start 2023-11-24T08:03:53.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.363+0000 7f8f80b6c640 1 -- start start 2023-11-24T08:03:53.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.364+0000 7f8f80b6c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f7c0b35a0 0x7f8f7c13a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:53.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.364+0000 7f8f80b6c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c144770 0x7f8f7c13b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:53.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.364+0000 7f8f80b6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f7c15f0a0 0x7f8f7c13b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:53.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.364+0000 7f8f80b6c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c13caf0 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.364+0000 7f8f80b6c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c13ac10 con 0x7f8f7c0b35a0 2023-11-24T08:03:53.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.364+0000 7f8f80b6c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c15ce10 con 0x7f8f7c144770 2023-11-24T08:03:53.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f7c15f0a0 0x7f8f7c13b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:53.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f7c15f0a0 0x7f8f7c13b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51172/0 (socket says 172.21.15.64:51172) 2023-11-24T08:03:53.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 -- 172.21.15.64:0/2435628594 learned_addr learned my addr 172.21.15.64:0/2435628594 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f72ffd640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c144770 0x7f8f7c13b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7b7fe640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f7c0b35a0 0x7f8f7c13a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 -- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c144770 msgr2=0x7f8f7c13b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c144770 0x7f8f7c13b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 -- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f7c0b35a0 msgr2=0x7f8f7c13a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f7c0b35a0 0x7f8f7c13a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 -- 172.21.15.64:0/2435628594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74009400 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.365+0000 7f8f7bfff640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f7c15f0a0 0x7f8f7c13b7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f7402b910 tx=0x7f8f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:53.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.366+0000 7f8f78ff9640 1 -- 172.21.15.64:0/2435628594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f740317b0 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.366+0000 7f8f78ff9640 1 -- 172.21.15.64:0/2435628594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7403b970 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.366+0000 7f8f78ff9640 1 -- 172.21.15.64:0/2435628594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7403bd20 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.366+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c13be70 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.366+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c13c430 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.367+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7c13cc30 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.368+0000 7f8f78ff9640 1 -- 172.21.15.64:0/2435628594 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f740453f0 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.369+0000 7f8f78ff9640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8f5c06ae70 0x7f8f5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:53.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.369+0000 7f8f78ff9640 1 -- 172.21.15.64:0/2435628594 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f740ab550 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.369+0000 7f8f7b7fe640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8f5c06ae70 0x7f8f5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.370+0000 7f8f7b7fe640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8f5c06ae70 0x7f8f5c06d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0cf660 tx=0x7f8f6c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.373+0000 7f8f78ff9640 1 -- 172.21.15.64:0/2435628594 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f7c13cc30 con 0x7f8f7c15f0a0 2023-11-24T08:03:53.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:53.572+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8f7c13cc30 con 0x7f8f7c15f0a0 2023-11-24T08:03:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.203+0000 7f8f78ff9640 1 -- 172.21.15.64:0/2435628594 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8f7c13cc30 con 0x7f8f7c15f0a0 2023-11-24T08:03:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-11-24T08:03:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.208+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8f5c06ae70 msgr2=0x7f8f5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.208+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8f5c06ae70 0x7f8f5c06d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0cf660 tx=0x7f8f6c00b040 comp rx=0 tx=0).stop 2023-11-24T08:03:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.208+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f7c15f0a0 msgr2=0x7f8f7c13b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.208+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f7c15f0a0 0x7f8f7c13b7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f7402b910 tx=0x7f8f740046c0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.209+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 shutdown_connections 2023-11-24T08:03:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.209+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8f5c06ae70 0x7f8f5c06d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.209+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f7c15f0a0 0x7f8f7c13b7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.209+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8f7c144770 0x7f8f7c13b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.209+0000 7f8f80b6c640 1 --2- 172.21.15.64:0/2435628594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8f7c0b35a0 0x7f8f7c13a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.209+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 >> 172.21.15.64:0/2435628594 conn(0x7f8f7c0cec10 msgr2=0x7f8f7c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:54.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.209+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 shutdown_connections 2023-11-24T08:03:54.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.210+0000 7f8f80b6c640 1 -- 172.21.15.64:0/2435628594 wait complete. 2023-11-24T08:03:54.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:03:54.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.457+0000 7f2cdbfff640 1 Processor -- start 2023-11-24T08:03:54.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.457+0000 7f2cdbfff640 1 -- start start 2023-11-24T08:03:54.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.458+0000 7f2cdbfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc15d3b0 0x7f2cdc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.458+0000 7f2cdbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cdc0d0fa0 0x7f2cdc0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cdbfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0d2010 con 0x7f2cdc0d1bc0 2023-11-24T08:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cdbfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0c6bf0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cdbfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0d1400 con 0x7f2cdc15d3b0 2023-11-24T08:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cda7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cdc0d0fa0 0x7f2cdc0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cda7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cdc0d0fa0 0x7f2cdc0d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51174/0 (socket says 172.21.15.64:51174) 2023-11-24T08:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cda7fc640 1 -- 172.21.15.64:0/2247929333 learned_addr learned my addr 172.21.15.64:0/2247929333 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:54.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cdaffd640 1 --2- 172.21.15.64:0/2247929333 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc15d3b0 0x7f2cdc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:54.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cda7fc640 1 -- 172.21.15.64:0/2247929333 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc15d3b0 msgr2=0x7f2cdc0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:54.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cda7fc640 1 --2- 172.21.15.64:0/2247929333 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc15d3b0 0x7f2cdc0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.459+0000 7f2cda7fc640 1 -- 172.21.15.64:0/2247929333 >> v1:172.21.15.91:6789/0 conn(0x7f2cdc0d1bc0 legacy=0x7f2cdc0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:54.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.460+0000 7f2cda7fc640 1 -- 172.21.15.64:0/2247929333 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0cedc0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.461+0000 7f2cda7fc640 1 --2- 172.21.15.64:0/2247929333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0d1680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009870 tx=0x7f2cd002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375b48cbb6a42b0d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:54.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.461+0000 7f2cd9ffb640 1 -- 172.21.15.64:0/2247929333 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0031640 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.461+0000 7f2cd9ffb640 1 -- 172.21.15.64:0/2247929333 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd00317a0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.461+0000 7f2cd9ffb640 1 -- 172.21.15.64:0/2247929333 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0031b30 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.462+0000 7f2cdbfff640 1 -- 172.21.15.64:0/2247929333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 msgr2=0x7f2cdc0d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.462+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/2247929333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0d1680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009870 tx=0x7f2cd002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.462+0000 7f2cdbfff640 1 -- 172.21.15.64:0/2247929333 shutdown_connections 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.462+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/2247929333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0d1680 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.462+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/2247929333 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc15d3b0 0x7f2cdc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.462+0000 7f2cdbfff640 1 -- 172.21.15.64:0/2247929333 >> 172.21.15.64:0/2247929333 conn(0x7f2cdc0d30a0 msgr2=0x7f2cdc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.462+0000 7f2cdbfff640 1 -- 172.21.15.64:0/2247929333 shutdown_connections 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.462+0000 7f2cdbfff640 1 -- 172.21.15.64:0/2247929333 wait complete. 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.464+0000 7f2cdbfff640 1 Processor -- start 2023-11-24T08:03:54.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.464+0000 7f2cdbfff640 1 -- start start 2023-11-24T08:03:54.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdbfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc0d1bc0 0x7f2cdc1f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdbfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cdc15d3b0 0x7f2cdc15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdbfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0b69e0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdbfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0b5d60 con 0x7f2cdc15d3b0 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdbfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0b6040 con 0x7f2cdc0d1bc0 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51176/0 (socket says 172.21.15.64:51176) 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cdaffd640 1 -- 172.21.15.64:0/1382751591 learned_addr learned my addr 172.21.15.64:0/1382751591 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.465+0000 7f2cda7fc640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc0d1bc0 0x7f2cdc1f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdaffd640 1 -- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc0d1bc0 msgr2=0x7f2cdc1f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdaffd640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc0d1bc0 0x7f2cdc1f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdb7fe640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cdc15d3b0 0x7f2cdc15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdaffd640 1 -- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cdc15d3b0 msgr2=0x7f2cdc15bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdaffd640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cdc15d3b0 0x7f2cdc15bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdaffd640 1 -- 172.21.15.64:0/1382751591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd00094d0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdaffd640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0cc5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc007ef0 tx=0x7f2ccc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cc3fff640 1 -- 172.21.15.64:0/1382751591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc014620 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cc3fff640 1 -- 172.21.15.64:0/1382751591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc014780 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cc3fff640 1 -- 172.21.15.64:0/1382751591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc014b30 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc0b6320 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.466+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc1f2d40 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.467+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc1f2ed0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.469+0000 7f2cc3fff640 1 -- 172.21.15.64:0/1382751591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ccc014c90 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.470+0000 7f2cc3fff640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2cc806ae40 0x7f2cc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:54.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.470+0000 7f2cc3fff640 1 -- 172.21.15.64:0/1382751591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2ccc089ed0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.470+0000 7f2cda7fc640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2cc806ae40 0x7f2cc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:54.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.471+0000 7f2cda7fc640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2cc806ae40 0x7f2cc806d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0036000 tx=0x7f2cd003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:54.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.475+0000 7f2cc3fff640 1 -- 172.21.15.64:0/1382751591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ccc0550c0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:54.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:54.669+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2cdc04e840 con 0x7f2cdc0d0fa0 2023-11-24T08:03:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.210+0000 7f2cc3fff640 1 -- 172.21.15.64:0/1382751591 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2ccc0586e0 con 0x7f2cdc0d0fa0 2023-11-24T08:03:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-11-24T08:03:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.215+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2cc806ae40 msgr2=0x7f2cc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:55.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.215+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2cc806ae40 0x7f2cc806d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0036000 tx=0x7f2cd003c040 comp rx=0 tx=0).stop 2023-11-24T08:03:55.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.215+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 msgr2=0x7f2cdc0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:55.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.215+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0cc5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc007ef0 tx=0x7f2ccc00e4b0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.220+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 shutdown_connections 2023-11-24T08:03:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.220+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2cc806ae40 0x7f2cc806d300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.220+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2cdc15d3b0 0x7f2cdc15bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.220+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2cdc0d1bc0 0x7f2cdc1f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.220+0000 7f2cdbfff640 1 --2- 172.21.15.64:0/1382751591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc0d0fa0 0x7f2cdc0cc5b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.220+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 >> 172.21.15.64:0/1382751591 conn(0x7f2cdc0d30a0 msgr2=0x7f2cdc0d2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:55.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.223+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 shutdown_connections 2023-11-24T08:03:55.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.223+0000 7f2cdbfff640 1 -- 172.21.15.64:0/1382751591 wait complete. 2023-11-24T08:03:55.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:03:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.450+0000 7f80bcd33640 1 Processor -- start 2023-11-24T08:03:55.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.450+0000 7f80bcd33640 1 -- start start 2023-11-24T08:03:55.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80bcd33640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80b809cb30 0x7f80b80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:55.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80bcd33640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b80adb60 0x7f80b80adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:55.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80bcd33640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b815b730 con 0x7f80b815b350 2023-11-24T08:03:55.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80bcd33640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80ac740 con 0x7f80b809cb30 2023-11-24T08:03:55.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80bcd33640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80ac460 con 0x7f80b80adb60 2023-11-24T08:03:55.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80b659c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f80b815b350 0x7f80b80b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58704/0 (socket says 172.21.15.64:58704) 2023-11-24T08:03:55.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80b659c640 1 -- 172.21.15.64:0/3514061643 learned_addr learned my addr 172.21.15.64:0/3514061643 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:55.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80b5d9b640 1 --2- 172.21.15.64:0/3514061643 >> v2:172.21.15.64:3300/0 conn(0x7f80b809cb30 0x7f80b80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.452+0000 7f80b6d9d640 1 --2- 172.21.15.64:0/3514061643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b80adb60 0x7f80b80adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.453+0000 7f80b559a640 1 -- 172.21.15.64:0/3514061643 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101889399 0 0) 0x7f80a8001110 con 0x7f80b815b350 2023-11-24T08:03:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.453+0000 7f80b559a640 1 -- 172.21.15.64:0/3514061643 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a0003520 con 0x7f80b815b350 2023-11-24T08:03:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.453+0000 7f80b5d9b640 1 -- 172.21.15.64:0/3514061643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b80adb60 msgr2=0x7f80b80adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.453+0000 7f80b5d9b640 1 --2- 172.21.15.64:0/3514061643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b80adb60 0x7f80b80adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.453+0000 7f80b5d9b640 1 -- 172.21.15.64:0/3514061643 >> v1:172.21.15.64:6789/0 conn(0x7f80b815b350 legacy=0x7f80b80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.453+0000 7f80b5d9b640 1 -- 172.21.15.64:0/3514061643 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80b2a50 con 0x7f80b809cb30 2023-11-24T08:03:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.454+0000 7f80b5d9b640 1 --2- 172.21.15.64:0/3514061643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b80abe80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0097a0 tx=0x7f80ac02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c073f527f7fb69d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.454+0000 7f80b559a640 1 -- 172.21.15.64:0/3514061643 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac02d040 con 0x7f80b809cb30 2023-11-24T08:03:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.454+0000 7f80b559a640 1 -- 172.21.15.64:0/3514061643 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac031810 con 0x7f80b809cb30 2023-11-24T08:03:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.454+0000 7f80b559a640 1 -- 172.21.15.64:0/3514061643 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac031ba0 con 0x7f80b809cb30 2023-11-24T08:03:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.455+0000 7f80bcd33640 1 -- 172.21.15.64:0/3514061643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 msgr2=0x7f80b80abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.455+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3514061643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b80abe80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0097a0 tx=0x7f80ac02b8b0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.455+0000 7f80bcd33640 1 -- 172.21.15.64:0/3514061643 shutdown_connections 2023-11-24T08:03:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.455+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3514061643 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b80adb60 0x7f80b80adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.455+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3514061643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b80abe80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.455+0000 7f80bcd33640 1 -- 172.21.15.64:0/3514061643 >> 172.21.15.64:0/3514061643 conn(0x7f80b80b0ba0 msgr2=0x7f80b80b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.455+0000 7f80bcd33640 1 -- 172.21.15.64:0/3514061643 shutdown_connections 2023-11-24T08:03:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.455+0000 7f80bcd33640 1 -- 172.21.15.64:0/3514061643 wait complete. 2023-11-24T08:03:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.457+0000 7f80bcd33640 1 Processor -- start 2023-11-24T08:03:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.457+0000 7f80bcd33640 1 -- start start 2023-11-24T08:03:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80bcd33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b81537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:55.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80bcd33640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80b80adb60 0x7f80b814ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:55.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80bcd33640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b815b350 0x7f80b81f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:55.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80bcd33640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b81ed160 con 0x7f80b809cb30 2023-11-24T08:03:55.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80bcd33640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8070490 con 0x7f80b80adb60 2023-11-24T08:03:55.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80bcd33640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80706e0 con 0x7f80b815b350 2023-11-24T08:03:55.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80b659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b81537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:55.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80b659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b81537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51202/0 (socket says 172.21.15.64:51202) 2023-11-24T08:03:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80b659c640 1 -- 172.21.15.64:0/3614184000 learned_addr learned my addr 172.21.15.64:0/3614184000 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80b5d9b640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80b80adb60 0x7f80b814ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.458+0000 7f80b6d9d640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b815b350 0x7f80b81f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80b659c640 1 -- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b815b350 msgr2=0x7f80b81f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80b659c640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b815b350 0x7f80b81f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80b659c640 1 -- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80b80adb60 msgr2=0x7f80b814ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80b659c640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80b80adb60 0x7f80b814ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80b659c640 1 -- 172.21.15.64:0/3614184000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac009400 con 0x7f80b809cb30 2023-11-24T08:03:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80b5d9b640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80b80adb60 0x7f80b814ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80b659c640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b81537e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c450 tx=0x7f80a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80a6ffd640 1 -- 172.21.15.64:0/3614184000 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8005ce0 con 0x7f80b809cb30 2023-11-24T08:03:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80a6ffd640 1 -- 172.21.15.64:0/3614184000 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a8005e40 con 0x7f80b809cb30 2023-11-24T08:03:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b80709c0 con 0x7f80b809cb30 2023-11-24T08:03:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80a6ffd640 1 -- 172.21.15.64:0/3614184000 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8007290 con 0x7f80b809cb30 2023-11-24T08:03:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.459+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b8070f80 con 0x7f80b809cb30 2023-11-24T08:03:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.460+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b8071170 con 0x7f80b809cb30 2023-11-24T08:03:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.462+0000 7f80a6ffd640 1 -- 172.21.15.64:0/3614184000 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80a80073f0 con 0x7f80b809cb30 2023-11-24T08:03:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.463+0000 7f80a6ffd640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f809806ae40 0x7f809806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.463+0000 7f80a6ffd640 1 -- 172.21.15.64:0/3614184000 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f80a8088a60 con 0x7f80b809cb30 2023-11-24T08:03:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.463+0000 7f80b5d9b640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f809806ae40 0x7f809806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:55.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.464+0000 7f80b5d9b640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f809806ae40 0x7f809806d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f80ac036000 tx=0x7f80ac03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:55.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.468+0000 7f80a6ffd640 1 -- 172.21.15.64:0/3614184000 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80a8053ad0 con 0x7f80b809cb30 2023-11-24T08:03:55.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:55.696+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f80b815b730 con 0x7f80b809cb30 2023-11-24T08:03:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.238+0000 7f80a6ffd640 1 -- 172.21.15.64:0/3614184000 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f80b815b730 con 0x7f80b809cb30 2023-11-24T08:03:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T08:03:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f809806ae40 msgr2=0x7f809806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f809806ae40 0x7f809806d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f80ac036000 tx=0x7f80ac03c040 comp rx=0 tx=0).stop 2023-11-24T08:03:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 msgr2=0x7f80b81537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b81537e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c450 tx=0x7f80a800c910 comp rx=0 tx=0).stop 2023-11-24T08:03:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 shutdown_connections 2023-11-24T08:03:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f809806ae40 0x7f809806d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80b815b350 0x7f80b81f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80b80adb60 0x7f80b814ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 --2- 172.21.15.64:0/3614184000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b809cb30 0x7f80b81537e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 >> 172.21.15.64:0/3614184000 conn(0x7f80b80b0ba0 msgr2=0x7f80b809bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 shutdown_connections 2023-11-24T08:03:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.242+0000 7f80bcd33640 1 -- 172.21.15.64:0/3614184000 wait complete. 2023-11-24T08:03:56.270 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:03:56.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.471+0000 7f715d75c640 1 Processor -- start 2023-11-24T08:03:56.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.471+0000 7f715d75c640 1 -- start start 2023-11-24T08:03:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f715d75c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f715813dfd0 0x7f715815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f715d75c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71580cf050 0x7f71580cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f715d75c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f715806c130 0x7f715806c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f715d75c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71580cfa10 con 0x7f715813dfd0 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f715d75c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715815fa70 con 0x7f715806c130 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f715d75c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715815ee60 con 0x7f71580cf050 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f7157fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f715813dfd0 0x7f715815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f7157fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f715813dfd0 0x7f715815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51216/0 (socket says 172.21.15.64:51216) 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f7157fff640 1 -- 172.21.15.64:0/3897240404 learned_addr learned my addr 172.21.15.64:0/3897240404 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f7157fff640 1 -- 172.21.15.64:0/3897240404 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71580cf050 msgr2=0x7f71580cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f7157fff640 1 --2- 172.21.15.64:0/3897240404 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71580cf050 0x7f71580cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f7157fff640 1 -- 172.21.15.64:0/3897240404 >> v2:172.21.15.91:3300/0 conn(0x7f715806c130 msgr2=0x7f715806c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f7157fff640 1 --2- 172.21.15.64:0/3897240404 >> v2:172.21.15.91:3300/0 conn(0x7f715806c130 0x7f715806c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.473+0000 7f7157fff640 1 -- 172.21.15.64:0/3897240404 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715815f140 con 0x7f715813dfd0 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.475+0000 7f7157fff640 1 --2- 172.21.15.64:0/3897240404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715813dfd0 0x7f715815be70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f715806d520 tx=0x7f714802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a34adfc68604a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.475+0000 7f71577fe640 1 -- 172.21.15.64:0/3897240404 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148031860 con 0x7f715813dfd0 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.475+0000 7f71577fe640 1 -- 172.21.15.64:0/3897240404 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71480319c0 con 0x7f715813dfd0 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.475+0000 7f71577fe640 1 -- 172.21.15.64:0/3897240404 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148031d50 con 0x7f715813dfd0 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 -- 172.21.15.64:0/3897240404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715813dfd0 msgr2=0x7f715815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 --2- 172.21.15.64:0/3897240404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715813dfd0 0x7f715815be70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f715806d520 tx=0x7f714802b900 comp rx=0 tx=0).stop 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 -- 172.21.15.64:0/3897240404 shutdown_connections 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 --2- 172.21.15.64:0/3897240404 >> v2:172.21.15.91:3300/0 conn(0x7f715806c130 0x7f715806c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 --2- 172.21.15.64:0/3897240404 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71580cf050 0x7f71580cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 --2- 172.21.15.64:0/3897240404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715813dfd0 0x7f715815be70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 -- 172.21.15.64:0/3897240404 >> 172.21.15.64:0/3897240404 conn(0x7f715806a860 msgr2=0x7f715806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:56.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 -- 172.21.15.64:0/3897240404 shutdown_connections 2023-11-24T08:03:56.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.476+0000 7f715d75c640 1 -- 172.21.15.64:0/3897240404 wait complete. 2023-11-24T08:03:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.478+0000 7f715d75c640 1 Processor -- start 2023-11-24T08:03:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.478+0000 7f715d75c640 1 -- start start 2023-11-24T08:03:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f715d75c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715806c130 0x7f71580b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f715d75c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71580cf050 0x7f715815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f715d75c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f715813dfd0 0x7f715815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f715d75c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581f6800 con 0x7f715806c130 2023-11-24T08:03:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f715d75c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581f6a10 con 0x7f71580cf050 2023-11-24T08:03:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f715d75c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71581f6cf0 con 0x7f715813dfd0 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f7157fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715806c130 0x7f71580b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f7157fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715806c130 0x7f71580b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51226/0 (socket says 172.21.15.64:51226) 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f7157fff640 1 -- 172.21.15.64:0/1818240497 learned_addr learned my addr 172.21.15.64:0/1818240497 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f7157fff640 1 -- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f715813dfd0 msgr2=0x7f715815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f7157fff640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f715813dfd0 0x7f715815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f714ffff640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71580cf050 0x7f715815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f7157fff640 1 -- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71580cf050 msgr2=0x7f715815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f7157fff640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71580cf050 0x7f715815a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f7157fff640 1 -- 172.21.15.64:0/1818240497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480094d0 con 0x7f715806c130 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.479+0000 7f714ffff640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71580cf050 0x7f715815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.480+0000 7f7157fff640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715806c130 0x7f71580b3950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7148000c00 tx=0x7f71480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.480+0000 7f71557fa640 1 -- 172.21.15.64:0/1818240497 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714803ad10 con 0x7f715806c130 2023-11-24T08:03:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.480+0000 7f71557fa640 1 -- 172.21.15.64:0/1818240497 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714803ae70 con 0x7f715806c130 2023-11-24T08:03:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.480+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71581f6fd0 con 0x7f715806c130 2023-11-24T08:03:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.480+0000 7f71557fa640 1 -- 172.21.15.64:0/1818240497 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148044610 con 0x7f715806c130 2023-11-24T08:03:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.480+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71581f7590 con 0x7f715806c130 2023-11-24T08:03:56.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.481+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71580c8dd0 con 0x7f715806c130 2023-11-24T08:03:56.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.482+0000 7f71557fa640 1 -- 172.21.15.64:0/1818240497 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f714804d450 con 0x7f715806c130 2023-11-24T08:03:56.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.483+0000 7f71557fa640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f713806ae40 0x7f713806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.483+0000 7f71557fa640 1 -- 172.21.15.64:0/1818240497 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71480abd70 con 0x7f715806c130 2023-11-24T08:03:56.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.484+0000 7f714ffff640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f713806ae40 0x7f713806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.485+0000 7f714ffff640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f713806ae40 0x7f713806d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7140006fd0 tx=0x7f7140008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:56.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.488+0000 7f71557fa640 1 -- 172.21.15.64:0/1818240497 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7148076d80 con 0x7f715806c130 2023-11-24T08:03:56.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.681+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f715804e840 con 0x7f715806c130 2023-11-24T08:03:56.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.682+0000 7f71557fa640 1 -- 172.21.15.64:0/1818240497 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f714807a3a0 con 0x7f715806c130 2023-11-24T08:03:56.685 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:03:56.685 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:55.698821+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:03:56.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.685+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f713806ae40 msgr2=0x7f713806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.685+0000 7f715d75c640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f713806ae40 0x7f713806d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7140006fd0 tx=0x7f7140008040 comp rx=0 tx=0).stop 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.685+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715806c130 msgr2=0x7f71580b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.685+0000 7f715d75c640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715806c130 0x7f71580b3950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7148000c00 tx=0x7f71480046c0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.685+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 shutdown_connections 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.685+0000 7f715d75c640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f713806ae40 0x7f713806d300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.686+0000 7f715d75c640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f715813dfd0 0x7f715815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.686+0000 7f715d75c640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71580cf050 0x7f715815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.686+0000 7f715d75c640 1 --2- 172.21.15.64:0/1818240497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715806c130 0x7f71580b3950 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.686+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 >> 172.21.15.64:0/1818240497 conn(0x7f715806a860 msgr2=0x7f7158071800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.686+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 shutdown_connections 2023-11-24T08:03:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.686+0000 7f715d75c640 1 -- 172.21.15.64:0/1818240497 wait complete. 2023-11-24T08:03:56.708 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:03:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.922+0000 7f741359e640 1 Processor -- start 2023-11-24T08:03:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.922+0000 7f741359e640 1 -- start start 2023-11-24T08:03:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f741359e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7414143570 0x7f74140cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f741359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7414143950 0x7f74140cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f741359e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7414071700 con 0x7f741415e6b0 2023-11-24T08:03:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f741359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741415b410 con 0x7f7414143950 2023-11-24T08:03:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f741359e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741415b130 con 0x7f7414143570 2023-11-24T08:03:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f7411d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7414143950 0x7f74140cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f7411d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7414143950 0x7f74140cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51234/0 (socket says 172.21.15.64:51234) 2023-11-24T08:03:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f7411d9b640 1 -- 172.21.15.64:0/815375296 learned_addr learned my addr 172.21.15.64:0/815375296 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f7411d9b640 1 -- 172.21.15.64:0/815375296 >> v2:172.21.15.91:3300/0 conn(0x7f7414143570 msgr2=0x7f74140cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f741259c640 1 --2- 172.21.15.64:0/815375296 >> v2:172.21.15.91:3300/0 conn(0x7f7414143570 0x7f74140cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f7411d9b640 1 --2- 172.21.15.64:0/815375296 >> v2:172.21.15.91:3300/0 conn(0x7f7414143570 0x7f74140cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f7411d9b640 1 -- 172.21.15.64:0/815375296 >> v1:172.21.15.91:6789/0 conn(0x7f741415e6b0 legacy=0x7f741415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.924+0000 7f7411d9b640 1 -- 172.21.15.64:0/815375296 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741415dbd0 con 0x7f7414143950 2023-11-24T08:03:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.926+0000 7f7411d9b640 1 --2- 172.21.15.64:0/815375296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7414143950 0x7f74140cf590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7414070c70 tx=0x7f740802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0004dd607eeca2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:56.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.926+0000 7f741159a640 1 -- 172.21.15.64:0/815375296 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74080317e0 con 0x7f7414143950 2023-11-24T08:03:56.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.926+0000 7f741159a640 1 -- 172.21.15.64:0/815375296 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7408031940 con 0x7f7414143950 2023-11-24T08:03:56.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.926+0000 7f741159a640 1 -- 172.21.15.64:0/815375296 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408031cd0 con 0x7f7414143950 2023-11-24T08:03:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.927+0000 7f741359e640 1 -- 172.21.15.64:0/815375296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7414143950 msgr2=0x7f74140cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.927+0000 7f741359e640 1 --2- 172.21.15.64:0/815375296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7414143950 0x7f74140cf590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7414070c70 tx=0x7f740802b880 comp rx=0 tx=0).stop 2023-11-24T08:03:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.927+0000 7f741359e640 1 -- 172.21.15.64:0/815375296 shutdown_connections 2023-11-24T08:03:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.927+0000 7f741359e640 1 --2- 172.21.15.64:0/815375296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7414143950 0x7f74140cf590 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.927+0000 7f741359e640 1 --2- 172.21.15.64:0/815375296 >> v2:172.21.15.91:3300/0 conn(0x7f7414143570 0x7f74140cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.927+0000 7f741359e640 1 -- 172.21.15.64:0/815375296 >> 172.21.15.64:0/815375296 conn(0x7f741406a860 msgr2=0x7f741406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.927+0000 7f741359e640 1 -- 172.21.15.64:0/815375296 shutdown_connections 2023-11-24T08:03:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.927+0000 7f741359e640 1 -- 172.21.15.64:0/815375296 wait complete. 2023-11-24T08:03:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.929+0000 7f741359e640 1 Processor -- start 2023-11-24T08:03:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.929+0000 7f741359e640 1 -- start start 2023-11-24T08:03:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f741359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7414143570 0x7f741413e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f741359e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7414143950 0x7f741413ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f741359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741415e6b0 0x7f74140b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f741359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741413b0a0 con 0x7f741415e6b0 2023-11-24T08:03:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f741359e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7414149190 con 0x7f7414143950 2023-11-24T08:03:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f741359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741413d3c0 con 0x7f7414143570 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f7412d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741415e6b0 0x7f74140b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f7412d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741415e6b0 0x7f74140b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51242/0 (socket says 172.21.15.64:51242) 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f7412d9d640 1 -- 172.21.15.64:0/3576582642 learned_addr learned my addr 172.21.15.64:0/3576582642 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f741259c640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7414143570 0x7f741413e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.930+0000 7f7411d9b640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7414143950 0x7f741413ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7412d9d640 1 -- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7414143570 msgr2=0x7f741413e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7412d9d640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7414143570 0x7f741413e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7412d9d640 1 -- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7414143950 msgr2=0x7f741413ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7412d9d640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7414143950 0x7f741413ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7412d9d640 1 -- 172.21.15.64:0/3576582642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080094d0 con 0x7f741415e6b0 2023-11-24T08:03:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7412d9d640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741415e6b0 0x7f74140b0820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f740c00c780 tx=0x7f740c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7402ffd640 1 -- 172.21.15.64:0/3576582642 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c004b60 con 0x7f741415e6b0 2023-11-24T08:03:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7402ffd640 1 -- 172.21.15.64:0/3576582642 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c00ce90 con 0x7f741415e6b0 2023-11-24T08:03:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f7402ffd640 1 -- 172.21.15.64:0/3576582642 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00a850 con 0x7f741415e6b0 2023-11-24T08:03:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741413d6a0 con 0x7f741415e6b0 2023-11-24T08:03:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.931+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741413dc60 con 0x7f741415e6b0 2023-11-24T08:03:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.933+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741413de50 con 0x7f741415e6b0 2023-11-24T08:03:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.934+0000 7f7402ffd640 1 -- 172.21.15.64:0/3576582642 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f740c00a9b0 con 0x7f741415e6b0 2023-11-24T08:03:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.934+0000 7f7402ffd640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73f406ae40 0x7f73f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.935+0000 7f7402ffd640 1 -- 172.21.15.64:0/3576582642 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f740c089800 con 0x7f741415e6b0 2023-11-24T08:03:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.935+0000 7f741259c640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73f406ae40 0x7f73f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.936+0000 7f741259c640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73f406ae40 0x7f73f406d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7404006fd0 tx=0x7f7404008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:56.939+0000 7f7402ffd640 1 -- 172.21.15.64:0/3576582642 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f740c0548a0 con 0x7f741415e6b0 2023-11-24T08:03:57.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.135+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f741404e840 con 0x7f741415e6b0 2023-11-24T08:03:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.137+0000 7f7402ffd640 1 -- 172.21.15.64:0/3576582642 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f740c057ec0 con 0x7f741415e6b0 2023-11-24T08:03:57.142 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:03:57.142 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:03:57.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73f406ae40 msgr2=0x7f73f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73f406ae40 0x7f73f406d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7404006fd0 tx=0x7f7404008040 comp rx=0 tx=0).stop 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741415e6b0 msgr2=0x7f74140b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741415e6b0 0x7f74140b0820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f740c00c780 tx=0x7f740c00cc40 comp rx=0 tx=0).stop 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 shutdown_connections 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73f406ae40 0x7f73f406d300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741415e6b0 0x7f74140b0820 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7414143950 0x7f741413ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 --2- 172.21.15.64:0/3576582642 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7414143570 0x7f741413e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 >> 172.21.15.64:0/3576582642 conn(0x7f741406a860 msgr2=0x7f74140ce470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 shutdown_connections 2023-11-24T08:03:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.141+0000 7f741359e640 1 -- 172.21.15.64:0/3576582642 wait complete. 2023-11-24T08:03:57.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:03:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.379+0000 7fc3439a5640 1 Processor -- start 2023-11-24T08:03:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.379+0000 7fc3439a5640 1 -- start start 2023-11-24T08:03:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.380+0000 7fc3439a5640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc33c15b350 0x7fc33c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.380+0000 7fc3439a5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c09cb30 0x7fc33c0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc3439a5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc33c0adb60 0x7fc33c0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc3439a5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c15b730 con 0x7fc33c0adb60 2023-11-24T08:03:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc3439a5640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c0ac740 con 0x7fc33c15b350 2023-11-24T08:03:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc3439a5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c0ac460 con 0x7fc33c09cb30 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341f42640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc33c0adb60 0x7fc33c0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341f42640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc33c0adb60 0x7fc33c0adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36592/0 (socket says 172.21.15.64:36592) 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341f42640 1 -- 172.21.15.64:0/1513485663 learned_addr learned my addr 172.21.15.64:0/1513485663 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341741640 1 --2- 172.21.15.64:0/1513485663 >> v2:172.21.15.91:3300/0 conn(0x7fc33c15b350 0x7fc33c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc340f40640 1 --2- 172.21.15.64:0/1513485663 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c09cb30 0x7fc33c0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341f42640 1 -- 172.21.15.64:0/1513485663 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c09cb30 msgr2=0x7fc33c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341f42640 1 --2- 172.21.15.64:0/1513485663 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c09cb30 0x7fc33c0b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341f42640 1 -- 172.21.15.64:0/1513485663 >> v2:172.21.15.91:3300/0 conn(0x7fc33c15b350 msgr2=0x7fc33c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341f42640 1 --2- 172.21.15.64:0/1513485663 >> v2:172.21.15.91:3300/0 conn(0x7fc33c15b350 0x7fc33c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.381+0000 7fc341f42640 1 -- 172.21.15.64:0/1513485663 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c0b2a50 con 0x7fc33c0adb60 2023-11-24T08:03:57.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.382+0000 7fc341f42640 1 --2- 172.21.15.64:0/1513485663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0adb60 0x7fc33c0adfb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0b5100 tx=0x7fc33402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4d22989b036daf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:57.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.383+0000 7fc333fff640 1 -- 172.21.15.64:0/1513485663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334031870 con 0x7fc33c0adb60 2023-11-24T08:03:57.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.383+0000 7fc333fff640 1 -- 172.21.15.64:0/1513485663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3340319d0 con 0x7fc33c0adb60 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.383+0000 7fc333fff640 1 -- 172.21.15.64:0/1513485663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334031d60 con 0x7fc33c0adb60 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.383+0000 7fc3439a5640 1 -- 172.21.15.64:0/1513485663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0adb60 msgr2=0x7fc33c0adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.383+0000 7fc3439a5640 1 --2- 172.21.15.64:0/1513485663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0adb60 0x7fc33c0adfb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0b5100 tx=0x7fc33402b910 comp rx=0 tx=0).stop 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.384+0000 7fc3439a5640 1 -- 172.21.15.64:0/1513485663 shutdown_connections 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.384+0000 7fc3439a5640 1 --2- 172.21.15.64:0/1513485663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0adb60 0x7fc33c0adfb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.384+0000 7fc3439a5640 1 --2- 172.21.15.64:0/1513485663 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c09cb30 0x7fc33c0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.384+0000 7fc3439a5640 1 --2- 172.21.15.64:0/1513485663 >> v2:172.21.15.91:3300/0 conn(0x7fc33c15b350 0x7fc33c0abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.384+0000 7fc3439a5640 1 -- 172.21.15.64:0/1513485663 >> 172.21.15.64:0/1513485663 conn(0x7fc33c0b0ba0 msgr2=0x7fc33c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:57.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.384+0000 7fc3439a5640 1 -- 172.21.15.64:0/1513485663 shutdown_connections 2023-11-24T08:03:57.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.384+0000 7fc3439a5640 1 -- 172.21.15.64:0/1513485663 wait complete. 2023-11-24T08:03:57.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.385+0000 7fc3439a5640 1 Processor -- start 2023-11-24T08:03:57.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.386+0000 7fc3439a5640 1 -- start start 2023-11-24T08:03:57.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.386+0000 7fc3439a5640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc33c09cb30 0x7fc33c1e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:57.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc3439a5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c0adb60 0x7fc33c1e1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:57.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc3439a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c15b350 0x7fc33c1404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:57.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc3439a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c142e40 con 0x7fc33c15b350 2023-11-24T08:03:57.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc3439a5640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c142230 con 0x7fc33c09cb30 2023-11-24T08:03:57.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc3439a5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c142510 con 0x7fc33c0adb60 2023-11-24T08:03:57.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341f42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c15b350 0x7fc33c1404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:57.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341f42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c15b350 0x7fc33c1404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36594/0 (socket says 172.21.15.64:36594) 2023-11-24T08:03:57.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341f42640 1 -- 172.21.15.64:0/3210588023 learned_addr learned my addr 172.21.15.64:0/3210588023 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc340f40640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c0adb60 0x7fc33c1e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341741640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc33c09cb30 0x7fc33c1e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341f42640 1 -- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c0adb60 msgr2=0x7fc33c1e1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341f42640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c0adb60 0x7fc33c1e1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341f42640 1 -- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc33c09cb30 msgr2=0x7fc33c1e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341f42640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc33c09cb30 0x7fc33c1e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:57.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.387+0000 7fc341f42640 1 -- 172.21.15.64:0/3210588023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340094d0 con 0x7fc33c15b350 2023-11-24T08:03:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.388+0000 7fc341f42640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c15b350 0x7fc33c1404a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3340090b0 tx=0x7fc3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.388+0000 7fc331ffb640 1 -- 172.21.15.64:0/3210588023 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33403ac60 con 0x7fc33c15b350 2023-11-24T08:03:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.388+0000 7fc331ffb640 1 -- 172.21.15.64:0/3210588023 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33403adc0 con 0x7fc33c15b350 2023-11-24T08:03:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.388+0000 7fc331ffb640 1 -- 172.21.15.64:0/3210588023 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334044630 con 0x7fc33c15b350 2023-11-24T08:03:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.388+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33c1427f0 con 0x7fc33c15b350 2023-11-24T08:03:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.388+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33c1e0b20 con 0x7fc33c15b350 2023-11-24T08:03:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.389+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33c142fb0 con 0x7fc33c15b350 2023-11-24T08:03:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.390+0000 7fc331ffb640 1 -- 172.21.15.64:0/3210588023 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc33404d450 con 0x7fc33c15b350 2023-11-24T08:03:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.391+0000 7fc331ffb640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc32006ae40 0x7fc32006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.391+0000 7fc331ffb640 1 -- 172.21.15.64:0/3210588023 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc33402d040 con 0x7fc33c15b350 2023-11-24T08:03:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.392+0000 7fc341741640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc32006ae40 0x7fc32006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.393+0000 7fc341741640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc32006ae40 0x7fc32006d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc338006fd0 tx=0x7fc338008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.397+0000 7fc331ffb640 1 -- 172.21.15.64:0/3210588023 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc334076cb0 con 0x7fc33c15b350 2023-11-24T08:03:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:57.631+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc33c09d200 con 0x7fc33c15b350 2023-11-24T08:03:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.259+0000 7fc331ffb640 1 -- 172.21.15.64:0/3210588023 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc334035070 con 0x7fc33c15b350 2023-11-24T08:03:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc32006ae40 msgr2=0x7fc32006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc32006ae40 0x7fc32006d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc338006fd0 tx=0x7fc338008040 comp rx=0 tx=0).stop 2023-11-24T08:03:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c15b350 msgr2=0x7fc33c1404a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c15b350 0x7fc33c1404a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3340090b0 tx=0x7fc3340046c0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 shutdown_connections 2023-11-24T08:03:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc32006ae40 0x7fc32006d300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c15b350 0x7fc33c1404a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc33c0adb60 0x7fc33c1e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 --2- 172.21.15.64:0/3210588023 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc33c09cb30 0x7fc33c1e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.263+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 >> 172.21.15.64:0/3210588023 conn(0x7fc33c0b0ba0 msgr2=0x7fc33c09bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.264+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 shutdown_connections 2023-11-24T08:03:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.264+0000 7fc3439a5640 1 -- 172.21.15.64:0/3210588023 wait complete. 2023-11-24T08:03:58.284 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:03:58.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:03:58.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.508+0000 7f94711ef640 1 Processor -- start 2023-11-24T08:03:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.508+0000 7f94711ef640 1 -- start start 2023-11-24T08:03:58.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.509+0000 7f94711ef640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f946c15afc0 0x7f946c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.509+0000 7f94711ef640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0cbd10 0x7f946c0cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.509+0000 7f94711ef640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c0cb870 con 0x7f946c0d0d10 2023-11-24T08:03:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f94711ef640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0ced50 con 0x7f946c15afc0 2023-11-24T08:03:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f94711ef640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0cdcc0 con 0x7f946c0cbd10 2023-11-24T08:03:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f946bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f946c15afc0 0x7f946c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f946b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f946c0d0d10 0x7f946c0d11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56594/0 (socket says 172.21.15.64:56594) 2023-11-24T08:03:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f946b7fe640 1 -- 172.21.15.64:0/1780739563 learned_addr learned my addr 172.21.15.64:0/1780739563 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f94709ee640 1 --2- 172.21.15.64:0/1780739563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0cbd10 0x7f946c0cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f946bfff640 1 -- 172.21.15.64:0/1780739563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0cbd10 msgr2=0x7f946c0cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f946bfff640 1 --2- 172.21.15.64:0/1780739563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0cbd10 0x7f946c0cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f946bfff640 1 -- 172.21.15.64:0/1780739563 >> v1:172.21.15.64:6789/0 conn(0x7f946c0d0d10 legacy=0x7f946c0d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f946bfff640 1 -- 172.21.15.64:0/1780739563 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0cdfa0 con 0x7f946c15afc0 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f946affd640 1 -- 172.21.15.64:0/1780739563 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273229607 0 0) 0x7f946c0cb870 con 0x7f946c0d0d10 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.510+0000 7f94709ee640 1 --2- 172.21.15.64:0/1780739563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0cbd10 0x7f946c0cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.512+0000 7f946bfff640 1 --2- 172.21.15.64:0/1780739563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c15afc0 0x7f946c0b21e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f945c009d90 tx=0x7f945c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad046fa9c86ec228 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.512+0000 7f946affd640 1 -- 172.21.15.64:0/1780739563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c02cc50 con 0x7f946c15afc0 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.512+0000 7f946affd640 1 -- 172.21.15.64:0/1780739563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c02cdb0 con 0x7f946c15afc0 2023-11-24T08:03:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.512+0000 7f946affd640 1 -- 172.21.15.64:0/1780739563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c034660 con 0x7f946c15afc0 2023-11-24T08:03:58.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.512+0000 7f94711ef640 1 -- 172.21.15.64:0/1780739563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c15afc0 msgr2=0x7f946c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:58.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.512+0000 7f94711ef640 1 --2- 172.21.15.64:0/1780739563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c15afc0 0x7f946c0b21e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f945c009d90 tx=0x7f945c02bea0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.513+0000 7f94711ef640 1 -- 172.21.15.64:0/1780739563 shutdown_connections 2023-11-24T08:03:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.513+0000 7f94711ef640 1 --2- 172.21.15.64:0/1780739563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0cbd10 0x7f946c0cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.513+0000 7f94711ef640 1 --2- 172.21.15.64:0/1780739563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c15afc0 0x7f946c0b21e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.513+0000 7f94711ef640 1 -- 172.21.15.64:0/1780739563 >> 172.21.15.64:0/1780739563 conn(0x7f946c099a00 msgr2=0x7f946c099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.513+0000 7f94711ef640 1 -- 172.21.15.64:0/1780739563 shutdown_connections 2023-11-24T08:03:58.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.513+0000 7f94711ef640 1 -- 172.21.15.64:0/1780739563 wait complete. 2023-11-24T08:03:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.514+0000 7f94711ef640 1 Processor -- start 2023-11-24T08:03:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.515+0000 7f94711ef640 1 -- start start 2023-11-24T08:03:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.515+0000 7f94711ef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c0cbd10 0x7f946c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f94711ef640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0d0d10 0x7f946c158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f94711ef640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c15afc0 0x7f946c0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:58.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f94711ef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c8a50 con 0x7f946c0cbd10 2023-11-24T08:03:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f94711ef640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c8120 con 0x7f946c15afc0 2023-11-24T08:03:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f94711ef640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c8400 con 0x7f946c0d0d10 2023-11-24T08:03:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c0cbd10 0x7f946c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c0cbd10 0x7f946c0b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36606/0 (socket says 172.21.15.64:36606) 2023-11-24T08:03:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946bfff640 1 -- 172.21.15.64:0/1481773275 learned_addr learned my addr 172.21.15.64:0/1481773275 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946b7fe640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0d0d10 0x7f946c158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f94709ee640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c15afc0 0x7f946c0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946bfff640 1 -- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0d0d10 msgr2=0x7f946c158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946bfff640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0d0d10 0x7f946c158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946bfff640 1 -- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c15afc0 msgr2=0x7f946c0c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946bfff640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c15afc0 0x7f946c0c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.516+0000 7f946bfff640 1 -- 172.21.15.64:0/1481773275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c009a70 con 0x7f946c0cbd10 2023-11-24T08:03:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.517+0000 7f946bfff640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c0cbd10 0x7f946c0b3af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f945c037040 tx=0x7f945c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.517+0000 7f9468ff9640 1 -- 172.21.15.64:0/1481773275 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c040650 con 0x7f946c0cbd10 2023-11-24T08:03:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.517+0000 7f9468ff9640 1 -- 172.21.15.64:0/1481773275 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c0407b0 con 0x7f946c0cbd10 2023-11-24T08:03:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.517+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c0c86e0 con 0x7f946c0cbd10 2023-11-24T08:03:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.517+0000 7f9468ff9640 1 -- 172.21.15.64:0/1481773275 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c040ba0 con 0x7f946c0cbd10 2023-11-24T08:03:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.517+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c158990 con 0x7f946c0cbd10 2023-11-24T08:03:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.518+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c04e840 con 0x7f946c0cbd10 2023-11-24T08:03:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.519+0000 7f9468ff9640 1 -- 172.21.15.64:0/1481773275 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f945c040d00 con 0x7f946c0cbd10 2023-11-24T08:03:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.520+0000 7f9468ff9640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f944c06ae70 0x7f944c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.520+0000 7f9468ff9640 1 -- 172.21.15.64:0/1481773275 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f945c0ac2a0 con 0x7f946c0cbd10 2023-11-24T08:03:58.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.521+0000 7f946b7fe640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f944c06ae70 0x7f944c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:58.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.522+0000 7f946b7fe640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f944c06ae70 0x7f944c06d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f946c06d140 tx=0x7f9454009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:58.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.525+0000 7f9468ff9640 1 -- 172.21.15.64:0/1481773275 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f945c077460 con 0x7f946c0cbd10 2023-11-24T08:03:58.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:58.758+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f946c0b1130 con 0x7f946c0cbd10 2023-11-24T08:03:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.262+0000 7f9468ff9640 1 -- 172.21.15.64:0/1481773275 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f945c07aa80 con 0x7f946c0cbd10 2023-11-24T08:03:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.266+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f944c06ae70 msgr2=0x7f944c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.266+0000 7f94711ef640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f944c06ae70 0x7f944c06d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f946c06d140 tx=0x7f9454009040 comp rx=0 tx=0).stop 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.266+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c0cbd10 msgr2=0x7f946c0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.266+0000 7f94711ef640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c0cbd10 0x7f946c0b3af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f945c037040 tx=0x7f945c004a40 comp rx=0 tx=0).stop 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.267+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 shutdown_connections 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.267+0000 7f94711ef640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f944c06ae70 0x7f944c06d330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.267+0000 7f94711ef640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f946c15afc0 0x7f946c0c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.267+0000 7f94711ef640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f946c0d0d10 0x7f946c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.267+0000 7f94711ef640 1 --2- 172.21.15.64:0/1481773275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f946c0cbd10 0x7f946c0b3af0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.267+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 >> 172.21.15.64:0/1481773275 conn(0x7f946c099a00 msgr2=0x7f946c0b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.267+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 shutdown_connections 2023-11-24T08:03:59.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.267+0000 7f94711ef640 1 -- 172.21.15.64:0/1481773275 wait complete. 2023-11-24T08:03:59.293 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:03:59.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:03:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.517+0000 7fa476409640 1 Processor -- start 2023-11-24T08:03:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.517+0000 7fa476409640 1 -- start start 2023-11-24T08:03:59.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa476409640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa4700cf050 0x7fa4700cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa476409640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4700cd7b0 0x7fa4700cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa476409640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47015fa70 con 0x7fa47013dfd0 2023-11-24T08:03:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa476409640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4700cfa40 con 0x7fa4700cf050 2023-11-24T08:03:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa476409640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47015c580 con 0x7fa4700cd7b0 2023-11-24T08:03:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa475407640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa47013dfd0 0x7fa47015be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56598/0 (socket says 172.21.15.64:56598) 2023-11-24T08:03:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa475407640 1 -- 172.21.15.64:0/3456828031 learned_addr learned my addr 172.21.15.64:0/3456828031 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:03:59.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa475c08640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4700cd7b0 0x7fa4700cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa474c06640 1 --2- 172.21.15.64:0/3456828031 >> v2:172.21.15.91:3300/0 conn(0x7fa4700cf050 0x7fa4700cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.519+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562744242 0 0) 0x7fa47015fa70 con 0x7fa47013dfd0 2023-11-24T08:03:59.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa45c003520 con 0x7fa47013dfd0 2023-11-24T08:03:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3529494105 0 0) 0x7fa45c003520 con 0x7fa47013dfd0 2023-11-24T08:03:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4700cd7b0 msgr2=0x7fa4700cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4700cd7b0 0x7fa4700cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 >> v2:172.21.15.91:3300/0 conn(0x7fa4700cf050 msgr2=0x7fa4700cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 --2- 172.21.15.64:0/3456828031 >> v2:172.21.15.91:3300/0 conn(0x7fa4700cf050 0x7fa4700cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470140900 con 0x7fa47013dfd0 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fa46c001690 con 0x7fa47013dfd0 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 >> v1:172.21.15.64:6789/0 conn(0x7fa47013dfd0 legacy=0x7fa47015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa45c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa45c0047c0 0x7fa45c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4700cd7b0 0x7fa45c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700cfa40 con 0x7fa45c003a10 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47015c580 con 0x7fa45c0047c0 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c00b9d0 con 0x7fa4700cd7b0 2023-11-24T08:03:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.520+0000 7fa475407640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa45c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.521+0000 7fa474c06640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa45c0047c0 0x7fa45c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.521+0000 7fa475c08640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4700cd7b0 0x7fa45c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.521+0000 7fa475407640 1 -- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4700cd7b0 msgr2=0x7fa45c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.521+0000 7fa475407640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4700cd7b0 0x7fa45c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.521+0000 7fa475407640 1 -- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa45c0047c0 msgr2=0x7fa45c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:03:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.521+0000 7fa475407640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa45c0047c0 0x7fa45c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.521+0000 7fa475407640 1 -- 172.21.15.64:0/3456828031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c003520 con 0x7fa45c003a10 2023-11-24T08:03:59.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.522+0000 7fa475407640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa45c003e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c0e0 tx=0x7fa45c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19e2533a4c2ce5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:59.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.522+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c005ad0 con 0x7fa45c003a10 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.522+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c005c30 con 0x7fa45c003a10 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.522+0000 7fa467fff640 1 -- 172.21.15.64:0/3456828031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c007070 con 0x7fa45c003a10 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.523+0000 7fa476409640 1 -- 172.21.15.64:0/3456828031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 msgr2=0x7fa45c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.523+0000 7fa476409640 1 --2- 172.21.15.64:0/3456828031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa45c003e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c0e0 tx=0x7fa45c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.523+0000 7fa475c08640 1 -- 172.21.15.64:0/3456828031 reap_dead start 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.523+0000 7fa476409640 1 -- 172.21.15.64:0/3456828031 shutdown_connections 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.523+0000 7fa476409640 1 -- 172.21.15.64:0/3456828031 >> 172.21.15.64:0/3456828031 conn(0x7fa47006a860 msgr2=0x7fa47006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.523+0000 7fa476409640 1 -- 172.21.15.64:0/3456828031 shutdown_connections 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.523+0000 7fa476409640 1 -- 172.21.15.64:0/3456828031 wait complete. 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.525+0000 7fa476409640 1 Processor -- start 2023-11-24T08:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.525+0000 7fa476409640 1 -- start start 2023-11-24T08:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.526+0000 7fa476409640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa470158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.526+0000 7fa476409640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4701593a0 0x7fa4701f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.526+0000 7fa476409640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4701f37d0 0x7fa4701f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.526+0000 7fa476409640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701f41c0 con 0x7fa45c003a10 2023-11-24T08:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.526+0000 7fa476409640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa470155700 con 0x7fa4701593a0 2023-11-24T08:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.526+0000 7fa476409640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa470159960 con 0x7fa4701f37d0 2023-11-24T08:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.526+0000 7fa475407640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa470158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475407640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa470158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36628/0 (socket says 172.21.15.64:36628) 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475407640 1 -- 172.21.15.64:0/2344482492 learned_addr learned my addr 172.21.15.64:0/2344482492 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475c08640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4701f37d0 0x7fa4701f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa474c06640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4701593a0 0x7fa4701f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475407640 1 -- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4701f37d0 msgr2=0x7fa4701f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475407640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4701f37d0 0x7fa4701f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475407640 1 -- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4701593a0 msgr2=0x7fa4701f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475407640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4701593a0 0x7fa4701f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475407640 1 -- 172.21.15.64:0/2344482492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470159c40 con 0x7fa45c003a10 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475c08640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4701f37d0 0x7fa4701f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa474c06640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4701593a0 0x7fa4701f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.527+0000 7fa475407640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa470158e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c0e0 tx=0x7fa470140b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.528+0000 7fa465ffb640 1 -- 172.21.15.64:0/2344482492 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c007480 con 0x7fa45c003a10 2023-11-24T08:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.528+0000 7fa465ffb640 1 -- 172.21.15.64:0/2344482492 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c0075e0 con 0x7fa45c003a10 2023-11-24T08:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.528+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa470159f20 con 0x7fa45c003a10 2023-11-24T08:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.528+0000 7fa465ffb640 1 -- 172.21.15.64:0/2344482492 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c02c990 con 0x7fa45c003a10 2023-11-24T08:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.528+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47015a4e0 con 0x7fa45c003a10 2023-11-24T08:03:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.529+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47004e840 con 0x7fa45c003a10 2023-11-24T08:03:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.530+0000 7fa465ffb640 1 -- 172.21.15.64:0/2344482492 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa46c032070 con 0x7fa45c003a10 2023-11-24T08:03:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.531+0000 7fa465ffb640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa45406ae70 0x7fa45406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:03:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.531+0000 7fa465ffb640 1 -- 172.21.15.64:0/2344482492 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa46c0aa510 con 0x7fa45c003a10 2023-11-24T08:03:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.531+0000 7fa474c06640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa45406ae70 0x7fa45406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:03:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.532+0000 7fa474c06640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa45406ae70 0x7fa45406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa470155cb0 tx=0x7fa460008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:03:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.535+0000 7fa465ffb640 1 -- 172.21.15.64:0/2344482492 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa46c075520 con 0x7fa45c003a10 2023-11-24T08:03:59.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.768+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa47006e0a0 con 0x7fa45c003a10 2023-11-24T08:03:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.770+0000 7fa465ffb640 1 -- 172.21.15.64:0/2344482492 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fa46c007750 con 0x7fa45c003a10 2023-11-24T08:03:59.771 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:03:59.771 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:03:55.699656+0000","modified":"2023-11-24T08:03:58.759467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:03:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T08:03:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa45406ae70 msgr2=0x7fa45406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa45406ae70 0x7fa45406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa470155cb0 tx=0x7fa460008040 comp rx=0 tx=0).stop 2023-11-24T08:03:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 msgr2=0x7fa470158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:03:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa470158e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c0e0 tx=0x7fa470140b50 comp rx=0 tx=0).stop 2023-11-24T08:03:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 shutdown_connections 2023-11-24T08:03:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa45406ae70 0x7fa45406d330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4701f37d0 0x7fa4701f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4701593a0 0x7fa4701f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 --2- 172.21.15.64:0/2344482492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c003a10 0x7fa470158e60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:03:59.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 >> 172.21.15.64:0/2344482492 conn(0x7fa47006a860 msgr2=0x7fa4700afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:03:59.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 shutdown_connections 2023-11-24T08:03:59.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:03:59.773+0000 7fa476409640 1 -- 172.21.15.64:0/2344482492 wait complete. 2023-11-24T08:03:59.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.016+0000 7fb886b3e640 1 Processor -- start 2023-11-24T08:04:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.016+0000 7fb886b3e640 1 -- start start 2023-11-24T08:04:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.018+0000 7fb886b3e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb8800d2920 0x7fb8800d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.018+0000 7fb886b3e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8800d1d50 con 0x7fb8800b1d20 2023-11-24T08:04:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.018+0000 7fb886b3e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8800d2030 con 0x7fb8800b4c80 2023-11-24T08:04:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.018+0000 7fb886b3e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8800d2310 con 0x7fb8800d2920 2023-11-24T08:04:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.018+0000 7fb8850db640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb8800b1d20 0x7fb8800b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56614/0 (socket says 172.21.15.64:56614) 2023-11-24T08:04:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.018+0000 7fb8850db640 1 -- 172.21.15.64:0/3123540650 learned_addr learned my addr 172.21.15.64:0/3123540650 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.018+0000 7fb87ffff640 1 --2- 172.21.15.64:0/3123540650 >> v2:172.21.15.91:3300/0 conn(0x7fb8800d2920 0x7fb8800d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247064550 0 0) 0x7fb8800d1d50 con 0x7fb8800b1d20 2023-11-24T08:04:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86c003520 con 0x7fb8800b1d20 2023-11-24T08:04:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1885706762 0 0) 0x7fb86c003520 con 0x7fb8800b1d20 2023-11-24T08:04:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 >> v2:172.21.15.91:3300/0 conn(0x7fb8800d2920 msgr2=0x7fb8800d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 --2- 172.21.15.64:0/3123540650 >> v2:172.21.15.91:3300/0 conn(0x7fb8800d2920 0x7fb8800d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 >> v1:172.21.15.91:6789/0 conn(0x7fb8800b4c80 legacy=0x7fb8800b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8800b65e0 con 0x7fb8800b1d20 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fb878001670 con 0x7fb8800b1d20 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 >> v1:172.21.15.64:6789/0 conn(0x7fb8800b1d20 legacy=0x7fb8800b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb86c0039c0 0x7fb86c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb86c0051d0 0x7fb86c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800d2310 con 0x7fb86c0047c0 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c00c120 con 0x7fb86c0039c0 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c00c2e0 con 0x7fb86c0051d0 2023-11-24T08:04:00.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.019+0000 7fb87ffff640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.020+0000 7fb8848da640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb86c0039c0 0x7fb86c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.020+0000 7fb87ffff640 1 -- 172.21.15.64:0/3123540650 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb86c0051d0 msgr2=0x7fb86c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.020+0000 7fb87ffff640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb86c0051d0 0x7fb86c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.020+0000 7fb87ffff640 1 -- 172.21.15.64:0/3123540650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb86c0039c0 msgr2=0x7fb86c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.020+0000 7fb87ffff640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb86c0039c0 0x7fb86c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.020+0000 7fb87ffff640 1 -- 172.21.15.64:0/3123540650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c003520 con 0x7fb86c0047c0 2023-11-24T08:04:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.020+0000 7fb8848da640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb86c0039c0 0x7fb86c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.020+0000 7fb8850db640 1 -- 172.21.15.64:0/3123540650 reap_dead start 2023-11-24T08:04:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.021+0000 7fb87ffff640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb8800d2d40 tx=0x7fb86c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa79ecb8cb6bd81 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.021+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86802c660 con 0x7fb86c0047c0 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.021+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86802c7c0 con 0x7fb86c0047c0 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.021+0000 7fb87f7fe640 1 -- 172.21.15.64:0/3123540650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86802cb50 con 0x7fb86c0047c0 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.022+0000 7fb886b3e640 1 -- 172.21.15.64:0/3123540650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 msgr2=0x7fb86c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.022+0000 7fb886b3e640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb8800d2d40 tx=0x7fb86c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.022+0000 7fb886b3e640 1 -- 172.21.15.64:0/3123540650 shutdown_connections 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.022+0000 7fb886b3e640 1 --2- 172.21.15.64:0/3123540650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.022+0000 7fb886b3e640 1 -- 172.21.15.64:0/3123540650 >> 172.21.15.64:0/3123540650 conn(0x7fb8800721a0 msgr2=0x7fb8800cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.022+0000 7fb886b3e640 1 -- 172.21.15.64:0/3123540650 shutdown_connections 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.022+0000 7fb886b3e640 1 -- 172.21.15.64:0/3123540650 wait complete. 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.024+0000 7fb886b3e640 1 Processor -- start 2023-11-24T08:04:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.024+0000 7fb886b3e640 1 -- start start 2023-11-24T08:04:00.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb886b3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb8801f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb886b3e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb88015f350 0x7fb8801f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb886b3e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8801f6330 0x7fb8801f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb886b3e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800c9a60 con 0x7fb86c0047c0 2023-11-24T08:04:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb886b3e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88015f7b0 con 0x7fb88015f350 2023-11-24T08:04:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb886b3e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8801f6cf0 con 0x7fb8801f6330 2023-11-24T08:04:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb8848da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb8801f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb8848da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb8801f2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36646/0 (socket says 172.21.15.64:36646) 2023-11-24T08:04:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.025+0000 7fb8848da640 1 -- 172.21.15.64:0/4125047586 learned_addr learned my addr 172.21.15.64:0/4125047586 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb8850db640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8801f6330 0x7fb8801f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb8848da640 1 -- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8801f6330 msgr2=0x7fb8801f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb87ffff640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb88015f350 0x7fb8801f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb8848da640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8801f6330 0x7fb8801f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb8848da640 1 -- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb88015f350 msgr2=0x7fb8801f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb8848da640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb88015f350 0x7fb8801f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb8848da640 1 -- 172.21.15.64:0/4125047586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8801605d0 con 0x7fb86c0047c0 2023-11-24T08:04:00.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb8850db640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8801f6330 0x7fb8801f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:00.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb8848da640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb8801f2850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb87000db10 tx=0x7fb8700020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:00.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb87d7fa640 1 -- 172.21.15.64:0/4125047586 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870003da0 con 0x7fb86c0047c0 2023-11-24T08:04:00.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb87d7fa640 1 -- 172.21.15.64:0/4125047586 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb870006b50 con 0x7fb86c0047c0 2023-11-24T08:04:00.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8801f3d00 con 0x7fb86c0047c0 2023-11-24T08:04:00.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.026+0000 7fb87d7fa640 1 -- 172.21.15.64:0/4125047586 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87000b6e0 con 0x7fb86c0047c0 2023-11-24T08:04:00.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.027+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb880160ab0 con 0x7fb86c0047c0 2023-11-24T08:04:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.028+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb880074770 con 0x7fb86c0047c0 2023-11-24T08:04:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.029+0000 7fb87d7fa640 1 -- 172.21.15.64:0/4125047586 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb87000b840 con 0x7fb86c0047c0 2023-11-24T08:04:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.030+0000 7fb87d7fa640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.030+0000 7fb87d7fa640 1 -- 172.21.15.64:0/4125047586 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb870088ab0 con 0x7fb86c0047c0 2023-11-24T08:04:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.030+0000 7fb87ffff640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.031+0000 7fb87ffff640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb868036000 tx=0x7fb868030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.034+0000 7fb87d7fa640 1 -- 172.21.15.64:0/4125047586 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb870053c40 con 0x7fb86c0047c0 2023-11-24T08:04:00.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.231+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8800d2f70 con 0x7fb86c0047c0 2023-11-24T08:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.234+0000 7fb87d7fa640 1 -- 172.21.15.64:0/4125047586 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fb870057260 con 0x7fb86c0047c0 2023-11-24T08:04:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:00.235 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 msgr2=0x7fb86006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb868036000 tx=0x7fb868030000 comp rx=0 tx=0).stop 2023-11-24T08:04:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 msgr2=0x7fb8801f2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb8801f2850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb87000db10 tx=0x7fb8700020c0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 shutdown_connections 2023-11-24T08:04:00.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8801f6330 0x7fb8801f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb88015f350 0x7fb8801f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 --2- 172.21.15.64:0/4125047586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb86c0047c0 0x7fb8801f2850 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 >> 172.21.15.64:0/4125047586 conn(0x7fb8800721a0 msgr2=0x7fb8800b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 shutdown_connections 2023-11-24T08:04:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.237+0000 7fb886b3e640 1 -- 172.21.15.64:0/4125047586 wait complete. 2023-11-24T08:04:00.258 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:04:00.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:04:00.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.478+0000 7f2d5ad33640 1 Processor -- start 2023-11-24T08:04:00.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.478+0000 7f2d5ad33640 1 -- start start 2023-11-24T08:04:00.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d5ad33640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d5415a100 0x7f2d540b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d5ad33640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2d540abff0 0x7f2d540b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d5ad33640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d540ae7e0 0x7f2d5406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d5ad33640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d540ac840 con 0x7f2d540ae7e0 2023-11-24T08:04:00.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d5ad33640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5413da30 con 0x7f2d540abff0 2023-11-24T08:04:00.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d5ad33640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540ac560 con 0x7f2d5415a100 2023-11-24T08:04:00.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d592d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d540ae7e0 0x7f2d5406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d592d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d540ae7e0 0x7f2d5406d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36648/0 (socket says 172.21.15.64:36648) 2023-11-24T08:04:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.480+0000 7f2d592d0640 1 -- 172.21.15.64:0/385474609 learned_addr learned my addr 172.21.15.64:0/385474609 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.481+0000 7f2d53fff640 1 --2- 172.21.15.64:0/385474609 >> v2:172.21.15.91:3300/0 conn(0x7f2d540abff0 0x7f2d540b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.481+0000 7f2d58acf640 1 --2- 172.21.15.64:0/385474609 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d5415a100 0x7f2d540b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.481+0000 7f2d592d0640 1 -- 172.21.15.64:0/385474609 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d5415a100 msgr2=0x7f2d540b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.481+0000 7f2d592d0640 1 --2- 172.21.15.64:0/385474609 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d5415a100 0x7f2d540b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.481+0000 7f2d592d0640 1 -- 172.21.15.64:0/385474609 >> v2:172.21.15.91:3300/0 conn(0x7f2d540abff0 msgr2=0x7f2d540b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.481+0000 7f2d592d0640 1 --2- 172.21.15.64:0/385474609 >> v2:172.21.15.91:3300/0 conn(0x7f2d540abff0 0x7f2d540b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.481+0000 7f2d592d0640 1 -- 172.21.15.64:0/385474609 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d54070650 con 0x7f2d540ae7e0 2023-11-24T08:04:00.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.482+0000 7f2d592d0640 1 --2- 172.21.15.64:0/385474609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d540ae7e0 0x7f2d5406d520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d5406b420 tx=0x7f2d4c02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac53d4fcb63b8471 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:00.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.482+0000 7f2d537fe640 1 -- 172.21.15.64:0/385474609 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c02e9f0 con 0x7f2d540ae7e0 2023-11-24T08:04:00.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.482+0000 7f2d537fe640 1 -- 172.21.15.64:0/385474609 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c02eb50 con 0x7f2d540ae7e0 2023-11-24T08:04:00.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.482+0000 7f2d537fe640 1 -- 172.21.15.64:0/385474609 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c009600 con 0x7f2d540ae7e0 2023-11-24T08:04:00.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 -- 172.21.15.64:0/385474609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d540ae7e0 msgr2=0x7f2d5406d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/385474609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d540ae7e0 0x7f2d5406d520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d5406b420 tx=0x7f2d4c02e050 comp rx=0 tx=0).stop 2023-11-24T08:04:00.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 -- 172.21.15.64:0/385474609 shutdown_connections 2023-11-24T08:04:00.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/385474609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d540ae7e0 0x7f2d5406d520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/385474609 >> v2:172.21.15.91:3300/0 conn(0x7f2d540abff0 0x7f2d540b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/385474609 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d5415a100 0x7f2d540b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 -- 172.21.15.64:0/385474609 >> 172.21.15.64:0/385474609 conn(0x7f2d540b0d80 msgr2=0x7f2d540b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:00.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 -- 172.21.15.64:0/385474609 shutdown_connections 2023-11-24T08:04:00.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.483+0000 7f2d5ad33640 1 -- 172.21.15.64:0/385474609 wait complete. 2023-11-24T08:04:00.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.485+0000 7f2d5ad33640 1 Processor -- start 2023-11-24T08:04:00.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.485+0000 7f2d5ad33640 1 -- start start 2023-11-24T08:04:00.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d5ad33640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d540abff0 0x7f2d54095760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d5ad33640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d540ae7e0 0x7f2d540981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d5ad33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5415a100 0x7f2d541f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d5ad33640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54098700 con 0x7f2d5415a100 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d5ad33640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d541f2ae0 con 0x7f2d540abff0 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d5ad33640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5415a700 con 0x7f2d540ae7e0 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d592d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5415a100 0x7f2d541f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d592d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5415a100 0x7f2d541f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36662/0 (socket says 172.21.15.64:36662) 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d592d0640 1 -- 172.21.15.64:0/710820436 learned_addr learned my addr 172.21.15.64:0/710820436 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d58acf640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d540abff0 0x7f2d54095760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d53fff640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d540ae7e0 0x7f2d540981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d592d0640 1 -- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d540ae7e0 msgr2=0x7f2d540981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d592d0640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d540ae7e0 0x7f2d540981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d592d0640 1 -- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d540abff0 msgr2=0x7f2d54095760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d592d0640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d540abff0 0x7f2d54095760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.486+0000 7f2d592d0640 1 -- 172.21.15.64:0/710820436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c00bd40 con 0x7f2d5415a100 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.487+0000 7f2d53fff640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d540ae7e0 0x7f2d540981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.487+0000 7f2d592d0640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5415a100 0x7f2d541f25a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c038040 tx=0x7f2d4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.487+0000 7f2d517fa640 1 -- 172.21.15.64:0/710820436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c02e9f0 con 0x7f2d5415a100 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.487+0000 7f2d517fa640 1 -- 172.21.15.64:0/710820436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c036800 con 0x7f2d5415a100 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.487+0000 7f2d517fa640 1 -- 172.21.15.64:0/710820436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c036bb0 con 0x7f2d5415a100 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.487+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5415a9e0 con 0x7f2d5415a100 2023-11-24T08:04:00.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.487+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d540b0a80 con 0x7f2d5415a100 2023-11-24T08:04:00.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.488+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5415c2b0 con 0x7f2d5415a100 2023-11-24T08:04:00.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.489+0000 7f2d517fa640 1 -- 172.21.15.64:0/710820436 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d4c036d10 con 0x7f2d5415a100 2023-11-24T08:04:00.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.491+0000 7f2d517fa640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d3806ae70 0x7f2d3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:00.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.491+0000 7f2d517fa640 1 -- 172.21.15.64:0/710820436 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d4c0ac990 con 0x7f2d5415a100 2023-11-24T08:04:00.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.491+0000 7f2d58acf640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d3806ae70 0x7f2d3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:00.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.492+0000 7f2d58acf640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d3806ae70 0x7f2d3806d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800a8f0 tx=0x7f2d48008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:00.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.495+0000 7f2d517fa640 1 -- 172.21.15.64:0/710820436 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d5415c2b0 con 0x7f2d5415a100 2023-11-24T08:04:00.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.689+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d5415a4e0 con 0x7f2d3806ae70 2023-11-24T08:04:00.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.714+0000 7f2d517fa640 1 -- 172.21.15.64:0/710820436 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264426 (secure 0 0 0) 0x7f2d4800c050 con 0x7f2d3806ae70 2023-11-24T08:04:00.716 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:04:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d3806ae70 msgr2=0x7f2d3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d3806ae70 0x7f2d3806d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800a8f0 tx=0x7f2d48008040 comp rx=0 tx=0).stop 2023-11-24T08:04:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5415a100 msgr2=0x7f2d541f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5415a100 0x7f2d541f25a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c038040 tx=0x7f2d4c004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 shutdown_connections 2023-11-24T08:04:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d3806ae70 0x7f2d3806d330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d5415a100 0x7f2d541f25a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d540ae7e0 0x7f2d540981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 --2- 172.21.15.64:0/710820436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d540abff0 0x7f2d54095760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 >> 172.21.15.64:0/710820436 conn(0x7f2d540b0d80 msgr2=0x7f2d540b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 shutdown_connections 2023-11-24T08:04:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:00.721+0000 7f2d5ad33640 1 -- 172.21.15.64:0/710820436 wait complete. 2023-11-24T08:04:00.743 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-24T08:04:00.012440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805499"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265413+0000","last_change":"2023-11-24T08:03:55.977134+0000","last_active":"2023-11-24T08:03:57.265413+0000","last_peered":"2023-11-24T08:03:57.265413+0000","last_clean":"2023-11-24T08:03:57.265413+0000","last_became_active":"2023-11-24T08:03:55.970402+0000","last_became_peered":"2023-11-24T08:03:55.970402+0000","last_unstale":"2023-11-24T08:03:57.265413+0000","last_undegraded":"2023-11-24T08:03:57.265413+0000","last_fullsized":"2023-11-24T08:03:57.265413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:47.677166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949561+0000","last_change":"2023-11-24T08:03:55.973687+0000","last_active":"2023-11-24T08:03:56.949561+0000","last_peered":"2023-11-24T08:03:56.949561+0000","last_clean":"2023-11-24T08:03:56.949561+0000","last_became_active":"2023-11-24T08:03:55.973369+0000","last_became_peered":"2023-11-24T08:03:55.973369+0000","last_unstale":"2023-11-24T08:03:56.949561+0000","last_undegraded":"2023-11-24T08:03:56.949561+0000","last_fullsized":"2023-11-24T08:03:56.949561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449190+0000","last_change":"2023-11-24T08:03:55.971194+0000","last_active":"2023-11-24T08:03:57.449190+0000","last_peered":"2023-11-24T08:03:57.449190+0000","last_clean":"2023-11-24T08:03:57.449190+0000","last_became_active":"2023-11-24T08:03:55.970751+0000","last_became_peered":"2023-11-24T08:03:55.970751+0000","last_unstale":"2023-11-24T08:03:57.449190+0000","last_undegraded":"2023-11-24T08:03:57.449190+0000","last_fullsized":"2023-11-24T08:03:57.449190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994604+0000","last_change":"2023-11-24T08:03:55.984822+0000","last_active":"2023-11-24T08:03:56.994604+0000","last_peered":"2023-11-24T08:03:56.994604+0000","last_clean":"2023-11-24T08:03:56.994604+0000","last_became_active":"2023-11-24T08:03:55.982969+0000","last_became_peered":"2023-11-24T08:03:55.982969+0000","last_unstale":"2023-11-24T08:03:56.994604+0000","last_undegraded":"2023-11-24T08:03:56.994604+0000","last_fullsized":"2023-11-24T08:03:56.994604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:43.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264555+0000","last_change":"2023-11-24T08:03:55.980587+0000","last_active":"2023-11-24T08:03:57.264555+0000","last_peered":"2023-11-24T08:03:57.264555+0000","last_clean":"2023-11-24T08:03:57.264555+0000","last_became_active":"2023-11-24T08:03:55.980331+0000","last_became_peered":"2023-11-24T08:03:55.980331+0000","last_unstale":"2023-11-24T08:03:57.264555+0000","last_undegraded":"2023-11-24T08:03:57.264555+0000","last_fullsized":"2023-11-24T08:03:57.264555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:16.719789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994145+0000","last_change":"2023-11-24T08:03:55.981917+0000","last_active":"2023-11-24T08:03:56.994145+0000","last_peered":"2023-11-24T08:03:56.994145+0000","last_clean":"2023-11-24T08:03:56.994145+0000","last_became_active":"2023-11-24T08:03:55.981300+0000","last_became_peered":"2023-11-24T08:03:55.981300+0000","last_unstale":"2023-11-24T08:03:56.994145+0000","last_undegraded":"2023-11-24T08:03:56.994145+0000","last_fullsized":"2023-11-24T08:03:56.994145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263992+0000","last_change":"2023-11-24T08:03:55.975783+0000","last_active":"2023-11-24T08:03:57.263992+0000","last_peered":"2023-11-24T08:03:57.263992+0000","last_clean":"2023-11-24T08:03:57.263992+0000","last_became_active":"2023-11-24T08:03:55.975452+0000","last_became_peered":"2023-11-24T08:03:55.975452+0000","last_unstale":"2023-11-24T08:03:57.263992+0000","last_undegraded":"2023-11-24T08:03:57.263992+0000","last_fullsized":"2023-11-24T08:03:57.263992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952790+0000","last_change":"2023-11-24T08:03:55.980610+0000","last_active":"2023-11-24T08:03:56.952790+0000","last_peered":"2023-11-24T08:03:56.952790+0000","last_clean":"2023-11-24T08:03:56.952790+0000","last_became_active":"2023-11-24T08:03:55.980222+0000","last_became_peered":"2023-11-24T08:03:55.980222+0000","last_unstale":"2023-11-24T08:03:56.952790+0000","last_undegraded":"2023-11-24T08:03:56.952790+0000","last_fullsized":"2023-11-24T08:03:56.952790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265494+0000","last_change":"2023-11-24T08:03:55.988607+0000","last_active":"2023-11-24T08:03:57.265494+0000","last_peered":"2023-11-24T08:03:57.265494+0000","last_clean":"2023-11-24T08:03:57.265494+0000","last_became_active":"2023-11-24T08:03:55.988074+0000","last_became_peered":"2023-11-24T08:03:55.988074+0000","last_unstale":"2023-11-24T08:03:57.265494+0000","last_undegraded":"2023-11-24T08:03:57.265494+0000","last_fullsized":"2023-11-24T08:03:57.265494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:08.202382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449121+0000","last_change":"2023-11-24T08:03:55.972034+0000","last_active":"2023-11-24T08:03:57.449121+0000","last_peered":"2023-11-24T08:03:57.449121+0000","last_clean":"2023-11-24T08:03:57.449121+0000","last_became_active":"2023-11-24T08:03:55.971627+0000","last_became_peered":"2023-11-24T08:03:55.971627+0000","last_unstale":"2023-11-24T08:03:57.449121+0000","last_undegraded":"2023-11-24T08:03:57.449121+0000","last_fullsized":"2023-11-24T08:03:57.449121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448738+0000","last_change":"2023-11-24T08:03:55.979788+0000","last_active":"2023-11-24T08:03:57.448738+0000","last_peered":"2023-11-24T08:03:57.448738+0000","last_clean":"2023-11-24T08:03:57.448738+0000","last_became_active":"2023-11-24T08:03:55.979429+0000","last_became_peered":"2023-11-24T08:03:55.979429+0000","last_unstale":"2023-11-24T08:03:57.448738+0000","last_undegraded":"2023-11-24T08:03:57.448738+0000","last_fullsized":"2023-11-24T08:03:57.448738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953475+0000","last_change":"2023-11-24T08:03:55.975330+0000","last_active":"2023-11-24T08:03:56.953475+0000","last_peered":"2023-11-24T08:03:56.953475+0000","last_clean":"2023-11-24T08:03:56.953475+0000","last_became_active":"2023-11-24T08:03:55.974721+0000","last_became_peered":"2023-11-24T08:03:55.974721+0000","last_unstale":"2023-11-24T08:03:56.953475+0000","last_undegraded":"2023-11-24T08:03:56.953475+0000","last_fullsized":"2023-11-24T08:03:56.953475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:35.456457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953384+0000","last_change":"2023-11-24T08:03:55.975039+0000","last_active":"2023-11-24T08:03:56.953384+0000","last_peered":"2023-11-24T08:03:56.953384+0000","last_clean":"2023-11-24T08:03:56.953384+0000","last_became_active":"2023-11-24T08:03:55.974171+0000","last_became_peered":"2023-11-24T08:03:55.974171+0000","last_unstale":"2023-11-24T08:03:56.953384+0000","last_undegraded":"2023-11-24T08:03:56.953384+0000","last_fullsized":"2023-11-24T08:03:56.953384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448417+0000","last_change":"2023-11-24T08:03:55.972138+0000","last_active":"2023-11-24T08:03:57.448417+0000","last_peered":"2023-11-24T08:03:57.448417+0000","last_clean":"2023-11-24T08:03:57.448417+0000","last_became_active":"2023-11-24T08:03:55.968993+0000","last_became_peered":"2023-11-24T08:03:55.968993+0000","last_unstale":"2023-11-24T08:03:57.448417+0000","last_undegraded":"2023-11-24T08:03:57.448417+0000","last_fullsized":"2023-11-24T08:03:57.448417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949358+0000","last_change":"2023-11-24T08:03:55.961573+0000","last_active":"2023-11-24T08:03:56.949358+0000","last_peered":"2023-11-24T08:03:56.949358+0000","last_clean":"2023-11-24T08:03:56.949358+0000","last_became_active":"2023-11-24T08:03:55.961135+0000","last_became_peered":"2023-11-24T08:03:55.961135+0000","last_unstale":"2023-11-24T08:03:56.949358+0000","last_undegraded":"2023-11-24T08:03:56.949358+0000","last_fullsized":"2023-11-24T08:03:56.949358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994390+0000","last_change":"2023-11-24T08:03:55.982805+0000","last_active":"2023-11-24T08:03:56.994390+0000","last_peered":"2023-11-24T08:03:56.994390+0000","last_clean":"2023-11-24T08:03:56.994390+0000","last_became_active":"2023-11-24T08:03:55.980530+0000","last_became_peered":"2023-11-24T08:03:55.980530+0000","last_unstale":"2023-11-24T08:03:56.994390+0000","last_undegraded":"2023-11-24T08:03:56.994390+0000","last_fullsized":"2023-11-24T08:03:56.994390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995521+0000","last_change":"2023-11-24T08:03:55.988078+0000","last_active":"2023-11-24T08:03:56.995521+0000","last_peered":"2023-11-24T08:03:56.995521+0000","last_clean":"2023-11-24T08:03:56.995521+0000","last_became_active":"2023-11-24T08:03:55.987802+0000","last_became_peered":"2023-11-24T08:03:55.987802+0000","last_unstale":"2023-11-24T08:03:56.995521+0000","last_undegraded":"2023-11-24T08:03:56.995521+0000","last_fullsized":"2023-11-24T08:03:56.995521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:19.891560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445715+0000","last_change":"2023-11-24T08:03:55.968360+0000","last_active":"2023-11-24T08:03:57.445715+0000","last_peered":"2023-11-24T08:03:57.445715+0000","last_clean":"2023-11-24T08:03:57.445715+0000","last_became_active":"2023-11-24T08:03:55.968089+0000","last_became_peered":"2023-11-24T08:03:55.968089+0000","last_unstale":"2023-11-24T08:03:57.445715+0000","last_undegraded":"2023-11-24T08:03:57.445715+0000","last_fullsized":"2023-11-24T08:03:57.445715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:04.042778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264933+0000","last_change":"2023-11-24T08:03:55.971957+0000","last_active":"2023-11-24T08:03:57.264933+0000","last_peered":"2023-11-24T08:03:57.264933+0000","last_clean":"2023-11-24T08:03:57.264933+0000","last_became_active":"2023-11-24T08:03:55.971638+0000","last_became_peered":"2023-11-24T08:03:55.971638+0000","last_unstale":"2023-11-24T08:03:57.264933+0000","last_undegraded":"2023-11-24T08:03:57.264933+0000","last_fullsized":"2023-11-24T08:03:57.264933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:01.176228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448754+0000","last_change":"2023-11-24T08:03:55.976704+0000","last_active":"2023-11-24T08:03:57.448754+0000","last_peered":"2023-11-24T08:03:57.448754+0000","last_clean":"2023-11-24T08:03:57.448754+0000","last_became_active":"2023-11-24T08:03:55.976410+0000","last_became_peered":"2023-11-24T08:03:55.976410+0000","last_unstale":"2023-11-24T08:03:57.448754+0000","last_undegraded":"2023-11-24T08:03:57.448754+0000","last_fullsized":"2023-11-24T08:03:57.448754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448488+0000","last_change":"2023-11-24T08:03:55.985384+0000","last_active":"2023-11-24T08:03:57.448488+0000","last_peered":"2023-11-24T08:03:57.448488+0000","last_clean":"2023-11-24T08:03:57.448488+0000","last_became_active":"2023-11-24T08:03:55.985155+0000","last_became_peered":"2023-11-24T08:03:55.985155+0000","last_unstale":"2023-11-24T08:03:57.448488+0000","last_undegraded":"2023-11-24T08:03:57.448488+0000","last_fullsized":"2023-11-24T08:03:57.448488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994409+0000","last_change":"2023-11-24T08:03:55.982133+0000","last_active":"2023-11-24T08:03:56.994409+0000","last_peered":"2023-11-24T08:03:56.994409+0000","last_clean":"2023-11-24T08:03:56.994409+0000","last_became_active":"2023-11-24T08:03:55.981629+0000","last_became_peered":"2023-11-24T08:03:55.981629+0000","last_unstale":"2023-11-24T08:03:56.994409+0000","last_undegraded":"2023-11-24T08:03:56.994409+0000","last_fullsized":"2023-11-24T08:03:56.994409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952961+0000","last_change":"2023-11-24T08:03:55.963916+0000","last_active":"2023-11-24T08:03:56.952961+0000","last_peered":"2023-11-24T08:03:56.952961+0000","last_clean":"2023-11-24T08:03:56.952961+0000","last_became_active":"2023-11-24T08:03:55.958460+0000","last_became_peered":"2023-11-24T08:03:55.958460+0000","last_unstale":"2023-11-24T08:03:56.952961+0000","last_undegraded":"2023-11-24T08:03:56.952961+0000","last_fullsized":"2023-11-24T08:03:56.952961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994284+0000","last_change":"2023-11-24T08:03:55.980845+0000","last_active":"2023-11-24T08:03:56.994284+0000","last_peered":"2023-11-24T08:03:56.994284+0000","last_clean":"2023-11-24T08:03:56.994284+0000","last_became_active":"2023-11-24T08:03:55.979729+0000","last_became_peered":"2023-11-24T08:03:55.979729+0000","last_unstale":"2023-11-24T08:03:56.994284+0000","last_undegraded":"2023-11-24T08:03:56.994284+0000","last_fullsized":"2023-11-24T08:03:56.994284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953148+0000","last_change":"2023-11-24T08:03:55.956896+0000","last_active":"2023-11-24T08:03:56.953148+0000","last_peered":"2023-11-24T08:03:56.953148+0000","last_clean":"2023-11-24T08:03:56.953148+0000","last_became_active":"2023-11-24T08:03:55.956452+0000","last_became_peered":"2023-11-24T08:03:55.956452+0000","last_unstale":"2023-11-24T08:03:56.953148+0000","last_undegraded":"2023-11-24T08:03:56.953148+0000","last_fullsized":"2023-11-24T08:03:56.953148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449134+0000","last_change":"2023-11-24T08:03:55.968031+0000","last_active":"2023-11-24T08:03:57.449134+0000","last_peered":"2023-11-24T08:03:57.449134+0000","last_clean":"2023-11-24T08:03:57.449134+0000","last_became_active":"2023-11-24T08:03:55.967733+0000","last_became_peered":"2023-11-24T08:03:55.967733+0000","last_unstale":"2023-11-24T08:03:57.449134+0000","last_undegraded":"2023-11-24T08:03:57.449134+0000","last_fullsized":"2023-11-24T08:03:57.449134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949553+0000","last_change":"2023-11-24T08:03:55.971172+0000","last_active":"2023-11-24T08:03:56.949553+0000","last_peered":"2023-11-24T08:03:56.949553+0000","last_clean":"2023-11-24T08:03:56.949553+0000","last_became_active":"2023-11-24T08:03:55.970829+0000","last_became_peered":"2023-11-24T08:03:55.970829+0000","last_unstale":"2023-11-24T08:03:56.949553+0000","last_undegraded":"2023-11-24T08:03:56.949553+0000","last_fullsized":"2023-11-24T08:03:56.949553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953446+0000","last_change":"2023-11-24T08:03:55.965485+0000","last_active":"2023-11-24T08:03:56.953446+0000","last_peered":"2023-11-24T08:03:56.953446+0000","last_clean":"2023-11-24T08:03:56.953446+0000","last_became_active":"2023-11-24T08:03:55.965220+0000","last_became_peered":"2023-11-24T08:03:55.965220+0000","last_unstale":"2023-11-24T08:03:56.953446+0000","last_undegraded":"2023-11-24T08:03:56.953446+0000","last_fullsized":"2023-11-24T08:03:56.953446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:43.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448732+0000","last_change":"2023-11-24T08:03:55.978834+0000","last_active":"2023-11-24T08:03:57.448732+0000","last_peered":"2023-11-24T08:03:57.448732+0000","last_clean":"2023-11-24T08:03:57.448732+0000","last_became_active":"2023-11-24T08:03:55.978604+0000","last_became_peered":"2023-11-24T08:03:55.978604+0000","last_unstale":"2023-11-24T08:03:57.448732+0000","last_undegraded":"2023-11-24T08:03:57.448732+0000","last_fullsized":"2023-11-24T08:03:57.448732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994520+0000","last_change":"2023-11-24T08:03:55.982758+0000","last_active":"2023-11-24T08:03:56.994520+0000","last_peered":"2023-11-24T08:03:56.994520+0000","last_clean":"2023-11-24T08:03:56.994520+0000","last_became_active":"2023-11-24T08:03:55.978909+0000","last_became_peered":"2023-11-24T08:03:55.978909+0000","last_unstale":"2023-11-24T08:03:56.994520+0000","last_undegraded":"2023-11-24T08:03:56.994520+0000","last_fullsized":"2023-11-24T08:03:56.994520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:55.841729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445961+0000","last_change":"2023-11-24T08:03:55.972047+0000","last_active":"2023-11-24T08:03:57.445961+0000","last_peered":"2023-11-24T08:03:57.445961+0000","last_clean":"2023-11-24T08:03:57.445961+0000","last_became_active":"2023-11-24T08:03:55.971770+0000","last_became_peered":"2023-11-24T08:03:55.971770+0000","last_unstale":"2023-11-24T08:03:57.445961+0000","last_undegraded":"2023-11-24T08:03:57.445961+0000","last_fullsized":"2023-11-24T08:03:57.445961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:35.456457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448439+0000","last_change":"2023-11-24T08:03:55.974157+0000","last_active":"2023-11-24T08:03:57.448439+0000","last_peered":"2023-11-24T08:03:57.448439+0000","last_clean":"2023-11-24T08:03:57.448439+0000","last_became_active":"2023-11-24T08:03:55.973864+0000","last_became_peered":"2023-11-24T08:03:55.973864+0000","last_unstale":"2023-11-24T08:03:57.448439+0000","last_undegraded":"2023-11-24T08:03:57.448439+0000","last_fullsized":"2023-11-24T08:03:57.448439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952874+0000","last_change":"2023-11-24T08:03:55.959866+0000","last_active":"2023-11-24T08:03:56.952874+0000","last_peered":"2023-11-24T08:03:56.952874+0000","last_clean":"2023-11-24T08:03:56.952874+0000","last_became_active":"2023-11-24T08:03:55.959194+0000","last_became_peered":"2023-11-24T08:03:55.959194+0000","last_unstale":"2023-11-24T08:03:56.952874+0000","last_undegraded":"2023-11-24T08:03:56.952874+0000","last_fullsized":"2023-11-24T08:03:56.952874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:13.909369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953570+0000","last_change":"2023-11-24T08:03:55.243163+0000","last_active":"2023-11-24T08:03:56.953570+0000","last_peered":"2023-11-24T08:03:56.953570+0000","last_clean":"2023-11-24T08:03:56.953570+0000","last_became_active":"2023-11-24T08:03:55.242828+0000","last_became_peered":"2023-11-24T08:03:55.242828+0000","last_unstale":"2023-11-24T08:03:56.953570+0000","last_undegraded":"2023-11-24T08:03:56.953570+0000","last_fullsized":"2023-11-24T08:03:56.953570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448964+0000","last_change":"2023-11-24T08:03:55.976360+0000","last_active":"2023-11-24T08:03:57.448964+0000","last_peered":"2023-11-24T08:03:57.448964+0000","last_clean":"2023-11-24T08:03:57.448964+0000","last_became_active":"2023-11-24T08:03:55.975680+0000","last_became_peered":"2023-11-24T08:03:55.975680+0000","last_unstale":"2023-11-24T08:03:57.448964+0000","last_undegraded":"2023-11-24T08:03:57.448964+0000","last_fullsized":"2023-11-24T08:03:57.448964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448633+0000","last_change":"2023-11-24T08:03:55.250034+0000","last_active":"2023-11-24T08:03:57.448633+0000","last_peered":"2023-11-24T08:03:57.448633+0000","last_clean":"2023-11-24T08:03:57.448633+0000","last_became_active":"2023-11-24T08:03:55.247004+0000","last_became_peered":"2023-11-24T08:03:55.247004+0000","last_unstale":"2023-11-24T08:03:57.448633+0000","last_undegraded":"2023-11-24T08:03:57.448633+0000","last_fullsized":"2023-11-24T08:03:57.448633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:58.276728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952866+0000","last_change":"2023-11-24T08:03:55.249873+0000","last_active":"2023-11-24T08:03:56.952866+0000","last_peered":"2023-11-24T08:03:56.952866+0000","last_clean":"2023-11-24T08:03:56.952866+0000","last_became_active":"2023-11-24T08:03:55.249613+0000","last_became_peered":"2023-11-24T08:03:55.249613+0000","last_unstale":"2023-11-24T08:03:56.952866+0000","last_undegraded":"2023-11-24T08:03:56.952866+0000","last_fullsized":"2023-11-24T08:03:56.952866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952860+0000","last_change":"2023-11-24T08:03:55.975878+0000","last_active":"2023-11-24T08:03:56.952860+0000","last_peered":"2023-11-24T08:03:56.952860+0000","last_clean":"2023-11-24T08:03:56.952860+0000","last_became_active":"2023-11-24T08:03:55.974080+0000","last_became_peered":"2023-11-24T08:03:55.974080+0000","last_unstale":"2023-11-24T08:03:56.952860+0000","last_undegraded":"2023-11-24T08:03:56.952860+0000","last_fullsized":"2023-11-24T08:03:56.952860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994534+0000","last_change":"2023-11-24T08:03:55.249828+0000","last_active":"2023-11-24T08:03:56.994534+0000","last_peered":"2023-11-24T08:03:56.994534+0000","last_clean":"2023-11-24T08:03:56.994534+0000","last_became_active":"2023-11-24T08:03:55.246927+0000","last_became_peered":"2023-11-24T08:03:55.246927+0000","last_unstale":"2023-11-24T08:03:56.994534+0000","last_undegraded":"2023-11-24T08:03:56.994534+0000","last_fullsized":"2023-11-24T08:03:56.994534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949045+0000","last_change":"2023-11-24T08:03:55.971036+0000","last_active":"2023-11-24T08:03:56.949045+0000","last_peered":"2023-11-24T08:03:56.949045+0000","last_clean":"2023-11-24T08:03:56.949045+0000","last_became_active":"2023-11-24T08:03:55.970127+0000","last_became_peered":"2023-11-24T08:03:55.970127+0000","last_unstale":"2023-11-24T08:03:56.949045+0000","last_undegraded":"2023-11-24T08:03:56.949045+0000","last_fullsized":"2023-11-24T08:03:56.949045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448936+0000","last_change":"2023-11-24T08:03:55.253888+0000","last_active":"2023-11-24T08:03:57.448936+0000","last_peered":"2023-11-24T08:03:57.448936+0000","last_clean":"2023-11-24T08:03:57.448936+0000","last_became_active":"2023-11-24T08:03:55.253583+0000","last_became_peered":"2023-11-24T08:03:55.253583+0000","last_unstale":"2023-11-24T08:03:57.448936+0000","last_undegraded":"2023-11-24T08:03:57.448936+0000","last_fullsized":"2023-11-24T08:03:57.448936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:01.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995133+0000","last_change":"2023-11-24T08:03:55.986660+0000","last_active":"2023-11-24T08:03:56.995133+0000","last_peered":"2023-11-24T08:03:56.995133+0000","last_clean":"2023-11-24T08:03:56.995133+0000","last_became_active":"2023-11-24T08:03:55.986351+0000","last_became_peered":"2023-11-24T08:03:55.986351+0000","last_unstale":"2023-11-24T08:03:56.995133+0000","last_undegraded":"2023-11-24T08:03:56.995133+0000","last_fullsized":"2023-11-24T08:03:56.995133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:16.719789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995284+0000","last_change":"2023-11-24T08:03:55.256523+0000","last_active":"2023-11-24T08:03:56.995284+0000","last_peered":"2023-11-24T08:03:56.995284+0000","last_clean":"2023-11-24T08:03:56.995284+0000","last_became_active":"2023-11-24T08:03:55.256208+0000","last_became_peered":"2023-11-24T08:03:55.256208+0000","last_unstale":"2023-11-24T08:03:56.995284+0000","last_undegraded":"2023-11-24T08:03:56.995284+0000","last_fullsized":"2023-11-24T08:03:56.995284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449400+0000","last_change":"2023-11-24T08:03:55.980794+0000","last_active":"2023-11-24T08:03:57.449400+0000","last_peered":"2023-11-24T08:03:57.449400+0000","last_clean":"2023-11-24T08:03:57.449400+0000","last_became_active":"2023-11-24T08:03:55.980463+0000","last_became_peered":"2023-11-24T08:03:55.980463+0000","last_unstale":"2023-11-24T08:03:57.449400+0000","last_undegraded":"2023-11-24T08:03:57.449400+0000","last_fullsized":"2023-11-24T08:03:57.449400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949486+0000","last_change":"2023-11-24T08:03:55.244567+0000","last_active":"2023-11-24T08:03:56.949486+0000","last_peered":"2023-11-24T08:03:56.949486+0000","last_clean":"2023-11-24T08:03:56.949486+0000","last_became_active":"2023-11-24T08:03:55.244286+0000","last_became_peered":"2023-11-24T08:03:55.244286+0000","last_unstale":"2023-11-24T08:03:56.949486+0000","last_undegraded":"2023-11-24T08:03:56.949486+0000","last_fullsized":"2023-11-24T08:03:56.949486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.896540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953786+0000","last_change":"2023-11-24T08:03:55.982726+0000","last_active":"2023-11-24T08:03:56.953786+0000","last_peered":"2023-11-24T08:03:56.953786+0000","last_clean":"2023-11-24T08:03:56.953786+0000","last_became_active":"2023-11-24T08:03:55.982437+0000","last_became_peered":"2023-11-24T08:03:55.982437+0000","last_unstale":"2023-11-24T08:03:56.953786+0000","last_undegraded":"2023-11-24T08:03:56.953786+0000","last_fullsized":"2023-11-24T08:03:56.953786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:59.289556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448900+0000","last_change":"2023-11-24T08:03:55.252214+0000","last_active":"2023-11-24T08:03:57.448900+0000","last_peered":"2023-11-24T08:03:57.448900+0000","last_clean":"2023-11-24T08:03:57.448900+0000","last_became_active":"2023-11-24T08:03:55.251878+0000","last_became_peered":"2023-11-24T08:03:55.251878+0000","last_unstale":"2023-11-24T08:03:57.448900+0000","last_undegraded":"2023-11-24T08:03:57.448900+0000","last_fullsized":"2023-11-24T08:03:57.448900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:59.543627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264565+0000","last_change":"2023-11-24T08:03:55.981213+0000","last_active":"2023-11-24T08:03:57.264565+0000","last_peered":"2023-11-24T08:03:57.264565+0000","last_clean":"2023-11-24T08:03:57.264565+0000","last_became_active":"2023-11-24T08:03:55.979791+0000","last_became_peered":"2023-11-24T08:03:55.979791+0000","last_unstale":"2023-11-24T08:03:57.264565+0000","last_undegraded":"2023-11-24T08:03:57.264565+0000","last_fullsized":"2023-11-24T08:03:57.264565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:03.856484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448918+0000","last_change":"2023-11-24T08:03:55.242184+0000","last_active":"2023-11-24T08:03:57.448918+0000","last_peered":"2023-11-24T08:03:57.448918+0000","last_clean":"2023-11-24T08:03:57.448918+0000","last_became_active":"2023-11-24T08:03:55.241829+0000","last_became_peered":"2023-11-24T08:03:55.241829+0000","last_unstale":"2023-11-24T08:03:57.448918+0000","last_undegraded":"2023-11-24T08:03:57.448918+0000","last_fullsized":"2023-11-24T08:03:57.448918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448374+0000","last_change":"2023-11-24T08:03:55.979303+0000","last_active":"2023-11-24T08:03:57.448374+0000","last_peered":"2023-11-24T08:03:57.448374+0000","last_clean":"2023-11-24T08:03:57.448374+0000","last_became_active":"2023-11-24T08:03:55.979044+0000","last_became_peered":"2023-11-24T08:03:55.979044+0000","last_unstale":"2023-11-24T08:03:57.448374+0000","last_undegraded":"2023-11-24T08:03:57.448374+0000","last_fullsized":"2023-11-24T08:03:57.448374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994583+0000","last_change":"2023-11-24T08:03:55.247945+0000","last_active":"2023-11-24T08:03:56.994583+0000","last_peered":"2023-11-24T08:03:56.994583+0000","last_clean":"2023-11-24T08:03:56.994583+0000","last_became_active":"2023-11-24T08:03:55.247662+0000","last_became_peered":"2023-11-24T08:03:55.247662+0000","last_unstale":"2023-11-24T08:03:56.994583+0000","last_undegraded":"2023-11-24T08:03:56.994583+0000","last_fullsized":"2023-11-24T08:03:56.994583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:47.114189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264673+0000","last_change":"2023-11-24T08:03:55.985619+0000","last_active":"2023-11-24T08:03:57.264673+0000","last_peered":"2023-11-24T08:03:57.264673+0000","last_clean":"2023-11-24T08:03:57.264673+0000","last_became_active":"2023-11-24T08:03:55.985179+0000","last_became_peered":"2023-11-24T08:03:55.985179+0000","last_unstale":"2023-11-24T08:03:57.264673+0000","last_undegraded":"2023-11-24T08:03:57.264673+0000","last_fullsized":"2023-11-24T08:03:57.264673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:19.891560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449005+0000","last_change":"2023-11-24T08:03:55.247678+0000","last_active":"2023-11-24T08:03:57.449005+0000","last_peered":"2023-11-24T08:03:57.449005+0000","last_clean":"2023-11-24T08:03:57.449005+0000","last_became_active":"2023-11-24T08:03:55.247366+0000","last_became_peered":"2023-11-24T08:03:55.247366+0000","last_unstale":"2023-11-24T08:03:57.449005+0000","last_undegraded":"2023-11-24T08:03:57.449005+0000","last_fullsized":"2023-11-24T08:03:57.449005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:21.976953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448482+0000","last_change":"2023-11-24T08:03:55.973556+0000","last_active":"2023-11-24T08:03:57.448482+0000","last_peered":"2023-11-24T08:03:57.448482+0000","last_clean":"2023-11-24T08:03:57.448482+0000","last_became_active":"2023-11-24T08:03:55.971826+0000","last_became_peered":"2023-11-24T08:03:55.971826+0000","last_unstale":"2023-11-24T08:03:57.448482+0000","last_undegraded":"2023-11-24T08:03:57.448482+0000","last_fullsized":"2023-11-24T08:03:57.448482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445426+0000","last_change":"2023-11-24T08:03:55.240904+0000","last_active":"2023-11-24T08:03:57.445426+0000","last_peered":"2023-11-24T08:03:57.445426+0000","last_clean":"2023-11-24T08:03:57.445426+0000","last_became_active":"2023-11-24T08:03:55.240317+0000","last_became_peered":"2023-11-24T08:03:55.240317+0000","last_unstale":"2023-11-24T08:03:57.445426+0000","last_undegraded":"2023-11-24T08:03:57.445426+0000","last_fullsized":"2023-11-24T08:03:57.445426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:25.132329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264211+0000","last_change":"2023-11-24T08:03:55.979396+0000","last_active":"2023-11-24T08:03:57.264211+0000","last_peered":"2023-11-24T08:03:57.264211+0000","last_clean":"2023-11-24T08:03:57.264211+0000","last_became_active":"2023-11-24T08:03:55.979048+0000","last_became_peered":"2023-11-24T08:03:55.979048+0000","last_unstale":"2023-11-24T08:03:57.264211+0000","last_undegraded":"2023-11-24T08:03:57.264211+0000","last_fullsized":"2023-11-24T08:03:57.264211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:14.268136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953570+0000","last_change":"2023-11-24T08:03:55.241439+0000","last_active":"2023-11-24T08:03:56.953570+0000","last_peered":"2023-11-24T08:03:56.953570+0000","last_clean":"2023-11-24T08:03:56.953570+0000","last_became_active":"2023-11-24T08:03:55.239231+0000","last_became_peered":"2023-11-24T08:03:55.239231+0000","last_unstale":"2023-11-24T08:03:56.953570+0000","last_undegraded":"2023-11-24T08:03:56.953570+0000","last_fullsized":"2023-11-24T08:03:56.953570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:25.132329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265081+0000","last_change":"2023-11-24T08:03:55.987779+0000","last_active":"2023-11-24T08:03:57.265081+0000","last_peered":"2023-11-24T08:03:57.265081+0000","last_clean":"2023-11-24T08:03:57.265081+0000","last_became_active":"2023-11-24T08:03:55.987443+0000","last_became_peered":"2023-11-24T08:03:55.987443+0000","last_unstale":"2023-11-24T08:03:57.265081+0000","last_undegraded":"2023-11-24T08:03:57.265081+0000","last_fullsized":"2023-11-24T08:03:57.265081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:18.291474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445818+0000","last_change":"2023-11-24T08:03:55.240840+0000","last_active":"2023-11-24T08:03:57.445818+0000","last_peered":"2023-11-24T08:03:57.445818+0000","last_clean":"2023-11-24T08:03:57.445818+0000","last_became_active":"2023-11-24T08:03:55.239508+0000","last_became_peered":"2023-11-24T08:03:55.239508+0000","last_unstale":"2023-11-24T08:03:57.445818+0000","last_undegraded":"2023-11-24T08:03:57.445818+0000","last_fullsized":"2023-11-24T08:03:57.445818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.896540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265617+0000","last_change":"2023-11-24T08:03:55.987770+0000","last_active":"2023-11-24T08:03:57.265617+0000","last_peered":"2023-11-24T08:03:57.265617+0000","last_clean":"2023-11-24T08:03:57.265617+0000","last_became_active":"2023-11-24T08:03:55.987481+0000","last_became_peered":"2023-11-24T08:03:55.987481+0000","last_unstale":"2023-11-24T08:03:57.265617+0000","last_undegraded":"2023-11-24T08:03:57.265617+0000","last_fullsized":"2023-11-24T08:03:57.265617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:36.996687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949593+0000","last_change":"2023-11-24T08:03:55.243684+0000","last_active":"2023-11-24T08:03:56.949593+0000","last_peered":"2023-11-24T08:03:56.949593+0000","last_clean":"2023-11-24T08:03:56.949593+0000","last_became_active":"2023-11-24T08:03:55.242363+0000","last_became_peered":"2023-11-24T08:03:55.242363+0000","last_unstale":"2023-11-24T08:03:56.949593+0000","last_undegraded":"2023-11-24T08:03:56.949593+0000","last_fullsized":"2023-11-24T08:03:56.949593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:34.443628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995391+0000","last_change":"2023-11-24T08:03:55.985203+0000","last_active":"2023-11-24T08:03:56.995391+0000","last_peered":"2023-11-24T08:03:56.995391+0000","last_clean":"2023-11-24T08:03:56.995391+0000","last_became_active":"2023-11-24T08:03:55.979146+0000","last_became_peered":"2023-11-24T08:03:55.979146+0000","last_unstale":"2023-11-24T08:03:56.995391+0000","last_undegraded":"2023-11-24T08:03:56.995391+0000","last_fullsized":"2023-11-24T08:03:56.995391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:47.350513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445560+0000","last_change":"2023-11-24T08:03:55.252157+0000","last_active":"2023-11-24T08:03:57.445560+0000","last_peered":"2023-11-24T08:03:57.445560+0000","last_clean":"2023-11-24T08:03:57.445560+0000","last_became_active":"2023-11-24T08:03:55.251772+0000","last_became_peered":"2023-11-24T08:03:55.251772+0000","last_unstale":"2023-11-24T08:03:57.445560+0000","last_undegraded":"2023-11-24T08:03:57.445560+0000","last_fullsized":"2023-11-24T08:03:57.445560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:01.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264581+0000","last_change":"2023-11-24T08:03:55.981873+0000","last_active":"2023-11-24T08:03:57.264581+0000","last_peered":"2023-11-24T08:03:57.264581+0000","last_clean":"2023-11-24T08:03:57.264581+0000","last_became_active":"2023-11-24T08:03:55.981562+0000","last_became_peered":"2023-11-24T08:03:55.981562+0000","last_unstale":"2023-11-24T08:03:57.264581+0000","last_undegraded":"2023-11-24T08:03:57.264581+0000","last_fullsized":"2023-11-24T08:03:57.264581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:47.350513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T08:03:55.953376+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:03:55.953376+0000","last_peered":"2023-11-24T08:03:55.953376+0000","last_clean":"2023-11-24T08:03:55.953376+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:03:55.953376+0000","last_undegraded":"2023-11-24T08:03:55.953376+0000","last_fullsized":"2023-11-24T08:03:55.953376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994403+0000","last_change":"2023-11-24T08:03:55.244940+0000","last_active":"2023-11-24T08:03:56.994403+0000","last_peered":"2023-11-24T08:03:56.994403+0000","last_clean":"2023-11-24T08:03:56.994403+0000","last_became_active":"2023-11-24T08:03:55.243981+0000","last_became_peered":"2023-11-24T08:03:55.243981+0000","last_unstale":"2023-11-24T08:03:56.994403+0000","last_undegraded":"2023-11-24T08:03:56.994403+0000","last_fullsized":"2023-11-24T08:03:56.994403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:30.740161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953049+0000","last_change":"2023-11-24T08:03:55.969277+0000","last_active":"2023-11-24T08:03:56.953049+0000","last_peered":"2023-11-24T08:03:56.953049+0000","last_clean":"2023-11-24T08:03:56.953049+0000","last_became_active":"2023-11-24T08:03:55.968980+0000","last_became_peered":"2023-11-24T08:03:55.968980+0000","last_unstale":"2023-11-24T08:03:56.953049+0000","last_undegraded":"2023-11-24T08:03:56.953049+0000","last_fullsized":"2023-11-24T08:03:56.953049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263854+0000","last_change":"2023-11-24T08:03:55.255880+0000","last_active":"2023-11-24T08:03:57.263854+0000","last_peered":"2023-11-24T08:03:57.263854+0000","last_clean":"2023-11-24T08:03:57.263854+0000","last_became_active":"2023-11-24T08:03:55.255584+0000","last_became_peered":"2023-11-24T08:03:55.255584+0000","last_unstale":"2023-11-24T08:03:57.263854+0000","last_undegraded":"2023-11-24T08:03:57.263854+0000","last_fullsized":"2023-11-24T08:03:57.263854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.458202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448244+0000","last_change":"2023-11-24T08:03:55.972919+0000","last_active":"2023-11-24T08:03:57.448244+0000","last_peered":"2023-11-24T08:03:57.448244+0000","last_clean":"2023-11-24T08:03:57.448244+0000","last_became_active":"2023-11-24T08:03:55.972754+0000","last_became_peered":"2023-11-24T08:03:55.972754+0000","last_unstale":"2023-11-24T08:03:57.448244+0000","last_undegraded":"2023-11-24T08:03:57.448244+0000","last_fullsized":"2023-11-24T08:03:57.448244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:01.646788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445758+0000","last_change":"2023-11-24T08:03:55.237610+0000","last_active":"2023-11-24T08:03:57.445758+0000","last_peered":"2023-11-24T08:03:57.445758+0000","last_clean":"2023-11-24T08:03:57.445758+0000","last_became_active":"2023-11-24T08:03:55.237273+0000","last_became_peered":"2023-11-24T08:03:55.237273+0000","last_unstale":"2023-11-24T08:03:57.445758+0000","last_undegraded":"2023-11-24T08:03:57.445758+0000","last_fullsized":"2023-11-24T08:03:57.445758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:24.389964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949322+0000","last_change":"2023-11-24T08:03:55.965491+0000","last_active":"2023-11-24T08:03:56.949322+0000","last_peered":"2023-11-24T08:03:56.949322+0000","last_clean":"2023-11-24T08:03:56.949322+0000","last_became_active":"2023-11-24T08:03:55.965148+0000","last_became_peered":"2023-11-24T08:03:55.965148+0000","last_unstale":"2023-11-24T08:03:56.949322+0000","last_undegraded":"2023-11-24T08:03:56.949322+0000","last_fullsized":"2023-11-24T08:03:56.949322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994656+0000","last_change":"2023-11-24T08:03:55.246876+0000","last_active":"2023-11-24T08:03:56.994656+0000","last_peered":"2023-11-24T08:03:56.994656+0000","last_clean":"2023-11-24T08:03:56.994656+0000","last_became_active":"2023-11-24T08:03:55.243530+0000","last_became_peered":"2023-11-24T08:03:55.243530+0000","last_unstale":"2023-11-24T08:03:56.994656+0000","last_undegraded":"2023-11-24T08:03:56.994656+0000","last_fullsized":"2023-11-24T08:03:56.994656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:27.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953302+0000","last_change":"2023-11-24T08:03:55.975512+0000","last_active":"2023-11-24T08:03:56.953302+0000","last_peered":"2023-11-24T08:03:56.953302+0000","last_clean":"2023-11-24T08:03:56.953302+0000","last_became_active":"2023-11-24T08:03:55.975099+0000","last_became_peered":"2023-11-24T08:03:55.975099+0000","last_unstale":"2023-11-24T08:03:56.953302+0000","last_undegraded":"2023-11-24T08:03:56.953302+0000","last_fullsized":"2023-11-24T08:03:56.953302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:25.402792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949417+0000","last_change":"2023-11-24T08:03:55.236722+0000","last_active":"2023-11-24T08:03:56.949417+0000","last_peered":"2023-11-24T08:03:56.949417+0000","last_clean":"2023-11-24T08:03:56.949417+0000","last_became_active":"2023-11-24T08:03:55.236383+0000","last_became_peered":"2023-11-24T08:03:55.236383+0000","last_unstale":"2023-11-24T08:03:56.949417+0000","last_undegraded":"2023-11-24T08:03:56.949417+0000","last_fullsized":"2023-11-24T08:03:56.949417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995298+0000","last_change":"2023-11-24T08:03:55.978747+0000","last_active":"2023-11-24T08:03:56.995298+0000","last_peered":"2023-11-24T08:03:56.995298+0000","last_clean":"2023-11-24T08:03:56.995298+0000","last_became_active":"2023-11-24T08:03:55.978454+0000","last_became_peered":"2023-11-24T08:03:55.978454+0000","last_unstale":"2023-11-24T08:03:56.995298+0000","last_undegraded":"2023-11-24T08:03:56.995298+0000","last_fullsized":"2023-11-24T08:03:56.995298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:03.856484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264892+0000","last_change":"2023-11-24T08:03:55.255119+0000","last_active":"2023-11-24T08:03:57.264892+0000","last_peered":"2023-11-24T08:03:57.264892+0000","last_clean":"2023-11-24T08:03:57.264892+0000","last_became_active":"2023-11-24T08:03:55.254773+0000","last_became_peered":"2023-11-24T08:03:55.254773+0000","last_unstale":"2023-11-24T08:03:57.264892+0000","last_undegraded":"2023-11-24T08:03:57.264892+0000","last_fullsized":"2023-11-24T08:03:57.264892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:27.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949334+0000","last_change":"2023-11-24T08:03:55.969522+0000","last_active":"2023-11-24T08:03:56.949334+0000","last_peered":"2023-11-24T08:03:56.949334+0000","last_clean":"2023-11-24T08:03:56.949334+0000","last_became_active":"2023-11-24T08:03:55.968807+0000","last_became_peered":"2023-11-24T08:03:55.968807+0000","last_unstale":"2023-11-24T08:03:56.949334+0000","last_undegraded":"2023-11-24T08:03:56.949334+0000","last_fullsized":"2023-11-24T08:03:56.949334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949596+0000","last_change":"2023-11-24T08:03:55.250820+0000","last_active":"2023-11-24T08:03:56.949596+0000","last_peered":"2023-11-24T08:03:56.949596+0000","last_clean":"2023-11-24T08:03:56.949596+0000","last_became_active":"2023-11-24T08:03:55.250479+0000","last_became_peered":"2023-11-24T08:03:55.250479+0000","last_unstale":"2023-11-24T08:03:56.949596+0000","last_undegraded":"2023-11-24T08:03:56.949596+0000","last_fullsized":"2023-11-24T08:03:56.949596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953296+0000","last_change":"2023-11-24T08:03:55.970065+0000","last_active":"2023-11-24T08:03:56.953296+0000","last_peered":"2023-11-24T08:03:56.953296+0000","last_clean":"2023-11-24T08:03:56.953296+0000","last_became_active":"2023-11-24T08:03:55.969811+0000","last_became_peered":"2023-11-24T08:03:55.969811+0000","last_unstale":"2023-11-24T08:03:56.953296+0000","last_undegraded":"2023-11-24T08:03:56.953296+0000","last_fullsized":"2023-11-24T08:03:56.953296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449201+0000","last_change":"2023-11-24T08:03:55.256073+0000","last_active":"2023-11-24T08:03:57.449201+0000","last_peered":"2023-11-24T08:03:57.449201+0000","last_clean":"2023-11-24T08:03:57.449201+0000","last_became_active":"2023-11-24T08:03:55.255797+0000","last_became_peered":"2023-11-24T08:03:55.255797+0000","last_unstale":"2023-11-24T08:03:57.449201+0000","last_undegraded":"2023-11-24T08:03:57.449201+0000","last_fullsized":"2023-11-24T08:03:57.449201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:24.389964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449472+0000","last_change":"2023-11-24T08:03:55.971989+0000","last_active":"2023-11-24T08:03:57.449472+0000","last_peered":"2023-11-24T08:03:57.449472+0000","last_clean":"2023-11-24T08:03:57.449472+0000","last_became_active":"2023-11-24T08:03:55.971690+0000","last_became_peered":"2023-11-24T08:03:55.971690+0000","last_unstale":"2023-11-24T08:03:57.449472+0000","last_undegraded":"2023-11-24T08:03:57.449472+0000","last_fullsized":"2023-11-24T08:03:57.449472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263829+0000","last_change":"2023-11-24T08:03:55.252987+0000","last_active":"2023-11-24T08:03:57.263829+0000","last_peered":"2023-11-24T08:03:57.263829+0000","last_clean":"2023-11-24T08:03:57.263829+0000","last_became_active":"2023-11-24T08:03:55.252708+0000","last_became_peered":"2023-11-24T08:03:55.252708+0000","last_unstale":"2023-11-24T08:03:57.263829+0000","last_undegraded":"2023-11-24T08:03:57.263829+0000","last_fullsized":"2023-11-24T08:03:57.263829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:34.443628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953362+0000","last_change":"2023-11-24T08:03:55.970570+0000","last_active":"2023-11-24T08:03:56.953362+0000","last_peered":"2023-11-24T08:03:56.953362+0000","last_clean":"2023-11-24T08:03:56.953362+0000","last_became_active":"2023-11-24T08:03:55.969759+0000","last_became_peered":"2023-11-24T08:03:55.969759+0000","last_unstale":"2023-11-24T08:03:56.953362+0000","last_undegraded":"2023-11-24T08:03:56.953362+0000","last_fullsized":"2023-11-24T08:03:56.953362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:04.042778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949401+0000","last_change":"2023-11-24T08:03:55.243962+0000","last_active":"2023-11-24T08:03:56.949401+0000","last_peered":"2023-11-24T08:03:56.949401+0000","last_clean":"2023-11-24T08:03:56.949401+0000","last_became_active":"2023-11-24T08:03:55.243548+0000","last_became_peered":"2023-11-24T08:03:55.243548+0000","last_unstale":"2023-11-24T08:03:56.949401+0000","last_undegraded":"2023-11-24T08:03:56.949401+0000","last_fullsized":"2023-11-24T08:03:56.949401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:59.543627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952949+0000","last_change":"2023-11-24T08:03:55.960318+0000","last_active":"2023-11-24T08:03:56.952949+0000","last_peered":"2023-11-24T08:03:56.952949+0000","last_clean":"2023-11-24T08:03:56.952949+0000","last_became_active":"2023-11-24T08:03:55.957342+0000","last_became_peered":"2023-11-24T08:03:55.957342+0000","last_unstale":"2023-11-24T08:03:56.952949+0000","last_undegraded":"2023-11-24T08:03:56.952949+0000","last_fullsized":"2023-11-24T08:03:56.952949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949250+0000","last_change":"2023-11-24T08:03:55.234991+0000","last_active":"2023-11-24T08:03:56.949250+0000","last_peered":"2023-11-24T08:03:56.949250+0000","last_clean":"2023-11-24T08:03:56.949250+0000","last_became_active":"2023-11-24T08:03:55.234480+0000","last_became_peered":"2023-11-24T08:03:55.234480+0000","last_unstale":"2023-11-24T08:03:56.949250+0000","last_undegraded":"2023-11-24T08:03:56.949250+0000","last_fullsized":"2023-11-24T08:03:56.949250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953403+0000","last_change":"2023-11-24T08:03:55.974916+0000","last_active":"2023-11-24T08:03:56.953403+0000","last_peered":"2023-11-24T08:03:56.953403+0000","last_clean":"2023-11-24T08:03:56.953403+0000","last_became_active":"2023-11-24T08:03:55.974533+0000","last_became_peered":"2023-11-24T08:03:55.974533+0000","last_unstale":"2023-11-24T08:03:56.953403+0000","last_undegraded":"2023-11-24T08:03:56.953403+0000","last_fullsized":"2023-11-24T08:03:56.953403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:55.841729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264803+0000","last_change":"2023-11-24T08:03:55.258218+0000","last_active":"2023-11-24T08:03:57.264803+0000","last_peered":"2023-11-24T08:03:57.264803+0000","last_clean":"2023-11-24T08:03:57.264803+0000","last_became_active":"2023-11-24T08:03:55.257904+0000","last_became_peered":"2023-11-24T08:03:55.257904+0000","last_unstale":"2023-11-24T08:03:57.264803+0000","last_undegraded":"2023-11-24T08:03:57.264803+0000","last_fullsized":"2023-11-24T08:03:57.264803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:42.858025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448404+0000","last_change":"2023-11-24T08:03:55.965022+0000","last_active":"2023-11-24T08:03:57.448404+0000","last_peered":"2023-11-24T08:03:57.448404+0000","last_clean":"2023-11-24T08:03:57.448404+0000","last_became_active":"2023-11-24T08:03:55.964652+0000","last_became_peered":"2023-11-24T08:03:55.964652+0000","last_unstale":"2023-11-24T08:03:57.448404+0000","last_undegraded":"2023-11-24T08:03:57.448404+0000","last_fullsized":"2023-11-24T08:03:57.448404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994904+0000","last_change":"2023-11-24T08:03:55.980852+0000","last_active":"2023-11-24T08:03:56.994904+0000","last_peered":"2023-11-24T08:03:56.994904+0000","last_clean":"2023-11-24T08:03:56.994904+0000","last_became_active":"2023-11-24T08:03:55.980467+0000","last_became_peered":"2023-11-24T08:03:55.980467+0000","last_unstale":"2023-11-24T08:03:56.994904+0000","last_undegraded":"2023-11-24T08:03:56.994904+0000","last_fullsized":"2023-11-24T08:03:56.994904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264433+0000","last_change":"2023-11-24T08:03:55.254837+0000","last_active":"2023-11-24T08:03:57.264433+0000","last_peered":"2023-11-24T08:03:57.264433+0000","last_clean":"2023-11-24T08:03:57.264433+0000","last_became_active":"2023-11-24T08:03:55.254593+0000","last_became_peered":"2023-11-24T08:03:55.254593+0000","last_unstale":"2023-11-24T08:03:57.264433+0000","last_undegraded":"2023-11-24T08:03:57.264433+0000","last_fullsized":"2023-11-24T08:03:57.264433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:47.114189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994917+0000","last_change":"2023-11-24T08:03:55.985005+0000","last_active":"2023-11-24T08:03:56.994917+0000","last_peered":"2023-11-24T08:03:56.994917+0000","last_clean":"2023-11-24T08:03:56.994917+0000","last_became_active":"2023-11-24T08:03:55.984533+0000","last_became_peered":"2023-11-24T08:03:55.984533+0000","last_unstale":"2023-11-24T08:03:56.994917+0000","last_undegraded":"2023-11-24T08:03:56.994917+0000","last_fullsized":"2023-11-24T08:03:56.994917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:14.268136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264754+0000","last_change":"2023-11-24T08:03:55.257482+0000","last_active":"2023-11-24T08:03:57.264754+0000","last_peered":"2023-11-24T08:03:57.264754+0000","last_clean":"2023-11-24T08:03:57.264754+0000","last_became_active":"2023-11-24T08:03:55.257267+0000","last_became_peered":"2023-11-24T08:03:55.257267+0000","last_unstale":"2023-11-24T08:03:57.264754+0000","last_undegraded":"2023-11-24T08:03:57.264754+0000","last_fullsized":"2023-11-24T08:03:57.264754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449374+0000","last_change":"2023-11-24T08:03:55.981373+0000","last_active":"2023-11-24T08:03:57.449374+0000","last_peered":"2023-11-24T08:03:57.449374+0000","last_clean":"2023-11-24T08:03:57.449374+0000","last_became_active":"2023-11-24T08:03:55.980504+0000","last_became_peered":"2023-11-24T08:03:55.980504+0000","last_unstale":"2023-11-24T08:03:57.449374+0000","last_undegraded":"2023-11-24T08:03:57.449374+0000","last_fullsized":"2023-11-24T08:03:57.449374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953650+0000","last_change":"2023-11-24T08:03:55.240395+0000","last_active":"2023-11-24T08:03:56.953650+0000","last_peered":"2023-11-24T08:03:56.953650+0000","last_clean":"2023-11-24T08:03:56.953650+0000","last_became_active":"2023-11-24T08:03:55.240021+0000","last_became_peered":"2023-11-24T08:03:55.240021+0000","last_unstale":"2023-11-24T08:03:56.953650+0000","last_undegraded":"2023-11-24T08:03:56.953650+0000","last_fullsized":"2023-11-24T08:03:56.953650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953580+0000","last_change":"2023-11-24T08:03:55.971086+0000","last_active":"2023-11-24T08:03:56.953580+0000","last_peered":"2023-11-24T08:03:56.953580+0000","last_clean":"2023-11-24T08:03:56.953580+0000","last_became_active":"2023-11-24T08:03:55.970765+0000","last_became_peered":"2023-11-24T08:03:55.970765+0000","last_unstale":"2023-11-24T08:03:56.953580+0000","last_undegraded":"2023-11-24T08:03:56.953580+0000","last_fullsized":"2023-11-24T08:03:56.953580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449518+0000","last_change":"2023-11-24T08:03:55.252498+0000","last_active":"2023-11-24T08:03:57.449518+0000","last_peered":"2023-11-24T08:03:57.449518+0000","last_clean":"2023-11-24T08:03:57.449518+0000","last_became_active":"2023-11-24T08:03:55.252172+0000","last_became_peered":"2023-11-24T08:03:55.252172+0000","last_unstale":"2023-11-24T08:03:57.449518+0000","last_undegraded":"2023-11-24T08:03:57.449518+0000","last_fullsized":"2023-11-24T08:03:57.449518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.458202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.835}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8240000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.78}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9239999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6319999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.794}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:04:00.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:04:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.018+0000 7f6c866ad640 1 Processor -- start 2023-11-24T08:04:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.019+0000 7f6c866ad640 1 -- start start 2023-11-24T08:04:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c866ad640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6c8006d520 0x7f6c8006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c866ad640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c800abff0 0x7f6c8015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c866ad640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8015c360 con 0x7f6c800ad720 2023-11-24T08:04:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c866ad640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c800b41a0 con 0x7f6c800abff0 2023-11-24T08:04:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c866ad640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8006de40 con 0x7f6c8006d520 2023-11-24T08:04:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c77fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6c800ad720 0x7f6c8015ecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56618/0 (socket says 172.21.15.64:56618) 2023-11-24T08:04:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c77fff640 1 -- 172.21.15.64:0/2918436358 learned_addr learned my addr 172.21.15.64:0/2918436358 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c84c4a640 1 --2- 172.21.15.64:0/2918436358 >> v2:172.21.15.64:3300/0 conn(0x7f6c800abff0 0x7f6c8015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.020+0000 7f6c7ffff640 1 --2- 172.21.15.64:0/2918436358 >> v2:172.21.15.91:3300/0 conn(0x7f6c8006d520 0x7f6c8006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.021+0000 7f6c7f7fe640 1 -- 172.21.15.64:0/2918436358 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216672577 0 0) 0x7f6c8015c360 con 0x7f6c800ad720 2023-11-24T08:04:01.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.021+0000 7f6c7f7fe640 1 -- 172.21.15.64:0/2918436358 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c6c003520 con 0x7f6c800ad720 2023-11-24T08:04:01.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.021+0000 7f6c84c4a640 1 -- 172.21.15.64:0/2918436358 >> v2:172.21.15.91:3300/0 conn(0x7f6c8006d520 msgr2=0x7f6c8006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:01.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.021+0000 7f6c84c4a640 1 --2- 172.21.15.64:0/2918436358 >> v2:172.21.15.91:3300/0 conn(0x7f6c8006d520 0x7f6c8006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.021+0000 7f6c84c4a640 1 -- 172.21.15.64:0/2918436358 >> v1:172.21.15.64:6789/0 conn(0x7f6c800ad720 legacy=0x7f6c8015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.021+0000 7f6c84c4a640 1 -- 172.21.15.64:0/2918436358 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8015ba30 con 0x7f6c800abff0 2023-11-24T08:04:01.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.021+0000 7f6c7ffff640 1 --2- 172.21.15.64:0/2918436358 >> v2:172.21.15.91:3300/0 conn(0x7f6c8006d520 0x7f6c8006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:01.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.022+0000 7f6c84c4a640 1 --2- 172.21.15.64:0/2918436358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c800abff0 0x7f6c8015f3e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c8006c070 tx=0x7f6c7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b69e742bdf4908d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:01.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.022+0000 7f6c7f7fe640 1 -- 172.21.15.64:0/2918436358 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7802cc80 con 0x7f6c800abff0 2023-11-24T08:04:01.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.022+0000 7f6c7f7fe640 1 -- 172.21.15.64:0/2918436358 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7802cde0 con 0x7f6c800abff0 2023-11-24T08:04:01.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.022+0000 7f6c7f7fe640 1 -- 172.21.15.64:0/2918436358 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78034660 con 0x7f6c800abff0 2023-11-24T08:04:01.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.023+0000 7f6c866ad640 1 -- 172.21.15.64:0/2918436358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c800abff0 msgr2=0x7f6c8015f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.023+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2918436358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c800abff0 0x7f6c8015f3e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c8006c070 tx=0x7f6c7802bed0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.023+0000 7f6c866ad640 1 -- 172.21.15.64:0/2918436358 shutdown_connections 2023-11-24T08:04:01.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.023+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2918436358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c800abff0 0x7f6c8015f3e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.023+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2918436358 >> v2:172.21.15.91:3300/0 conn(0x7f6c8006d520 0x7f6c8006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.023+0000 7f6c866ad640 1 -- 172.21.15.64:0/2918436358 >> 172.21.15.64:0/2918436358 conn(0x7f6c800b0d80 msgr2=0x7f6c800b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:01.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.023+0000 7f6c866ad640 1 -- 172.21.15.64:0/2918436358 shutdown_connections 2023-11-24T08:04:01.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.023+0000 7f6c866ad640 1 -- 172.21.15.64:0/2918436358 wait complete. 2023-11-24T08:04:01.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.025+0000 7f6c866ad640 1 Processor -- start 2023-11-24T08:04:01.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.025+0000 7f6c866ad640 1 -- start start 2023-11-24T08:04:01.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c866ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c8006d520 0x7f6c80147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c866ad640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c800abff0 0x7f6c800c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c866ad640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c800ad720 0x7f6c800ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c866ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8015d810 con 0x7f6c8006d520 2023-11-24T08:04:01.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c866ad640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80147650 con 0x7f6c800abff0 2023-11-24T08:04:01.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c866ad640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8015cb90 con 0x7f6c800ad720 2023-11-24T08:04:01.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c7ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c8006d520 0x7f6c80147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c7ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c8006d520 0x7f6c80147110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36688/0 (socket says 172.21.15.64:36688) 2023-11-24T08:04:01.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.026+0000 7f6c7ffff640 1 -- 172.21.15.64:0/2273449478 learned_addr learned my addr 172.21.15.64:0/2273449478 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:01.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c77fff640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c800abff0 0x7f6c800c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c7ffff640 1 -- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c800ad720 msgr2=0x7f6c800ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c84c4a640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c800ad720 0x7f6c800ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c7ffff640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c800ad720 0x7f6c800ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c7ffff640 1 -- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c800abff0 msgr2=0x7f6c800c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c7ffff640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c800abff0 0x7f6c800c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c7ffff640 1 -- 172.21.15.64:0/2273449478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78009a70 con 0x7f6c8006d520 2023-11-24T08:04:01.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c7ffff640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c8006d520 0x7f6c80147110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000e7e0 tx=0x7f6c7000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:01.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c7d7fa640 1 -- 172.21.15.64:0/2273449478 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70004030 con 0x7f6c8006d520 2023-11-24T08:04:01.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c7d7fa640 1 -- 172.21.15.64:0/2273449478 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c70004190 con 0x7f6c8006d520 2023-11-24T08:04:01.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.027+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8015ce70 con 0x7f6c8006d520 2023-11-24T08:04:01.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.028+0000 7f6c7d7fa640 1 -- 172.21.15.64:0/2273449478 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70004520 con 0x7f6c8006d520 2023-11-24T08:04:01.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.028+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8015d430 con 0x7f6c8006d520 2023-11-24T08:04:01.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.029+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c800ca430 con 0x7f6c8006d520 2023-11-24T08:04:01.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.030+0000 7f6c7d7fa640 1 -- 172.21.15.64:0/2273449478 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c7000c6d0 con 0x7f6c8006d520 2023-11-24T08:04:01.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.031+0000 7f6c7d7fa640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c6006ae40 0x7f6c6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.031+0000 7f6c7d7fa640 1 -- 172.21.15.64:0/2273449478 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c70089240 con 0x7f6c8006d520 2023-11-24T08:04:01.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.031+0000 7f6c77fff640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c6006ae40 0x7f6c6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.032+0000 7f6c77fff640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c6006ae40 0x7f6c6006d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c8014dfe0 tx=0x7f6c68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:01.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.035+0000 7f6c7d7fa640 1 -- 172.21.15.64:0/2273449478 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c70054380 con 0x7f6c8006d520 2023-11-24T08:04:01.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.226+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c8015b6e0 con 0x7f6c6006ae40 2023-11-24T08:04:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.247+0000 7f6c7d7fa640 1 -- 172.21.15.64:0/2273449478 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264426 (secure 0 0 0) 0x7f6c68005890 con 0x7f6c6006ae40 2023-11-24T08:04:01.251 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:01.253 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.253+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c6006ae40 msgr2=0x7f6c6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.253+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c6006ae40 0x7f6c6006d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c8014dfe0 tx=0x7f6c68009040 comp rx=0 tx=0).stop 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.253+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c8006d520 msgr2=0x7f6c80147110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.253+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c8006d520 0x7f6c80147110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000e7e0 tx=0x7f6c7000eca0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.254+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 shutdown_connections 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.254+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c6006ae40 0x7f6c6006d300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.254+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c800ad720 0x7f6c800ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.254+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c800abff0 0x7f6c800c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.254+0000 7f6c866ad640 1 --2- 172.21.15.64:0/2273449478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c8006d520 0x7f6c80147110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.254+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 >> 172.21.15.64:0/2273449478 conn(0x7f6c800b0d80 msgr2=0x7f6c800aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:01.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.254+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 shutdown_connections 2023-11-24T08:04:01.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.254+0000 7f6c866ad640 1 -- 172.21.15.64:0/2273449478 wait complete. 2023-11-24T08:04:01.275 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-24T08:04:00.012440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805499"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265413+0000","last_change":"2023-11-24T08:03:55.977134+0000","last_active":"2023-11-24T08:03:57.265413+0000","last_peered":"2023-11-24T08:03:57.265413+0000","last_clean":"2023-11-24T08:03:57.265413+0000","last_became_active":"2023-11-24T08:03:55.970402+0000","last_became_peered":"2023-11-24T08:03:55.970402+0000","last_unstale":"2023-11-24T08:03:57.265413+0000","last_undegraded":"2023-11-24T08:03:57.265413+0000","last_fullsized":"2023-11-24T08:03:57.265413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:47.677166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949561+0000","last_change":"2023-11-24T08:03:55.973687+0000","last_active":"2023-11-24T08:03:56.949561+0000","last_peered":"2023-11-24T08:03:56.949561+0000","last_clean":"2023-11-24T08:03:56.949561+0000","last_became_active":"2023-11-24T08:03:55.973369+0000","last_became_peered":"2023-11-24T08:03:55.973369+0000","last_unstale":"2023-11-24T08:03:56.949561+0000","last_undegraded":"2023-11-24T08:03:56.949561+0000","last_fullsized":"2023-11-24T08:03:56.949561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449190+0000","last_change":"2023-11-24T08:03:55.971194+0000","last_active":"2023-11-24T08:03:57.449190+0000","last_peered":"2023-11-24T08:03:57.449190+0000","last_clean":"2023-11-24T08:03:57.449190+0000","last_became_active":"2023-11-24T08:03:55.970751+0000","last_became_peered":"2023-11-24T08:03:55.970751+0000","last_unstale":"2023-11-24T08:03:57.449190+0000","last_undegraded":"2023-11-24T08:03:57.449190+0000","last_fullsized":"2023-11-24T08:03:57.449190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994604+0000","last_change":"2023-11-24T08:03:55.984822+0000","last_active":"2023-11-24T08:03:56.994604+0000","last_peered":"2023-11-24T08:03:56.994604+0000","last_clean":"2023-11-24T08:03:56.994604+0000","last_became_active":"2023-11-24T08:03:55.982969+0000","last_became_peered":"2023-11-24T08:03:55.982969+0000","last_unstale":"2023-11-24T08:03:56.994604+0000","last_undegraded":"2023-11-24T08:03:56.994604+0000","last_fullsized":"2023-11-24T08:03:56.994604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:43.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264555+0000","last_change":"2023-11-24T08:03:55.980587+0000","last_active":"2023-11-24T08:03:57.264555+0000","last_peered":"2023-11-24T08:03:57.264555+0000","last_clean":"2023-11-24T08:03:57.264555+0000","last_became_active":"2023-11-24T08:03:55.980331+0000","last_became_peered":"2023-11-24T08:03:55.980331+0000","last_unstale":"2023-11-24T08:03:57.264555+0000","last_undegraded":"2023-11-24T08:03:57.264555+0000","last_fullsized":"2023-11-24T08:03:57.264555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:16.719789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994145+0000","last_change":"2023-11-24T08:03:55.981917+0000","last_active":"2023-11-24T08:03:56.994145+0000","last_peered":"2023-11-24T08:03:56.994145+0000","last_clean":"2023-11-24T08:03:56.994145+0000","last_became_active":"2023-11-24T08:03:55.981300+0000","last_became_peered":"2023-11-24T08:03:55.981300+0000","last_unstale":"2023-11-24T08:03:56.994145+0000","last_undegraded":"2023-11-24T08:03:56.994145+0000","last_fullsized":"2023-11-24T08:03:56.994145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263992+0000","last_change":"2023-11-24T08:03:55.975783+0000","last_active":"2023-11-24T08:03:57.263992+0000","last_peered":"2023-11-24T08:03:57.263992+0000","last_clean":"2023-11-24T08:03:57.263992+0000","last_became_active":"2023-11-24T08:03:55.975452+0000","last_became_peered":"2023-11-24T08:03:55.975452+0000","last_unstale":"2023-11-24T08:03:57.263992+0000","last_undegraded":"2023-11-24T08:03:57.263992+0000","last_fullsized":"2023-11-24T08:03:57.263992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952790+0000","last_change":"2023-11-24T08:03:55.980610+0000","last_active":"2023-11-24T08:03:56.952790+0000","last_peered":"2023-11-24T08:03:56.952790+0000","last_clean":"2023-11-24T08:03:56.952790+0000","last_became_active":"2023-11-24T08:03:55.980222+0000","last_became_peered":"2023-11-24T08:03:55.980222+0000","last_unstale":"2023-11-24T08:03:56.952790+0000","last_undegraded":"2023-11-24T08:03:56.952790+0000","last_fullsized":"2023-11-24T08:03:56.952790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265494+0000","last_change":"2023-11-24T08:03:55.988607+0000","last_active":"2023-11-24T08:03:57.265494+0000","last_peered":"2023-11-24T08:03:57.265494+0000","last_clean":"2023-11-24T08:03:57.265494+0000","last_became_active":"2023-11-24T08:03:55.988074+0000","last_became_peered":"2023-11-24T08:03:55.988074+0000","last_unstale":"2023-11-24T08:03:57.265494+0000","last_undegraded":"2023-11-24T08:03:57.265494+0000","last_fullsized":"2023-11-24T08:03:57.265494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:08.202382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449121+0000","last_change":"2023-11-24T08:03:55.972034+0000","last_active":"2023-11-24T08:03:57.449121+0000","last_peered":"2023-11-24T08:03:57.449121+0000","last_clean":"2023-11-24T08:03:57.449121+0000","last_became_active":"2023-11-24T08:03:55.971627+0000","last_became_peered":"2023-11-24T08:03:55.971627+0000","last_unstale":"2023-11-24T08:03:57.449121+0000","last_undegraded":"2023-11-24T08:03:57.449121+0000","last_fullsized":"2023-11-24T08:03:57.449121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448738+0000","last_change":"2023-11-24T08:03:55.979788+0000","last_active":"2023-11-24T08:03:57.448738+0000","last_peered":"2023-11-24T08:03:57.448738+0000","last_clean":"2023-11-24T08:03:57.448738+0000","last_became_active":"2023-11-24T08:03:55.979429+0000","last_became_peered":"2023-11-24T08:03:55.979429+0000","last_unstale":"2023-11-24T08:03:57.448738+0000","last_undegraded":"2023-11-24T08:03:57.448738+0000","last_fullsized":"2023-11-24T08:03:57.448738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953475+0000","last_change":"2023-11-24T08:03:55.975330+0000","last_active":"2023-11-24T08:03:56.953475+0000","last_peered":"2023-11-24T08:03:56.953475+0000","last_clean":"2023-11-24T08:03:56.953475+0000","last_became_active":"2023-11-24T08:03:55.974721+0000","last_became_peered":"2023-11-24T08:03:55.974721+0000","last_unstale":"2023-11-24T08:03:56.953475+0000","last_undegraded":"2023-11-24T08:03:56.953475+0000","last_fullsized":"2023-11-24T08:03:56.953475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:35.456457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953384+0000","last_change":"2023-11-24T08:03:55.975039+0000","last_active":"2023-11-24T08:03:56.953384+0000","last_peered":"2023-11-24T08:03:56.953384+0000","last_clean":"2023-11-24T08:03:56.953384+0000","last_became_active":"2023-11-24T08:03:55.974171+0000","last_became_peered":"2023-11-24T08:03:55.974171+0000","last_unstale":"2023-11-24T08:03:56.953384+0000","last_undegraded":"2023-11-24T08:03:56.953384+0000","last_fullsized":"2023-11-24T08:03:56.953384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448417+0000","last_change":"2023-11-24T08:03:55.972138+0000","last_active":"2023-11-24T08:03:57.448417+0000","last_peered":"2023-11-24T08:03:57.448417+0000","last_clean":"2023-11-24T08:03:57.448417+0000","last_became_active":"2023-11-24T08:03:55.968993+0000","last_became_peered":"2023-11-24T08:03:55.968993+0000","last_unstale":"2023-11-24T08:03:57.448417+0000","last_undegraded":"2023-11-24T08:03:57.448417+0000","last_fullsized":"2023-11-24T08:03:57.448417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949358+0000","last_change":"2023-11-24T08:03:55.961573+0000","last_active":"2023-11-24T08:03:56.949358+0000","last_peered":"2023-11-24T08:03:56.949358+0000","last_clean":"2023-11-24T08:03:56.949358+0000","last_became_active":"2023-11-24T08:03:55.961135+0000","last_became_peered":"2023-11-24T08:03:55.961135+0000","last_unstale":"2023-11-24T08:03:56.949358+0000","last_undegraded":"2023-11-24T08:03:56.949358+0000","last_fullsized":"2023-11-24T08:03:56.949358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994390+0000","last_change":"2023-11-24T08:03:55.982805+0000","last_active":"2023-11-24T08:03:56.994390+0000","last_peered":"2023-11-24T08:03:56.994390+0000","last_clean":"2023-11-24T08:03:56.994390+0000","last_became_active":"2023-11-24T08:03:55.980530+0000","last_became_peered":"2023-11-24T08:03:55.980530+0000","last_unstale":"2023-11-24T08:03:56.994390+0000","last_undegraded":"2023-11-24T08:03:56.994390+0000","last_fullsized":"2023-11-24T08:03:56.994390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995521+0000","last_change":"2023-11-24T08:03:55.988078+0000","last_active":"2023-11-24T08:03:56.995521+0000","last_peered":"2023-11-24T08:03:56.995521+0000","last_clean":"2023-11-24T08:03:56.995521+0000","last_became_active":"2023-11-24T08:03:55.987802+0000","last_became_peered":"2023-11-24T08:03:55.987802+0000","last_unstale":"2023-11-24T08:03:56.995521+0000","last_undegraded":"2023-11-24T08:03:56.995521+0000","last_fullsized":"2023-11-24T08:03:56.995521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:19.891560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445715+0000","last_change":"2023-11-24T08:03:55.968360+0000","last_active":"2023-11-24T08:03:57.445715+0000","last_peered":"2023-11-24T08:03:57.445715+0000","last_clean":"2023-11-24T08:03:57.445715+0000","last_became_active":"2023-11-24T08:03:55.968089+0000","last_became_peered":"2023-11-24T08:03:55.968089+0000","last_unstale":"2023-11-24T08:03:57.445715+0000","last_undegraded":"2023-11-24T08:03:57.445715+0000","last_fullsized":"2023-11-24T08:03:57.445715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:04.042778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264933+0000","last_change":"2023-11-24T08:03:55.971957+0000","last_active":"2023-11-24T08:03:57.264933+0000","last_peered":"2023-11-24T08:03:57.264933+0000","last_clean":"2023-11-24T08:03:57.264933+0000","last_became_active":"2023-11-24T08:03:55.971638+0000","last_became_peered":"2023-11-24T08:03:55.971638+0000","last_unstale":"2023-11-24T08:03:57.264933+0000","last_undegraded":"2023-11-24T08:03:57.264933+0000","last_fullsized":"2023-11-24T08:03:57.264933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:01.176228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448754+0000","last_change":"2023-11-24T08:03:55.976704+0000","last_active":"2023-11-24T08:03:57.448754+0000","last_peered":"2023-11-24T08:03:57.448754+0000","last_clean":"2023-11-24T08:03:57.448754+0000","last_became_active":"2023-11-24T08:03:55.976410+0000","last_became_peered":"2023-11-24T08:03:55.976410+0000","last_unstale":"2023-11-24T08:03:57.448754+0000","last_undegraded":"2023-11-24T08:03:57.448754+0000","last_fullsized":"2023-11-24T08:03:57.448754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448488+0000","last_change":"2023-11-24T08:03:55.985384+0000","last_active":"2023-11-24T08:03:57.448488+0000","last_peered":"2023-11-24T08:03:57.448488+0000","last_clean":"2023-11-24T08:03:57.448488+0000","last_became_active":"2023-11-24T08:03:55.985155+0000","last_became_peered":"2023-11-24T08:03:55.985155+0000","last_unstale":"2023-11-24T08:03:57.448488+0000","last_undegraded":"2023-11-24T08:03:57.448488+0000","last_fullsized":"2023-11-24T08:03:57.448488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994409+0000","last_change":"2023-11-24T08:03:55.982133+0000","last_active":"2023-11-24T08:03:56.994409+0000","last_peered":"2023-11-24T08:03:56.994409+0000","last_clean":"2023-11-24T08:03:56.994409+0000","last_became_active":"2023-11-24T08:03:55.981629+0000","last_became_peered":"2023-11-24T08:03:55.981629+0000","last_unstale":"2023-11-24T08:03:56.994409+0000","last_undegraded":"2023-11-24T08:03:56.994409+0000","last_fullsized":"2023-11-24T08:03:56.994409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952961+0000","last_change":"2023-11-24T08:03:55.963916+0000","last_active":"2023-11-24T08:03:56.952961+0000","last_peered":"2023-11-24T08:03:56.952961+0000","last_clean":"2023-11-24T08:03:56.952961+0000","last_became_active":"2023-11-24T08:03:55.958460+0000","last_became_peered":"2023-11-24T08:03:55.958460+0000","last_unstale":"2023-11-24T08:03:56.952961+0000","last_undegraded":"2023-11-24T08:03:56.952961+0000","last_fullsized":"2023-11-24T08:03:56.952961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994284+0000","last_change":"2023-11-24T08:03:55.980845+0000","last_active":"2023-11-24T08:03:56.994284+0000","last_peered":"2023-11-24T08:03:56.994284+0000","last_clean":"2023-11-24T08:03:56.994284+0000","last_became_active":"2023-11-24T08:03:55.979729+0000","last_became_peered":"2023-11-24T08:03:55.979729+0000","last_unstale":"2023-11-24T08:03:56.994284+0000","last_undegraded":"2023-11-24T08:03:56.994284+0000","last_fullsized":"2023-11-24T08:03:56.994284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953148+0000","last_change":"2023-11-24T08:03:55.956896+0000","last_active":"2023-11-24T08:03:56.953148+0000","last_peered":"2023-11-24T08:03:56.953148+0000","last_clean":"2023-11-24T08:03:56.953148+0000","last_became_active":"2023-11-24T08:03:55.956452+0000","last_became_peered":"2023-11-24T08:03:55.956452+0000","last_unstale":"2023-11-24T08:03:56.953148+0000","last_undegraded":"2023-11-24T08:03:56.953148+0000","last_fullsized":"2023-11-24T08:03:56.953148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449134+0000","last_change":"2023-11-24T08:03:55.968031+0000","last_active":"2023-11-24T08:03:57.449134+0000","last_peered":"2023-11-24T08:03:57.449134+0000","last_clean":"2023-11-24T08:03:57.449134+0000","last_became_active":"2023-11-24T08:03:55.967733+0000","last_became_peered":"2023-11-24T08:03:55.967733+0000","last_unstale":"2023-11-24T08:03:57.449134+0000","last_undegraded":"2023-11-24T08:03:57.449134+0000","last_fullsized":"2023-11-24T08:03:57.449134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949553+0000","last_change":"2023-11-24T08:03:55.971172+0000","last_active":"2023-11-24T08:03:56.949553+0000","last_peered":"2023-11-24T08:03:56.949553+0000","last_clean":"2023-11-24T08:03:56.949553+0000","last_became_active":"2023-11-24T08:03:55.970829+0000","last_became_peered":"2023-11-24T08:03:55.970829+0000","last_unstale":"2023-11-24T08:03:56.949553+0000","last_undegraded":"2023-11-24T08:03:56.949553+0000","last_fullsized":"2023-11-24T08:03:56.949553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953446+0000","last_change":"2023-11-24T08:03:55.965485+0000","last_active":"2023-11-24T08:03:56.953446+0000","last_peered":"2023-11-24T08:03:56.953446+0000","last_clean":"2023-11-24T08:03:56.953446+0000","last_became_active":"2023-11-24T08:03:55.965220+0000","last_became_peered":"2023-11-24T08:03:55.965220+0000","last_unstale":"2023-11-24T08:03:56.953446+0000","last_undegraded":"2023-11-24T08:03:56.953446+0000","last_fullsized":"2023-11-24T08:03:56.953446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:43.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448732+0000","last_change":"2023-11-24T08:03:55.978834+0000","last_active":"2023-11-24T08:03:57.448732+0000","last_peered":"2023-11-24T08:03:57.448732+0000","last_clean":"2023-11-24T08:03:57.448732+0000","last_became_active":"2023-11-24T08:03:55.978604+0000","last_became_peered":"2023-11-24T08:03:55.978604+0000","last_unstale":"2023-11-24T08:03:57.448732+0000","last_undegraded":"2023-11-24T08:03:57.448732+0000","last_fullsized":"2023-11-24T08:03:57.448732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994520+0000","last_change":"2023-11-24T08:03:55.982758+0000","last_active":"2023-11-24T08:03:56.994520+0000","last_peered":"2023-11-24T08:03:56.994520+0000","last_clean":"2023-11-24T08:03:56.994520+0000","last_became_active":"2023-11-24T08:03:55.978909+0000","last_became_peered":"2023-11-24T08:03:55.978909+0000","last_unstale":"2023-11-24T08:03:56.994520+0000","last_undegraded":"2023-11-24T08:03:56.994520+0000","last_fullsized":"2023-11-24T08:03:56.994520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:55.841729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445961+0000","last_change":"2023-11-24T08:03:55.972047+0000","last_active":"2023-11-24T08:03:57.445961+0000","last_peered":"2023-11-24T08:03:57.445961+0000","last_clean":"2023-11-24T08:03:57.445961+0000","last_became_active":"2023-11-24T08:03:55.971770+0000","last_became_peered":"2023-11-24T08:03:55.971770+0000","last_unstale":"2023-11-24T08:03:57.445961+0000","last_undegraded":"2023-11-24T08:03:57.445961+0000","last_fullsized":"2023-11-24T08:03:57.445961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:35.456457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448439+0000","last_change":"2023-11-24T08:03:55.974157+0000","last_active":"2023-11-24T08:03:57.448439+0000","last_peered":"2023-11-24T08:03:57.448439+0000","last_clean":"2023-11-24T08:03:57.448439+0000","last_became_active":"2023-11-24T08:03:55.973864+0000","last_became_peered":"2023-11-24T08:03:55.973864+0000","last_unstale":"2023-11-24T08:03:57.448439+0000","last_undegraded":"2023-11-24T08:03:57.448439+0000","last_fullsized":"2023-11-24T08:03:57.448439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952874+0000","last_change":"2023-11-24T08:03:55.959866+0000","last_active":"2023-11-24T08:03:56.952874+0000","last_peered":"2023-11-24T08:03:56.952874+0000","last_clean":"2023-11-24T08:03:56.952874+0000","last_became_active":"2023-11-24T08:03:55.959194+0000","last_became_peered":"2023-11-24T08:03:55.959194+0000","last_unstale":"2023-11-24T08:03:56.952874+0000","last_undegraded":"2023-11-24T08:03:56.952874+0000","last_fullsized":"2023-11-24T08:03:56.952874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:13.909369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953570+0000","last_change":"2023-11-24T08:03:55.243163+0000","last_active":"2023-11-24T08:03:56.953570+0000","last_peered":"2023-11-24T08:03:56.953570+0000","last_clean":"2023-11-24T08:03:56.953570+0000","last_became_active":"2023-11-24T08:03:55.242828+0000","last_became_peered":"2023-11-24T08:03:55.242828+0000","last_unstale":"2023-11-24T08:03:56.953570+0000","last_undegraded":"2023-11-24T08:03:56.953570+0000","last_fullsized":"2023-11-24T08:03:56.953570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448964+0000","last_change":"2023-11-24T08:03:55.976360+0000","last_active":"2023-11-24T08:03:57.448964+0000","last_peered":"2023-11-24T08:03:57.448964+0000","last_clean":"2023-11-24T08:03:57.448964+0000","last_became_active":"2023-11-24T08:03:55.975680+0000","last_became_peered":"2023-11-24T08:03:55.975680+0000","last_unstale":"2023-11-24T08:03:57.448964+0000","last_undegraded":"2023-11-24T08:03:57.448964+0000","last_fullsized":"2023-11-24T08:03:57.448964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448633+0000","last_change":"2023-11-24T08:03:55.250034+0000","last_active":"2023-11-24T08:03:57.448633+0000","last_peered":"2023-11-24T08:03:57.448633+0000","last_clean":"2023-11-24T08:03:57.448633+0000","last_became_active":"2023-11-24T08:03:55.247004+0000","last_became_peered":"2023-11-24T08:03:55.247004+0000","last_unstale":"2023-11-24T08:03:57.448633+0000","last_undegraded":"2023-11-24T08:03:57.448633+0000","last_fullsized":"2023-11-24T08:03:57.448633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:58.276728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952866+0000","last_change":"2023-11-24T08:03:55.249873+0000","last_active":"2023-11-24T08:03:56.952866+0000","last_peered":"2023-11-24T08:03:56.952866+0000","last_clean":"2023-11-24T08:03:56.952866+0000","last_became_active":"2023-11-24T08:03:55.249613+0000","last_became_peered":"2023-11-24T08:03:55.249613+0000","last_unstale":"2023-11-24T08:03:56.952866+0000","last_undegraded":"2023-11-24T08:03:56.952866+0000","last_fullsized":"2023-11-24T08:03:56.952866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952860+0000","last_change":"2023-11-24T08:03:55.975878+0000","last_active":"2023-11-24T08:03:56.952860+0000","last_peered":"2023-11-24T08:03:56.952860+0000","last_clean":"2023-11-24T08:03:56.952860+0000","last_became_active":"2023-11-24T08:03:55.974080+0000","last_became_peered":"2023-11-24T08:03:55.974080+0000","last_unstale":"2023-11-24T08:03:56.952860+0000","last_undegraded":"2023-11-24T08:03:56.952860+0000","last_fullsized":"2023-11-24T08:03:56.952860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994534+0000","last_change":"2023-11-24T08:03:55.249828+0000","last_active":"2023-11-24T08:03:56.994534+0000","last_peered":"2023-11-24T08:03:56.994534+0000","last_clean":"2023-11-24T08:03:56.994534+0000","last_became_active":"2023-11-24T08:03:55.246927+0000","last_became_peered":"2023-11-24T08:03:55.246927+0000","last_unstale":"2023-11-24T08:03:56.994534+0000","last_undegraded":"2023-11-24T08:03:56.994534+0000","last_fullsized":"2023-11-24T08:03:56.994534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949045+0000","last_change":"2023-11-24T08:03:55.971036+0000","last_active":"2023-11-24T08:03:56.949045+0000","last_peered":"2023-11-24T08:03:56.949045+0000","last_clean":"2023-11-24T08:03:56.949045+0000","last_became_active":"2023-11-24T08:03:55.970127+0000","last_became_peered":"2023-11-24T08:03:55.970127+0000","last_unstale":"2023-11-24T08:03:56.949045+0000","last_undegraded":"2023-11-24T08:03:56.949045+0000","last_fullsized":"2023-11-24T08:03:56.949045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448936+0000","last_change":"2023-11-24T08:03:55.253888+0000","last_active":"2023-11-24T08:03:57.448936+0000","last_peered":"2023-11-24T08:03:57.448936+0000","last_clean":"2023-11-24T08:03:57.448936+0000","last_became_active":"2023-11-24T08:03:55.253583+0000","last_became_peered":"2023-11-24T08:03:55.253583+0000","last_unstale":"2023-11-24T08:03:57.448936+0000","last_undegraded":"2023-11-24T08:03:57.448936+0000","last_fullsized":"2023-11-24T08:03:57.448936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:01.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995133+0000","last_change":"2023-11-24T08:03:55.986660+0000","last_active":"2023-11-24T08:03:56.995133+0000","last_peered":"2023-11-24T08:03:56.995133+0000","last_clean":"2023-11-24T08:03:56.995133+0000","last_became_active":"2023-11-24T08:03:55.986351+0000","last_became_peered":"2023-11-24T08:03:55.986351+0000","last_unstale":"2023-11-24T08:03:56.995133+0000","last_undegraded":"2023-11-24T08:03:56.995133+0000","last_fullsized":"2023-11-24T08:03:56.995133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:16.719789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995284+0000","last_change":"2023-11-24T08:03:55.256523+0000","last_active":"2023-11-24T08:03:56.995284+0000","last_peered":"2023-11-24T08:03:56.995284+0000","last_clean":"2023-11-24T08:03:56.995284+0000","last_became_active":"2023-11-24T08:03:55.256208+0000","last_became_peered":"2023-11-24T08:03:55.256208+0000","last_unstale":"2023-11-24T08:03:56.995284+0000","last_undegraded":"2023-11-24T08:03:56.995284+0000","last_fullsized":"2023-11-24T08:03:56.995284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449400+0000","last_change":"2023-11-24T08:03:55.980794+0000","last_active":"2023-11-24T08:03:57.449400+0000","last_peered":"2023-11-24T08:03:57.449400+0000","last_clean":"2023-11-24T08:03:57.449400+0000","last_became_active":"2023-11-24T08:03:55.980463+0000","last_became_peered":"2023-11-24T08:03:55.980463+0000","last_unstale":"2023-11-24T08:03:57.449400+0000","last_undegraded":"2023-11-24T08:03:57.449400+0000","last_fullsized":"2023-11-24T08:03:57.449400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949486+0000","last_change":"2023-11-24T08:03:55.244567+0000","last_active":"2023-11-24T08:03:56.949486+0000","last_peered":"2023-11-24T08:03:56.949486+0000","last_clean":"2023-11-24T08:03:56.949486+0000","last_became_active":"2023-11-24T08:03:55.244286+0000","last_became_peered":"2023-11-24T08:03:55.244286+0000","last_unstale":"2023-11-24T08:03:56.949486+0000","last_undegraded":"2023-11-24T08:03:56.949486+0000","last_fullsized":"2023-11-24T08:03:56.949486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.896540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953786+0000","last_change":"2023-11-24T08:03:55.982726+0000","last_active":"2023-11-24T08:03:56.953786+0000","last_peered":"2023-11-24T08:03:56.953786+0000","last_clean":"2023-11-24T08:03:56.953786+0000","last_became_active":"2023-11-24T08:03:55.982437+0000","last_became_peered":"2023-11-24T08:03:55.982437+0000","last_unstale":"2023-11-24T08:03:56.953786+0000","last_undegraded":"2023-11-24T08:03:56.953786+0000","last_fullsized":"2023-11-24T08:03:56.953786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:59.289556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448900+0000","last_change":"2023-11-24T08:03:55.252214+0000","last_active":"2023-11-24T08:03:57.448900+0000","last_peered":"2023-11-24T08:03:57.448900+0000","last_clean":"2023-11-24T08:03:57.448900+0000","last_became_active":"2023-11-24T08:03:55.251878+0000","last_became_peered":"2023-11-24T08:03:55.251878+0000","last_unstale":"2023-11-24T08:03:57.448900+0000","last_undegraded":"2023-11-24T08:03:57.448900+0000","last_fullsized":"2023-11-24T08:03:57.448900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:59.543627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264565+0000","last_change":"2023-11-24T08:03:55.981213+0000","last_active":"2023-11-24T08:03:57.264565+0000","last_peered":"2023-11-24T08:03:57.264565+0000","last_clean":"2023-11-24T08:03:57.264565+0000","last_became_active":"2023-11-24T08:03:55.979791+0000","last_became_peered":"2023-11-24T08:03:55.979791+0000","last_unstale":"2023-11-24T08:03:57.264565+0000","last_undegraded":"2023-11-24T08:03:57.264565+0000","last_fullsized":"2023-11-24T08:03:57.264565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:03.856484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448918+0000","last_change":"2023-11-24T08:03:55.242184+0000","last_active":"2023-11-24T08:03:57.448918+0000","last_peered":"2023-11-24T08:03:57.448918+0000","last_clean":"2023-11-24T08:03:57.448918+0000","last_became_active":"2023-11-24T08:03:55.241829+0000","last_became_peered":"2023-11-24T08:03:55.241829+0000","last_unstale":"2023-11-24T08:03:57.448918+0000","last_undegraded":"2023-11-24T08:03:57.448918+0000","last_fullsized":"2023-11-24T08:03:57.448918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448374+0000","last_change":"2023-11-24T08:03:55.979303+0000","last_active":"2023-11-24T08:03:57.448374+0000","last_peered":"2023-11-24T08:03:57.448374+0000","last_clean":"2023-11-24T08:03:57.448374+0000","last_became_active":"2023-11-24T08:03:55.979044+0000","last_became_peered":"2023-11-24T08:03:55.979044+0000","last_unstale":"2023-11-24T08:03:57.448374+0000","last_undegraded":"2023-11-24T08:03:57.448374+0000","last_fullsized":"2023-11-24T08:03:57.448374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994583+0000","last_change":"2023-11-24T08:03:55.247945+0000","last_active":"2023-11-24T08:03:56.994583+0000","last_peered":"2023-11-24T08:03:56.994583+0000","last_clean":"2023-11-24T08:03:56.994583+0000","last_became_active":"2023-11-24T08:03:55.247662+0000","last_became_peered":"2023-11-24T08:03:55.247662+0000","last_unstale":"2023-11-24T08:03:56.994583+0000","last_undegraded":"2023-11-24T08:03:56.994583+0000","last_fullsized":"2023-11-24T08:03:56.994583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:47.114189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264673+0000","last_change":"2023-11-24T08:03:55.985619+0000","last_active":"2023-11-24T08:03:57.264673+0000","last_peered":"2023-11-24T08:03:57.264673+0000","last_clean":"2023-11-24T08:03:57.264673+0000","last_became_active":"2023-11-24T08:03:55.985179+0000","last_became_peered":"2023-11-24T08:03:55.985179+0000","last_unstale":"2023-11-24T08:03:57.264673+0000","last_undegraded":"2023-11-24T08:03:57.264673+0000","last_fullsized":"2023-11-24T08:03:57.264673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:19.891560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449005+0000","last_change":"2023-11-24T08:03:55.247678+0000","last_active":"2023-11-24T08:03:57.449005+0000","last_peered":"2023-11-24T08:03:57.449005+0000","last_clean":"2023-11-24T08:03:57.449005+0000","last_became_active":"2023-11-24T08:03:55.247366+0000","last_became_peered":"2023-11-24T08:03:55.247366+0000","last_unstale":"2023-11-24T08:03:57.449005+0000","last_undegraded":"2023-11-24T08:03:57.449005+0000","last_fullsized":"2023-11-24T08:03:57.449005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:21.976953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448482+0000","last_change":"2023-11-24T08:03:55.973556+0000","last_active":"2023-11-24T08:03:57.448482+0000","last_peered":"2023-11-24T08:03:57.448482+0000","last_clean":"2023-11-24T08:03:57.448482+0000","last_became_active":"2023-11-24T08:03:55.971826+0000","last_became_peered":"2023-11-24T08:03:55.971826+0000","last_unstale":"2023-11-24T08:03:57.448482+0000","last_undegraded":"2023-11-24T08:03:57.448482+0000","last_fullsized":"2023-11-24T08:03:57.448482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445426+0000","last_change":"2023-11-24T08:03:55.240904+0000","last_active":"2023-11-24T08:03:57.445426+0000","last_peered":"2023-11-24T08:03:57.445426+0000","last_clean":"2023-11-24T08:03:57.445426+0000","last_became_active":"2023-11-24T08:03:55.240317+0000","last_became_peered":"2023-11-24T08:03:55.240317+0000","last_unstale":"2023-11-24T08:03:57.445426+0000","last_undegraded":"2023-11-24T08:03:57.445426+0000","last_fullsized":"2023-11-24T08:03:57.445426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:25.132329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264211+0000","last_change":"2023-11-24T08:03:55.979396+0000","last_active":"2023-11-24T08:03:57.264211+0000","last_peered":"2023-11-24T08:03:57.264211+0000","last_clean":"2023-11-24T08:03:57.264211+0000","last_became_active":"2023-11-24T08:03:55.979048+0000","last_became_peered":"2023-11-24T08:03:55.979048+0000","last_unstale":"2023-11-24T08:03:57.264211+0000","last_undegraded":"2023-11-24T08:03:57.264211+0000","last_fullsized":"2023-11-24T08:03:57.264211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:14.268136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953570+0000","last_change":"2023-11-24T08:03:55.241439+0000","last_active":"2023-11-24T08:03:56.953570+0000","last_peered":"2023-11-24T08:03:56.953570+0000","last_clean":"2023-11-24T08:03:56.953570+0000","last_became_active":"2023-11-24T08:03:55.239231+0000","last_became_peered":"2023-11-24T08:03:55.239231+0000","last_unstale":"2023-11-24T08:03:56.953570+0000","last_undegraded":"2023-11-24T08:03:56.953570+0000","last_fullsized":"2023-11-24T08:03:56.953570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:25.132329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265081+0000","last_change":"2023-11-24T08:03:55.987779+0000","last_active":"2023-11-24T08:03:57.265081+0000","last_peered":"2023-11-24T08:03:57.265081+0000","last_clean":"2023-11-24T08:03:57.265081+0000","last_became_active":"2023-11-24T08:03:55.987443+0000","last_became_peered":"2023-11-24T08:03:55.987443+0000","last_unstale":"2023-11-24T08:03:57.265081+0000","last_undegraded":"2023-11-24T08:03:57.265081+0000","last_fullsized":"2023-11-24T08:03:57.265081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:18.291474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445818+0000","last_change":"2023-11-24T08:03:55.240840+0000","last_active":"2023-11-24T08:03:57.445818+0000","last_peered":"2023-11-24T08:03:57.445818+0000","last_clean":"2023-11-24T08:03:57.445818+0000","last_became_active":"2023-11-24T08:03:55.239508+0000","last_became_peered":"2023-11-24T08:03:55.239508+0000","last_unstale":"2023-11-24T08:03:57.445818+0000","last_undegraded":"2023-11-24T08:03:57.445818+0000","last_fullsized":"2023-11-24T08:03:57.445818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.896540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265617+0000","last_change":"2023-11-24T08:03:55.987770+0000","last_active":"2023-11-24T08:03:57.265617+0000","last_peered":"2023-11-24T08:03:57.265617+0000","last_clean":"2023-11-24T08:03:57.265617+0000","last_became_active":"2023-11-24T08:03:55.987481+0000","last_became_peered":"2023-11-24T08:03:55.987481+0000","last_unstale":"2023-11-24T08:03:57.265617+0000","last_undegraded":"2023-11-24T08:03:57.265617+0000","last_fullsized":"2023-11-24T08:03:57.265617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:36.996687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949593+0000","last_change":"2023-11-24T08:03:55.243684+0000","last_active":"2023-11-24T08:03:56.949593+0000","last_peered":"2023-11-24T08:03:56.949593+0000","last_clean":"2023-11-24T08:03:56.949593+0000","last_became_active":"2023-11-24T08:03:55.242363+0000","last_became_peered":"2023-11-24T08:03:55.242363+0000","last_unstale":"2023-11-24T08:03:56.949593+0000","last_undegraded":"2023-11-24T08:03:56.949593+0000","last_fullsized":"2023-11-24T08:03:56.949593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:34.443628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995391+0000","last_change":"2023-11-24T08:03:55.985203+0000","last_active":"2023-11-24T08:03:56.995391+0000","last_peered":"2023-11-24T08:03:56.995391+0000","last_clean":"2023-11-24T08:03:56.995391+0000","last_became_active":"2023-11-24T08:03:55.979146+0000","last_became_peered":"2023-11-24T08:03:55.979146+0000","last_unstale":"2023-11-24T08:03:56.995391+0000","last_undegraded":"2023-11-24T08:03:56.995391+0000","last_fullsized":"2023-11-24T08:03:56.995391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:47.350513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445560+0000","last_change":"2023-11-24T08:03:55.252157+0000","last_active":"2023-11-24T08:03:57.445560+0000","last_peered":"2023-11-24T08:03:57.445560+0000","last_clean":"2023-11-24T08:03:57.445560+0000","last_became_active":"2023-11-24T08:03:55.251772+0000","last_became_peered":"2023-11-24T08:03:55.251772+0000","last_unstale":"2023-11-24T08:03:57.445560+0000","last_undegraded":"2023-11-24T08:03:57.445560+0000","last_fullsized":"2023-11-24T08:03:57.445560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:01.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264581+0000","last_change":"2023-11-24T08:03:55.981873+0000","last_active":"2023-11-24T08:03:57.264581+0000","last_peered":"2023-11-24T08:03:57.264581+0000","last_clean":"2023-11-24T08:03:57.264581+0000","last_became_active":"2023-11-24T08:03:55.981562+0000","last_became_peered":"2023-11-24T08:03:55.981562+0000","last_unstale":"2023-11-24T08:03:57.264581+0000","last_undegraded":"2023-11-24T08:03:57.264581+0000","last_fullsized":"2023-11-24T08:03:57.264581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:47.350513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T08:03:55.953376+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:03:55.953376+0000","last_peered":"2023-11-24T08:03:55.953376+0000","last_clean":"2023-11-24T08:03:55.953376+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:03:55.953376+0000","last_undegraded":"2023-11-24T08:03:55.953376+0000","last_fullsized":"2023-11-24T08:03:55.953376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994403+0000","last_change":"2023-11-24T08:03:55.244940+0000","last_active":"2023-11-24T08:03:56.994403+0000","last_peered":"2023-11-24T08:03:56.994403+0000","last_clean":"2023-11-24T08:03:56.994403+0000","last_became_active":"2023-11-24T08:03:55.243981+0000","last_became_peered":"2023-11-24T08:03:55.243981+0000","last_unstale":"2023-11-24T08:03:56.994403+0000","last_undegraded":"2023-11-24T08:03:56.994403+0000","last_fullsized":"2023-11-24T08:03:56.994403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:30.740161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953049+0000","last_change":"2023-11-24T08:03:55.969277+0000","last_active":"2023-11-24T08:03:56.953049+0000","last_peered":"2023-11-24T08:03:56.953049+0000","last_clean":"2023-11-24T08:03:56.953049+0000","last_became_active":"2023-11-24T08:03:55.968980+0000","last_became_peered":"2023-11-24T08:03:55.968980+0000","last_unstale":"2023-11-24T08:03:56.953049+0000","last_undegraded":"2023-11-24T08:03:56.953049+0000","last_fullsized":"2023-11-24T08:03:56.953049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263854+0000","last_change":"2023-11-24T08:03:55.255880+0000","last_active":"2023-11-24T08:03:57.263854+0000","last_peered":"2023-11-24T08:03:57.263854+0000","last_clean":"2023-11-24T08:03:57.263854+0000","last_became_active":"2023-11-24T08:03:55.255584+0000","last_became_peered":"2023-11-24T08:03:55.255584+0000","last_unstale":"2023-11-24T08:03:57.263854+0000","last_undegraded":"2023-11-24T08:03:57.263854+0000","last_fullsized":"2023-11-24T08:03:57.263854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.458202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448244+0000","last_change":"2023-11-24T08:03:55.972919+0000","last_active":"2023-11-24T08:03:57.448244+0000","last_peered":"2023-11-24T08:03:57.448244+0000","last_clean":"2023-11-24T08:03:57.448244+0000","last_became_active":"2023-11-24T08:03:55.972754+0000","last_became_peered":"2023-11-24T08:03:55.972754+0000","last_unstale":"2023-11-24T08:03:57.448244+0000","last_undegraded":"2023-11-24T08:03:57.448244+0000","last_fullsized":"2023-11-24T08:03:57.448244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:01.646788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445758+0000","last_change":"2023-11-24T08:03:55.237610+0000","last_active":"2023-11-24T08:03:57.445758+0000","last_peered":"2023-11-24T08:03:57.445758+0000","last_clean":"2023-11-24T08:03:57.445758+0000","last_became_active":"2023-11-24T08:03:55.237273+0000","last_became_peered":"2023-11-24T08:03:55.237273+0000","last_unstale":"2023-11-24T08:03:57.445758+0000","last_undegraded":"2023-11-24T08:03:57.445758+0000","last_fullsized":"2023-11-24T08:03:57.445758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:24.389964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949322+0000","last_change":"2023-11-24T08:03:55.965491+0000","last_active":"2023-11-24T08:03:56.949322+0000","last_peered":"2023-11-24T08:03:56.949322+0000","last_clean":"2023-11-24T08:03:56.949322+0000","last_became_active":"2023-11-24T08:03:55.965148+0000","last_became_peered":"2023-11-24T08:03:55.965148+0000","last_unstale":"2023-11-24T08:03:56.949322+0000","last_undegraded":"2023-11-24T08:03:56.949322+0000","last_fullsized":"2023-11-24T08:03:56.949322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994656+0000","last_change":"2023-11-24T08:03:55.246876+0000","last_active":"2023-11-24T08:03:56.994656+0000","last_peered":"2023-11-24T08:03:56.994656+0000","last_clean":"2023-11-24T08:03:56.994656+0000","last_became_active":"2023-11-24T08:03:55.243530+0000","last_became_peered":"2023-11-24T08:03:55.243530+0000","last_unstale":"2023-11-24T08:03:56.994656+0000","last_undegraded":"2023-11-24T08:03:56.994656+0000","last_fullsized":"2023-11-24T08:03:56.994656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:27.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953302+0000","last_change":"2023-11-24T08:03:55.975512+0000","last_active":"2023-11-24T08:03:56.953302+0000","last_peered":"2023-11-24T08:03:56.953302+0000","last_clean":"2023-11-24T08:03:56.953302+0000","last_became_active":"2023-11-24T08:03:55.975099+0000","last_became_peered":"2023-11-24T08:03:55.975099+0000","last_unstale":"2023-11-24T08:03:56.953302+0000","last_undegraded":"2023-11-24T08:03:56.953302+0000","last_fullsized":"2023-11-24T08:03:56.953302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:25.402792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949417+0000","last_change":"2023-11-24T08:03:55.236722+0000","last_active":"2023-11-24T08:03:56.949417+0000","last_peered":"2023-11-24T08:03:56.949417+0000","last_clean":"2023-11-24T08:03:56.949417+0000","last_became_active":"2023-11-24T08:03:55.236383+0000","last_became_peered":"2023-11-24T08:03:55.236383+0000","last_unstale":"2023-11-24T08:03:56.949417+0000","last_undegraded":"2023-11-24T08:03:56.949417+0000","last_fullsized":"2023-11-24T08:03:56.949417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995298+0000","last_change":"2023-11-24T08:03:55.978747+0000","last_active":"2023-11-24T08:03:56.995298+0000","last_peered":"2023-11-24T08:03:56.995298+0000","last_clean":"2023-11-24T08:03:56.995298+0000","last_became_active":"2023-11-24T08:03:55.978454+0000","last_became_peered":"2023-11-24T08:03:55.978454+0000","last_unstale":"2023-11-24T08:03:56.995298+0000","last_undegraded":"2023-11-24T08:03:56.995298+0000","last_fullsized":"2023-11-24T08:03:56.995298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:03.856484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264892+0000","last_change":"2023-11-24T08:03:55.255119+0000","last_active":"2023-11-24T08:03:57.264892+0000","last_peered":"2023-11-24T08:03:57.264892+0000","last_clean":"2023-11-24T08:03:57.264892+0000","last_became_active":"2023-11-24T08:03:55.254773+0000","last_became_peered":"2023-11-24T08:03:55.254773+0000","last_unstale":"2023-11-24T08:03:57.264892+0000","last_undegraded":"2023-11-24T08:03:57.264892+0000","last_fullsized":"2023-11-24T08:03:57.264892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:27.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949334+0000","last_change":"2023-11-24T08:03:55.969522+0000","last_active":"2023-11-24T08:03:56.949334+0000","last_peered":"2023-11-24T08:03:56.949334+0000","last_clean":"2023-11-24T08:03:56.949334+0000","last_became_active":"2023-11-24T08:03:55.968807+0000","last_became_peered":"2023-11-24T08:03:55.968807+0000","last_unstale":"2023-11-24T08:03:56.949334+0000","last_undegraded":"2023-11-24T08:03:56.949334+0000","last_fullsized":"2023-11-24T08:03:56.949334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949596+0000","last_change":"2023-11-24T08:03:55.250820+0000","last_active":"2023-11-24T08:03:56.949596+0000","last_peered":"2023-11-24T08:03:56.949596+0000","last_clean":"2023-11-24T08:03:56.949596+0000","last_became_active":"2023-11-24T08:03:55.250479+0000","last_became_peered":"2023-11-24T08:03:55.250479+0000","last_unstale":"2023-11-24T08:03:56.949596+0000","last_undegraded":"2023-11-24T08:03:56.949596+0000","last_fullsized":"2023-11-24T08:03:56.949596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953296+0000","last_change":"2023-11-24T08:03:55.970065+0000","last_active":"2023-11-24T08:03:56.953296+0000","last_peered":"2023-11-24T08:03:56.953296+0000","last_clean":"2023-11-24T08:03:56.953296+0000","last_became_active":"2023-11-24T08:03:55.969811+0000","last_became_peered":"2023-11-24T08:03:55.969811+0000","last_unstale":"2023-11-24T08:03:56.953296+0000","last_undegraded":"2023-11-24T08:03:56.953296+0000","last_fullsized":"2023-11-24T08:03:56.953296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449201+0000","last_change":"2023-11-24T08:03:55.256073+0000","last_active":"2023-11-24T08:03:57.449201+0000","last_peered":"2023-11-24T08:03:57.449201+0000","last_clean":"2023-11-24T08:03:57.449201+0000","last_became_active":"2023-11-24T08:03:55.255797+0000","last_became_peered":"2023-11-24T08:03:55.255797+0000","last_unstale":"2023-11-24T08:03:57.449201+0000","last_undegraded":"2023-11-24T08:03:57.449201+0000","last_fullsized":"2023-11-24T08:03:57.449201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:24.389964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449472+0000","last_change":"2023-11-24T08:03:55.971989+0000","last_active":"2023-11-24T08:03:57.449472+0000","last_peered":"2023-11-24T08:03:57.449472+0000","last_clean":"2023-11-24T08:03:57.449472+0000","last_became_active":"2023-11-24T08:03:55.971690+0000","last_became_peered":"2023-11-24T08:03:55.971690+0000","last_unstale":"2023-11-24T08:03:57.449472+0000","last_undegraded":"2023-11-24T08:03:57.449472+0000","last_fullsized":"2023-11-24T08:03:57.449472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263829+0000","last_change":"2023-11-24T08:03:55.252987+0000","last_active":"2023-11-24T08:03:57.263829+0000","last_peered":"2023-11-24T08:03:57.263829+0000","last_clean":"2023-11-24T08:03:57.263829+0000","last_became_active":"2023-11-24T08:03:55.252708+0000","last_became_peered":"2023-11-24T08:03:55.252708+0000","last_unstale":"2023-11-24T08:03:57.263829+0000","last_undegraded":"2023-11-24T08:03:57.263829+0000","last_fullsized":"2023-11-24T08:03:57.263829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:34.443628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953362+0000","last_change":"2023-11-24T08:03:55.970570+0000","last_active":"2023-11-24T08:03:56.953362+0000","last_peered":"2023-11-24T08:03:56.953362+0000","last_clean":"2023-11-24T08:03:56.953362+0000","last_became_active":"2023-11-24T08:03:55.969759+0000","last_became_peered":"2023-11-24T08:03:55.969759+0000","last_unstale":"2023-11-24T08:03:56.953362+0000","last_undegraded":"2023-11-24T08:03:56.953362+0000","last_fullsized":"2023-11-24T08:03:56.953362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:04.042778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949401+0000","last_change":"2023-11-24T08:03:55.243962+0000","last_active":"2023-11-24T08:03:56.949401+0000","last_peered":"2023-11-24T08:03:56.949401+0000","last_clean":"2023-11-24T08:03:56.949401+0000","last_became_active":"2023-11-24T08:03:55.243548+0000","last_became_peered":"2023-11-24T08:03:55.243548+0000","last_unstale":"2023-11-24T08:03:56.949401+0000","last_undegraded":"2023-11-24T08:03:56.949401+0000","last_fullsized":"2023-11-24T08:03:56.949401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:59.543627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952949+0000","last_change":"2023-11-24T08:03:55.960318+0000","last_active":"2023-11-24T08:03:56.952949+0000","last_peered":"2023-11-24T08:03:56.952949+0000","last_clean":"2023-11-24T08:03:56.952949+0000","last_became_active":"2023-11-24T08:03:55.957342+0000","last_became_peered":"2023-11-24T08:03:55.957342+0000","last_unstale":"2023-11-24T08:03:56.952949+0000","last_undegraded":"2023-11-24T08:03:56.952949+0000","last_fullsized":"2023-11-24T08:03:56.952949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949250+0000","last_change":"2023-11-24T08:03:55.234991+0000","last_active":"2023-11-24T08:03:56.949250+0000","last_peered":"2023-11-24T08:03:56.949250+0000","last_clean":"2023-11-24T08:03:56.949250+0000","last_became_active":"2023-11-24T08:03:55.234480+0000","last_became_peered":"2023-11-24T08:03:55.234480+0000","last_unstale":"2023-11-24T08:03:56.949250+0000","last_undegraded":"2023-11-24T08:03:56.949250+0000","last_fullsized":"2023-11-24T08:03:56.949250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953403+0000","last_change":"2023-11-24T08:03:55.974916+0000","last_active":"2023-11-24T08:03:56.953403+0000","last_peered":"2023-11-24T08:03:56.953403+0000","last_clean":"2023-11-24T08:03:56.953403+0000","last_became_active":"2023-11-24T08:03:55.974533+0000","last_became_peered":"2023-11-24T08:03:55.974533+0000","last_unstale":"2023-11-24T08:03:56.953403+0000","last_undegraded":"2023-11-24T08:03:56.953403+0000","last_fullsized":"2023-11-24T08:03:56.953403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:55.841729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264803+0000","last_change":"2023-11-24T08:03:55.258218+0000","last_active":"2023-11-24T08:03:57.264803+0000","last_peered":"2023-11-24T08:03:57.264803+0000","last_clean":"2023-11-24T08:03:57.264803+0000","last_became_active":"2023-11-24T08:03:55.257904+0000","last_became_peered":"2023-11-24T08:03:55.257904+0000","last_unstale":"2023-11-24T08:03:57.264803+0000","last_undegraded":"2023-11-24T08:03:57.264803+0000","last_fullsized":"2023-11-24T08:03:57.264803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:42.858025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448404+0000","last_change":"2023-11-24T08:03:55.965022+0000","last_active":"2023-11-24T08:03:57.448404+0000","last_peered":"2023-11-24T08:03:57.448404+0000","last_clean":"2023-11-24T08:03:57.448404+0000","last_became_active":"2023-11-24T08:03:55.964652+0000","last_became_peered":"2023-11-24T08:03:55.964652+0000","last_unstale":"2023-11-24T08:03:57.448404+0000","last_undegraded":"2023-11-24T08:03:57.448404+0000","last_fullsized":"2023-11-24T08:03:57.448404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994904+0000","last_change":"2023-11-24T08:03:55.980852+0000","last_active":"2023-11-24T08:03:56.994904+0000","last_peered":"2023-11-24T08:03:56.994904+0000","last_clean":"2023-11-24T08:03:56.994904+0000","last_became_active":"2023-11-24T08:03:55.980467+0000","last_became_peered":"2023-11-24T08:03:55.980467+0000","last_unstale":"2023-11-24T08:03:56.994904+0000","last_undegraded":"2023-11-24T08:03:56.994904+0000","last_fullsized":"2023-11-24T08:03:56.994904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264433+0000","last_change":"2023-11-24T08:03:55.254837+0000","last_active":"2023-11-24T08:03:57.264433+0000","last_peered":"2023-11-24T08:03:57.264433+0000","last_clean":"2023-11-24T08:03:57.264433+0000","last_became_active":"2023-11-24T08:03:55.254593+0000","last_became_peered":"2023-11-24T08:03:55.254593+0000","last_unstale":"2023-11-24T08:03:57.264433+0000","last_undegraded":"2023-11-24T08:03:57.264433+0000","last_fullsized":"2023-11-24T08:03:57.264433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:47.114189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994917+0000","last_change":"2023-11-24T08:03:55.985005+0000","last_active":"2023-11-24T08:03:56.994917+0000","last_peered":"2023-11-24T08:03:56.994917+0000","last_clean":"2023-11-24T08:03:56.994917+0000","last_became_active":"2023-11-24T08:03:55.984533+0000","last_became_peered":"2023-11-24T08:03:55.984533+0000","last_unstale":"2023-11-24T08:03:56.994917+0000","last_undegraded":"2023-11-24T08:03:56.994917+0000","last_fullsized":"2023-11-24T08:03:56.994917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:14.268136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264754+0000","last_change":"2023-11-24T08:03:55.257482+0000","last_active":"2023-11-24T08:03:57.264754+0000","last_peered":"2023-11-24T08:03:57.264754+0000","last_clean":"2023-11-24T08:03:57.264754+0000","last_became_active":"2023-11-24T08:03:55.257267+0000","last_became_peered":"2023-11-24T08:03:55.257267+0000","last_unstale":"2023-11-24T08:03:57.264754+0000","last_undegraded":"2023-11-24T08:03:57.264754+0000","last_fullsized":"2023-11-24T08:03:57.264754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449374+0000","last_change":"2023-11-24T08:03:55.981373+0000","last_active":"2023-11-24T08:03:57.449374+0000","last_peered":"2023-11-24T08:03:57.449374+0000","last_clean":"2023-11-24T08:03:57.449374+0000","last_became_active":"2023-11-24T08:03:55.980504+0000","last_became_peered":"2023-11-24T08:03:55.980504+0000","last_unstale":"2023-11-24T08:03:57.449374+0000","last_undegraded":"2023-11-24T08:03:57.449374+0000","last_fullsized":"2023-11-24T08:03:57.449374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953650+0000","last_change":"2023-11-24T08:03:55.240395+0000","last_active":"2023-11-24T08:03:56.953650+0000","last_peered":"2023-11-24T08:03:56.953650+0000","last_clean":"2023-11-24T08:03:56.953650+0000","last_became_active":"2023-11-24T08:03:55.240021+0000","last_became_peered":"2023-11-24T08:03:55.240021+0000","last_unstale":"2023-11-24T08:03:56.953650+0000","last_undegraded":"2023-11-24T08:03:56.953650+0000","last_fullsized":"2023-11-24T08:03:56.953650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953580+0000","last_change":"2023-11-24T08:03:55.971086+0000","last_active":"2023-11-24T08:03:56.953580+0000","last_peered":"2023-11-24T08:03:56.953580+0000","last_clean":"2023-11-24T08:03:56.953580+0000","last_became_active":"2023-11-24T08:03:55.970765+0000","last_became_peered":"2023-11-24T08:03:55.970765+0000","last_unstale":"2023-11-24T08:03:56.953580+0000","last_undegraded":"2023-11-24T08:03:56.953580+0000","last_fullsized":"2023-11-24T08:03:56.953580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449518+0000","last_change":"2023-11-24T08:03:55.252498+0000","last_active":"2023-11-24T08:03:57.449518+0000","last_peered":"2023-11-24T08:03:57.449518+0000","last_clean":"2023-11-24T08:03:57.449518+0000","last_became_active":"2023-11-24T08:03:55.252172+0000","last_became_peered":"2023-11-24T08:03:55.252172+0000","last_unstale":"2023-11-24T08:03:57.449518+0000","last_undegraded":"2023-11-24T08:03:57.449518+0000","last_fullsized":"2023-11-24T08:03:57.449518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.458202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.835}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8240000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.78}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9239999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6319999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.794}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:04:01.279 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:04:01.279 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T08:04:01.280 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T08:04:01.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T08:04:01.280 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T08:04:01.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.546+0000 7f95c91ef640 1 Processor -- start 2023-11-24T08:04:01.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.546+0000 7f95c91ef640 1 -- start start 2023-11-24T08:04:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.548+0000 7f95c91ef640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f95c40cf500 0x7f95c40cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.548+0000 7f95c91ef640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c4140900 con 0x7f95c40cf050 2023-11-24T08:04:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.548+0000 7f95c91ef640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c415fa70 con 0x7f95c413dfd0 2023-11-24T08:04:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.548+0000 7f95c91ef640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c415c580 con 0x7f95c40cf500 2023-11-24T08:04:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.548+0000 7f95bbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f95c40cf050 0x7f95c40cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56624/0 (socket says 172.21.15.64:56624) 2023-11-24T08:04:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.548+0000 7f95bbfff640 1 -- 172.21.15.64:0/1496091696 learned_addr learned my addr 172.21.15.64:0/1496091696 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c89ee640 1 --2- 172.21.15.64:0/1496091696 >> v2:172.21.15.91:3300/0 conn(0x7f95c40cf500 0x7f95c40cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644214225 0 0) 0x7f95c4140900 con 0x7f95c40cf050 2023-11-24T08:04:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b0003520 con 0x7f95c40cf050 2023-11-24T08:04:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1973680035 0 0) 0x7f95b0003520 con 0x7f95c40cf050 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 >> v2:172.21.15.91:3300/0 conn(0x7f95c40cf500 msgr2=0x7f95c40cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 --2- 172.21.15.64:0/1496091696 >> v2:172.21.15.91:3300/0 conn(0x7f95c40cf500 0x7f95c40cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 >> v1:172.21.15.91:6789/0 conn(0x7f95c413dfd0 legacy=0x7f95c415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c415ee60 con 0x7f95c40cf050 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f95ac001670 con 0x7f95c40cf050 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 >> v1:172.21.15.64:6789/0 conn(0x7f95c40cf050 legacy=0x7f95c40cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00039e0 0x7f95b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b00047c0 0x7f95b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95b0005180 0x7f95b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c415c580 con 0x7f95b00047c0 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b000c0b0 con 0x7f95b0005180 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.549+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b000c270 con 0x7f95b00039e0 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95bbfff640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b00047c0 0x7f95b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95c3fff640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00039e0 0x7f95b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95c89ee640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95b0005180 0x7f95b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95bbfff640 1 -- 172.21.15.64:0/1496091696 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00039e0 msgr2=0x7f95b0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95bbfff640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00039e0 0x7f95b0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95bbfff640 1 -- 172.21.15.64:0/1496091696 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95b0005180 msgr2=0x7f95b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95bbfff640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95b0005180 0x7f95b000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95c89ee640 1 -- 172.21.15.64:0/1496091696 reap_dead start 2023-11-24T08:04:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.550+0000 7f95bbfff640 1 -- 172.21.15.64:0/1496091696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0003520 con 0x7f95b00047c0 2023-11-24T08:04:01.554 INFO:tasks.ceph.mds.b.smithi091.stderr:2023-11-24T08:04:01.549+0000 7fb6ad423640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:04:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.551+0000 7f95bbfff640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b00047c0 0x7f95b0004c40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c160 tx=0x7f95b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c58e8b150c6c899 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac005ad0 con 0x7f95b00047c0 2023-11-24T08:04:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac005c50 con 0x7f95b00047c0 2023-11-24T08:04:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c37fe640 1 -- 172.21.15.64:0/1496091696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac02c850 con 0x7f95b00047c0 2023-11-24T08:04:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c91ef640 1 -- 172.21.15.64:0/1496091696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b00047c0 msgr2=0x7f95b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c91ef640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b00047c0 0x7f95b0004c40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c160 tx=0x7f95b00057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c91ef640 1 -- 172.21.15.64:0/1496091696 shutdown_connections 2023-11-24T08:04:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c91ef640 1 --2- 172.21.15.64:0/1496091696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b00047c0 0x7f95b0004c40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c91ef640 1 -- 172.21.15.64:0/1496091696 >> 172.21.15.64:0/1496091696 conn(0x7f95c406a860 msgr2=0x7f95c406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c91ef640 1 -- 172.21.15.64:0/1496091696 shutdown_connections 2023-11-24T08:04:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.552+0000 7f95c91ef640 1 -- 172.21.15.64:0/1496091696 wait complete. 2023-11-24T08:04:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.554+0000 7f95c91ef640 1 Processor -- start 2023-11-24T08:04:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.554+0000 7f95c91ef640 1 -- start start 2023-11-24T08:04:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.555+0000 7f95c91ef640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00047c0 0x7f95c41f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.555+0000 7f95c91ef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95c4139630 0x7f95c4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.555+0000 7f95c91ef640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95c415ac20 0x7f95c40af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95c91ef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c413fb40 con 0x7f95c4139630 2023-11-24T08:04:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95c91ef640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c415b0b0 con 0x7f95c415ac20 2023-11-24T08:04:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95c91ef640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c415b390 con 0x7f95b00047c0 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95c4139630 0x7f95c4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95c4139630 0x7f95c4071ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36696/0 (socket says 172.21.15.64:36696) 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95bbfff640 1 -- 172.21.15.64:0/1407275824 learned_addr learned my addr 172.21.15.64:0/1407275824 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95c89ee640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95c415ac20 0x7f95c40af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95c3fff640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00047c0 0x7f95c41f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95bbfff640 1 -- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00047c0 msgr2=0x7f95c41f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95bbfff640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00047c0 0x7f95c41f75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95bbfff640 1 -- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95c415ac20 msgr2=0x7f95c40af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.556+0000 7f95bbfff640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95c415ac20 0x7f95c40af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.557+0000 7f95bbfff640 1 -- 172.21.15.64:0/1407275824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c41f6b00 con 0x7f95c4139630 2023-11-24T08:04:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.557+0000 7f95c89ee640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95c415ac20 0x7f95c40af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.557+0000 7f95bbfff640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95c4139630 0x7f95c4071ab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95ac02c3f0 tx=0x7f95ac000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.557+0000 7f95c17fa640 1 -- 172.21.15.64:0/1407275824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac03c040 con 0x7f95c4139630 2023-11-24T08:04:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.557+0000 7f95c17fa640 1 -- 172.21.15.64:0/1407275824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac02cc60 con 0x7f95c4139630 2023-11-24T08:04:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.557+0000 7f95c17fa640 1 -- 172.21.15.64:0/1407275824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac007070 con 0x7f95c4139630 2023-11-24T08:04:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.557+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c41f6de0 con 0x7f95c4139630 2023-11-24T08:04:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.557+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c4071ff0 con 0x7f95c4139630 2023-11-24T08:04:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.558+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c404e840 con 0x7f95c4139630 2023-11-24T08:04:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.559+0000 7f95c17fa640 1 -- 172.21.15.64:0/1407275824 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95ac032070 con 0x7f95c4139630 2023-11-24T08:04:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.560+0000 7f95c17fa640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95a406ae70 0x7f95a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.560+0000 7f95c17fa640 1 -- 172.21.15.64:0/1407275824 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95ac0aa860 con 0x7f95c4139630 2023-11-24T08:04:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.561+0000 7f95c3fff640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95a406ae70 0x7f95a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.562+0000 7f95c3fff640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95a406ae70 0x7f95a406d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95b4000c00 tx=0x7f95b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:01.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.565+0000 7f95c17fa640 1 -- 172.21.15.64:0/1407275824 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95ac075980 con 0x7f95c4139630 2023-11-24T08:04:01.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.813+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f95c40ce9a0 con 0x7f95c4139630 2023-11-24T08:04:01.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.822+0000 7f95c17fa640 1 -- 172.21.15.64:0/1407275824 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171027 (secure 0 0 0) 0x7f95ac078fa0 con 0x7f95c4139630 2023-11-24T08:04:01.825 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95a406ae70 msgr2=0x7f95a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95a406ae70 0x7f95a406d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95b4000c00 tx=0x7f95b4009040 comp rx=0 tx=0).stop 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95c4139630 msgr2=0x7f95c4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95c4139630 0x7f95c4071ab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95ac02c3f0 tx=0x7f95ac000fe0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 shutdown_connections 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95a406ae70 0x7f95a406d330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95c415ac20 0x7f95c40af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95c4139630 0x7f95c4071ab0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 --2- 172.21.15.64:0/1407275824 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95b00047c0 0x7f95c41f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 >> 172.21.15.64:0/1407275824 conn(0x7f95c406a860 msgr2=0x7f95c413c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:01.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 shutdown_connections 2023-11-24T08:04:01.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:01.827+0000 7f95c91ef640 1 -- 172.21.15.64:0/1407275824 wait complete. 2023-11-24T08:04:01.848 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"active_gid":4113,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":401100545},{"type":"v1","addr":"172.21.15.91:6833","nonce":401100545}]},"active_addr":"172.21.15.91:6833/401100545","active_change":"2023-11-24T08:03:35.990698+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.91:0","nonce":1800554389}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.91:0","nonce":670178181}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.91:0","nonce":2728697298}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.91:0","nonce":612484242}]}]} 2023-11-24T08:04:01.851 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T08:04:01.851 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T08:04:01.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:02.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.078+0000 7f180d632640 1 Processor -- start 2023-11-24T08:04:02.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.079+0000 7f180d632640 1 -- start start 2023-11-24T08:04:02.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.080+0000 7f180d632640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f180815b130 0x7f18080b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.080+0000 7f180d632640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f18080b2a50 0x7f180806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.080+0000 7f180d632640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f180806cfa0 0x7f180815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f180d632640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180815b510 con 0x7f180815b130 2023-11-24T08:04:02.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f180d632640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180806d410 con 0x7f18080b2a50 2023-11-24T08:04:02.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f180d632640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18080b41d0 con 0x7f180806cfa0 2023-11-24T08:04:02.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f1807fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f180815b130 0x7f18080b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f1807fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f180815b130 0x7f18080b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36702/0 (socket says 172.21.15.64:36702) 2023-11-24T08:04:02.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f1807fff640 1 -- 172.21.15.64:0/1267007547 learned_addr learned my addr 172.21.15.64:0/1267007547 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:02.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f17fffff640 1 --2- 172.21.15.64:0/1267007547 >> v2:172.21.15.91:3300/0 conn(0x7f18080b2a50 0x7f180806ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f180ce31640 1 --2- 172.21.15.64:0/1267007547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f180806cfa0 0x7f180815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f1807fff640 1 -- 172.21.15.64:0/1267007547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f180806cfa0 msgr2=0x7f180815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:02.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f1807fff640 1 --2- 172.21.15.64:0/1267007547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f180806cfa0 0x7f180815ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f1807fff640 1 -- 172.21.15.64:0/1267007547 >> v2:172.21.15.91:3300/0 conn(0x7f18080b2a50 msgr2=0x7f180806ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:02.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f1807fff640 1 --2- 172.21.15.64:0/1267007547 >> v2:172.21.15.91:3300/0 conn(0x7f18080b2a50 0x7f180806ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f1807fff640 1 -- 172.21.15.64:0/1267007547 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180806d840 con 0x7f180815b130 2023-11-24T08:04:02.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.081+0000 7f180ce31640 1 --2- 172.21.15.64:0/1267007547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f180806cfa0 0x7f180815ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.082+0000 7f1807fff640 1 --2- 172.21.15.64:0/1267007547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180815b130 0x7f18080b2510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1808072120 tx=0x7f17f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fd4807831350e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.083+0000 7f18077fe640 1 -- 172.21.15.64:0/1267007547 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f8031640 con 0x7f180815b130 2023-11-24T08:04:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.083+0000 7f18077fe640 1 -- 172.21.15.64:0/1267007547 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f80317a0 con 0x7f180815b130 2023-11-24T08:04:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.083+0000 7f18077fe640 1 -- 172.21.15.64:0/1267007547 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f8031b30 con 0x7f180815b130 2023-11-24T08:04:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.083+0000 7f180d632640 1 -- 172.21.15.64:0/1267007547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180815b130 msgr2=0x7f18080b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.083+0000 7f180d632640 1 --2- 172.21.15.64:0/1267007547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180815b130 0x7f18080b2510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1808072120 tx=0x7f17f802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.084+0000 7f180d632640 1 -- 172.21.15.64:0/1267007547 shutdown_connections 2023-11-24T08:04:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.084+0000 7f180d632640 1 --2- 172.21.15.64:0/1267007547 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f180806cfa0 0x7f180815ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.084+0000 7f180d632640 1 --2- 172.21.15.64:0/1267007547 >> v2:172.21.15.91:3300/0 conn(0x7f18080b2a50 0x7f180806ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.084+0000 7f180d632640 1 --2- 172.21.15.64:0/1267007547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180815b130 0x7f18080b2510 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.084+0000 7f180d632640 1 -- 172.21.15.64:0/1267007547 >> 172.21.15.64:0/1267007547 conn(0x7f18080cbf60 msgr2=0x7f18080cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.084+0000 7f180d632640 1 -- 172.21.15.64:0/1267007547 shutdown_connections 2023-11-24T08:04:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.084+0000 7f180d632640 1 -- 172.21.15.64:0/1267007547 wait complete. 2023-11-24T08:04:02.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.085+0000 7f180d632640 1 Processor -- start 2023-11-24T08:04:02.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.086+0000 7f180d632640 1 -- start start 2023-11-24T08:04:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f180d632640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180806cfa0 0x7f1808146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f180d632640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18080b2a50 0x7f180815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f180d632640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f180815b130 0x7f18080ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f180d632640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18080adcb0 con 0x7f180806cfa0 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f180d632640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18080ad030 con 0x7f180815b130 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f180d632640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18080ad310 con 0x7f18080b2a50 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f1807fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180806cfa0 0x7f1808146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f1807fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180806cfa0 0x7f1808146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36714/0 (socket says 172.21.15.64:36714) 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f1807fff640 1 -- 172.21.15.64:0/2091704499 learned_addr learned my addr 172.21.15.64:0/2091704499 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f17fffff640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18080b2a50 0x7f180815dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f1807fff640 1 -- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18080b2a50 msgr2=0x7f180815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f1807fff640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18080b2a50 0x7f180815dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f1807fff640 1 -- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f180815b130 msgr2=0x7f18080ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f1807fff640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f180815b130 0x7f18080ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f1807fff640 1 -- 172.21.15.64:0/2091704499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f80094d0 con 0x7f180806cfa0 2023-11-24T08:04:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.087+0000 7f17fffff640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18080b2a50 0x7f180815dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.088+0000 7f1807fff640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180806cfa0 0x7f1808146070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f17f8000c00 tx=0x7f17f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.088+0000 7f18057fa640 1 -- 172.21.15.64:0/2091704499 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f803ac60 con 0x7f180806cfa0 2023-11-24T08:04:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.088+0000 7f18057fa640 1 -- 172.21.15.64:0/2091704499 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f803adc0 con 0x7f180806cfa0 2023-11-24T08:04:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.088+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18080ad5f0 con 0x7f180806cfa0 2023-11-24T08:04:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.088+0000 7f18057fa640 1 -- 172.21.15.64:0/2091704499 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f8044630 con 0x7f180806cfa0 2023-11-24T08:04:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.088+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180815e4f0 con 0x7f180806cfa0 2023-11-24T08:04:02.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.089+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18080668f0 con 0x7f180806cfa0 2023-11-24T08:04:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.090+0000 7f18057fa640 1 -- 172.21.15.64:0/2091704499 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17f802f050 con 0x7f180806cfa0 2023-11-24T08:04:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.091+0000 7f18057fa640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f17e806ae40 0x7f17e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.092+0000 7f18057fa640 1 -- 172.21.15.64:0/2091704499 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17f80ab550 con 0x7f180806cfa0 2023-11-24T08:04:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.092+0000 7f17fffff640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f17e806ae40 0x7f17e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.093+0000 7f17fffff640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f17e806ae40 0x7f17e806d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18080aee00 tx=0x7f17f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.096+0000 7f18057fa640 1 -- 172.21.15.64:0/2091704499 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17f8076560 con 0x7f180806cfa0 2023-11-24T08:04:02.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.290+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18080adf20 con 0x7f180806cfa0 2023-11-24T08:04:02.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.292+0000 7f18057fa640 1 -- 172.21.15.64:0/2091704499 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f17f8031e60 con 0x7f180806cfa0 2023-11-24T08:04:02.294 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:02.294 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f17e806ae40 msgr2=0x7f17e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f17e806ae40 0x7f17e806d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18080aee00 tx=0x7f17f0008040 comp rx=0 tx=0).stop 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180806cfa0 msgr2=0x7f1808146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180806cfa0 0x7f1808146070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f17f8000c00 tx=0x7f17f80046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 shutdown_connections 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f17e806ae40 0x7f17e806d300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f180815b130 0x7f18080ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18080b2a50 0x7f180815dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 --2- 172.21.15.64:0/2091704499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f180806cfa0 0x7f1808146070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.295+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 >> 172.21.15.64:0/2091704499 conn(0x7f18080cbf60 msgr2=0x7f180806dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.296+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 shutdown_connections 2023-11-24T08:04:02.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.296+0000 7f180d632640 1 -- 172.21.15.64:0/2091704499 wait complete. 2023-11-24T08:04:02.314 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T08:04:02.314 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:02.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.539+0000 7f0d7ed27640 1 Processor -- start 2023-11-24T08:04:02.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.540+0000 7f0d7ed27640 1 -- start start 2023-11-24T08:04:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.541+0000 7f0d7ed27640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d7815d5b0 0x7f0d780d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.541+0000 7f0d7ed27640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d780b1650 0x7f0d780d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7ed27640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d780cf0c0 con 0x7f0d780d14a0 2023-11-24T08:04:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7ed27640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d780b01b0 con 0x7f0d780b1650 2023-11-24T08:04:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7ed27640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7815d990 con 0x7f0d7815d5b0 2023-11-24T08:04:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7d524640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0d780d14a0 0x7f0d780cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56626/0 (socket says 172.21.15.64:56626) 2023-11-24T08:04:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7d524640 1 -- 172.21.15.64:0/2127463525 learned_addr learned my addr 172.21.15.64:0/2127463525 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:02.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7dd25640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d7815d5b0 0x7f0d780d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7e526640 1 --2- 172.21.15.64:0/2127463525 >> v2:172.21.15.91:3300/0 conn(0x7f0d780b1650 0x7f0d780d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773345402 0 0) 0x7f0d780cf0c0 con 0x7f0d780d14a0 2023-11-24T08:04:02.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.542+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d64003520 con 0x7f0d780d14a0 2023-11-24T08:04:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3878466752 0 0) 0x7f0d64003520 con 0x7f0d780d14a0 2023-11-24T08:04:02.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d7815d5b0 msgr2=0x7f0d780d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d7815d5b0 0x7f0d780d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 >> v2:172.21.15.91:3300/0 conn(0x7f0d780b1650 msgr2=0x7f0d780d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 --2- 172.21.15.64:0/2127463525 >> v2:172.21.15.91:3300/0 conn(0x7f0d780b1650 0x7f0d780d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780d1e40 con 0x7f0d780d14a0 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0d68001670 con 0x7f0d780d14a0 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 >> v1:172.21.15.64:6789/0 conn(0x7f0d780d14a0 legacy=0x7f0d780cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d640039e0 0x7f0d64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d7815d5b0 0x7f0d640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d780b01b0 con 0x7f0d64004d00 2023-11-24T08:04:02.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7815d990 con 0x7f0d640039e0 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6400b980 con 0x7f0d7815d5b0 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7e526640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7dd25640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d640039e0 0x7f0d64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7e526640 1 -- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d7815d5b0 msgr2=0x7f0d640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7d524640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d7815d5b0 0x7f0d640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.543+0000 7f0d7e526640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d7815d5b0 0x7f0d640047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.544+0000 7f0d7e526640 1 -- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d640039e0 msgr2=0x7f0d64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.544+0000 7f0d7e526640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d640039e0 0x7f0d64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.544+0000 7f0d7e526640 1 -- 172.21.15.64:0/2127463525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d64003520 con 0x7f0d64004d00 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.545+0000 7f0d7e526640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d64005180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009fc0 tx=0x7f0d640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e930d35f66bbed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.545+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7002cc60 con 0x7f0d64004d00 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.545+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7000f040 con 0x7f0d64004d00 2023-11-24T08:04:02.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.545+0000 7f0d7cd23640 1 -- 172.21.15.64:0/2127463525 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70034440 con 0x7f0d64004d00 2023-11-24T08:04:02.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.545+0000 7f0d7ed27640 1 -- 172.21.15.64:0/2127463525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 msgr2=0x7f0d64005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.545+0000 7f0d7ed27640 1 --2- 172.21.15.64:0/2127463525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d64005180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009fc0 tx=0x7f0d640057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.546+0000 7f0d7e526640 1 -- 172.21.15.64:0/2127463525 reap_dead start 2023-11-24T08:04:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.546+0000 7f0d7ed27640 1 -- 172.21.15.64:0/2127463525 shutdown_connections 2023-11-24T08:04:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.546+0000 7f0d7ed27640 1 -- 172.21.15.64:0/2127463525 >> 172.21.15.64:0/2127463525 conn(0x7f0d7809c3b0 msgr2=0x7f0d780d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.546+0000 7f0d7ed27640 1 -- 172.21.15.64:0/2127463525 shutdown_connections 2023-11-24T08:04:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.546+0000 7f0d7ed27640 1 -- 172.21.15.64:0/2127463525 wait complete. 2023-11-24T08:04:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.548+0000 7f0d7ed27640 1 Processor -- start 2023-11-24T08:04:02.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.548+0000 7f0d7ed27640 1 -- start start 2023-11-24T08:04:02.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7ed27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d780c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7ed27640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d780b5480 0x7f0d7815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7ed27640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d780c8300 0x7f0d780cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7ed27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7815a790 con 0x7f0d64004d00 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7ed27640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d780c7260 con 0x7f0d780b5480 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7ed27640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d780c7540 con 0x7f0d780c8300 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7dd25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d780c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7dd25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d780c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36728/0 (socket says 172.21.15.64:36728) 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.549+0000 7f0d7dd25640 1 -- 172.21.15.64:0/1476033409 learned_addr learned my addr 172.21.15.64:0/1476033409 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7d524640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d780b5480 0x7f0d7815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7e526640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d780c8300 0x7f0d780cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7dd25640 1 -- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d780c8300 msgr2=0x7f0d780cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7dd25640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d780c8300 0x7f0d780cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7dd25640 1 -- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d780b5480 msgr2=0x7f0d7815a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7dd25640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d780b5480 0x7f0d7815a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7dd25640 1 -- 172.21.15.64:0/1476033409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780c7820 con 0x7f0d64004d00 2023-11-24T08:04:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7d524640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d780b5480 0x7f0d7815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7dd25640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d780c6ba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d74011540 tx=0x7f0d74011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:02.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d6e7fc640 1 -- 172.21.15.64:0/1476033409 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74015930 con 0x7f0d64004d00 2023-11-24T08:04:02.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d6e7fc640 1 -- 172.21.15.64:0/1476033409 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d74015a90 con 0x7f0d64004d00 2023-11-24T08:04:02.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d6e7fc640 1 -- 172.21.15.64:0/1476033409 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74015e60 con 0x7f0d64004d00 2023-11-24T08:04:02.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.550+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d780c7b00 con 0x7f0d64004d00 2023-11-24T08:04:02.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.551+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d780b5900 con 0x7f0d64004d00 2023-11-24T08:04:02.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.552+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7804e840 con 0x7f0d64004d00 2023-11-24T08:04:02.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.552+0000 7f0d6e7fc640 1 -- 172.21.15.64:0/1476033409 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d7400f460 con 0x7f0d64004d00 2023-11-24T08:04:02.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.553+0000 7f0d6e7fc640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d5c06ae70 0x7f0d5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:02.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.554+0000 7f0d6e7fc640 1 -- 172.21.15.64:0/1476033409 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d74089eb0 con 0x7f0d64004d00 2023-11-24T08:04:02.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.554+0000 7f0d7d524640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d5c06ae70 0x7f0d5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.555+0000 7f0d7d524640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d5c06ae70 0x7f0d5c06d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0d68001eb0 tx=0x7f0d68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.558+0000 7f0d6e7fc640 1 -- 172.21.15.64:0/1476033409 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d74054ec0 con 0x7f0d64004d00 2023-11-24T08:04:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.751+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d780678d0 con 0x7f0d64004d00 2023-11-24T08:04:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.754+0000 7f0d6e7fc640 1 -- 172.21.15.64:0/1476033409 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f0d740584e0 con 0x7f0d64004d00 2023-11-24T08:04:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:02.756 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:02.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d5c06ae70 msgr2=0x7f0d5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d5c06ae70 0x7f0d5c06d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0d68001eb0 tx=0x7f0d68009040 comp rx=0 tx=0).stop 2023-11-24T08:04:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 msgr2=0x7f0d780c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d780c6ba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d74011540 tx=0x7f0d74011a00 comp rx=0 tx=0).stop 2023-11-24T08:04:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 shutdown_connections 2023-11-24T08:04:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d5c06ae70 0x7f0d5c06d330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d780c8300 0x7f0d780cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d780b5480 0x7f0d7815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 --2- 172.21.15.64:0/1476033409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d64004d00 0x7f0d780c6ba0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 >> 172.21.15.64:0/1476033409 conn(0x7f0d7809c3b0 msgr2=0x7f0d780d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 shutdown_connections 2023-11-24T08:04:02.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:02.757+0000 7f0d7ed27640 1 -- 172.21.15.64:0/1476033409 wait complete. 2023-11-24T08:04:02.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T08:04:02.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T08:04:02.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T08:04:02.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T08:04:02.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T08:04:02.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T08:04:02.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T08:04:02.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T08:04:03.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.094+0000 7f57bcd3c640 1 Processor -- start 2023-11-24T08:04:03.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.094+0000 7f57bcd3c640 1 -- start start 2023-11-24T08:04:03.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.096+0000 7f57bcd3c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f57b8070ce0 0x7f57b80ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.096+0000 7f57bcd3c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b80b2f60 con 0x7f57b815cea0 2023-11-24T08:04:03.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.096+0000 7f57bcd3c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b80cb680 con 0x7f57b8156ff0 2023-11-24T08:04:03.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.096+0000 7f57bcd3c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b806c860 con 0x7f57b8070ce0 2023-11-24T08:04:03.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.097+0000 7f57b6d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f57b815cea0 0x7f57b80b33f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56640/0 (socket says 172.21.15.64:56640) 2023-11-24T08:04:03.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.098+0000 7f57b6d9d640 1 -- 172.21.15.64:0/1979037059 learned_addr learned my addr 172.21.15.64:0/1979037059 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.099+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908644752 0 0) 0x7f57b80b2f60 con 0x7f57b815cea0 2023-11-24T08:04:03.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.099+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a0003520 con 0x7f57b815cea0 2023-11-24T08:04:03.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.099+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3622364452 0 0) 0x7f57a0003520 con 0x7f57b815cea0 2023-11-24T08:04:03.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.099+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 >> v2:172.21.15.91:3300/0 conn(0x7f57b8070ce0 msgr2=0x7f57b80ce250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T08:04:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 --2- 172.21.15.64:0/1979037059 >> v2:172.21.15.91:3300/0 conn(0x7f57b8070ce0 0x7f57b80ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 >> v1:172.21.15.91:6789/0 conn(0x7f57b8156ff0 legacy=0x7f57b806c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8071150 con 0x7f57b815cea0 2023-11-24T08:04:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f57ac001670 con 0x7f57b815cea0 2023-11-24T08:04:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 >> v1:172.21.15.64:6789/0 conn(0x7f57b815cea0 legacy=0x7f57b80b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.099+0000 7f6941b89640 1 Processor -- start 2023-11-24T08:04:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f6941b89640 1 -- start start 2023-11-24T08:04:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57a0004270 0x7f57a0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a00047c0 0x7f57a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57a0005380 0x7f57a000ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b806c860 con 0x7f57a00047c0 2023-11-24T08:04:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a000d300 con 0x7f57a0005380 2023-11-24T08:04:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.100+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a000d4c0 con 0x7f57a0004270 2023-11-24T08:04:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.101+0000 7f57b5d9b640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a00047c0 0x7f57a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.101+0000 7f57b6d9d640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57a0005380 0x7f57a000ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.103+0000 7f6941b89640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f693c0acc20 0x7f693c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.103+0000 7f6941b89640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0b2080 0x7f693c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.103+0000 7f6941b89640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f693c161d80 con 0x7f693c1619a0 2023-11-24T08:04:03.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.103+0000 7f6941b89640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693c0ad170 con 0x7f693c0acc20 2023-11-24T08:04:03.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.103+0000 7f6941b89640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693c0ad450 con 0x7f693c0b2080 2023-11-24T08:04:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.104+0000 7f693affd640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f693c0acc20 0x7f693c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.104+0000 7fba7a91d640 1 Processor -- start 2023-11-24T08:04:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.104+0000 7f693bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0b2080 0x7f693c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.101+0000 7f57b5d9b640 1 -- 172.21.15.64:0/1979037059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57a0004270 msgr2=0x7f57a0003970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T08:04:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.101+0000 7f57b5d9b640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57a0004270 0x7f57a0003970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.101+0000 7f57b5d9b640 1 -- 172.21.15.64:0/1979037059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57a0005380 msgr2=0x7f57a000ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.101+0000 7f57b5d9b640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57a0005380 0x7f57a000ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.101+0000 7f57b5d9b640 1 -- 172.21.15.64:0/1979037059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8071150 con 0x7f57a00047c0 2023-11-24T08:04:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.103+0000 7f57b6d9d640 1 -- 172.21.15.64:0/1979037059 reap_dead start 2023-11-24T08:04:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.104+0000 7f57b5d9b640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a00047c0 0x7f57a0004c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f57b0007280 tx=0x7f57a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3fcebf6f409338 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.104+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0030ba0 con 0x7f57a00047c0 2023-11-24T08:04:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.104+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0030d00 con 0x7f57a00047c0 2023-11-24T08:04:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.105+0000 7fba7a91d640 1 -- start start 2023-11-24T08:04:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.104+0000 7f693bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0b2080 0x7f693c0b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54530/0 (socket says 172.21.15.64:54530) 2023-11-24T08:04:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.105+0000 7f693affd640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f693c0acc20 0x7f693c0b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34292/0 (socket says 172.21.15.64:34292) 2023-11-24T08:04:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.105+0000 7f693affd640 1 -- 172.21.15.64:0/3476701755 learned_addr learned my addr 172.21.15.64:0/3476701755 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.105+0000 7f57bcd3c640 1 -- 172.21.15.64:0/1979037059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a00047c0 msgr2=0x7f57a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.105+0000 7f57b559a640 1 -- 172.21.15.64:0/1979037059 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0030ba0 con 0x7f57a00047c0 2023-11-24T08:04:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.105+0000 7f57bcd3c640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a00047c0 0x7f57a0004c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f57b0007280 tx=0x7f57a00057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.106+0000 7f57bcd3c640 1 -- 172.21.15.64:0/1979037059 shutdown_connections 2023-11-24T08:04:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.106+0000 7f57bcd3c640 1 --2- 172.21.15.64:0/1979037059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a00047c0 0x7f57a0004c40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.106+0000 7f57bcd3c640 1 -- 172.21.15.64:0/1979037059 >> 172.21.15.64:0/1979037059 conn(0x7f57b80b0840 msgr2=0x7f57b80b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.106+0000 7f57bcd3c640 1 -- 172.21.15.64:0/1979037059 shutdown_connections 2023-11-24T08:04:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.106+0000 7f57bcd3c640 1 -- 172.21.15.64:0/1979037059 wait complete. 2023-11-24T08:04:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.107+0000 7f693a7fc640 1 -- 172.21.15.64:0/3476701755 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019026871 0 0) 0x7f693c161d80 con 0x7f693c1619a0 2023-11-24T08:04:03.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.107+0000 7fba7a91d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fba74156ff0 0x7fba74070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.107+0000 7fba7a91d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fba740ce250 0x7fba7406c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.107+0000 7f693a7fc640 1 -- 172.21.15.64:0/3476701755 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6924003520 con 0x7f693c1619a0 2023-11-24T08:04:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.108+0000 7fba7a91d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba740b2f60 con 0x7fba7415cea0 2023-11-24T08:04:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.108+0000 7fba7a91d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba740cb680 con 0x7fba740ce250 2023-11-24T08:04:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.109+0000 7fba7a91d640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba740ce6c0 con 0x7fba74156ff0 2023-11-24T08:04:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.109+0000 7fba78eba640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fba7415cea0 0x7fba740b33f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:50112/0 (socket says 172.21.15.64:50112) 2023-11-24T08:04:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.109+0000 7fba6b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fba740ce250 0x7fba7406c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.107+0000 7f693affd640 1 -- 172.21.15.64:0/3476701755 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0b2080 msgr2=0x7f693c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.107+0000 7f693affd640 1 --2- 172.21.15.64:0/3476701755 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0b2080 0x7f693c0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.107+0000 7f693affd640 1 -- 172.21.15.64:0/3476701755 >> v1:172.21.15.64:6789/0 conn(0x7f693c1619a0 legacy=0x7f693c0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.107+0000 7f693affd640 1 -- 172.21.15.64:0/3476701755 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c0ad730 con 0x7f693c0acc20 2023-11-24T08:04:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.109+0000 7fba6b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fba740ce250 0x7fba7406c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36750/0 (socket says 172.21.15.64:36750) 2023-11-24T08:04:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.109+0000 7fba6b7fe640 1 -- 172.21.15.64:0/1490101429 learned_addr learned my addr 172.21.15.64:0/1490101429 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.109+0000 7f57bcd3c640 1 Processor -- start 2023-11-24T08:04:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.110+0000 7f57bcd3c640 1 -- start start 2023-11-24T08:04:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.110+0000 7f693affd640 1 --2- 172.21.15.64:0/3476701755 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c0acc20 0x7f693c0b32a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6930009870 tx=0x7f693002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=165281faac5fab0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.110+0000 7fba6b7fe640 1 -- 172.21.15.64:0/1490101429 >> v2:172.21.15.91:3300/0 conn(0x7fba74156ff0 msgr2=0x7fba74070ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T08:04:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.110+0000 7f693a7fc640 1 -- 172.21.15.64:0/3476701755 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930031640 con 0x7f693c0acc20 2023-11-24T08:04:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.110+0000 7fba6b7fe640 1 --2- 172.21.15.64:0/1490101429 >> v2:172.21.15.91:3300/0 conn(0x7fba74156ff0 0x7fba74070ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.110+0000 7fba6b7fe640 1 -- 172.21.15.64:0/1490101429 >> v1:172.21.15.91:6789/0 conn(0x7fba7415cea0 legacy=0x7fba740b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.112+0000 7f57bcd3c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57a00047c0 0x7f57b8150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.111+0000 7fba6b7fe640 1 -- 172.21.15.64:0/1490101429 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba740ce950 con 0x7fba740ce250 2023-11-24T08:04:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.111+0000 7f693a7fc640 1 -- 172.21.15.64:0/3476701755 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69300317a0 con 0x7f693c0acc20 2023-11-24T08:04:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.111+0000 7f693a7fc640 1 -- 172.21.15.64:0/3476701755 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930031b30 con 0x7f693c0acc20 2023-11-24T08:04:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.112+0000 7f6941b89640 1 -- 172.21.15.64:0/3476701755 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c0acc20 msgr2=0x7f693c0b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.112+0000 7f6941b89640 1 --2- 172.21.15.64:0/3476701755 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c0acc20 0x7f693c0b32a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6930009870 tx=0x7f693002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.112+0000 7f57bcd3c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b81eea60 0x7f57b80ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.112+0000 7fba6b7fe640 1 --2- 172.21.15.64:0/1490101429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba740ce250 0x7fba7406c150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fba7406bc30 tx=0x7fba5c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16b1a06174a6181 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.112+0000 7f57b659c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57a00047c0 0x7f57b8150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7f57b659c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57a00047c0 0x7f57b8150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54546/0 (socket says 172.21.15.64:54546) 2023-11-24T08:04:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7f57b5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b81eea60 0x7f57b80ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7f57b5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b81eea60 0x7f57b80ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36760/0 (socket says 172.21.15.64:36760) 2023-11-24T08:04:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7f57b659c640 1 -- 172.21.15.64:0/574508041 learned_addr learned my addr 172.21.15.64:0/574508041 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7f57bcd3c640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b80aecf0 0x7f57b815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7fba737fe640 1 -- 172.21.15.64:0/1490101429 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c031890 con 0x7fba740ce250 2023-11-24T08:04:03.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7fba737fe640 1 -- 172.21.15.64:0/1490101429 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c0319f0 con 0x7fba740ce250 2023-11-24T08:04:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8153920 con 0x7f57b81eea60 2023-11-24T08:04:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80af170 con 0x7f57b80aecf0 2023-11-24T08:04:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8152ca0 con 0x7f57a00047c0 2023-11-24T08:04:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f57b6d9d640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b80aecf0 0x7f57b815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f6941b89640 1 -- 172.21.15.64:0/3476701755 shutdown_connections 2023-11-24T08:04:03.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f6941b89640 1 --2- 172.21.15.64:0/3476701755 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0b2080 0x7f693c0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f6941b89640 1 --2- 172.21.15.64:0/3476701755 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c0acc20 0x7f693c0b32a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f6941b89640 1 -- 172.21.15.64:0/3476701755 >> 172.21.15.64:0/3476701755 conn(0x7f693c0709f0 msgr2=0x7f693c0718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.113+0000 7fba737fe640 1 -- 172.21.15.64:0/1490101429 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c031d80 con 0x7fba740ce250 2023-11-24T08:04:03.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f6941b89640 1 -- 172.21.15.64:0/3476701755 shutdown_connections 2023-11-24T08:04:03.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f57b5d9b640 1 -- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57a00047c0 msgr2=0x7f57b8150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f57b5d9b640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f57a00047c0 0x7f57b8150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f57b5d9b640 1 -- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b80aecf0 msgr2=0x7f57b815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f57b5d9b640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57b80aecf0 0x7f57b815bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f57b5d9b640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8152f80 con 0x7f57b81eea60 2023-11-24T08:04:03.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f57b5d9b640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b81eea60 0x7f57b80ae7b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f57b0004000 tx=0x7f57b0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7fba7a91d640 1 -- 172.21.15.64:0/1490101429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba740ce250 msgr2=0x7fba7406c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.115+0000 7fba7a91d640 1 --2- 172.21.15.64:0/1490101429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba740ce250 0x7fba7406c150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fba7406bc30 tx=0x7fba5c02b930 comp rx=0 tx=0).stop 2023-11-24T08:04:03.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.114+0000 7f6941b89640 1 -- 172.21.15.64:0/3476701755 wait complete. 2023-11-24T08:04:03.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.115+0000 7fba7a91d640 1 -- 172.21.15.64:0/1490101429 shutdown_connections 2023-11-24T08:04:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.115+0000 7fba7a91d640 1 --2- 172.21.15.64:0/1490101429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba740ce250 0x7fba7406c150 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.115+0000 7fba7a91d640 1 --2- 172.21.15.64:0/1490101429 >> v2:172.21.15.91:3300/0 conn(0x7fba74156ff0 0x7fba74070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.115+0000 7fba7a91d640 1 -- 172.21.15.64:0/1490101429 >> 172.21.15.64:0/1490101429 conn(0x7fba740b0840 msgr2=0x7fba740b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.116+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0044bd0 con 0x7f57b81eea60 2023-11-24T08:04:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.116+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0044d30 con 0x7f57b81eea60 2023-11-24T08:04:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.116+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0044bd0 con 0x7f57b81eea60 2023-11-24T08:04:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.116+0000 7fba7a91d640 1 -- 172.21.15.64:0/1490101429 shutdown_connections 2023-11-24T08:04:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.116+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b8153260 con 0x7f57b81eea60 2023-11-24T08:04:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.116+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b81eeee0 con 0x7f57b81eea60 2023-11-24T08:04:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.117+0000 7fba7a91d640 1 -- 172.21.15.64:0/1490101429 wait complete. 2023-11-24T08:04:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.117+0000 7f783001c640 1 Processor -- start 2023-11-24T08:04:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.117+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57b81590a0 con 0x7f57b81eea60 2023-11-24T08:04:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.118+0000 7f6941b89640 1 Processor -- start 2023-11-24T08:04:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.118+0000 7f783001c640 1 -- start start 2023-11-24T08:04:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.118+0000 7f6941b89640 1 -- start start 2023-11-24T08:04:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.120+0000 7f6941b89640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0acc20 0x7f693c1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.119+0000 7f783001c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7820053770 0x7f7820053b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.120+0000 7f783001c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7820007860 con 0x7f782003d0c0 2023-11-24T08:04:03.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.120+0000 7f783001c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7820058170 con 0x7f78200061d0 2023-11-24T08:04:03.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.120+0000 7f783001c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7820054090 con 0x7f7820053770 2023-11-24T08:04:03.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.120+0000 7fba7a91d640 1 Processor -- start 2023-11-24T08:04:03.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.120+0000 7f6941b89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c0b2080 0x7f693c14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f6941b89640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c1619a0 0x7f693c14f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f6941b89640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c1f6a00 con 0x7f693c0b2080 2023-11-24T08:04:03.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f6941b89640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c14de20 con 0x7f693c1619a0 2023-11-24T08:04:03.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f6941b89640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693c14e100 con 0x7f693c0acc20 2023-11-24T08:04:03.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0acc20 0x7f693c1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0acc20 0x7f693c1f64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54562/0 (socket says 172.21.15.64:54562) 2023-11-24T08:04:03.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f693b7fe640 1 -- 172.21.15.64:0/2687794466 learned_addr learned my addr 172.21.15.64:0/2687794466 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f693bfff640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c1619a0 0x7f693c14f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7fba7a91d640 1 -- start start 2023-11-24T08:04:03.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f693affd640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c0b2080 0x7f693c14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f693b7fe640 1 -- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c1619a0 msgr2=0x7f693c14f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f693b7fe640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c1619a0 0x7f693c14f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f782d5b7640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f78200061d0 0x7f7820057350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:50118/0 (socket says 172.21.15.64:50118) 2023-11-24T08:04:03.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.121+0000 7f782d5b7640 1 -- 172.21.15.64:0/2315649561 learned_addr learned my addr 172.21.15.64:0/2315649561 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f782ddb8640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7820053770 0x7f7820053b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f693b7fe640 1 -- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c0b2080 msgr2=0x7f693c14ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f693b7fe640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c0b2080 0x7f693c14ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f693b7fe640 1 -- 172.21.15.64:0/2687794466 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69300094d0 con 0x7f693c0acc20 2023-11-24T08:04:03.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f693bfff640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f693c1619a0 0x7f693c14f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:03.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f693affd640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c0b2080 0x7f693c14ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:03.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941646416 0 0) 0x7f7820007860 con 0x7f782003d0c0 2023-11-24T08:04:03.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7814003520 con 0x7f782003d0c0 2023-11-24T08:04:03.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7fba7a91d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba740ce250 0x7fba7414c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.122+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57b004a020 con 0x7f57b81eea60 2023-11-24T08:04:03.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7fba7a91d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba74156ff0 0x7fba740af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681115676 0 0) 0x7f7820058170 con 0x7f78200061d0 2023-11-24T08:04:03.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7820007860 con 0x7f78200061d0 2023-11-24T08:04:03.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7fba7a91d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7415cea0 0x7fba740afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f693b7fe640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0acc20 0x7f693c1f64c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f692c00c460 tx=0x7f692c00c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7fba7a91d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7414c5d0 con 0x7fba7415cea0 2023-11-24T08:04:03.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 484212783 0 0) 0x7f7814003520 con 0x7f782003d0c0 2023-11-24T08:04:03.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7820053770 msgr2=0x7f7820053b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7820053770 0x7f7820053b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 >> v1:172.21.15.91:6789/0 conn(0x7f78200061d0 legacy=0x7f7820057350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7820006c50 con 0x7f782003d0c0 2023-11-24T08:04:03.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.123+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f7828068590 con 0x7f782003d0c0 2023-11-24T08:04:03.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 >> v1:172.21.15.64:6789/0 conn(0x7f782003d0c0 legacy=0x7f7820057a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f782cdb6640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7814003b30 0x7f7814003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f782cdb6640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f7814004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f782cdb6640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7814005180 0x7f7814014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7820054090 con 0x7f78140047c0 2023-11-24T08:04:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7814014780 con 0x7f7814003b30 2023-11-24T08:04:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7814014910 con 0x7f7814005180 2023-11-24T08:04:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f57a6ffd640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f579806af30 0x7f579806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57b00b3b00 con 0x7f57b81eea60 2023-11-24T08:04:03.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7fba73fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba740ce250 0x7fba7414c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7fba73fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba740ce250 0x7fba7414c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54578/0 (socket says 172.21.15.64:54578) 2023-11-24T08:04:03.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c005480 con 0x7f693c0acc20 2023-11-24T08:04:03.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7fba73fff640 1 -- 172.21.15.64:0/2209212211 learned_addr learned my addr 172.21.15.64:0/2209212211 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba741509a0 con 0x7fba74156ff0 2023-11-24T08:04:03.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7414fd90 con 0x7fba740ce250 2023-11-24T08:04:03.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f782d5b7640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f7814004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7f57a6ffd640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f57980730d0 0x7f57980754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5798075ba0 con 0x7f57980730d0 2023-11-24T08:04:03.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b0044bd0 con 0x7f57b81eea60 2023-11-24T08:04:03.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.124+0000 7fba6b7fe640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba74156ff0 0x7fba740af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f782e5b9640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7814005180 0x7f7814014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f57b6d9d640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f57980730d0 0x7f57980754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693c14e3e0 con 0x7f693c0acc20 2023-11-24T08:04:03.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c14e9a0 con 0x7f693c0acc20 2023-11-24T08:04:03.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.126+0000 7f57b6d9d640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f57980730d0 0x7f57980754f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.126+0000 7fba78eba640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7415cea0 0x7fba740afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f782d5b7640 1 -- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7814005180 msgr2=0x7f7814014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f782d5b7640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7814005180 0x7f7814014190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f782d5b7640 1 -- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7814003b30 msgr2=0x7f7814003f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T08:04:03.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f782d5b7640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7814003b30 0x7f7814003f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c0035f0 con 0x7f693c0acc20 2023-11-24T08:04:03.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.126+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c0039e0 con 0x7f693c0acc20 2023-11-24T08:04:03.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.125+0000 7f782d5b7640 1 -- 172.21.15.64:0/2315649561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7820006c50 con 0x7f78140047c0 2023-11-24T08:04:03.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f691c000fc0 con 0x7f693c0acc20 2023-11-24T08:04:03.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7f782d5b7640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f7814004c40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f78240019d0 tx=0x7f78140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4946f4887de010f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7f57b659c640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f579806af30 0x7f579806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7f1ccf883640 1 Processor -- start 2023-11-24T08:04:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7fba78eba640 1 -- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba740ce250 msgr2=0x7fba7414c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7fba78eba640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fba740ce250 0x7fba7414c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7fba78eba640 1 -- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba74156ff0 msgr2=0x7fba740af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7fba78eba640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fba74156ff0 0x7fba740af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7fba78eba640 1 -- 172.21.15.64:0/2209212211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0094d0 con 0x7fba7415cea0 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.128+0000 7fba78eba640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7415cea0 0x7fba740afa20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c390 tx=0x7fba6c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.128+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0074e0 con 0x7fba7415cea0 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.128+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c007640 con 0x7fba7415cea0 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.128+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004a00 con 0x7fba7415cea0 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.128+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba74150070 con 0x7fba7415cea0 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.128+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824004aa0 con 0x7f78140047c0 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.128+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7824004c20 con 0x7f78140047c0 2023-11-24T08:04:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.128+0000 7f782cdb6640 1 -- 172.21.15.64:0/2315649561 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824007070 con 0x7f78140047c0 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f783001c640 1 -- 172.21.15.64:0/2315649561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 msgr2=0x7f7814004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f783001c640 1 --2- 172.21.15.64:0/2315649561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f7814004c40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f78240019d0 tx=0x7f78140057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f782e5b9640 1 -- 172.21.15.64:0/2315649561 reap_dead start 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f783001c640 1 -- 172.21.15.64:0/2315649561 shutdown_connections 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f783001c640 1 -- 172.21.15.64:0/2315649561 >> 172.21.15.64:0/2315649561 conn(0x7f782003aa00 msgr2=0x7f782003ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.127+0000 7f1ccf883640 1 -- start start 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f1ccf883640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1cc815d770 0x7f1cc80d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f1ccf883640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80d0b40 con 0x7f1cc80d3980 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f1ccf883640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc815db50 con 0x7f1cc80d2740 2023-11-24T08:04:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7f1ccf883640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc80d0860 con 0x7f1cc815d770 2023-11-24T08:04:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.129+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba74150630 con 0x7fba7415cea0 2023-11-24T08:04:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.130+0000 7f1ccde20640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1cc80d3980 0x7f1cc80b6760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56672/0 (socket says 172.21.15.64:56672) 2023-11-24T08:04:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.130+0000 7f1ccde20640 1 -- 172.21.15.64:0/769684462 learned_addr learned my addr 172.21.15.64:0/769684462 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.130+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 <== osd.0 v2:172.21.15.64:6800/1432182325 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5798075ba0 con 0x7f57980730d0 2023-11-24T08:04:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.130+0000 7f57b659c640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f579806af30 0x7f579806d3f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f57a8007830 tx=0x7f57a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.130+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba58000fc0 con 0x7fba7415cea0 2023-11-24T08:04:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.131+0000 7f1ccd61f640 1 --2- 172.21.15.64:0/769684462 >> v2:172.21.15.91:3300/0 conn(0x7f1cc815d770 0x7f1cc80d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.131+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f692c003b40 con 0x7f693c0acc20 2023-11-24T08:04:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.130+0000 7f783001c640 1 -- 172.21.15.64:0/2315649561 shutdown_connections 2023-11-24T08:04:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.130+0000 7f783001c640 1 -- 172.21.15.64:0/2315649561 wait complete. 2023-11-24T08:04:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.132+0000 7f783001c640 1 Processor -- start 2023-11-24T08:04:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.131+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071059039 0 0) 0x7f1cc80d0b40 con 0x7f1cc80d3980 2023-11-24T08:04:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb8003520 con 0x7f1cc80d3980 2023-11-24T08:04:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f6923fff640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f692806af40 0x7f692806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625292971 0 0) 0x7f1cc815db50 con 0x7f1cc80d2740 2023-11-24T08:04:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc80d0b40 con 0x7f1cc80d2740 2023-11-24T08:04:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f783001c640 1 -- start start 2023-11-24T08:04:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f692c089380 con 0x7f693c0acc20 2023-11-24T08:04:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f6923fff640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7f6928073160 0x7f6928075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 --> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6928075c30 con 0x7f6928073160 2023-11-24T08:04:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f692c089730 con 0x7f693c0acc20 2023-11-24T08:04:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1480579530 0 0) 0x7f1cb8003520 con 0x7f1cc80d3980 2023-11-24T08:04:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 >> v2:172.21.15.91:3300/0 conn(0x7f1cc815d770 msgr2=0x7f1cc80d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 --2- 172.21.15.64:0/769684462 >> v2:172.21.15.91:3300/0 conn(0x7f1cc815d770 0x7f1cc80d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 >> v1:172.21.15.91:6789/0 conn(0x7f1cc80d2740 legacy=0x7f1cc80d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80d1720 con 0x7f1cc80d3980 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f1cc0001670 con 0x7f1cc80d3980 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.133+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 >> v1:172.21.15.64:6789/0 conn(0x7f1cc80d3980 legacy=0x7f1cc80b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f1cc7fff640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cb8003b30 0x7f1cb8003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f1cc7fff640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cb80047c0 0x7f1cb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f1cc7fff640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb8005180 0x7f1cb8014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb4001230 con 0x7f1cb8005180 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80d0860 con 0x7f1cb8003b30 2023-11-24T08:04:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80149b0 con 0x7f1cb80047c0 2023-11-24T08:04:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f783001c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f78200d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f693affd640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f692806af40 0x7f692806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f693bfff640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7f6928073160 0x7f6928075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f783001c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78200e54f0 0x7f78200e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f783001c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7820038c80 0x7f7820039100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f783001c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200e4940 con 0x7f78140047c0 2023-11-24T08:04:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f783001c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200d09f0 con 0x7f78200e54f0 2023-11-24T08:04:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f783001c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7820039640 con 0x7f7820038c80 2023-11-24T08:04:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f1ccce1e640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cb80047c0 0x7f1cb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.134+0000 7f1ccde20640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb8005180 0x7f1cb8014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f1ccde20640 1 -- 172.21.15.64:0/769684462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cb80047c0 msgr2=0x7f1cb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7fd1dcba0640 1 Processor -- start 2023-11-24T08:04:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7fd1dcba0640 1 -- start start 2023-11-24T08:04:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f1ccde20640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cb80047c0 0x7f1cb8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f1ccde20640 1 -- 172.21.15.64:0/769684462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cb8003b30 msgr2=0x7f1cb8003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f1ccde20640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cb8003b30 0x7f1cb8003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f1ccde20640 1 -- 172.21.15.64:0/769684462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb8003520 con 0x7f1cb8005180 2023-11-24T08:04:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f782ddb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f78200d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f782ddb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f78200d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36792/0 (socket says 172.21.15.64:36792) 2023-11-24T08:04:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.135+0000 7f782ddb8640 1 -- 172.21.15.64:0/1596341498 learned_addr learned my addr 172.21.15.64:0/1596341498 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.136+0000 7f782e5b9640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7820038c80 0x7f7820039100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.136+0000 7f693affd640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f692806af40 0x7f692806d400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6930036000 tx=0x7f693003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.136+0000 7f782ddb8640 1 -- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7820038c80 msgr2=0x7f7820039100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.136+0000 7f782ddb8640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7820038c80 0x7f7820039100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.136+0000 7f782ddb8640 1 -- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78200e54f0 msgr2=0x7f78200e5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.136+0000 7f782ddb8640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78200e54f0 0x7f78200e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.136+0000 7f782ddb8640 1 -- 172.21.15.64:0/1596341498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200e38a0 con 0x7f78140047c0 2023-11-24T08:04:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.136+0000 7f782ddb8640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f78200d04b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f781800d7d0 tx=0x7f781800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.137+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781800de80 con 0x7f78140047c0 2023-11-24T08:04:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.137+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781800b660 con 0x7f78140047c0 2023-11-24T08:04:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.137+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781800b9f0 con 0x7f78140047c0 2023-11-24T08:04:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.137+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78200e3b80 con 0x7f78140047c0 2023-11-24T08:04:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.137+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78200e4140 con 0x7f78140047c0 2023-11-24T08:04:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.137+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba6c004b60 con 0x7fba7415cea0 2023-11-24T08:04:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.137+0000 7f693bfff640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7f6928073160 0x7f6928075580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.138+0000 7fd1dcba0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d00061e0 0x7fd1d0006640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.138+0000 7fd1dcba0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d0007870 con 0x7fd1d0053780 2023-11-24T08:04:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.138+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 <== osd.2 v2:172.21.15.64:6816/492491626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6928075c30 con 0x7f6928073160 2023-11-24T08:04:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.138+0000 7f1ccde20640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb8005180 0x7f1cb8014190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80cd9e0 tx=0x7f1cb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360d27225ad4b237 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.138+0000 7f1ccde20640 1 -- 172.21.15.64:0/769684462 reap_dead start 2023-11-24T08:04:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.138+0000 7fd1dcba0640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d0006c60 con 0x7fd1d003c0c0 2023-11-24T08:04:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.138+0000 7fd1dcba0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0006f40 con 0x7fd1d00061e0 2023-11-24T08:04:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.138+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d00061e0 0x7fd1d0006640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.139+0000 7fd1d77fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd1d0053780 0x7fd1d0053b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56684/0 (socket says 172.21.15.64:56684) 2023-11-24T08:04:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.139+0000 7fd1d77fe640 1 -- 172.21.15.64:0/1283906132 learned_addr learned my addr 172.21.15.64:0/1283906132 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.139+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f782000ffa0 con 0x7f78140047c0 2023-11-24T08:04:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.139+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0005ad0 con 0x7f1cb8005180 2023-11-24T08:04:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.140+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0005c30 con 0x7f1cb8005180 2023-11-24T08:04:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.140+0000 7f1cc7fff640 1 -- 172.21.15.64:0/769684462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0005ad0 con 0x7f1cb8005180 2023-11-24T08:04:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.139+0000 7fd1d6ffd640 1 -- 172.21.15.64:0/1283906132 >> v1:172.21.15.91:6789/0 conn(0x7fd1d003c0c0 legacy=0x7fd1d003c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.140+0000 7fd1d6ffd640 1 -- 172.21.15.64:0/1283906132 >> v1:172.21.15.64:6789/0 conn(0x7fd1d0053780 legacy=0x7fd1d0053b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.140+0000 7fd1d6ffd640 1 -- 172.21.15.64:0/1283906132 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0007220 con 0x7fd1d00061e0 2023-11-24T08:04:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.141+0000 7fba717fa640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fba5406af10 0x7fba5406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.141+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba6c088ca0 con 0x7fba7415cea0 2023-11-24T08:04:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.141+0000 7f1ccf883640 1 -- 172.21.15.64:0/769684462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb8005180 msgr2=0x7f1cb8014190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.141+0000 7f1ccf883640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb8005180 0x7f1cb8014190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80cd9e0 tx=0x7f1cb80057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.141+0000 7fba717fa640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fba54073130 0x7fba54075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.141+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 --> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba54075c00 con 0x7fba54073130 2023-11-24T08:04:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.141+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6c08b2c0 con 0x7fba7415cea0 2023-11-24T08:04:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.142+0000 7f1ccf883640 1 -- 172.21.15.64:0/769684462 shutdown_connections 2023-11-24T08:04:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.142+0000 7f1ccf883640 1 --2- 172.21.15.64:0/769684462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb8005180 0x7f1cb8014190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.142+0000 7f1ccf883640 1 -- 172.21.15.64:0/769684462 >> 172.21.15.64:0/769684462 conn(0x7f1cc80721a0 msgr2=0x7f1cc806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.142+0000 7fba73fff640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fba5406af10 0x7fba5406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.142+0000 7fba6b7fe640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fba54073130 0x7fba54075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.142+0000 7fd1d6ffd640 1 --2- 172.21.15.64:0/1283906132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d00061e0 0x7fd1d0006640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40097d0 tx=0x7fd1c402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9dacaaf6ef9d52f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.142+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f781800bb50 con 0x7f78140047c0 2023-11-24T08:04:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.143+0000 7facb51e5640 1 Processor -- start 2023-11-24T08:04:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.143+0000 7f1ccf883640 1 -- 172.21.15.64:0/769684462 shutdown_connections 2023-11-24T08:04:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.143+0000 7f1ccf883640 1 -- 172.21.15.64:0/769684462 wait complete. 2023-11-24T08:04:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.143+0000 7fd1d67fc640 1 -- 172.21.15.64:0/1283906132 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4030800 con 0x7fd1d00061e0 2023-11-24T08:04:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.143+0000 7fd1d67fc640 1 -- 172.21.15.64:0/1283906132 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c4030960 con 0x7fd1d00061e0 2023-11-24T08:04:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.143+0000 7fd1d67fc640 1 -- 172.21.15.64:0/1283906132 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4030cf0 con 0x7fd1d00061e0 2023-11-24T08:04:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.143+0000 7fba6b7fe640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fba54073130 0x7fba54075550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.144+0000 7fd1dcba0640 1 -- 172.21.15.64:0/1283906132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d00061e0 msgr2=0x7fd1d0006640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.144+0000 7fd1dcba0640 1 --2- 172.21.15.64:0/1283906132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d00061e0 0x7fd1d0006640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40097d0 tx=0x7fd1c402b940 comp rx=0 tx=0).stop 2023-11-24T08:04:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.143+0000 7facb51e5640 1 -- start start 2023-11-24T08:04:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 <== osd.1 v2:172.21.15.64:6808/2214220706 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fba54075c00 con 0x7fba54073130 2023-11-24T08:04:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7fba73fff640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fba5406af10 0x7fba5406d3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba741f1df0 tx=0x7fba6400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7facb51e5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb015d770 0x7facb00d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7facb51e5640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7facb00d2740 0x7facb00d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7facb51e5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7facb00d37b0 0x7facb00b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7facb51e5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb00d3c30 con 0x7facb00d37b0 2023-11-24T08:04:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7facb51e5640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb00d0b40 con 0x7facb00d2740 2023-11-24T08:04:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7facb51e5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb015db50 con 0x7facb015d770 2023-11-24T08:04:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7f781e7fc640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f780c06ae40 0x7f780c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f781808a490 con 0x7f78140047c0 2023-11-24T08:04:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7f781e7fc640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f780c073060 0x7f780c075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 --> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f780c075b30 con 0x7f780c073060 2023-11-24T08:04:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.145+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f781808a840 con 0x7f78140047c0 2023-11-24T08:04:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7f782d5b7640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f780c06ae40 0x7f780c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7facaed9d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb015d770 0x7facb00d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7facaf59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7facb00d37b0 0x7facb00b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7f2ebd648640 1 Processor -- start 2023-11-24T08:04:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7faca659c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7facb00d2740 0x7facb00d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7faca659c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7facb00d2740 0x7facb00d3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34354/0 (socket says 172.21.15.64:34354) 2023-11-24T08:04:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7faca659c640 1 -- 172.21.15.64:0/3702147272 learned_addr learned my addr 172.21.15.64:0/3702147272 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7f2ebd648640 1 -- start start 2023-11-24T08:04:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.146+0000 7f782e5b9640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f780c073060 0x7f780c075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7fd1dcba0640 1 -- 172.21.15.64:0/1283906132 shutdown_connections 2023-11-24T08:04:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7fd1dcba0640 1 --2- 172.21.15.64:0/1283906132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d00061e0 0x7fd1d0006640 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7fd1dcba0640 1 -- 172.21.15.64:0/1283906132 >> 172.21.15.64:0/1283906132 conn(0x7fd1d003aa10 msgr2=0x7fd1d003ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7faca659c640 1 -- 172.21.15.64:0/3702147272 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb015d770 msgr2=0x7facb00d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7faca659c640 1 --2- 172.21.15.64:0/3702147272 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb015d770 0x7facb00d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7faca659c640 1 -- 172.21.15.64:0/3702147272 >> v2:172.21.15.64:3300/0 conn(0x7facb00d37b0 msgr2=0x7facb00b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7faca659c640 1 --2- 172.21.15.64:0/3702147272 >> v2:172.21.15.64:3300/0 conn(0x7facb00d37b0 0x7facb00b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7faca659c640 1 -- 172.21.15.64:0/3702147272 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb00d0860 con 0x7facb00d2740 2023-11-24T08:04:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7facaf59e640 1 --2- 172.21.15.64:0/3702147272 >> v2:172.21.15.64:3300/0 conn(0x7facb00d37b0 0x7facb00b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:04:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f2ebd648640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2eb815d6d0 0x7f2eb80d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f2ebd648640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2eb80d2d80 0x7f2eb80b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f2ebd648640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb80d1300 con 0x7f2eb80b51c0 2023-11-24T08:04:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f2ebd648640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb80d21d0 con 0x7f2eb815d6d0 2023-11-24T08:04:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f2ebd648640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb80b5670 con 0x7f2eb80d2d80 2023-11-24T08:04:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.149+0000 7fd1dcba0640 1 -- 172.21.15.64:0/1283906132 shutdown_connections 2023-11-24T08:04:03.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.149+0000 7fd1dcba0640 1 -- 172.21.15.64:0/1283906132 wait complete. 2023-11-24T08:04:03.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f2eb6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2eb815d6d0 0x7f2eb80d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.149+0000 7f2eb77fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2eb80b51c0 0x7f2eb806b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56686/0 (socket says 172.21.15.64:56686) 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.149+0000 7f2eb77fe640 1 -- 172.21.15.64:0/3699093514 learned_addr learned my addr 172.21.15.64:0/3699093514 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.149+0000 7f2eb5ffb640 1 -- 172.21.15.64:0/3699093514 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293950839 0 0) 0x7f2eb80d1300 con 0x7f2eb80b51c0 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.150+0000 7f2eb5ffb640 1 -- 172.21.15.64:0/3699093514 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea0003520 con 0x7f2eb80b51c0 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.150+0000 7f2eb6ffd640 1 -- 172.21.15.64:0/3699093514 >> v2:172.21.15.91:3300/0 conn(0x7f2eb80d2d80 msgr2=0x7f2eb80b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.150+0000 7f2eb6ffd640 1 --2- 172.21.15.64:0/3699093514 >> v2:172.21.15.91:3300/0 conn(0x7f2eb80d2d80 0x7f2eb80b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.150+0000 7f2eb6ffd640 1 -- 172.21.15.64:0/3699093514 >> v1:172.21.15.64:6789/0 conn(0x7f2eb80b51c0 legacy=0x7f2eb806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.150+0000 7f2eb6ffd640 1 -- 172.21.15.64:0/3699093514 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80b2830 con 0x7f2eb815d6d0 2023-11-24T08:04:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.149+0000 7f782e5b9640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f780c073060 0x7f780c075480 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.151+0000 7facaed9d640 1 --2- 172.21.15.64:0/3702147272 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb015d770 0x7facb00d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.151+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 <== osd.3 v2:172.21.15.64:6822/3251355202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f780c075b30 con 0x7f780c073060 2023-11-24T08:04:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.151+0000 7f782d5b7640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f780c06ae40 0x7f780c06d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7824033000 tx=0x7f7824007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7f1ccf883640 1 Processor -- start 2023-11-24T08:04:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.147+0000 7f1ccf883640 1 -- start start 2023-11-24T08:04:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.151+0000 7f2eb6ffd640 1 --2- 172.21.15.64:0/3699093514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb815d6d0 0x7f2eb80d2840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80cec10 tx=0x7f2ea802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb800b27d8bc12cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f1ccf883640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cb8005180 0x7f1cc81f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f1ccf883640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc8157620 0x7f1cc81ee510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f1ccf883640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cc81561c0 0x7f1cc8156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f1ccf883640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81f3230 con 0x7f1cc8157620 2023-11-24T08:04:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f1ccf883640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8155da0 con 0x7f1cc81561c0 2023-11-24T08:04:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.148+0000 7f1ccf883640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81f2990 con 0x7f1cb8005180 2023-11-24T08:04:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.152+0000 7faca659c640 1 --2- 172.21.15.64:0/3702147272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facb00d2740 0x7facb00d3270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7facb00d43e0 tx=0x7fac9802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4fe3678475ddd1e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.151+0000 7fd1dcba0640 1 Processor -- start 2023-11-24T08:04:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.152+0000 7f2eb5ffb640 1 -- 172.21.15.64:0/3699093514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8030840 con 0x7f2eb815d6d0 2023-11-24T08:04:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.152+0000 7f2eb5ffb640 1 -- 172.21.15.64:0/3699093514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea80309a0 con 0x7f2eb815d6d0 2023-11-24T08:04:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.152+0000 7f2eb5ffb640 1 -- 172.21.15.64:0/3699093514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8030d50 con 0x7f2eb815d6d0 2023-11-24T08:04:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.152+0000 7fd1dcba0640 1 -- start start 2023-11-24T08:04:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.152+0000 7f1ccce1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc8157620 0x7f1cc81ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.152+0000 7f1ccd61f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cb8005180 0x7f1cc81f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.152+0000 7f1ccde20640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cc81561c0 0x7f1cc8156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.153+0000 7f1ccde20640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cc81561c0 0x7f1cc8156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34378/0 (socket says 172.21.15.64:34378) 2023-11-24T08:04:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.153+0000 7f1ccde20640 1 -- 172.21.15.64:0/830668512 learned_addr learned my addr 172.21.15.64:0/830668512 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.153+0000 7facae59c640 1 -- 172.21.15.64:0/3702147272 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98030800 con 0x7facb00d2740 2023-11-24T08:04:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.153+0000 7facae59c640 1 -- 172.21.15.64:0/3702147272 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98030960 con 0x7facb00d2740 2023-11-24T08:04:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.153+0000 7facae59c640 1 -- 172.21.15.64:0/3702147272 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98030d30 con 0x7facb00d2740 2023-11-24T08:04:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.153+0000 7f2ebd648640 1 -- 172.21.15.64:0/3699093514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb815d6d0 msgr2=0x7f2eb80d2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.153+0000 7f2ebd648640 1 --2- 172.21.15.64:0/3699093514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb815d6d0 0x7f2eb80d2840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80cec10 tx=0x7f2ea802b9a0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.153+0000 7f1ccde20640 1 -- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cb8005180 msgr2=0x7f1cc81f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7f1ccde20640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1cb8005180 0x7f1cc81f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7f1ccde20640 1 -- 172.21.15.64:0/830668512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc8157620 msgr2=0x7f1cc81ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7facb51e5640 1 -- 172.21.15.64:0/3702147272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facb00d2740 msgr2=0x7facb00d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7facb51e5640 1 --2- 172.21.15.64:0/3702147272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facb00d2740 0x7facb00d3270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7facb00d43e0 tx=0x7fac9802b940 comp rx=0 tx=0).stop 2023-11-24T08:04:03.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7f1ccde20640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc8157620 0x7f1cc81ee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7facb51e5640 1 -- 172.21.15.64:0/3702147272 shutdown_connections 2023-11-24T08:04:03.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7facb51e5640 1 --2- 172.21.15.64:0/3702147272 >> v2:172.21.15.64:3300/0 conn(0x7facb00d37b0 0x7facb00b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7facb51e5640 1 --2- 172.21.15.64:0/3702147272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facb00d2740 0x7facb00d3270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7facb51e5640 1 --2- 172.21.15.64:0/3702147272 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb015d770 0x7facb00d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7fd1dcba0640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1d00061e0 0x7fd1d00e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7fd1dcba0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d003c0c0 0x7fd1d0052900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7fd1dcba0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d0053780 0x7fd1d00e4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7fd1dcba0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0052e40 con 0x7fd1d003c0c0 2023-11-24T08:04:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7fd1dcba0640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d017e6e0 con 0x7fd1d00061e0 2023-11-24T08:04:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7fd1dcba0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d017db60 con 0x7fd1d0053780 2023-11-24T08:04:03.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.154+0000 7facb51e5640 1 -- 172.21.15.64:0/3702147272 >> 172.21.15.64:0/3702147272 conn(0x7facb00721a0 msgr2=0x7facb006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7facb51e5640 1 -- 172.21.15.64:0/3702147272 shutdown_connections 2023-11-24T08:04:03.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7f1ccde20640 1 -- 172.21.15.64:0/830668512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc81f2c70 con 0x7f1cc81561c0 2023-11-24T08:04:03.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1d00061e0 0x7fd1d00e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1d00061e0 0x7fd1d00e3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34382/0 (socket says 172.21.15.64:34382) 2023-11-24T08:04:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7fd1d77fe640 1 -- 172.21.15.64:0/9463817 learned_addr learned my addr 172.21.15.64:0/9463817 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7fd1d6ffd640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d003c0c0 0x7fd1d0052900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7facb51e5640 1 -- 172.21.15.64:0/3702147272 wait complete. 2023-11-24T08:04:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7f2ebd648640 1 -- 172.21.15.64:0/3699093514 shutdown_connections 2023-11-24T08:04:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7f2ebd648640 1 --2- 172.21.15.64:0/3699093514 >> v2:172.21.15.91:3300/0 conn(0x7f2eb80d2d80 0x7f2eb80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7f2ebd648640 1 --2- 172.21.15.64:0/3699093514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb815d6d0 0x7f2eb80d2840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7f2ebd648640 1 -- 172.21.15.64:0/3699093514 >> 172.21.15.64:0/3699093514 conn(0x7f2eb80721a0 msgr2=0x7f2eb80cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7f2ebd648640 1 -- 172.21.15.64:0/3699093514 shutdown_connections 2023-11-24T08:04:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7f1ccce1e640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc8157620 0x7f1cc81ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.156+0000 7fd1d77fe640 1 -- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d0053780 msgr2=0x7fd1d00e4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.156+0000 7fd1d77fe640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd1d0053780 0x7fd1d00e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.156+0000 7fd1d77fe640 1 -- 172.21.15.64:0/9463817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d003c0c0 msgr2=0x7fd1d0052900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.155+0000 7f2ebd648640 1 -- 172.21.15.64:0/3699093514 wait complete. 2023-11-24T08:04:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.157+0000 7f1ccde20640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cc81561c0 0x7f1cc8156640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1cc002c2d0 tx=0x7f1cc002e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.157+0000 7fd1d77fe640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d003c0c0 0x7fd1d0052900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.157+0000 7fd1d77fe640 1 -- 172.21.15.64:0/9463817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c4009400 con 0x7fd1d00061e0 2023-11-24T08:04:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.157+0000 7fd1d6ffd640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d003c0c0 0x7fd1d0052900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:04:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.157+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc003c040 con 0x7f1cc81561c0 2023-11-24T08:04:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.157+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0007480 con 0x7f1cc81561c0 2023-11-24T08:04:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.157+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc002c730 con 0x7f1cc81561c0 2023-11-24T08:04:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.158+0000 7facb51e5640 1 Processor -- start 2023-11-24T08:04:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.158+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc81f2f50 con 0x7f1cc81561c0 2023-11-24T08:04:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.158+0000 7facb51e5640 1 -- start start 2023-11-24T08:04:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.158+0000 7fd1d77fe640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1d00061e0 0x7fd1d00e3800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00c3c0 tx=0x7fd1cc00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.158+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc81eea50 con 0x7f1cc81561c0 2023-11-24T08:04:03.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.159+0000 7f2ebd648640 1 Processor -- start 2023-11-24T08:04:03.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.159+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc005ce0 con 0x7fd1d00061e0 2023-11-24T08:04:03.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.159+0000 7f2ebd648640 1 -- start start 2023-11-24T08:04:03.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.159+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1cc80d0440 con 0x7f1cc81561c0 2023-11-24T08:04:03.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.159+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d017de40 con 0x7fd1d00061e0 2023-11-24T08:04:03.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.159+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d017e400 con 0x7fd1d00061e0 2023-11-24T08:04:03.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.160+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc005e40 con 0x7fd1d00061e0 2023-11-24T08:04:03.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.160+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc0072d0 con 0x7fd1d00061e0 2023-11-24T08:04:03.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.161+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 --> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f691c002cb0 con 0x7f6928073160 2023-11-24T08:04:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.161+0000 7f2ebd648640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b51c0 0x7f2eb81f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.161+0000 7f2ebd648640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2eb80d2d80 0x7f2eb80cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.161+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1b8000fc0 con 0x7fd1d00061e0 2023-11-24T08:04:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.161+0000 7f2ebd648640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2eb815d6d0 0x7f2eb80d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.161+0000 7f2ebd648640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81f6eb0 con 0x7f2eb80b51c0 2023-11-24T08:04:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.161+0000 7f2ebd648640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80c8940 con 0x7f2eb815d6d0 2023-11-24T08:04:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.161+0000 7f2ebd648640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81f6230 con 0x7f2eb80d2d80 2023-11-24T08:04:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.162+0000 7f6923fff640 1 -- 172.21.15.64:0/2687794466 <== osd.2 v2:172.21.15.64:6816/492491626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f691c002cb0 con 0x7f6928073160 2023-11-24T08:04:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.162+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b51c0 0x7f2eb81f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b51c0 0x7f2eb81f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36826/0 (socket says 172.21.15.64:36826) 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.162+0000 7facb51e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb00d37b0 0x7facb0148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.162+0000 7facb51e5640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facb015d770 0x7facb00ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.162+0000 7facb51e5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb00cada0 0x7facb00c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.162+0000 7facb51e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb00cb620 con 0x7facb00d37b0 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.162+0000 7facb51e5640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb00ca3b0 con 0x7facb015d770 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.162+0000 7facb51e5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb0148850 con 0x7facb00cada0 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb00d37b0 0x7facb0148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb00d37b0 0x7facb0148310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36836/0 (socket says 172.21.15.64:36836) 2023-11-24T08:04:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaed9d640 1 -- 172.21.15.64:0/1980242243 learned_addr learned my addr 172.21.15.64:0/1980242243 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cc0032070 con 0x7f1cc81561c0 2023-11-24T08:04:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7f2eb6ffd640 1 -- 172.21.15.64:0/2499389891 learned_addr learned my addr 172.21.15.64:0/2499389891 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7f2eb67fc640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2eb80d2d80 0x7f2eb80cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7f2eb77fe640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2eb815d6d0 0x7f2eb80d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f2eb6ffd640 1 -- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2eb80d2d80 msgr2=0x7f2eb80cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f2eb6ffd640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2eb80d2d80 0x7f2eb80cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f2eb6ffd640 1 -- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2eb815d6d0 msgr2=0x7f2eb80d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f2eb6ffd640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2eb815d6d0 0x7f2eb80d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f2eb6ffd640 1 -- 172.21.15.64:0/2499389891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8009400 con 0x7f2eb80b51c0 2023-11-24T08:04:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f1cc5ffb640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cac06ae90 0x7f1cac06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cc00aa510 con 0x7f1cc81561c0 2023-11-24T08:04:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f2eb77fe640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2eb815d6d0 0x7f2eb80d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.164+0000 7f2eb6ffd640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b51c0 0x7f2eb81f2720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ea802bfb0 tx=0x7f2ea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f1cc5ffb640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f1cac0730b0 0x7f1cac0754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 --> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cac075b80 con 0x7f1cac0730b0 2023-11-24T08:04:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cc00aa880 con 0x7f1cc81561c0 2023-11-24T08:04:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f1ccd61f640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cac06ae90 0x7f1cac06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea803e070 con 0x7f2eb80b51c0 2023-11-24T08:04:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea8003820 con 0x7f2eb80b51c0 2023-11-24T08:04:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8003bd0 con 0x7f2eb80b51c0 2023-11-24T08:04:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb81f6510 con 0x7f2eb80b51c0 2023-11-24T08:04:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f1ccce1e640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f1cac0730b0 0x7f1cac0754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb81f6ad0 con 0x7f2eb80b51c0 2023-11-24T08:04:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaf59e640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb00cada0 0x7facb00c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaed9d640 1 -- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb00cada0 msgr2=0x7facb00c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaed9d640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facb00cada0 0x7facb00c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaed9d640 1 -- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facb015d770 msgr2=0x7facb00ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaed9d640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facb015d770 0x7facb00ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.163+0000 7facaed9d640 1 -- 172.21.15.64:0/1980242243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8009400 con 0x7facb00d37b0 2023-11-24T08:04:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1cc0074f0 con 0x7fd1d00061e0 2023-11-24T08:04:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7facaed9d640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb00d37b0 0x7facb0148310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faca000ce00 tx=0x7faca000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f1ccd61f640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cac06ae90 0x7f1cac06d350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc004b30 tx=0x7f1cbc00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.165+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 --> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57b8070520 con 0x7f57980730d0 2023-11-24T08:04:03.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f57a6ffd640 1 -- 172.21.15.64:0/574508041 <== osd.0 v2:172.21.15.64:6800/1432182325 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57ac005850 con 0x7f57980730d0 2023-11-24T08:04:03.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7f6928073160 msgr2=0x7f6928075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f6941b89640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7f6928073160 0x7f6928075580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f692806af40 msgr2=0x7f692806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f6941b89640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f692806af40 0x7f692806d400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6930036000 tx=0x7f693003c040 comp rx=0 tx=0).stop 2023-11-24T08:04:03.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0acc20 msgr2=0x7f693c1f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f6941b89640 1 --2- 172.21.15.64:0/2687794466 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f693c0acc20 0x7f693c1f64c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f692c00c460 tx=0x7f692c00c920 comp rx=0 tx=0).stop 2023-11-24T08:04:03.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f693bfff640 1 -- 172.21.15.64:0/2687794466 reap_dead start 2023-11-24T08:04:03.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 shutdown_connections 2023-11-24T08:04:03.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 >> 172.21.15.64:0/2687794466 conn(0x7f693c0709f0 msgr2=0x7f693c164070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 shutdown_connections 2023-11-24T08:04:03.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca000f890 con 0x7facb00d37b0 2023-11-24T08:04:03.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca000f9f0 con 0x7facb00d37b0 2023-11-24T08:04:03.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.166+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca000fdc0 con 0x7facb00d37b0 2023-11-24T08:04:03.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2eb80d1b10 con 0x7f2eb80b51c0 2023-11-24T08:04:03.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f57980730d0 msgr2=0x7f57980754f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57bcd3c640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f57980730d0 0x7f57980754f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f579806af30 msgr2=0x7f579806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57bcd3c640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f579806af30 0x7f579806d3f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f57a8007830 tx=0x7f57a8009040 comp rx=0 tx=0).stop 2023-11-24T08:04:03.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b81eea60 msgr2=0x7f57b80ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57bcd3c640 1 --2- 172.21.15.64:0/574508041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b81eea60 0x7f57b80ae7b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f57b0004000 tx=0x7f57b0004b50 comp rx=0 tx=0).stop 2023-11-24T08:04:03.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f6941b89640 1 -- 172.21.15.64:0/2687794466 wait complete. 2023-11-24T08:04:03.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57b6d9d640 1 -- 172.21.15.64:0/574508041 reap_dead start 2023-11-24T08:04:03.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 shutdown_connections 2023-11-24T08:04:03.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.167+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 >> 172.21.15.64:0/574508041 conn(0x7f57b80b0840 msgr2=0x7f57b81512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.168+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 shutdown_connections 2023-11-24T08:04:03.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.168+0000 7f1ccce1e640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f1cac0730b0 0x7f1cac0754d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.168+0000 7f57bcd3c640 1 -- 172.21.15.64:0/574508041 wait complete. 2023-11-24T08:04:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.168+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac980094d0 con 0x7facb00d37b0 2023-11-24T08:04:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.168+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb00ca600 con 0x7facb00d37b0 2023-11-24T08:04:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.168+0000 7fd1c3fff640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd1bc06ae40 0x7fd1bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.169+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1cc089200 con 0x7fd1d00061e0 2023-11-24T08:04:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.169+0000 7fd1c3fff640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7fd1bc073060 0x7fd1bc075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.169+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 --> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1bc075b30 con 0x7fd1bc073060 2023-11-24T08:04:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.169+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1cc0895b0 con 0x7fd1d00061e0 2023-11-24T08:04:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.170+0000 7fd1d6ffd640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd1bc06ae40 0x7fd1bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.170+0000 7fd1d7fff640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7fd1bc073060 0x7fd1bc075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.171+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 <== osd.4 v2:172.21.15.91:6821/3295720288 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1cac075b80 con 0x7f1cac0730b0 2023-11-24T08:04:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.171+0000 7fd1d6ffd640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd1bc06ae40 0x7fd1bc06d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4036000 tx=0x7fd1c4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.170+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ea8003d30 con 0x7f2eb80b51c0 2023-11-24T08:04:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.171+0000 7f2e9f7fe640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ea406af10 0x7f2ea406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.170+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7facb00d1ce0 con 0x7facb00d37b0 2023-11-24T08:04:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.172+0000 7fd1d7fff640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7fd1bc073060 0x7fd1bc075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.172+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ea80abcf0 con 0x7f2eb80b51c0 2023-11-24T08:04:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.172+0000 7f2eb67fc640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ea406af10 0x7f2ea406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.175+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 <== osd.5 v2:172.21.15.91:6816/3653724610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd1bc075b30 con 0x7fd1bc073060 2023-11-24T08:04:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.173+0000 7f2e9f7fe640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f2ea4073130 0x7f2ea4075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.173+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 --> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ea4075c00 con 0x7f2ea4073130 2023-11-24T08:04:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.173+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ea80ae310 con 0x7f2eb80b51c0 2023-11-24T08:04:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.173+0000 7f2eb67fc640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ea406af10 0x7f2ea406d3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80b6b20 tx=0x7f2eac009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.174+0000 7f2eb77fe640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f2ea4073130 0x7f2ea4075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.174+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faca000b040 con 0x7facb00d37b0 2023-11-24T08:04:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.175+0000 7f2eb77fe640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f2ea4073130 0x7f2ea4075550 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.177+0000 7faca7fff640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac9006aec0 0x7fac9006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.177+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faca0089e80 con 0x7facb00d37b0 2023-11-24T08:04:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.178+0000 7faca659c640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac9006aec0 0x7fac9006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.178+0000 7faca7fff640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fac900730e0 0x7fac90075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.178+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 --> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac90075bb0 con 0x7fac900730e0 2023-11-24T08:04:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.178+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faca008c590 con 0x7facb00d37b0 2023-11-24T08:04:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.178+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 --> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fba58002cb0 con 0x7fba54073130 2023-11-24T08:04:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.179+0000 7facaf59e640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fac900730e0 0x7fac90075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.179+0000 7faca659c640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac9006aec0 0x7fac9006d380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac98036000 tx=0x7fac9803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.179+0000 7fba717fa640 1 -- 172.21.15.64:0/2209212211 <== osd.1 v2:172.21.15.64:6808/2214220706 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fba58002cb0 con 0x7fba54073130 2023-11-24T08:04:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.179+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 <== osd.7 v2:172.21.15.91:6808/4224688629 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2ea4075c00 con 0x7f2ea4073130 2023-11-24T08:04:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.180+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fba54073130 msgr2=0x7fba54075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.180+0000 7fba7a91d640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fba54073130 0x7fba54075550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.180+0000 7facaf59e640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fac900730e0 0x7fac90075500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.180+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fba5406af10 msgr2=0x7fba5406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.180+0000 7fba7a91d640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fba5406af10 0x7fba5406d3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba741f1df0 tx=0x7fba6400b040 comp rx=0 tx=0).stop 2023-11-24T08:04:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.181+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7415cea0 msgr2=0x7fba740afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.181+0000 7fba7a91d640 1 --2- 172.21.15.64:0/2209212211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7415cea0 0x7fba740afa20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c390 tx=0x7fba6c00c850 comp rx=0 tx=0).stop 2023-11-24T08:04:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.181+0000 7fba78eba640 1 -- 172.21.15.64:0/2209212211 reap_dead start 2023-11-24T08:04:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.182+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 shutdown_connections 2023-11-24T08:04:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.182+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 >> 172.21.15.64:0/2209212211 conn(0x7fba740b0840 msgr2=0x7fba740b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.182+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 shutdown_connections 2023-11-24T08:04:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.182+0000 7fba7a91d640 1 -- 172.21.15.64:0/2209212211 wait complete. 2023-11-24T08:04:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.183+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 <== osd.6 v2:172.21.15.91:6800/3533292378 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fac90075bb0 con 0x7fac900730e0 2023-11-24T08:04:03.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.190+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 --> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78200074f0 con 0x7f780c073060 2023-11-24T08:04:03.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.192+0000 7f781e7fc640 1 -- 172.21.15.64:0/1596341498 <== osd.3 v2:172.21.15.64:6822/3251355202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f782804f730 con 0x7f780c073060 2023-11-24T08:04:03.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.193+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f780c073060 msgr2=0x7f780c075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.193+0000 7f783001c640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f780c073060 0x7f780c075480 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.193+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f780c06ae40 msgr2=0x7f780c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.193+0000 7f783001c640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f780c06ae40 0x7f780c06d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7824033000 tx=0x7f7824007000 comp rx=0 tx=0).stop 2023-11-24T08:04:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.193+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 msgr2=0x7f78200d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.193+0000 7f783001c640 1 --2- 172.21.15.64:0/1596341498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78140047c0 0x7f78200d04b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f781800d7d0 tx=0x7f781800dc90 comp rx=0 tx=0).stop 2023-11-24T08:04:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.193+0000 7f782e5b9640 1 -- 172.21.15.64:0/1596341498 reap_dead start 2023-11-24T08:04:03.217 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:03.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T08:04:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.194+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 shutdown_connections 2023-11-24T08:04:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.194+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 >> 172.21.15.64:0/1596341498 conn(0x7f782003aa00 msgr2=0x7f78200048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.194+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 shutdown_connections 2023-11-24T08:04:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.194+0000 7f783001c640 1 -- 172.21.15.64:0/1596341498 wait complete. 2023-11-24T08:04:03.218 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:03.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T08:04:03.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.205+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 --> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1cc80b6030 con 0x7f1cac0730b0 2023-11-24T08:04:03.220 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:03.220 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T08:04:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1cc5ffb640 1 -- 172.21.15.64:0/830668512 <== osd.4 v2:172.21.15.91:6821/3295720288 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1cc80b6030 con 0x7f1cac0730b0 2023-11-24T08:04:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f1cac0730b0 msgr2=0x7f1cac0754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccf883640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f1cac0730b0 0x7f1cac0754d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cac06ae90 msgr2=0x7f1cac06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccf883640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cac06ae90 0x7f1cac06d350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc004b30 tx=0x7f1cbc00b040 comp rx=0 tx=0).stop 2023-11-24T08:04:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cc81561c0 msgr2=0x7f1cc8156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccf883640 1 --2- 172.21.15.64:0/830668512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1cc81561c0 0x7f1cc8156640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1cc002c2d0 tx=0x7f1cc002e070 comp rx=0 tx=0).stop 2023-11-24T08:04:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 --> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd1b8002cb0 con 0x7fd1bc073060 2023-11-24T08:04:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccde20640 1 -- 172.21.15.64:0/830668512 reap_dead start 2023-11-24T08:04:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 shutdown_connections 2023-11-24T08:04:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.207+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 >> 172.21.15.64:0/830668512 conn(0x7f1cc80721a0 msgr2=0x7f1cc80cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.208+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 shutdown_connections 2023-11-24T08:04:03.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.208+0000 7f1ccf883640 1 -- 172.21.15.64:0/830668512 wait complete. 2023-11-24T08:04:03.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.208+0000 7fd1c3fff640 1 -- 172.21.15.64:0/9463817 <== osd.5 v2:172.21.15.91:6816/3653724610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd1b8002cb0 con 0x7fd1bc073060 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7fd1bc073060 msgr2=0x7fd1bc075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1dcba0640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7fd1bc073060 0x7fd1bc075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd1bc06ae40 msgr2=0x7fd1bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1dcba0640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd1bc06ae40 0x7fd1bc06d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4036000 tx=0x7fd1c4008030 comp rx=0 tx=0).stop 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1d00061e0 msgr2=0x7fd1d00e3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1dcba0640 1 --2- 172.21.15.64:0/9463817 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd1d00061e0 0x7fd1d00e3800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00c3c0 tx=0x7fd1cc00c880 comp rx=0 tx=0).stop 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1d7fff640 1 -- 172.21.15.64:0/9463817 reap_dead start 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 shutdown_connections 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.209+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 >> 172.21.15.64:0/9463817 conn(0x7fd1d003aa10 msgr2=0x7fd1d003dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.210+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 shutdown_connections 2023-11-24T08:04:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.210+0000 7fd1dcba0640 1 -- 172.21.15.64:0/9463817 wait complete. 2023-11-24T08:04:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.211+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 --> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2eb80d1830 con 0x7f2ea4073130 2023-11-24T08:04:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.213+0000 7f2e9f7fe640 1 -- 172.21.15.64:0/2499389891 <== osd.7 v2:172.21.15.91:6808/4224688629 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2eb0001e30 con 0x7f2ea4073130 2023-11-24T08:04:03.226 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:03.226 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T08:04:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.213+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f2ea4073130 msgr2=0x7f2ea4075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.213+0000 7f2ebd648640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f2ea4073130 0x7f2ea4075550 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.213+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ea406af10 msgr2=0x7f2ea406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.213+0000 7f2ebd648640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ea406af10 0x7f2ea406d3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80b6b20 tx=0x7f2eac009110 comp rx=0 tx=0).stop 2023-11-24T08:04:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.213+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b51c0 msgr2=0x7f2eb81f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.213+0000 7f2ebd648640 1 --2- 172.21.15.64:0/2499389891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b51c0 0x7f2eb81f2720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ea802bfb0 tx=0x7f2ea80046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.213+0000 7f2eb77fe640 1 -- 172.21.15.64:0/2499389891 reap_dead start 2023-11-24T08:04:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.214+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 shutdown_connections 2023-11-24T08:04:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.214+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 >> 172.21.15.64:0/2499389891 conn(0x7f2eb80721a0 msgr2=0x7f2eb80b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.214+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 shutdown_connections 2023-11-24T08:04:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.214+0000 7f2ebd648640 1 -- 172.21.15.64:0/2499389891 wait complete. 2023-11-24T08:04:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.214+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 --> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7facb00d1a00 con 0x7fac900730e0 2023-11-24T08:04:03.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.215+0000 7faca7fff640 1 -- 172.21.15.64:0/1980242243 <== osd.6 v2:172.21.15.91:6800/3533292378 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faca8009870 con 0x7fac900730e0 2023-11-24T08:04:03.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fac900730e0 msgr2=0x7fac90075500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fac900730e0 0x7fac90075500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac9006aec0 msgr2=0x7fac9006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac9006aec0 0x7fac9006d380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac98036000 tx=0x7fac9803b040 comp rx=0 tx=0).stop 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb00d37b0 msgr2=0x7facb0148310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 --2- 172.21.15.64:0/1980242243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb00d37b0 0x7facb0148310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faca000ce00 tx=0x7faca000a650 comp rx=0 tx=0).stop 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facaf59e640 1 -- 172.21.15.64:0/1980242243 reap_dead start 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 shutdown_connections 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 >> 172.21.15.64:0/1980242243 conn(0x7facb00721a0 msgr2=0x7facb00cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 shutdown_connections 2023-11-24T08:04:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.216+0000 7facb51e5640 1 -- 172.21.15.64:0/1980242243 wait complete. 2023-11-24T08:04:03.232 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2023-11-24T08:04:03.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T08:04:03.237 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:03.237 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:03.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T08:04:03.238 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:03.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T08:04:03.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T08:04:03.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.531+0000 7f6f2353c640 1 Processor -- start 2023-11-24T08:04:03.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.532+0000 7f495359e640 1 Processor -- start 2023-11-24T08:04:03.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.532+0000 7f495359e640 1 -- start start 2023-11-24T08:04:03.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.534+0000 7f495359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49540cb6d0 0x7f49540cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.534+0000 7f495359e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4954071010 con 0x7f49540b3a50 2023-11-24T08:04:03.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.534+0000 7f495359e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49540ce5f0 con 0x7f49540b1360 2023-11-24T08:04:03.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.534+0000 7f495359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4954159850 con 0x7f49540cb6d0 2023-11-24T08:04:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.532+0000 7f6f2353c640 1 -- start start 2023-11-24T08:04:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.534+0000 7f6f2353c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14005eb0 0x7f6f140062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.534+0000 7f6f2353c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6f14006830 0x7f6f14006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.535+0000 7f6f2253a640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6f140eacd0 0x7f6f1404fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56696/0 (socket says 172.21.15.64:56696) 2023-11-24T08:04:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.536+0000 7f6f2253a640 1 -- 172.21.15.64:0/1113775265 learned_addr learned my addr 172.21.15.64:0/1113775265 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.535+0000 7f4952d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49540cb6d0 0x7f49540cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.535+0000 7f4952d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49540cb6d0 0x7f49540cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36842/0 (socket says 172.21.15.64:36842) 2023-11-24T08:04:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.535+0000 7f4952d9d640 1 -- 172.21.15.64:0/216528091 learned_addr learned my addr 172.21.15.64:0/216528091 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.536+0000 7f6f21d39640 1 --2- 172.21.15.64:0/1113775265 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14005eb0 0x7f6f140062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.534+0000 7f6f2353c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f14054300 con 0x7f6f140eacd0 2023-11-24T08:04:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.536+0000 7f4952d9d640 1 -- 172.21.15.64:0/216528091 >> v1:172.21.15.91:6789/0 conn(0x7f49540b1360 legacy=0x7f49540702a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T08:04:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.537+0000 7f4952d9d640 1 -- 172.21.15.64:0/216528091 >> v1:172.21.15.64:6789/0 conn(0x7f49540b3a50 legacy=0x7f4954159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.537+0000 7f4952d9d640 1 -- 172.21.15.64:0/216528091 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4954070ca0 con 0x7f49540cb6d0 2023-11-24T08:04:03.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.537+0000 7f6f2353c640 1 -- 172.21.15.64:0/1113775265 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f140536f0 con 0x7f6f14006830 2023-11-24T08:04:03.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.537+0000 7f09a8d1d640 1 Processor -- start 2023-11-24T08:04:03.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.537+0000 7f6f2353c640 1 -- 172.21.15.64:0/1113775265 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140539d0 con 0x7f6f14005eb0 2023-11-24T08:04:03.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.538+0000 7f09a8d1d640 1 -- start start 2023-11-24T08:04:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.537+0000 7f6f22d3b640 1 --2- 172.21.15.64:0/1113775265 >> v2:172.21.15.64:3300/0 conn(0x7f6f14006830 0x7f6f14006cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.538+0000 7f6f22d3b640 1 -- 172.21.15.64:0/1113775265 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14005eb0 msgr2=0x7f6f140062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.538+0000 7f6f22d3b640 1 --2- 172.21.15.64:0/1113775265 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14005eb0 0x7f6f140062f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.538+0000 7f6f22d3b640 1 -- 172.21.15.64:0/1113775265 >> v1:172.21.15.64:6789/0 conn(0x7f6f140eacd0 legacy=0x7f6f1404fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.538+0000 7f6f22d3b640 1 -- 172.21.15.64:0/1113775265 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f14053cb0 con 0x7f6f14006830 2023-11-24T08:04:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.539+0000 7f4952d9d640 1 --2- 172.21.15.64:0/216528091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f49540cbb20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4944009ed0 tx=0x7f4944030400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde8676312194160 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.540+0000 7f6f22d3b640 1 --2- 172.21.15.64:0/1113775265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14006830 0x7f6f14006cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0681c0 tx=0x7f6f1c0a5c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b744193bd29bc885 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.540+0000 7f09a8d1d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f099c00c060 0x7f099c038040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.540+0000 7f09a8d1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f099c053c30 0x7f099c0df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.540+0000 7f09a8d1d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c054480 con 0x7f099c033eb0 2023-11-24T08:04:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.540+0000 7f09a8d1d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c0541a0 con 0x7f099c053c30 2023-11-24T08:04:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.540+0000 7f09a8d1d640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c005f40 con 0x7f099c00c060 2023-11-24T08:04:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.540+0000 7f6f21538640 1 -- 172.21.15.64:0/1113775265 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0a6a10 con 0x7f6f14006830 2023-11-24T08:04:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.541+0000 7f09a2d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f099c033eb0 0x7f099c04fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56710/0 (socket says 172.21.15.64:56710) 2023-11-24T08:04:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.541+0000 7f09a2d9d640 1 -- 172.21.15.64:0/1379913410 learned_addr learned my addr 172.21.15.64:0/1379913410 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.541+0000 7f09a1d9b640 1 --2- 172.21.15.64:0/1379913410 >> v2:172.21.15.64:3300/0 conn(0x7f099c053c30 0x7f099c0df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.541+0000 7f09a259c640 1 --2- 172.21.15.64:0/1379913410 >> v2:172.21.15.91:3300/0 conn(0x7f099c00c060 0x7f099c038040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.541+0000 7f495159a640 1 -- 172.21.15.64:0/216528091 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944037600 con 0x7f49540cb6d0 2023-11-24T08:04:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.541+0000 7f6f21538640 1 -- 172.21.15.64:0/1113775265 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c0a6b70 con 0x7f6f14006830 2023-11-24T08:04:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.541+0000 7f6f21538640 1 -- 172.21.15.64:0/1113775265 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0ae440 con 0x7f6f14006830 2023-11-24T08:04:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7f6f2353c640 1 -- 172.21.15.64:0/1113775265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14006830 msgr2=0x7f6f14006cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7f6f2353c640 1 --2- 172.21.15.64:0/1113775265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14006830 0x7f6f14006cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0681c0 tx=0x7f6f1c0a5c20 comp rx=0 tx=0).stop 2023-11-24T08:04:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7f09a259c640 1 -- 172.21.15.64:0/1379913410 >> v2:172.21.15.64:3300/0 conn(0x7f099c053c30 msgr2=0x7f099c0df150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7f09a259c640 1 --2- 172.21.15.64:0/1379913410 >> v2:172.21.15.64:3300/0 conn(0x7f099c053c30 0x7f099c0df150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7f09a259c640 1 -- 172.21.15.64:0/1379913410 >> v1:172.21.15.64:6789/0 conn(0x7f099c033eb0 legacy=0x7f099c04fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7fd2b8ebb640 1 Processor -- start 2023-11-24T08:04:03.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7f495159a640 1 -- 172.21.15.64:0/216528091 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4944033050 con 0x7f49540cb6d0 2023-11-24T08:04:03.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7f495159a640 1 -- 172.21.15.64:0/216528091 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944037980 con 0x7f49540cb6d0 2023-11-24T08:04:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.542+0000 7f09a259c640 1 -- 172.21.15.64:0/1379913410 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c006220 con 0x7f099c00c060 2023-11-24T08:04:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7fd2b8ebb640 1 -- start start 2023-11-24T08:04:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7f6f2353c640 1 -- 172.21.15.64:0/1113775265 shutdown_connections 2023-11-24T08:04:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7f6f2353c640 1 --2- 172.21.15.64:0/1113775265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14006830 0x7f6f14006cc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7f6f2353c640 1 --2- 172.21.15.64:0/1113775265 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14005eb0 0x7f6f140062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7f6f2353c640 1 -- 172.21.15.64:0/1113775265 >> 172.21.15.64:0/1113775265 conn(0x7f6f14009f80 msgr2=0x7f6f1400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7f495359e640 1 -- 172.21.15.64:0/216528091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 msgr2=0x7f49540cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7f495359e640 1 --2- 172.21.15.64:0/216528091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f49540cbb20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4944009ed0 tx=0x7f4944030400 comp rx=0 tx=0).stop 2023-11-24T08:04:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7f495359e640 1 -- 172.21.15.64:0/216528091 shutdown_connections 2023-11-24T08:04:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.543+0000 7f495359e640 1 --2- 172.21.15.64:0/216528091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f49540cbb20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7f495359e640 1 -- 172.21.15.64:0/216528091 >> 172.21.15.64:0/216528091 conn(0x7f49540b0840 msgr2=0x7f49540b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7f6f2353c640 1 -- 172.21.15.64:0/1113775265 shutdown_connections 2023-11-24T08:04:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7f495359e640 1 -- 172.21.15.64:0/216528091 shutdown_connections 2023-11-24T08:04:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7f495359e640 1 -- 172.21.15.64:0/216528091 wait complete. 2023-11-24T08:04:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7fd2b8ebb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2b415d530 0x7fd2b40d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7fd2b8ebb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd2b40d1570 0x7fd2b40b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7fd2b8ebb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b40d3220 0x7fd2b40d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7fd2b8ebb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b40d30c0 con 0x7fd2b415d530 2023-11-24T08:04:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7fd2b8ebb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b415d910 con 0x7fd2b40d1570 2023-11-24T08:04:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.544+0000 7fd2b8ebb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40d3b90 con 0x7fd2b40d3220 2023-11-24T08:04:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.545+0000 7fd2b259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2b415d530 0x7fd2b40d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.545+0000 7fd2a9d9b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd2b40d1570 0x7fd2b40b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.545+0000 7fd2a9d9b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd2b40d1570 0x7fd2b40b8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34416/0 (socket says 172.21.15.64:34416) 2023-11-24T08:04:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.545+0000 7f09a259c640 1 --2- 172.21.15.64:0/1379913410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 0x7f099c038040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b060 tx=0x7f099402fce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db29fe4a57471255 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.545+0000 7fd2a9d9b640 1 -- 172.21.15.64:0/719438971 learned_addr learned my addr 172.21.15.64:0/719438971 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.545+0000 7fd2b2d9d640 1 --2- 172.21.15.64:0/719438971 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b40d3220 0x7fd2b40d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.545+0000 7fd2b259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2b415d530 0x7fd2b40d2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36878/0 (socket says 172.21.15.64:36878) 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.546+0000 7fd2a9d9b640 1 -- 172.21.15.64:0/719438971 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b40d3220 msgr2=0x7fd2b40d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.546+0000 7fd2a9d9b640 1 --2- 172.21.15.64:0/719438971 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b40d3220 0x7fd2b40d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.546+0000 7fd2a9d9b640 1 -- 172.21.15.64:0/719438971 >> v2:172.21.15.64:3300/0 conn(0x7fd2b415d530 msgr2=0x7fd2b40d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.546+0000 7fd2a9d9b640 1 --2- 172.21.15.64:0/719438971 >> v2:172.21.15.64:3300/0 conn(0x7fd2b415d530 0x7fd2b40d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.546+0000 7fd2a9d9b640 1 -- 172.21.15.64:0/719438971 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40d1ae0 con 0x7fd2b40d1570 2023-11-24T08:04:03.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.546+0000 7fd2b259c640 1 --2- 172.21.15.64:0/719438971 >> v2:172.21.15.64:3300/0 conn(0x7fd2b415d530 0x7fd2b40d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:04:03.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.546+0000 7f09a159a640 1 -- 172.21.15.64:0/1379913410 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994030a90 con 0x7f099c00c060 2023-11-24T08:04:03.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.547+0000 7f09a159a640 1 -- 172.21.15.64:0/1379913410 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0994030bf0 con 0x7f099c00c060 2023-11-24T08:04:03.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.547+0000 7f09a159a640 1 -- 172.21.15.64:0/1379913410 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994038440 con 0x7f099c00c060 2023-11-24T08:04:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.547+0000 7f97b8090640 1 Processor -- start 2023-11-24T08:04:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.547+0000 7f495359e640 1 Processor -- start 2023-11-24T08:04:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.548+0000 7f495359e640 1 -- start start 2023-11-24T08:04:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.547+0000 7f6f2353c640 1 -- 172.21.15.64:0/1113775265 wait complete. 2023-11-24T08:04:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.548+0000 7f09a8d1d640 1 -- 172.21.15.64:0/1379913410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 msgr2=0x7f099c038040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.548+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/1379913410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 0x7f099c038040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b060 tx=0x7f099402fce0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.547+0000 7f97b8090640 1 -- start start 2023-11-24T08:04:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.548+0000 7f09a8d1d640 1 -- 172.21.15.64:0/1379913410 shutdown_connections 2023-11-24T08:04:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.548+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/1379913410 >> v2:172.21.15.64:3300/0 conn(0x7f099c053c30 0x7f099c0df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.548+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/1379913410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 0x7f099c038040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.548+0000 7f09a8d1d640 1 -- 172.21.15.64:0/1379913410 >> 172.21.15.64:0/1379913410 conn(0x7f099c009f70 msgr2=0x7f099c00a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.548+0000 7f75a07dc640 1 Processor -- start 2023-11-24T08:04:03.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f75a07dc640 1 -- start start 2023-11-24T08:04:03.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f97b8090640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f97b00ccb40 0x7f97b00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f09a8d1d640 1 -- 172.21.15.64:0/1379913410 shutdown_connections 2023-11-24T08:04:03.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f97b8090640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 0x7f97b00c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f97b8090640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b00cf8c0 con 0x7f97b015ead0 2023-11-24T08:04:03.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f97b8090640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b00ce7c0 con 0x7f97b00ccb40 2023-11-24T08:04:03.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f97b8090640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00cf590 con 0x7f97b00d19a0 2023-11-24T08:04:03.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f97b5e2c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f97b015ead0 0x7f97b00cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56716/0 (socket says 172.21.15.64:56716) 2023-11-24T08:04:03.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f97b5e2c640 1 -- 172.21.15.64:0/4131309474 learned_addr learned my addr 172.21.15.64:0/4131309474 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7fd2a9d9b640 1 --2- 172.21.15.64:0/719438971 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b40d1570 0x7fd2b40b8c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40b5d90 tx=0x7fd29c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e19e51073b408cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f6f2353c640 1 Processor -- start 2023-11-24T08:04:03.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f97b562b640 1 --2- 172.21.15.64:0/4131309474 >> v2:172.21.15.91:3300/0 conn(0x7f97b00ccb40 0x7f97b00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.549+0000 7f09a8d1d640 1 -- 172.21.15.64:0/1379913410 wait complete. 2023-11-24T08:04:03.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f97b662d640 1 --2- 172.21.15.64:0/4131309474 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 0x7f97b00c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f97b4e2a640 1 -- 172.21.15.64:0/4131309474 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197656523 0 0) 0x7f97b00cf8c0 con 0x7f97b015ead0 2023-11-24T08:04:03.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f75a07dc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f75900e6620 0x7f759003e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f75a07dc640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f75900068b0 0x7f7590053b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f75a07dc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f75900e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f75a07dc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7590054530 con 0x7f75900e6620 2023-11-24T08:04:03.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f75a07dc640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75900e6a00 con 0x7f75900068b0 2023-11-24T08:04:03.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f75a07dc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7590056a60 con 0x7f75900540a0 2023-11-24T08:04:03.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f495359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49540b1360 0x7f49540a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f759dd77640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f75900068b0 0x7f7590053b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f759dd77640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f75900068b0 0x7f7590053b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34436/0 (socket says 172.21.15.64:34436) 2023-11-24T08:04:03.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7fd2b1d9b640 1 -- 172.21.15.64:0/719438971 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c031870 con 0x7fd2b40d1570 2023-11-24T08:04:03.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7fd2b1d9b640 1 -- 172.21.15.64:0/719438971 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c0319d0 con 0x7fd2b40d1570 2023-11-24T08:04:03.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7fd2b1d9b640 1 -- 172.21.15.64:0/719438971 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c031d60 con 0x7fd2b40d1570 2023-11-24T08:04:03.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f759dd77640 1 -- 172.21.15.64:0/1278838638 learned_addr learned my addr 172.21.15.64:0/1278838638 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f97b4e2a640 1 -- 172.21.15.64:0/4131309474 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f979c003520 con 0x7f97b015ead0 2023-11-24T08:04:03.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.550+0000 7f6f2353c640 1 -- start start 2023-11-24T08:04:03.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f495359e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49540b3a50 0x7f495406cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f495359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f495406d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f495359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954159990 con 0x7f49540cb6d0 2023-11-24T08:04:03.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f495359e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49541449b0 con 0x7f49540b3a50 2023-11-24T08:04:03.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.551+0000 7f495359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49540a89a0 con 0x7f49540b1360 2023-11-24T08:04:03.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f97b562b640 1 -- 172.21.15.64:0/4131309474 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 msgr2=0x7f97b00c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7fd2b8ebb640 1 -- 172.21.15.64:0/719438971 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b40d1570 msgr2=0x7fd2b40b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7fd2b8ebb640 1 --2- 172.21.15.64:0/719438971 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b40d1570 0x7fd2b40b8c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40b5d90 tx=0x7fd29c02b910 comp rx=0 tx=0).stop 2023-11-24T08:04:03.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f97b562b640 1 --2- 172.21.15.64:0/4131309474 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 0x7f97b00c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f6f2353c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14005eb0 0x7f6f14044340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f759ed79640 1 --2- 172.21.15.64:0/1278838638 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f75900e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f6f2353c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14006830 0x7f6f14044880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7fd2b8ebb640 1 -- 172.21.15.64:0/719438971 shutdown_connections 2023-11-24T08:04:03.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7fd2b8ebb640 1 --2- 172.21.15.64:0/719438971 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b40d3220 0x7fd2b40d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7fd2b8ebb640 1 --2- 172.21.15.64:0/719438971 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b40d1570 0x7fd2b40b8c20 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40b5d90 tx=0x7fd29c02b910 comp rx=0 tx=0).stop 2023-11-24T08:04:03.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f6f2353c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f140eacd0 0x7f6f14044dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7fd2b8ebb640 1 --2- 172.21.15.64:0/719438971 >> v2:172.21.15.64:3300/0 conn(0x7fd2b415d530 0x7fd2b40d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7fd2b8ebb640 1 -- 172.21.15.64:0/719438971 >> 172.21.15.64:0/719438971 conn(0x7fd2b4071f90 msgr2=0x7fd2b40723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f6f2353c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14043650 con 0x7f6f14005eb0 2023-11-24T08:04:03.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f6f2353c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14035c50 con 0x7f6f140eacd0 2023-11-24T08:04:03.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f6f2353c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14043860 con 0x7f6f14006830 2023-11-24T08:04:03.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f759dd77640 1 -- 172.21.15.64:0/1278838638 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 msgr2=0x7f75900e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f759dd77640 1 --2- 172.21.15.64:0/1278838638 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f75900e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f759dd77640 1 -- 172.21.15.64:0/1278838638 >> v2:172.21.15.64:3300/0 conn(0x7f75900e6620 msgr2=0x7f759003e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f759dd77640 1 --2- 172.21.15.64:0/1278838638 >> v2:172.21.15.64:3300/0 conn(0x7f75900e6620 0x7f759003e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f759dd77640 1 -- 172.21.15.64:0/1278838638 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7590006e20 con 0x7f75900068b0 2023-11-24T08:04:03.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f4952d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f495406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f4952d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f495406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36894/0 (socket says 172.21.15.64:36894) 2023-11-24T08:04:03.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f4952d9d640 1 -- 172.21.15.64:0/973067436 learned_addr learned my addr 172.21.15.64:0/973067436 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f4951d9b640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49540b3a50 0x7f495406cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f4952d9d640 1 -- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49540b1360 msgr2=0x7f49540a8460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T08:04:03.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f6f21d39640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14006830 0x7f6f14044880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7fd2b8ebb640 1 -- 172.21.15.64:0/719438971 shutdown_connections 2023-11-24T08:04:03.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f6f21d39640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14006830 0x7f6f14044880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54714/0 (socket says 172.21.15.64:54714) 2023-11-24T08:04:03.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f6f21d39640 1 -- 172.21.15.64:0/301322731 learned_addr learned my addr 172.21.15.64:0/301322731 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f4952d9d640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49540b1360 0x7f49540a8460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f4952d9d640 1 -- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49540b3a50 msgr2=0x7f495406cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f4952d9d640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49540b3a50 0x7f495406cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f4952d9d640 1 -- 172.21.15.64:0/973067436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944009b30 con 0x7f49540cb6d0 2023-11-24T08:04:03.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f6f22d3b640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f140eacd0 0x7f6f14044dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.552+0000 7f97b562b640 1 -- 172.21.15.64:0/4131309474 >> v1:172.21.15.64:6789/0 conn(0x7f97b015ead0 legacy=0x7f97b00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f97b562b640 1 -- 172.21.15.64:0/4131309474 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00ce4e0 con 0x7f97b00ccb40 2023-11-24T08:04:03.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f6f2253a640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14005eb0 0x7f6f14044340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f6f22d3b640 1 -- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14006830 msgr2=0x7f6f14044880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f6f22d3b640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14006830 0x7f6f14044880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f6f22d3b640 1 -- 172.21.15.64:0/301322731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14005eb0 msgr2=0x7f6f14044340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f6f22d3b640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14005eb0 0x7f6f14044340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f6f22d3b640 1 -- 172.21.15.64:0/301322731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c04f730 con 0x7f6f140eacd0 2023-11-24T08:04:03.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7f4952d9d640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f495406d310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4944009ed0 tx=0x7f4944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.553+0000 7fd2b8ebb640 1 -- 172.21.15.64:0/719438971 wait complete. 2023-11-24T08:04:03.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f759dd77640 1 --2- 172.21.15.64:0/1278838638 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 0x7f7590053b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f75900586b0 tx=0x7f758802b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbdf80802d60bb24 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f09a8d1d640 1 Processor -- start 2023-11-24T08:04:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f4942ffd640 1 -- 172.21.15.64:0/973067436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944041600 con 0x7f49540cb6d0 2023-11-24T08:04:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f4942ffd640 1 -- 172.21.15.64:0/973067436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4944041760 con 0x7f49540cb6d0 2023-11-24T08:04:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f4942ffd640 1 -- 172.21.15.64:0/973067436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944041ad0 con 0x7f49540cb6d0 2023-11-24T08:04:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49541446d0 con 0x7f49540cb6d0 2023-11-24T08:04:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4954141e10 con 0x7f49540cb6d0 2023-11-24T08:04:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.554+0000 7f09a8d1d640 1 -- start start 2023-11-24T08:04:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.555+0000 7f759d576640 1 -- 172.21.15.64:0/1278838638 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880317f0 con 0x7f75900068b0 2023-11-24T08:04:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.555+0000 7f759d576640 1 -- 172.21.15.64:0/1278838638 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588031950 con 0x7f75900068b0 2023-11-24T08:04:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.555+0000 7f759d576640 1 -- 172.21.15.64:0/1278838638 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588031ce0 con 0x7f75900068b0 2023-11-24T08:04:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.555+0000 7f97b562b640 1 --2- 172.21.15.64:0/4131309474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b00ccb40 0x7f97b00c49d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a00097a0 tx=0x7f97a002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fd206156bd48e12 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.555+0000 7f6f22d3b640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f140eacd0 0x7f6f14044dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c050020 tx=0x7f6f1c068480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.555+0000 7f97b4e2a640 1 -- 172.21.15.64:0/4131309474 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a002e070 con 0x7f97b00ccb40 2023-11-24T08:04:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.555+0000 7f97b4e2a640 1 -- 172.21.15.64:0/4131309474 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a00315a0 con 0x7f97b00ccb40 2023-11-24T08:04:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.555+0000 7f97b4e2a640 1 -- 172.21.15.64:0/4131309474 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0031930 con 0x7f97b00ccb40 2023-11-24T08:04:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f75a07dc640 1 -- 172.21.15.64:0/1278838638 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 msgr2=0x7f7590053b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1278838638 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 0x7f7590053b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f75900586b0 tx=0x7f758802b890 comp rx=0 tx=0).stop 2023-11-24T08:04:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7fd2b8ebb640 1 Processor -- start 2023-11-24T08:04:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f75a07dc640 1 -- 172.21.15.64:0/1278838638 shutdown_connections 2023-11-24T08:04:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1278838638 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f75900e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1278838638 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 0x7f7590053b60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1278838638 >> v2:172.21.15.64:3300/0 conn(0x7f75900e6620 0x7f759003e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f75a07dc640 1 -- 172.21.15.64:0/1278838638 >> 172.21.15.64:0/1278838638 conn(0x7f7590058e90 msgr2=0x7f7590059270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f6f12ffd640 1 -- 172.21.15.64:0/301322731 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0ba650 con 0x7f6f140eacd0 2023-11-24T08:04:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f75a07dc640 1 -- 172.21.15.64:0/1278838638 shutdown_connections 2023-11-24T08:04:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f6f2353c640 1 -- 172.21.15.64:0/301322731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f14043b40 con 0x7f6f140eacd0 2023-11-24T08:04:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f6f2353c640 1 -- 172.21.15.64:0/301322731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f14045300 con 0x7f6f140eacd0 2023-11-24T08:04:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f97b8090640 1 -- 172.21.15.64:0/4131309474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b00ccb40 msgr2=0x7f97b00c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f97b8090640 1 --2- 172.21.15.64:0/4131309474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b00ccb40 0x7f97b00c49d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a00097a0 tx=0x7f97a002b840 comp rx=0 tx=0).stop 2023-11-24T08:04:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f09a8d1d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 0x7f099c04d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.556+0000 7f09a8d1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f099c033eb0 0x7f099c04a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f09a8d1d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f099c053c30 0x7f099c04a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f09a8d1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c04ea90 con 0x7f099c033eb0 2023-11-24T08:04:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f09a8d1d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c04e600 con 0x7f099c00c060 2023-11-24T08:04:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f09a8d1d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c04d750 con 0x7f099c053c30 2023-11-24T08:04:03.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7fd2b8ebb640 1 -- start start 2023-11-24T08:04:03.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f97b8090640 1 -- 172.21.15.64:0/4131309474 shutdown_connections 2023-11-24T08:04:03.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f97b8090640 1 --2- 172.21.15.64:0/4131309474 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 0x7f97b00c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f97b8090640 1 --2- 172.21.15.64:0/4131309474 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b00ccb40 0x7f97b00c49d0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a00097a0 tx=0x7f97a002b840 comp rx=0 tx=0).stop 2023-11-24T08:04:03.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f97b8090640 1 -- 172.21.15.64:0/4131309474 >> 172.21.15.64:0/4131309474 conn(0x7f97b00d0e80 msgr2=0x7f97b00d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f09a259c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 0x7f099c04d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7f97b8090640 1 -- 172.21.15.64:0/4131309474 shutdown_connections 2023-11-24T08:04:03.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7f97b8090640 1 -- 172.21.15.64:0/4131309474 wait complete. 2023-11-24T08:04:03.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7f09a1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f099c033eb0 0x7f099c04a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7f09a259c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 0x7f099c04d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34460/0 (socket says 172.21.15.64:34460) 2023-11-24T08:04:03.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7f09a259c640 1 -- 172.21.15.64:0/2546534385 learned_addr learned my addr 172.21.15.64:0/2546534385 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4954141990 con 0x7f49540cb6d0 2023-11-24T08:04:03.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7f09a2d9d640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f099c053c30 0x7f099c04a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7fd2b8ebb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2b40d3220 0x7fd2b414d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7fd2b8ebb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b415d530 0x7fd2b40b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7fd2b8ebb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b41f3070 0x7fd2b40b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7fd2b8ebb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4157e30 con 0x7fd2b40d3220 2023-11-24T08:04:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7fd2b8ebb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40b5bd0 con 0x7fd2b41f3070 2023-11-24T08:04:03.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.558+0000 7fd2b8ebb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b414dc10 con 0x7fd2b415d530 2023-11-24T08:04:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f4942ffd640 1 -- 172.21.15.64:0/973067436 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f494403e070 con 0x7f49540cb6d0 2023-11-24T08:04:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f44c24da640 1 Processor -- start 2023-11-24T08:04:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f44c24da640 1 -- start start 2023-11-24T08:04:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f09a1d9b640 1 -- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f099c053c30 msgr2=0x7f099c04a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f09a1d9b640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f099c053c30 0x7f099c04a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f09a1d9b640 1 -- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 msgr2=0x7f099c04d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f09a1d9b640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 0x7f099c04d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.557+0000 7f75a07dc640 1 -- 172.21.15.64:0/1278838638 wait complete. 2023-11-24T08:04:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7fd2b259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2b40d3220 0x7fd2b414d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f09a1d9b640 1 -- 172.21.15.64:0/2546534385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099400f610 con 0x7f099c033eb0 2023-11-24T08:04:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7fd2a9d9b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b415d530 0x7fd2b40b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7fd2a9d9b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b415d530 0x7fd2b40b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54730/0 (socket says 172.21.15.64:54730) 2023-11-24T08:04:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7fd2a9d9b640 1 -- 172.21.15.64:0/3659508756 learned_addr learned my addr 172.21.15.64:0/3659508756 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7f75a07dc640 1 Processor -- start 2023-11-24T08:04:03.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.559+0000 7f6f2353c640 1 -- 172.21.15.64:0/301322731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f14044060 con 0x7f6f140eacd0 2023-11-24T08:04:03.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7f6f12ffd640 1 -- 172.21.15.64:0/301322731 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c0ba7b0 con 0x7f6f140eacd0 2023-11-24T08:04:03.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7f6f12ffd640 1 -- 172.21.15.64:0/301322731 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0ba650 con 0x7f6f140eacd0 2023-11-24T08:04:03.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7f09a1d9b640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f099c033eb0 0x7f099c04a3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f099800e9b0 tx=0x7f099800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7fd2b2d9d640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b41f3070 0x7fd2b40b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7f75a07dc640 1 -- start start 2023-11-24T08:04:03.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7f4942ffd640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f493406af40 0x7f493406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7f97b8090640 1 Processor -- start 2023-11-24T08:04:03.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7f4942ffd640 1 -- 172.21.15.64:0/973067436 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49440ab6a0 con 0x7f49540cb6d0 2023-11-24T08:04:03.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.561+0000 7f97b8090640 1 -- start start 2023-11-24T08:04:03.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7fd2b259c640 1 -- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b415d530 msgr2=0x7fd2b40b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.560+0000 7fd2b259c640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b415d530 0x7fd2b40b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.561+0000 7fd2b259c640 1 -- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b41f3070 msgr2=0x7fd2b40b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.561+0000 7fd2b259c640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b41f3070 0x7fd2b40b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.561+0000 7fd2b259c640 1 -- 172.21.15.64:0/3659508756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4009a70 con 0x7fd2b40d3220 2023-11-24T08:04:03.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.561+0000 7f44c24da640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f44bc1619a0 0x7f44bc0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.561+0000 7f495259c640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f493406af40 0x7f493406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.561+0000 7fd2b259c640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2b40d3220 0x7fd2b414d6d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009e60 tx=0x7fd2a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f44c24da640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc161d80 con 0x7f44bc0ae750 2023-11-24T08:04:03.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f44bb7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f44bc0ae750 0x7f44bc0b32a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56732/0 (socket says 172.21.15.64:56732) 2023-11-24T08:04:03.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f44bb7fe640 1 -- 172.21.15.64:0/1498168751 learned_addr learned my addr 172.21.15.64:0/1498168751 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:03.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f44c24da640 1 -- 172.21.15.64:0/1498168751 --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc0ad170 con 0x7f44bc0b39b0 2023-11-24T08:04:03.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f75a07dc640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 0x7f7590052790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f97b8090640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 0x7f97b00c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f97b8090640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b015ead0 0x7f97b014db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f97b8090640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b014e0a0 0x7f97b00c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f97b8090640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00b4450 con 0x7f97b015ead0 2023-11-24T08:04:03.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f97b8090640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b014e520 con 0x7f97b014e0a0 2023-11-24T08:04:03.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f97b8090640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00b4650 con 0x7f97b00d19a0 2023-11-24T08:04:03.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f75a07dc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f7590185120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f44c24da640 1 -- 172.21.15.64:0/1498168751 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc0ad450 con 0x7f44bc1619a0 2023-11-24T08:04:03.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f75a07dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75900e6620 0x7f7590050970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f44bbfff640 1 --2- 172.21.15.64:0/1498168751 >> v2:172.21.15.64:3300/0 conn(0x7f44bc1619a0 0x7f44bc0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f75a07dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7590185660 con 0x7f75900e6620 2023-11-24T08:04:03.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b562b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b015ead0 0x7f97b014db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f75a07dc640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75900e59b0 con 0x7f75900068b0 2023-11-24T08:04:03.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b562b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b015ead0 0x7f97b014db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36948/0 (socket says 172.21.15.64:36948) 2023-11-24T08:04:03.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b562b640 1 -- 172.21.15.64:0/358075639 learned_addr learned my addr 172.21.15.64:0/358075639 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f75a07dc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7590051730 con 0x7f75900540a0 2023-11-24T08:04:03.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b5e2c640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 0x7f97b00c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f759dd77640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f7590185120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f759dd77640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f7590185120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54762/0 (socket says 172.21.15.64:54762) 2023-11-24T08:04:03.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f759dd77640 1 -- 172.21.15.64:0/1123075868 learned_addr learned my addr 172.21.15.64:0/1123075868 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f759ed79640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75900e6620 0x7f7590050970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b662d640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b014e0a0 0x7f97b00c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f37de251640 1 Processor -- start 2023-11-24T08:04:03.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f759e578640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 0x7f7590052790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f44bbfff640 1 -- 172.21.15.64:0/1498168751 >> v1:172.21.15.91:6789/0 conn(0x7f44bc0b39b0 legacy=0x7f44bc06c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b562b640 1 -- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 msgr2=0x7f97b00c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b562b640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 0x7f97b00c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b562b640 1 -- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b014e0a0 msgr2=0x7f97b00c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b562b640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b014e0a0 0x7f97b00c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f97b562b640 1 -- 172.21.15.64:0/358075639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009400 con 0x7f97b015ead0 2023-11-24T08:04:03.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.564+0000 7f44bbfff640 1 -- 172.21.15.64:0/1498168751 >> v1:172.21.15.64:6789/0 conn(0x7f44bc0ae750 legacy=0x7f44bc0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.564+0000 7f44bbfff640 1 -- 172.21.15.64:0/1498168751 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0ad730 con 0x7f44bc1619a0 2023-11-24T08:04:03.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.564+0000 7f97b562b640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b015ead0 0x7f97b014db60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f97a002bdd0 tx=0x7f97a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.562+0000 7f495259c640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f493406af40 0x7f493406d400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f494c009670 tx=0x7f494c006bd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.563+0000 7f37de251640 1 -- start start 2023-11-24T08:04:03.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.564+0000 7f759dd77640 1 -- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 msgr2=0x7f7590052790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.564+0000 7f759dd77640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 0x7f7590052790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.564+0000 7f759dd77640 1 -- 172.21.15.64:0/1123075868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75900e6620 msgr2=0x7f7590050970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.564+0000 7f759dd77640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75900e6620 0x7f7590050970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.564+0000 7f759dd77640 1 -- 172.21.15.64:0/1123075868 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880094d0 con 0x7f75900540a0 2023-11-24T08:04:03.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.565+0000 7f759dd77640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f7590185120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f75880090b0 tx=0x7f75880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.565+0000 7f37de251640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37d80719c0 0x7f37d80ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.565+0000 7f37de251640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d80ad510 con 0x7f37d806d520 2023-11-24T08:04:03.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.565+0000 7f37de251640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d80acc50 con 0x7f37d8161930 2023-11-24T08:04:03.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.565+0000 7f37de251640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d8071e60 con 0x7f37d80719c0 2023-11-24T08:04:03.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.565+0000 7f758effd640 1 -- 172.21.15.64:0/1123075868 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588032890 con 0x7f75900540a0 2023-11-24T08:04:03.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.566+0000 7f758effd640 1 -- 172.21.15.64:0/1123075868 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75880329f0 con 0x7f75900540a0 2023-11-24T08:04:03.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.566+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75900519d0 con 0x7f75900540a0 2023-11-24T08:04:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.566+0000 7f758effd640 1 -- 172.21.15.64:0/1123075868 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588032da0 con 0x7f75900540a0 2023-11-24T08:04:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.566+0000 7f37d6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37d80719c0 0x7f37d80ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.566+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7590051f90 con 0x7f75900540a0 2023-11-24T08:04:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.566+0000 7f44bbfff640 1 --2- 172.21.15.64:0/1498168751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc1619a0 0x7f44bc0ae210 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44b00097a0 tx=0x7f44b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd1696bfb7d7ec1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.565+0000 7f6f12ffd640 1 -- 172.21.15.64:0/301322731 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f1c0ba920 con 0x7f6f140eacd0 2023-11-24T08:04:03.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.566+0000 7f37d6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37d80719c0 0x7f37d80ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36964/0 (socket says 172.21.15.64:36964) 2023-11-24T08:04:03.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.566+0000 7f37d6ffd640 1 -- 172.21.15.64:0/3967636630 learned_addr learned my addr 172.21.15.64:0/3967636630 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.567+0000 7f37d6ffd640 1 -- 172.21.15.64:0/3967636630 >> v1:172.21.15.91:6789/0 conn(0x7f37d8161930 legacy=0x7f37d80ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:03.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.567+0000 7f6f12ffd640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f0006aea0 0x7f6f0006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.567+0000 7f6f12ffd640 1 -- 172.21.15.64:0/301322731 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f1c125d10 con 0x7f6f140eacd0 2023-11-24T08:04:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.567+0000 7f37d6ffd640 1 -- 172.21.15.64:0/3967636630 >> v1:172.21.15.64:6789/0 conn(0x7f37d806d520 legacy=0x7f37d806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.567+0000 7f37d6ffd640 1 -- 172.21.15.64:0/3967636630 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80acec0 con 0x7f37d80719c0 2023-11-24T08:04:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.567+0000 7f6f2253a640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f0006aea0 0x7f6f0006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.567+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7590005280 con 0x7f75900540a0 2023-11-24T08:04:03.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7f37d6ffd640 1 --2- 172.21.15.64:0/3967636630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d80719c0 0x7f37d80ae6a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009870 tx=0x7f37c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff461fd6ac941122 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7f44baffd640 1 -- 172.21.15.64:0/1498168751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0031830 con 0x7f44bc1619a0 2023-11-24T08:04:03.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7f44baffd640 1 -- 172.21.15.64:0/1498168751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b0031990 con 0x7f44bc1619a0 2023-11-24T08:04:03.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7f44baffd640 1 -- 172.21.15.64:0/1498168751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0031830 con 0x7f44bc1619a0 2023-11-24T08:04:03.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7fd2ab7fe640 1 -- 172.21.15.64:0/3659508756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4014680 con 0x7fd2b40d3220 2023-11-24T08:04:03.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7fd2ab7fe640 1 -- 172.21.15.64:0/3659508756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a400f040 con 0x7fd2b40d3220 2023-11-24T08:04:03.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7fd2ab7fe640 1 -- 172.21.15.64:0/3659508756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4014680 con 0x7fd2b40d3220 2023-11-24T08:04:03.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f4942ffd640 1 -- 172.21.15.64:0/973067436 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4954141990 con 0x7f49540cb6d0 2023-11-24T08:04:03.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f97a67fc640 1 -- 172.21.15.64:0/358075639 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0004030 con 0x7f97b015ead0 2023-11-24T08:04:03.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f97a67fc640 1 -- 172.21.15.64:0/358075639 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a0004190 con 0x7f97b015ead0 2023-11-24T08:04:03.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f97a67fc640 1 -- 172.21.15.64:0/358075639 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0004500 con 0x7f97b015ead0 2023-11-24T08:04:03.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b00b4930 con 0x7f97b015ead0 2023-11-24T08:04:03.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b00b4ef0 con 0x7f97b015ead0 2023-11-24T08:04:03.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7f0992ffd640 1 -- 172.21.15.64:0/2546534385 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800c790 con 0x7f099c033eb0 2023-11-24T08:04:03.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f6f2253a640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f0006aea0 0x7f6f0006d360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c007fd0 tx=0x7f6f0c007c20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.568+0000 7f37d67fc640 1 -- 172.21.15.64:0/3967636630 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8030860 con 0x7f37d80719c0 2023-11-24T08:04:03.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f37d67fc640 1 -- 172.21.15.64:0/3967636630 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c80309c0 con 0x7f37d80719c0 2023-11-24T08:04:03.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f37d67fc640 1 -- 172.21.15.64:0/3967636630 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8030d50 con 0x7f37d80719c0 2023-11-24T08:04:03.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7fd2b8ebb640 1 -- 172.21.15.64:0/3659508756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c0094d0 con 0x7fd2b40d3220 2023-11-24T08:04:03.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f44c24da640 1 -- 172.21.15.64:0/1498168751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc1619a0 msgr2=0x7f44bc0ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f44c24da640 1 --2- 172.21.15.64:0/1498168751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc1619a0 0x7f44bc0ae210 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44b00097a0 tx=0x7f44b002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f44c24da640 1 -- 172.21.15.64:0/1498168751 shutdown_connections 2023-11-24T08:04:03.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f44c24da640 1 --2- 172.21.15.64:0/1498168751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc1619a0 0x7f44bc0ae210 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f44c24da640 1 -- 172.21.15.64:0/1498168751 >> 172.21.15.64:0/1498168751 conn(0x7f44bc0709f0 msgr2=0x7f44bc0718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f44c24da640 1 -- 172.21.15.64:0/1498168751 shutdown_connections 2023-11-24T08:04:03.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.571+0000 7f37de251640 1 -- 172.21.15.64:0/3967636630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d80719c0 msgr2=0x7f37d80ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.571+0000 7fd2b8ebb640 1 -- 172.21.15.64:0/3659508756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b4157770 con 0x7fd2b40d3220 2023-11-24T08:04:03.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.571+0000 7f44c24da640 1 -- 172.21.15.64:0/1498168751 wait complete. 2023-11-24T08:04:03.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b004e840 con 0x7f97b015ead0 2023-11-24T08:04:03.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.571+0000 7f37de251640 1 --2- 172.21.15.64:0/3967636630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d80719c0 0x7f37d80ae6a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009870 tx=0x7f37c802b9a0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f0992ffd640 1 -- 172.21.15.64:0/2546534385 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099800c8f0 con 0x7f099c033eb0 2023-11-24T08:04:03.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f0992ffd640 1 -- 172.21.15.64:0/2546534385 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800cca0 con 0x7f099c033eb0 2023-11-24T08:04:03.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.569+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c04d9f0 con 0x7f099c033eb0 2023-11-24T08:04:03.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c04dfb0 con 0x7f099c033eb0 2023-11-24T08:04:03.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.571+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c04e770 con 0x7f099c033eb0 2023-11-24T08:04:03.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.570+0000 7f758effd640 1 -- 172.21.15.64:0/1123075868 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f758804d3f0 con 0x7f75900540a0 2023-11-24T08:04:03.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.572+0000 7f37de251640 1 -- 172.21.15.64:0/3967636630 shutdown_connections 2023-11-24T08:04:03.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.572+0000 7f37de251640 1 --2- 172.21.15.64:0/3967636630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d80719c0 0x7f37d80ae6a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.572+0000 7f37de251640 1 -- 172.21.15.64:0/3967636630 >> 172.21.15.64:0/3967636630 conn(0x7f37d80b0d80 msgr2=0x7f37d80b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:03.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.572+0000 7f37de251640 1 -- 172.21.15.64:0/3967636630 shutdown_connections 2023-11-24T08:04:03.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.572+0000 7f37de251640 1 -- 172.21.15.64:0/3967636630 wait complete. 2023-11-24T08:04:03.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.572+0000 7f758effd640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757c06ae40 0x7f757c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.572+0000 7f758effd640 1 -- 172.21.15.64:0/1123075868 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75880ab550 con 0x7f75900540a0 2023-11-24T08:04:03.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.573+0000 7f44c24da640 1 Processor -- start 2023-11-24T08:04:03.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.573+0000 7f44c24da640 1 -- start start 2023-11-24T08:04:03.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.574+0000 7f759e578640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757c06ae40 0x7f757c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.572+0000 7fd2a8d99640 1 -- 172.21.15.64:0/3659508756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28c005250 con 0x7fd2b40d3220 2023-11-24T08:04:03.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.574+0000 7f44c24da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc0ae750 0x7f44bc09ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.574+0000 7f44c24da640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44bc0b39b0 0x7f44bc14b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.574+0000 7f44c24da640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44bc1619a0 0x7f44bc099170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.574+0000 7f44c24da640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc149f80 con 0x7f44bc0ae750 2023-11-24T08:04:03.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.574+0000 7f44c24da640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc149370 con 0x7f44bc1619a0 2023-11-24T08:04:03.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.574+0000 7f44c24da640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc149650 con 0x7f44bc0b39b0 2023-11-24T08:04:03.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.574+0000 7f37de251640 1 Processor -- start 2023-11-24T08:04:03.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.575+0000 7f37de251640 1 -- start start 2023-11-24T08:04:03.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.575+0000 7f44bb7fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44bc0b39b0 0x7f44bc14b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.575+0000 7f44bb7fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44bc0b39b0 0x7f44bc14b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54774/0 (socket says 172.21.15.64:54774) 2023-11-24T08:04:03.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.575+0000 7f44bb7fe640 1 -- 172.21.15.64:0/1249782997 learned_addr learned my addr 172.21.15.64:0/1249782997 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.575+0000 7f759e578640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757c06ae40 0x7f757c06d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7594009990 tx=0x7f7594009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.575+0000 7f44c0a77640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44bc1619a0 0x7f44bc099170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f44bbfff640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc0ae750 0x7f44bc09ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f0992ffd640 1 -- 172.21.15.64:0/2546534385 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0998013470 con 0x7f099c033eb0 2023-11-24T08:04:03.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f97a67fc640 1 -- 172.21.15.64:0/358075639 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f97a0043070 con 0x7f97b015ead0 2023-11-24T08:04:03.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f37de251640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37d806d520 0x7f37d8160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f44bbfff640 1 -- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44bc0b39b0 msgr2=0x7f44bc14b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f37de251640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37d80719c0 0x7f37d81faa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f44bbfff640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44bc0b39b0 0x7f44bc14b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f44bbfff640 1 -- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44bc1619a0 msgr2=0x7f44bc099170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f44bbfff640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44bc1619a0 0x7f44bc099170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f44bbfff640 1 -- 172.21.15.64:0/1249782997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0009400 con 0x7f44bc0ae750 2023-11-24T08:04:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f37de251640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d8161930 0x7f37d80ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f37de251640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d815f650 con 0x7f37d8161930 2023-11-24T08:04:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f37de251640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81faf50 con 0x7f37d806d520 2023-11-24T08:04:03.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7f37de251640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80cb860 con 0x7f37d80719c0 2023-11-24T08:04:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.575+0000 7fd2ab7fe640 1 -- 172.21.15.64:0/3659508756 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2a40147e0 con 0x7fd2b40d3220 2023-11-24T08:04:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7fd2ab7fe640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd29406aed0 0x7fd29406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.576+0000 7fd2ab7fe640 1 -- 172.21.15.64:0/3659508756 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2a4089cf0 con 0x7fd2b40d3220 2023-11-24T08:04:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.577+0000 7f44bbfff640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc0ae750 0x7f44bc09ad30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44b0033000 tx=0x7f44b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.577+0000 7f0992ffd640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f098406ae90 0x7f098406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.577+0000 7f97a67fc640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f979406af10 0x7f979406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.577+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d8161930 0x7f37d80ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.577+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d8161930 0x7f37d80ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36978/0 (socket says 172.21.15.64:36978) 2023-11-24T08:04:03.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.577+0000 7f37d7fff640 1 -- 172.21.15.64:0/159516550 learned_addr learned my addr 172.21.15.64:0/159516550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:03.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.577+0000 7f37d6ffd640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37d80719c0 0x7f37d81faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.577+0000 7f37d77fe640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37d806d520 0x7f37d8160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f97a67fc640 1 -- 172.21.15.64:0/358075639 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97a00ab1f0 con 0x7f97b015ead0 2023-11-24T08:04:03.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f37d7fff640 1 -- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37d80719c0 msgr2=0x7f37d81faa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f37d7fff640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37d80719c0 0x7f37d81faa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f37d7fff640 1 -- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37d806d520 msgr2=0x7f37d8160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:03.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f37d7fff640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37d806d520 0x7f37d8160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:03.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f37d7fff640 1 -- 172.21.15.64:0/159516550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80094d0 con 0x7f37d8161930 2023-11-24T08:04:03.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7fd2a9d9b640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd29406aed0 0x7fd29406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f09a259c640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f098406ae90 0x7f098406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f37d7fff640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d8161930 0x7f37d80ce470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f37c0001eb0 tx=0x7f37c000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.578+0000 7f97b5e2c640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f979406af10 0x7f979406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.579+0000 7fd2a9d9b640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd29406aed0 0x7fd29406d390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd29c02b950 tx=0x7fd29c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.579+0000 7f09a259c640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f098406ae90 0x7f098406d350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f099400f4c0 tx=0x7f0994034000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.580+0000 7f97b5e2c640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f979406af10 0x7f979406d3d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0011e0 tx=0x7f97ac007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.583+0000 7f0992ffd640 1 -- 172.21.15.64:0/2546534385 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0998058bc0 con 0x7f099c033eb0 2023-11-24T08:04:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.583+0000 7f97a67fc640 1 -- 172.21.15.64:0/358075639 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97a0076430 con 0x7f97b015ead0 2023-11-24T08:04:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.592+0000 7f44b8ff9640 1 -- 172.21.15.64:0/1249782997 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0032e10 con 0x7f44bc0ae750 2023-11-24T08:04:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.592+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc149930 con 0x7f44bc0ae750 2023-11-24T08:04:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.592+0000 7f37cffff640 1 -- 172.21.15.64:0/159516550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c00052d0 con 0x7f37d8161930 2023-11-24T08:04:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.592+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d81fb1f0 con 0x7f37d8161930 2023-11-24T08:04:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.592+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d81fb7b0 con 0x7f37d8161930 2023-11-24T08:04:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.592+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc09b270 con 0x7f44bc0ae750 2023-11-24T08:04:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.593+0000 7f6f12ffd640 1 -- 172.21.15.64:0/301322731 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f1c0f0ee0 con 0x7f6f140eacd0 2023-11-24T08:04:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.593+0000 7fd2ab7fe640 1 -- 172.21.15.64:0/3659508756 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2a4054e40 con 0x7fd2b40d3220 2023-11-24T08:04:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.594+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d8074770 con 0x7f37d8161930 2023-11-24T08:04:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.594+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc074770 con 0x7f44bc0ae750 2023-11-24T08:04:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.598+0000 7f758effd640 1 -- 172.21.15.64:0/1123075868 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75880765e0 con 0x7f75900540a0 2023-11-24T08:04:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.598+0000 7f37cffff640 1 -- 172.21.15.64:0/159516550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c0005430 con 0x7f37d8161930 2023-11-24T08:04:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.598+0000 7f37cffff640 1 -- 172.21.15.64:0/159516550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c0007250 con 0x7f37d8161930 2023-11-24T08:04:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.598+0000 7f44b8ff9640 1 -- 172.21.15.64:0/1249782997 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b0003610 con 0x7f44bc0ae750 2023-11-24T08:04:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.599+0000 7f44b8ff9640 1 -- 172.21.15.64:0/1249782997 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00039e0 con 0x7f44bc0ae750 2023-11-24T08:04:03.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.599+0000 7f0992ffd640 1 -- 172.21.15.64:0/2546534385 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0998054f00 con 0x7f099c033eb0 2023-11-24T08:04:03.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.602+0000 7f44b8ff9640 1 -- 172.21.15.64:0/1249782997 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44b0003b40 con 0x7f44bc0ae750 2023-11-24T08:04:03.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.603+0000 7f44b8ff9640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f44a806ae70 0x7f44a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.603+0000 7f44b8ff9640 1 -- 172.21.15.64:0/1249782997 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44b00ab810 con 0x7f44bc0ae750 2023-11-24T08:04:03.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.604+0000 7f44bb7fe640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f44a806ae70 0x7f44a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.602+0000 7f37cffff640 1 -- 172.21.15.64:0/159516550 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37c00073b0 con 0x7f37d8161930 2023-11-24T08:04:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.604+0000 7f37cffff640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37b806ae40 0x7f37b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.604+0000 7f37cffff640 1 -- 172.21.15.64:0/159516550 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37c0088a10 con 0x7f37d8161930 2023-11-24T08:04:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.605+0000 7f37d77fe640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37b806ae40 0x7f37b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.606+0000 7f44bb7fe640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f44a806ae70 0x7f44a806d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44bc1fa8d0 tx=0x7f44ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.606+0000 7f37d77fe640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37b806ae40 0x7f37b806d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b450 tx=0x7f37d000d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.607+0000 7f44b8ff9640 1 -- 172.21.15.64:0/1249782997 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44b0076960 con 0x7f44bc0ae750 2023-11-24T08:04:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:03.615+0000 7f37cffff640 1 -- 172.21.15.64:0/159516550 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37c0053b60 con 0x7f37d8161930 2023-11-24T08:04:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.002+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4954141990 con 0x7f49540cb6d0 2023-11-24T08:04:04.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.006+0000 7f4942ffd640 1 -- 172.21.15.64:0/973067436 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4944046020 con 0x7f49540cb6d0 2023-11-24T08:04:04.011 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:04.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.010+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f97b0151a10 con 0x7f97b015ead0 2023-11-24T08:04:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.011+0000 7f97a67fc640 1 -- 172.21.15.64:0/358075639 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97a0079a50 con 0x7f97b015ead0 2023-11-24T08:04:04.013 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:04.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f493406af40 msgr2=0x7f493406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f493406af40 0x7f493406d400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f494c009670 tx=0x7f494c006bd0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 msgr2=0x7f495406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f495406d310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4944009ed0 tx=0x7f4944004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 shutdown_connections 2023-11-24T08:04:04.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f493406af40 0x7f493406d400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49540cb6d0 0x7f495406d310 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f49540b3a50 0x7f495406cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 --2- 172.21.15.64:0/973067436 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f49540b1360 0x7f49540a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.013+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 >> 172.21.15.64:0/973067436 conn(0x7f49540b0840 msgr2=0x7f495406c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.014+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f979406af10 msgr2=0x7f979406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.014+0000 7f97b8090640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f979406af10 0x7f979406d3d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0011e0 tx=0x7f97ac007d30 comp rx=0 tx=0).stop 2023-11-24T08:04:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b015ead0 msgr2=0x7f97b014db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b015ead0 0x7f97b014db60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f97a002bdd0 tx=0x7f97a00046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 shutdown_connections 2023-11-24T08:04:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f979406af10 0x7f979406d3d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97b014e0a0 0x7f97b00c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b015ead0 0x7f97b014db60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 --2- 172.21.15.64:0/358075639 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97b00d19a0 0x7f97b00c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 >> 172.21.15.64:0/358075639 conn(0x7f97b00d0e80 msgr2=0x7f97b006bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 shutdown_connections 2023-11-24T08:04:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.015+0000 7f97b8090640 1 -- 172.21.15.64:0/358075639 wait complete. 2023-11-24T08:04:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.016+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 shutdown_connections 2023-11-24T08:04:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.016+0000 7f495359e640 1 -- 172.21.15.64:0/973067436 wait complete. 2023-11-24T08:04:04.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.022+0000 7f6f2353c640 1 -- 172.21.15.64:0/301322731 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f14044060 con 0x7f6f140eacd0 2023-11-24T08:04:04.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.023+0000 7f6f12ffd640 1 -- 172.21.15.64:0/301322731 <== mon.1 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f14044060 con 0x7f6f140eacd0 2023-11-24T08:04:04.027 INFO:teuthology.orchestra.run.smithi064.stdout:55834574853 2023-11-24T08:04:04.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 -- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f0006aea0 msgr2=0x7f6f0006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f0006aea0 0x7f6f0006d360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c007fd0 tx=0x7f6f0c007c20 comp rx=0 tx=0).stop 2023-11-24T08:04:04.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 -- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f140eacd0 msgr2=0x7f6f14044dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f140eacd0 0x7f6f14044dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c050020 tx=0x7f6f1c068480 comp rx=0 tx=0).stop 2023-11-24T08:04:04.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 -- 172.21.15.64:0/301322731 shutdown_connections 2023-11-24T08:04:04.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f0006aea0 0x7f6f0006d360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f140eacd0 0x7f6f14044dc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f14006830 0x7f6f14044880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 --2- 172.21.15.64:0/301322731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f14005eb0 0x7f6f14044340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.027+0000 7f6f10ff9640 1 -- 172.21.15.64:0/301322731 >> 172.21.15.64:0/301322731 conn(0x7f6f14009f80 msgr2=0x7f6f14049750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.028+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f759003e3f0 con 0x7f75900540a0 2023-11-24T08:04:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.029+0000 7f6f10ff9640 1 -- 172.21.15.64:0/301322731 shutdown_connections 2023-11-24T08:04:04.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.029+0000 7f6f10ff9640 1 -- 172.21.15.64:0/301322731 wait complete. 2023-11-24T08:04:04.034 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:04.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.029+0000 7f758effd640 1 -- 172.21.15.64:0/1123075868 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7588079c00 con 0x7f75900540a0 2023-11-24T08:04:04.035 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:04.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.031+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f44bc0b4330 con 0x7f44bc0ae750 2023-11-24T08:04:04.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.031+0000 7f44b8ff9640 1 -- 172.21.15.64:0/1249782997 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44b0048020 con 0x7f44bc0ae750 2023-11-24T08:04:04.036 INFO:teuthology.orchestra.run.smithi064.stdout:55834574853 2023-11-24T08:04:04.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.030+0000 7fd2a8d99640 1 -- 172.21.15.64:0/3659508756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd28c005500 con 0x7fd2b40d3220 2023-11-24T08:04:04.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.031+0000 7fd2ab7fe640 1 -- 172.21.15.64:0/3659508756 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd2a4058460 con 0x7fd2b40d3220 2023-11-24T08:04:04.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757c06ae40 msgr2=0x7f757c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757c06ae40 0x7f757c06d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7594009990 tx=0x7f7594009110 comp rx=0 tx=0).stop 2023-11-24T08:04:04.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 msgr2=0x7f7590185120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f7590185120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f75880090b0 tx=0x7f75880046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 shutdown_connections 2023-11-24T08:04:04.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757c06ae40 0x7f757c06d300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75900e6620 0x7f7590050970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75900540a0 0x7f7590185120 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 --2- 172.21.15.64:0/1123075868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75900068b0 0x7f7590052790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 >> 172.21.15.64:0/1123075868 conn(0x7f7590058e90 msgr2=0x7f75900395b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:04.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7fd2a8d99640 1 -- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd29406aed0 msgr2=0x7fd29406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7fd2a8d99640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd29406aed0 0x7fd29406d390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd29c02b950 tx=0x7fd29c03c040 comp rx=0 tx=0).stop 2023-11-24T08:04:04.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7fd2a8d99640 1 -- 172.21.15.64:0/3659508756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2b40d3220 msgr2=0x7fd2b414d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.034+0000 7fd2a8d99640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2b40d3220 0x7fd2b414d6d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009e60 tx=0x7fd2a400c490 comp rx=0 tx=0).stop 2023-11-24T08:04:04.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 shutdown_connections 2023-11-24T08:04:04.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f44a806ae70 msgr2=0x7f44a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7f44c24da640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f44a806ae70 0x7f44a806d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44bc1fa8d0 tx=0x7f44ac009040 comp rx=0 tx=0).stop 2023-11-24T08:04:04.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7fd2a8d99640 1 -- 172.21.15.64:0/3659508756 shutdown_connections 2023-11-24T08:04:04.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7fd2a8d99640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd29406aed0 0x7fd29406d390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7fd2a8d99640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2b41f3070 0x7fd2b40b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7fd2a8d99640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2b415d530 0x7fd2b40b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7fd2a8d99640 1 --2- 172.21.15.64:0/3659508756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2b40d3220 0x7fd2b414d6d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7fd2a8d99640 1 -- 172.21.15.64:0/3659508756 >> 172.21.15.64:0/3659508756 conn(0x7fd2b4071f90 msgr2=0x7fd2b40b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:04.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc0ae750 msgr2=0x7f44bc09ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7f44c24da640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc0ae750 0x7f44bc09ad30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44b0033000 tx=0x7f44b00046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7f75a07dc640 1 -- 172.21.15.64:0/1123075868 wait complete. 2023-11-24T08:04:04.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.036+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 shutdown_connections 2023-11-24T08:04:04.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.036+0000 7f44c24da640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f44a806ae70 0x7f44a806d330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.036+0000 7f44c24da640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44bc1619a0 0x7f44bc099170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.036+0000 7f44c24da640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44bc0b39b0 0x7f44bc14b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.036+0000 7f44c24da640 1 --2- 172.21.15.64:0/1249782997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44bc0ae750 0x7f44bc09ad30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.036+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 >> 172.21.15.64:0/1249782997 conn(0x7f44bc0709f0 msgr2=0x7f44bc164760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:04.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7fd2a8d99640 1 -- 172.21.15.64:0/3659508756 shutdown_connections 2023-11-24T08:04:04.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.035+0000 7fd2a8d99640 1 -- 172.21.15.64:0/3659508756 wait complete. 2023-11-24T08:04:04.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.036+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 shutdown_connections 2023-11-24T08:04:04.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.036+0000 7f44c24da640 1 -- 172.21.15.64:0/1249782997 wait complete. 2023-11-24T08:04:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.040+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f099c005280 con 0x7f099c033eb0 2023-11-24T08:04:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.041+0000 7f0992ffd640 1 -- 172.21.15.64:0/2546534385 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0998017090 con 0x7f099c033eb0 2023-11-24T08:04:04.048 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.045+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f098406ae90 msgr2=0x7f098406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.045+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f098406ae90 0x7f098406d350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f099400f4c0 tx=0x7f0994034000 comp rx=0 tx=0).stop 2023-11-24T08:04:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.045+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f099c033eb0 msgr2=0x7f099c04a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.045+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f099c033eb0 0x7f099c04a3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f099800e9b0 tx=0x7f099800ee70 comp rx=0 tx=0).stop 2023-11-24T08:04:04.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-11-24T08:04:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.046+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 shutdown_connections 2023-11-24T08:04:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.046+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f098406ae90 0x7f098406d350 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.046+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f099c053c30 0x7f099c04a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.046+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f099c033eb0 0x7f099c04a3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.046+0000 7f09a8d1d640 1 --2- 172.21.15.64:0/2546534385 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f099c00c060 0x7f099c04d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.046+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 >> 172.21.15.64:0/2546534385 conn(0x7f099c009f70 msgr2=0x7f099c0487d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.046+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 shutdown_connections 2023-11-24T08:04:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.047+0000 7f09a8d1d640 1 -- 172.21.15.64:0/2546534385 wait complete. 2023-11-24T08:04:04.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-11-24T08:04:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.057+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f37d806e290 con 0x7f37d8161930 2023-11-24T08:04:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.057+0000 7f37cffff640 1 -- 172.21.15.64:0/159516550 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f37c0057180 con 0x7f37d8161930 2023-11-24T08:04:04.059 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:04.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-11-24T08:04:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37b806ae40 msgr2=0x7f37b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37b806ae40 0x7f37b806d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b450 tx=0x7f37d000d040 comp rx=0 tx=0).stop 2023-11-24T08:04:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d8161930 msgr2=0x7f37d80ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d8161930 0x7f37d80ce470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f37c0001eb0 tx=0x7f37c000c890 comp rx=0 tx=0).stop 2023-11-24T08:04:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 shutdown_connections 2023-11-24T08:04:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37b806ae40 0x7f37b806d300 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b450 tx=0x7f37d000d040 comp rx=0 tx=0).stop 2023-11-24T08:04:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37d8161930 0x7f37d80ce470 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f37d80719c0 0x7f37d81faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 --2- 172.21.15.64:0/159516550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f37d806d520 0x7f37d8160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 >> 172.21.15.64:0/159516550 conn(0x7f37d80b0d80 msgr2=0x7f37d80b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 shutdown_connections 2023-11-24T08:04:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:04.060+0000 7f37de251640 1 -- 172.21.15.64:0/159516550 wait complete. 2023-11-24T08:04:04.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-11-24T08:04:04.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-11-24T08:04:04.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-11-24T08:04:04.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-11-24T08:04:04.078 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-11-24T08:04:05.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T08:04:05.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T08:04:05.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T08:04:05.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T08:04:05.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T08:04:05.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T08:04:05.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T08:04:05.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T08:04:05.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.324+0000 7fd39d4bb640 1 Processor -- start 2023-11-24T08:04:05.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.324+0000 7fd39d4bb640 1 -- start start 2023-11-24T08:04:05.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.326+0000 7fd39d4bb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd390036170 0x7fd390039b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.326+0000 7fd39d4bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390052e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.326+0000 7fd39d4bb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39003b300 con 0x7fd39004a2e0 2023-11-24T08:04:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.326+0000 7fd39d4bb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd390053380 con 0x7fd390036170 2023-11-24T08:04:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.326+0000 7fd39d4bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39003b5a0 con 0x7fd3900529b0 2023-11-24T08:04:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.327+0000 7fd396ffd640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fd39004a2e0 0x7fd39003c360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:50182/0 (socket says 172.21.15.64:50182) 2023-11-24T08:04:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.327+0000 7fd396ffd640 1 -- 172.21.15.64:0/2833140667 learned_addr learned my addr 172.21.15.64:0/2833140667 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.328+0000 7fd3977fe640 1 --2- 172.21.15.64:0/2833140667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390052e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.328+0000 7fd3967fc640 1 --2- 172.21.15.64:0/2833140667 >> v2:172.21.15.91:3300/0 conn(0x7fd390036170 0x7fd390039b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.329+0000 7fd3967fc640 1 -- 172.21.15.64:0/2833140667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 msgr2=0x7fd390052e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.329+0000 7fd3967fc640 1 --2- 172.21.15.64:0/2833140667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390052e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.329+0000 7fd3967fc640 1 -- 172.21.15.64:0/2833140667 >> v1:172.21.15.91:6789/0 conn(0x7fd39004a2e0 legacy=0x7fd39003c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.329+0000 7fd3967fc640 1 -- 172.21.15.64:0/2833140667 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39003b880 con 0x7fd390036170 2023-11-24T08:04:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.330+0000 7f93c6cea640 1 Processor -- start 2023-11-24T08:04:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.331+0000 7fd3967fc640 1 --2- 172.21.15.64:0/2833140667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 0x7fd390039b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0097a0 tx=0x7fd38c02b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9aa792ce74bd9a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.331+0000 7f93c6cea640 1 -- start start 2023-11-24T08:04:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.331+0000 7fd395ffb640 1 -- 172.21.15.64:0/2833140667 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c030710 con 0x7fd390036170 2023-11-24T08:04:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.332+0000 7fd395ffb640 1 -- 172.21.15.64:0/2833140667 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38c02e050 con 0x7fd390036170 2023-11-24T08:04:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.332+0000 7fd395ffb640 1 -- 172.21.15.64:0/2833140667 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c030a90 con 0x7fd390036170 2023-11-24T08:04:05.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.333+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2833140667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 msgr2=0x7fd390039b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.333+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2833140667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 0x7fd390039b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0097a0 tx=0x7fd38c02b6f0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.333+0000 7f93c6cea640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f93c01449d0 0x7f93c00700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.333+0000 7f93c6cea640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93c0070610 0x7f93c006c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.333+0000 7f93c6cea640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c0070a70 con 0x7f93c006c810 2023-11-24T08:04:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.333+0000 7f93c6cea640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c0144ed0 con 0x7f93c0070610 2023-11-24T08:04:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.333+0000 7f93c6cea640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c006d040 con 0x7f93c01449d0 2023-11-24T08:04:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.334+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2833140667 shutdown_connections 2023-11-24T08:04:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.334+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2833140667 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390052e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.334+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2833140667 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 0x7fd390039b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.334+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2833140667 >> 172.21.15.64:0/2833140667 conn(0x7fd390053770 msgr2=0x7fd390053b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.334+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2833140667 shutdown_connections 2023-11-24T08:04:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.334+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2833140667 wait complete. 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.335+0000 7f93c64e9640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f93c006c810 0x7f93c0146f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:50188/0 (socket says 172.21.15.64:50188) 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.335+0000 7f93c64e9640 1 -- 172.21.15.64:0/2306597 learned_addr learned my addr 172.21.15.64:0/2306597 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.335+0000 7f93c5ce8640 1 --2- 172.21.15.64:0/2306597 >> v2:172.21.15.91:3300/0 conn(0x7f93c01449d0 0x7f93c00700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.335+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226832512 0 0) 0x7f93c0070a70 con 0x7f93c006c810 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93a8003520 con 0x7f93c006c810 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 <== mon.1 v1:172.21.15.91:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4068495537 0 0) 0x7f93a8003520 con 0x7f93c006c810 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 >> v2:172.21.15.91:3300/0 conn(0x7f93c01449d0 msgr2=0x7f93c00700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 --2- 172.21.15.64:0/2306597 >> v2:172.21.15.91:3300/0 conn(0x7f93c01449d0 0x7f93c00700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 >> v2:172.21.15.64:3300/0 conn(0x7f93c0070610 msgr2=0x7f93c006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 --2- 172.21.15.64:0/2306597 >> v2:172.21.15.64:3300/0 conn(0x7f93c0070610 0x7f93c006c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 --> v1:172.21.15.91:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c006d320 con 0x7f93c006c810 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 <== mon.1 v1:172.21.15.91:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f93b8001650 con 0x7f93c006c810 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 >> v1:172.21.15.91:6789/0 conn(0x7f93c006c810 legacy=0x7f93c0146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93a80039e0 0x7f93a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a80047c0 0x7f93a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a8005180 0x7f93a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0144ed0 con 0x7f93a80047c0 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c006d040 con 0x7f93a80039e0 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.336+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a800c1c0 con 0x7f93a8005180 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.337+0000 7f93c54e7640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a80047c0 0x7f93a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.338+0000 7f93c64e9640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a8005180 0x7f93a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.338+0000 7f93c5ce8640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93a80039e0 0x7f93a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.339+0000 7fd39d4bb640 1 Processor -- start 2023-11-24T08:04:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.339+0000 7f93c5ce8640 1 -- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a8005180 msgr2=0x7f93a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.339+0000 7f93c5ce8640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a8005180 0x7f93a800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.339+0000 7f93c5ce8640 1 -- 172.21.15.64:0/2306597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a80047c0 msgr2=0x7f93a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.339+0000 7f93c5ce8640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a80047c0 0x7f93a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.339+0000 7f93c5ce8640 1 -- 172.21.15.64:0/2306597 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c006d320 con 0x7f93a80039e0 2023-11-24T08:04:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.340+0000 7f93c64e9640 1 -- 172.21.15.64:0/2306597 reap_dead start 2023-11-24T08:04:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.340+0000 7fd39d4bb640 1 -- start start 2023-11-24T08:04:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.341+0000 7f93c5ce8640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93a80039e0 0x7f93a8003e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009a90 tx=0x7f93a80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef3e041b640b09fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.341+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b402c690 con 0x7f93a80039e0 2023-11-24T08:04:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.341+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b402c7f0 con 0x7f93a80039e0 2023-11-24T08:04:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.342+0000 7f93c4ce6640 1 -- 172.21.15.64:0/2306597 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b402cb80 con 0x7f93a80039e0 2023-11-24T08:04:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.342+0000 7f93c6cea640 1 -- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93a80039e0 msgr2=0x7f93a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.342+0000 7f93c6cea640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93a80039e0 0x7f93a8003e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009a90 tx=0x7f93a80057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.344+0000 7fd39d4bb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 0x7fd3900e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.344+0000 7fd39d4bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39004a2e0 0x7fd3900e0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.344+0000 7fd39d4bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390180d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7fd396ffd640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 0x7fd3900e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7fd3967fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39004a2e0 0x7fd3900e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7fd3967fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39004a2e0 0x7fd3900e0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37014/0 (socket says 172.21.15.64:37014) 2023-11-24T08:04:05.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7fd396ffd640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 0x7fd3900e03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34520/0 (socket says 172.21.15.64:34520) 2023-11-24T08:04:05.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7fd3967fc640 1 -- 172.21.15.64:0/2578543312 learned_addr learned my addr 172.21.15.64:0/2578543312 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7f93c6cea640 1 -- 172.21.15.64:0/2306597 shutdown_connections 2023-11-24T08:04:05.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7f93c6cea640 1 --2- 172.21.15.64:0/2306597 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93a80039e0 0x7f93a8003e20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7f93c6cea640 1 -- 172.21.15.64:0/2306597 >> 172.21.15.64:0/2306597 conn(0x7f93c00b0840 msgr2=0x7f93c00b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.345+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd390009340 con 0x7fd39004a2e0 2023-11-24T08:04:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.346+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900e0e50 con 0x7fd390036170 2023-11-24T08:04:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.346+0000 7fd3977fe640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390180d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.346+0000 7f93c6cea640 1 -- 172.21.15.64:0/2306597 shutdown_connections 2023-11-24T08:04:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.346+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900089a0 con 0x7fd3900529b0 2023-11-24T08:04:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.346+0000 7f93c6cea640 1 -- 172.21.15.64:0/2306597 wait complete. 2023-11-24T08:04:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.349+0000 7f93c6cea640 1 Processor -- start 2023-11-24T08:04:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.350+0000 7fd3977fe640 1 -- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 msgr2=0x7fd3900e03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.350+0000 7fd3977fe640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 0x7fd3900e03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.350+0000 7fd3977fe640 1 -- 172.21.15.64:0/2578543312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39004a2e0 msgr2=0x7fd3900e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.350+0000 7fd3977fe640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39004a2e0 0x7fd3900e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.350+0000 7fd3977fe640 1 -- 172.21.15.64:0/2578543312 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c009400 con 0x7fd3900529b0 2023-11-24T08:04:05.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.351+0000 7f93c6cea640 1 -- start start 2023-11-24T08:04:05.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.351+0000 7fd3977fe640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390180d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd38800e7f0 tx=0x7fd38800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.352+0000 7fd37f7fe640 1 -- 172.21.15.64:0/2578543312 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800c6d0 con 0x7fd3900529b0 2023-11-24T08:04:05.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.352+0000 7fd37f7fe640 1 -- 172.21.15.64:0/2578543312 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38800c830 con 0x7fd3900529b0 2023-11-24T08:04:05.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.352+0000 7fd37f7fe640 1 -- 172.21.15.64:0/2578543312 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800cc20 con 0x7fd3900529b0 2023-11-24T08:04:05.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.352+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd390008c80 con 0x7fd3900529b0 2023-11-24T08:04:05.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.352+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd390009760 con 0x7fd3900529b0 2023-11-24T08:04:05.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.353+0000 7f93c6cea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a80039e0 0x7f93c01f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.353+0000 7f93c6cea640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c01526a0 0x7f93c0152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.353+0000 7f93c6cea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c015e150 0x7f93c015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c54e7640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c01526a0 0x7f93c0152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c54e7640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c01526a0 0x7f93c0152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34528/0 (socket says 172.21.15.64:34528) 2023-11-24T08:04:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c64e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c015e150 0x7f93c015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c64e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c015e150 0x7f93c015e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37028/0 (socket says 172.21.15.64:37028) 2023-11-24T08:04:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c54e7640 1 -- 172.21.15.64:0/3131349960 learned_addr learned my addr 172.21.15.64:0/3131349960 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0156d90 con 0x7f93c015e150 2023-11-24T08:04:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c5ce8640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a80039e0 0x7f93c01f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0153060 con 0x7f93c01526a0 2023-11-24T08:04:05.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.354+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c015f1d0 con 0x7f93a80039e0 2023-11-24T08:04:05.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.355+0000 7f93c64e9640 1 -- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a80039e0 msgr2=0x7f93c01f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.355+0000 7f93c64e9640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a80039e0 0x7f93c01f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.355+0000 7f93c64e9640 1 -- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c01526a0 msgr2=0x7f93c0152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.355+0000 7f93c64e9640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c01526a0 0x7f93c0152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.355+0000 7f93c64e9640 1 -- 172.21.15.64:0/3131349960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c0158fb0 con 0x7f93c015e150 2023-11-24T08:04:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.356+0000 7f93c64e9640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c015e150 0x7f93c015e5d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93c00cfc20 tx=0x7f93b8007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.356+0000 7f93be7fc640 1 -- 172.21.15.64:0/3131349960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b800f6a0 con 0x7f93c015e150 2023-11-24T08:04:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.357+0000 7f93be7fc640 1 -- 172.21.15.64:0/3131349960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b800f820 con 0x7f93c015e150 2023-11-24T08:04:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.357+0000 7f93be7fc640 1 -- 172.21.15.64:0/3131349960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b800fbd0 con 0x7f93c015e150 2023-11-24T08:04:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.357+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c015eef0 con 0x7f93c015e150 2023-11-24T08:04:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.357+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd390009480 con 0x7fd3900529b0 2023-11-24T08:04:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.359+0000 7fd37f7fe640 1 -- 172.21.15.64:0/2578543312 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd388013470 con 0x7fd3900529b0 2023-11-24T08:04:05.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.360+0000 7fd37f7fe640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd38406ae90 0x7fd38406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.363+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c01569b0 con 0x7f93c015e150 2023-11-24T08:04:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.361+0000 7fd37f7fe640 1 -- 172.21.15.64:0/2578543312 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3880897e0 con 0x7fd3900529b0 2023-11-24T08:04:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.364+0000 7fd396ffd640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd38406ae90 0x7fd38406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.363+0000 7f93be7fc640 1 -- 172.21.15.64:0/3131349960 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93b800fd30 con 0x7f93c015e150 2023-11-24T08:04:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.365+0000 7fd396ffd640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd38406ae90 0x7fd38406d350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd398063e60 tx=0x7fd39808af90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.365+0000 7f93be7fc640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93ac06ae90 0x7f93ac06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.365+0000 7f93be7fc640 1 -- 172.21.15.64:0/3131349960 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93b808c070 con 0x7f93c015e150 2023-11-24T08:04:05.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.366+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c0071970 con 0x7f93c015e150 2023-11-24T08:04:05.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.366+0000 7f93c5ce8640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93ac06ae90 0x7f93ac06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.367+0000 7f93c5ce8640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93ac06ae90 0x7f93ac06d350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93b4036000 tx=0x7f93b4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.375+0000 7fd37f7fe640 1 -- 172.21.15.64:0/2578543312 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd390009480 con 0x7fd3900529b0 2023-11-24T08:04:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.376+0000 7f93be7fc640 1 -- 172.21.15.64:0/3131349960 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93b801a020 con 0x7f93c015e150 2023-11-24T08:04:05.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.400+0000 7f072338a640 1 Processor -- start 2023-11-24T08:04:05.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.400+0000 7fe4eb61c640 1 Processor -- start 2023-11-24T08:04:05.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.400+0000 7f072338a640 1 -- start start 2023-11-24T08:04:05.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.400+0000 7fe4eb61c640 1 -- start start 2023-11-24T08:04:05.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.402+0000 7fe4eb61c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe4e415d4d0 0x7fe4e40cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.402+0000 7fe4eb61c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40d1f60 0x7fe4e40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.402+0000 7f072338a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f071c15abe0 0x7f071c0ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.402+0000 7f072338a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.402+0000 7f072338a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c06c550 con 0x7f071c071fb0 2023-11-24T08:04:05.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.402+0000 7f072338a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c153f70 con 0x7f071c15abe0 2023-11-24T08:04:05.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.402+0000 7f072338a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c06c830 con 0x7f071c159a20 2023-11-24T08:04:05.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.403+0000 7fe4eb61c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e415d8b0 con 0x7fe4e40ce260 2023-11-24T08:04:05.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.403+0000 7f0721126640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f071c071fb0 0x7f071c0adc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:50198/0 (socket says 172.21.15.64:50198) 2023-11-24T08:04:05.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.403+0000 7f0721126640 1 -- 172.21.15.64:0/1171229236 learned_addr learned my addr 172.21.15.64:0/1171229236 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:05.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7f0720925640 1 --2- 172.21.15.64:0/1171229236 >> v2:172.21.15.91:3300/0 conn(0x7f071c15abe0 0x7f071c0ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7fe4e9bb9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40d1f60 0x7fe4e40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7fe4e9bb9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40d1f60 0x7fe4e40b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54814/0 (socket says 172.21.15.64:54814) 2023-11-24T08:04:05.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7fe4e9bb9640 1 -- 172.21.15.64:0/810551624 learned_addr learned my addr 172.21.15.64:0/810551624 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7f0721927640 1 --2- 172.21.15.64:0/1171229236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7fe4eb61c640 1 -- 172.21.15.64:0/810551624 --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e40b91b0 con 0x7fe4e415d4d0 2023-11-24T08:04:05.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7fe4eb61c640 1 -- 172.21.15.64:0/810551624 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40cdfb0 con 0x7fe4e40d1f60 2023-11-24T08:04:05.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.403+0000 7fe4e8bb7640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fe4e40ce260 0x7fe4e40d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:50210/0 (socket says 172.21.15.64:50210) 2023-11-24T08:04:05.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.403+0000 7fe4e93b8640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe4e415d4d0 0x7fe4e40cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7f0713fff640 1 -- 172.21.15.64:0/1171229236 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 788891668 0 0) 0x7f071c06c550 con 0x7f071c071fb0 2023-11-24T08:04:05.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.404+0000 7f0713fff640 1 -- 172.21.15.64:0/1171229236 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0708003520 con 0x7f071c071fb0 2023-11-24T08:04:05.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.405+0000 7f0720925640 1 -- 172.21.15.64:0/1171229236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 msgr2=0x7f071c159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.405+0000 7f0720925640 1 --2- 172.21.15.64:0/1171229236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.405+0000 7f0720925640 1 -- 172.21.15.64:0/1171229236 >> v1:172.21.15.91:6789/0 conn(0x7f071c071fb0 legacy=0x7f071c0adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.405+0000 7f0720925640 1 -- 172.21.15.64:0/1171229236 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c06cb10 con 0x7f071c15abe0 2023-11-24T08:04:05.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.405+0000 7fe4e9bb9640 1 -- 172.21.15.64:0/810551624 >> v2:172.21.15.91:3300/0 conn(0x7fe4e415d4d0 msgr2=0x7fe4e40cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.405+0000 7fe4e9bb9640 1 --2- 172.21.15.64:0/810551624 >> v2:172.21.15.91:3300/0 conn(0x7fe4e415d4d0 0x7fe4e40cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.405+0000 7fe4e9bb9640 1 -- 172.21.15.64:0/810551624 >> v1:172.21.15.91:6789/0 conn(0x7fe4e40ce260 legacy=0x7fe4e40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.405+0000 7fe4e9bb9640 1 -- 172.21.15.64:0/810551624 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e40b2400 con 0x7fe4e40d1f60 2023-11-24T08:04:05.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.407+0000 7f0720925640 1 --2- 172.21.15.64:0/1171229236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c15abe0 0x7f071c0ae330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f070c0097a0 tx=0x7f070c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2125061206cf9c80 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.407+0000 7fe4e9bb9640 1 --2- 172.21.15.64:0/810551624 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40d1f60 0x7fe4e40b4cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009e50 tx=0x7fe4dc02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dca21b04bd415 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.407+0000 7fe4dbfff640 1 -- 172.21.15.64:0/810551624 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc02cb20 con 0x7fe4e40d1f60 2023-11-24T08:04:05.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.407+0000 7fe4dbfff640 1 -- 172.21.15.64:0/810551624 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc02cc80 con 0x7fe4e40d1f60 2023-11-24T08:04:05.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.408+0000 7f0713fff640 1 -- 172.21.15.64:0/1171229236 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c030840 con 0x7f071c15abe0 2023-11-24T08:04:05.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.408+0000 7f0713fff640 1 -- 172.21.15.64:0/1171229236 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c0309a0 con 0x7f071c15abe0 2023-11-24T08:04:05.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.408+0000 7f0713fff640 1 -- 172.21.15.64:0/1171229236 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c030d70 con 0x7f071c15abe0 2023-11-24T08:04:05.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.408+0000 7fe4eb61c640 1 -- 172.21.15.64:0/810551624 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40d1f60 msgr2=0x7fe4e40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.408+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/810551624 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40d1f60 0x7fe4e40b4cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009e50 tx=0x7fe4dc02bfa0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.409+0000 7fe4eb61c640 1 -- 172.21.15.64:0/810551624 shutdown_connections 2023-11-24T08:04:05.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.408+0000 7f3b790c9640 1 Processor -- start 2023-11-24T08:04:05.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.409+0000 7f3b790c9640 1 -- start start 2023-11-24T08:04:05.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.409+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/810551624 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40d1f60 0x7fe4e40b4cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.409+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/810551624 >> v2:172.21.15.91:3300/0 conn(0x7fe4e415d4d0 0x7fe4e40cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.409+0000 7f072338a640 1 -- 172.21.15.64:0/1171229236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c15abe0 msgr2=0x7f071c0ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.409+0000 7f072338a640 1 --2- 172.21.15.64:0/1171229236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c15abe0 0x7f071c0ae330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f070c0097a0 tx=0x7f070c02b980 comp rx=0 tx=0).stop 2023-11-24T08:04:05.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.409+0000 7fe4eb61c640 1 -- 172.21.15.64:0/810551624 >> 172.21.15.64:0/810551624 conn(0x7fe4e4071f90 msgr2=0x7fe4e40723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.410+0000 7f072338a640 1 -- 172.21.15.64:0/1171229236 shutdown_connections 2023-11-24T08:04:05.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.410+0000 7f072338a640 1 --2- 172.21.15.64:0/1171229236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.410+0000 7f072338a640 1 --2- 172.21.15.64:0/1171229236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c15abe0 0x7f071c0ae330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.410+0000 7f072338a640 1 -- 172.21.15.64:0/1171229236 >> 172.21.15.64:0/1171229236 conn(0x7f071c0709f0 msgr2=0x7f071c0718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.410+0000 7fe4eb61c640 1 -- 172.21.15.64:0/810551624 shutdown_connections 2023-11-24T08:04:05.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.410+0000 7fe4eb61c640 1 -- 172.21.15.64:0/810551624 wait complete. 2023-11-24T08:04:05.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.411+0000 7f3b790c9640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3b740d0210 0x7f3b740d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.411+0000 7f3b790c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b740d05f0 0x7f3b74161f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.411+0000 7f3b790c9640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b740d3780 con 0x7f3b74162440 2023-11-24T08:04:05.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.411+0000 7f3b790c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b740d4150 con 0x7f3b740d05f0 2023-11-24T08:04:05.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.411+0000 7f3b790c9640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7415c270 con 0x7f3b740d0210 2023-11-24T08:04:05.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.410+0000 7f072338a640 1 -- 172.21.15.64:0/1171229236 shutdown_connections 2023-11-24T08:04:05.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.410+0000 7f072338a640 1 -- 172.21.15.64:0/1171229236 wait complete. 2023-11-24T08:04:05.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.412+0000 7f3b72d9d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3b740d0210 0x7f3b740d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.412+0000 7f3b72d9d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3b740d0210 0x7f3b740d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34550/0 (socket says 172.21.15.64:34550) 2023-11-24T08:04:05.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.412+0000 7f3b72d9d640 1 -- 172.21.15.64:0/3635703670 learned_addr learned my addr 172.21.15.64:0/3635703670 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.412+0000 7f3b7259c640 1 --2- 172.21.15.64:0/3635703670 >> v2:172.21.15.64:3300/0 conn(0x7f3b740d05f0 0x7f3b74161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.413+0000 7f3b71d9b640 1 -- 172.21.15.64:0/3635703670 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893386661 0 0) 0x7f3b740d3780 con 0x7f3b74162440 2023-11-24T08:04:05.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.413+0000 7f3b7259c640 1 -- 172.21.15.64:0/3635703670 >> v2:172.21.15.91:3300/0 conn(0x7f3b740d0210 msgr2=0x7f3b740d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.413+0000 7f3b7259c640 1 --2- 172.21.15.64:0/3635703670 >> v2:172.21.15.91:3300/0 conn(0x7f3b740d0210 0x7f3b740d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.413+0000 7f3b7259c640 1 -- 172.21.15.64:0/3635703670 >> v1:172.21.15.91:6789/0 conn(0x7f3b74162440 legacy=0x7f3b740d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.413+0000 7f3b7259c640 1 -- 172.21.15.64:0/3635703670 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b74162ca0 con 0x7f3b740d05f0 2023-11-24T08:04:05.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.414+0000 7fe4eb61c640 1 Processor -- start 2023-11-24T08:04:05.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.415+0000 7fe4eb61c640 1 -- start start 2023-11-24T08:04:05.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.414+0000 7f072338a640 1 Processor -- start 2023-11-24T08:04:05.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.414+0000 7f072338a640 1 -- start start 2023-11-24T08:04:05.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.415+0000 7f072338a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c071fb0 0x7f071c146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.415+0000 7f3b7259c640 1 --2- 172.21.15.64:0/3635703670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740d05f0 0x7f3b74161f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b740b2500 tx=0x7f3b6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a429ff4a71c9a09e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7f072338a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c096260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7f3b71d9b640 1 -- 172.21.15.64:0/3635703670 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68031830 con 0x7f3b740d05f0 2023-11-24T08:04:05.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7f072338a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c15abe0 0x7f071c145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7f3b71d9b640 1 -- 172.21.15.64:0/3635703670 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b68031990 con 0x7f3b740d05f0 2023-11-24T08:04:05.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7f3b71d9b640 1 -- 172.21.15.64:0/3635703670 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68031830 con 0x7f3b740d05f0 2023-11-24T08:04:05.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7f0721126640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c071fb0 0x7f071c146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7f0721126640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c071fb0 0x7f071c146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34560/0 (socket says 172.21.15.64:34560) 2023-11-24T08:04:05.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7fe4eb61c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40ce260 0x7fe4e40cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.416+0000 7fe4eb61c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4e40d1f60 0x7fe4e40b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.417+0000 7fe4eb61c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e415d4d0 0x7fe4e40b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.417+0000 7fe4e93b8640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40ce260 0x7fe4e40cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.417+0000 7f3b790c9640 1 -- 172.21.15.64:0/3635703670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740d05f0 msgr2=0x7f3b74161f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.417+0000 7f3b790c9640 1 --2- 172.21.15.64:0/3635703670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740d05f0 0x7f3b74161f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b740b2500 tx=0x7f3b6802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.417+0000 7f0721927640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c15abe0 0x7f071c145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f0721927640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c15abe0 0x7f071c145200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37040/0 (socket says 172.21.15.64:37040) 2023-11-24T08:04:05.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.417+0000 7fe4e93b8640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40ce260 0x7fe4e40cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54844/0 (socket says 172.21.15.64:54844) 2023-11-24T08:04:05.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.417+0000 7fe4e8bb7640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4e40d1f60 0x7fe4e40b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7fe4e8bb7640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4e40d1f60 0x7fe4e40b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34568/0 (socket says 172.21.15.64:34568) 2023-11-24T08:04:05.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f0720925640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c096260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f0720925640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c096260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54840/0 (socket says 172.21.15.64:54840) 2023-11-24T08:04:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7fe4e9bb9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e415d4d0 0x7fe4e40b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7fe4e93b8640 1 -- 172.21.15.64:0/2593686790 learned_addr learned my addr 172.21.15.64:0/2593686790 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40cc460 con 0x7fe4e415d4d0 2023-11-24T08:04:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e4150240 con 0x7fe4e40d1f60 2023-11-24T08:04:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40cba90 con 0x7fe4e40ce260 2023-11-24T08:04:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f3b790c9640 1 -- 172.21.15.64:0/3635703670 shutdown_connections 2023-11-24T08:04:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f3b790c9640 1 --2- 172.21.15.64:0/3635703670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740d05f0 0x7f3b74161f00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f3b790c9640 1 --2- 172.21.15.64:0/3635703670 >> v2:172.21.15.91:3300/0 conn(0x7f3b740d0210 0x7f3b740d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f3b790c9640 1 -- 172.21.15.64:0/3635703670 >> 172.21.15.64:0/3635703670 conn(0x7f3b74071f90 msgr2=0x7f3b740723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f0721126640 1 -- 172.21.15.64:0/1023098340 learned_addr learned my addr 172.21.15.64:0/1023098340 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f3b790c9640 1 -- 172.21.15.64:0/3635703670 shutdown_connections 2023-11-24T08:04:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c144d70 con 0x7f071c15abe0 2023-11-24T08:04:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c146100 con 0x7f071c071fb0 2023-11-24T08:04:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c144440 con 0x7f071c159a20 2023-11-24T08:04:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7fe4e8bb7640 1 -- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40ce260 msgr2=0x7fe4e40cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7fe4e8bb7640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40ce260 0x7fe4e40cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7fe4e8bb7640 1 -- 172.21.15.64:0/2593686790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e415d4d0 msgr2=0x7fe4e40b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7fe4e8bb7640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e415d4d0 0x7fe4e40b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7fe4e8bb7640 1 -- 172.21.15.64:0/2593686790 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc009b30 con 0x7fe4e40d1f60 2023-11-24T08:04:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7f0721927640 1 -- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 msgr2=0x7f071c096260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7f0721927640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c096260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7f0721927640 1 -- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c071fb0 msgr2=0x7f071c146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.419+0000 7f0721927640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c071fb0 0x7f071c146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.420+0000 7f0721927640 1 -- 172.21.15.64:0/1023098340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c009400 con 0x7f071c15abe0 2023-11-24T08:04:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.420+0000 7f0721126640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c071fb0 0x7f071c146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.418+0000 7f3b790c9640 1 -- 172.21.15.64:0/3635703670 wait complete. 2023-11-24T08:04:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.420+0000 7fe4e8bb7640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4e40d1f60 0x7fe4e40b3dc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000c430 tx=0x7fe4e000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.420+0000 7f0721927640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c15abe0 0x7f071c145200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f071400efa0 tx=0x7f071400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7fe4d9ffb640 1 -- 172.21.15.64:0/2593686790 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00074e0 con 0x7fe4e40d1f60 2023-11-24T08:04:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7fe4d9ffb640 1 -- 172.21.15.64:0/2593686790 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0007640 con 0x7fe4e40d1f60 2023-11-24T08:04:05.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7fe4d9ffb640 1 -- 172.21.15.64:0/2593686790 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0004a00 con 0x7fe4e40d1f60 2023-11-24T08:04:05.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7f0711ffb640 1 -- 172.21.15.64:0/1023098340 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714014480 con 0x7f071c15abe0 2023-11-24T08:04:05.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7f0711ffb640 1 -- 172.21.15.64:0/1023098340 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071400f040 con 0x7f071c15abe0 2023-11-24T08:04:05.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7f0711ffb640 1 -- 172.21.15.64:0/1023098340 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714014820 con 0x7f071c15abe0 2023-11-24T08:04:05.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c144720 con 0x7f071c15abe0 2023-11-24T08:04:05.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.422+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c146350 con 0x7f071c15abe0 2023-11-24T08:04:05.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.423+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc005250 con 0x7f071c15abe0 2023-11-24T08:04:05.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e40caec0 con 0x7fe4e40d1f60 2023-11-24T08:04:05.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.421+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e40cb440 con 0x7fe4e40d1f60 2023-11-24T08:04:05.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.424+0000 7f3b790c9640 1 Processor -- start 2023-11-24T08:04:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.424+0000 7f3b790c9640 1 -- start start 2023-11-24T08:04:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.424+0000 7f535e209640 1 Processor -- start 2023-11-24T08:04:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.425+0000 7f535e209640 1 -- start start 2023-11-24T08:04:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.425+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e40cbc20 con 0x7fe4e40d1f60 2023-11-24T08:04:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.426+0000 7f3b790c9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b740d0210 0x7f3b741ff910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.426+0000 7f3b790c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740d05f0 0x7f3b74202650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.426+0000 7f3b790c9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b74162440 0x7f3b740cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.426+0000 7f3b790c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b740b71e0 con 0x7f3b740d05f0 2023-11-24T08:04:05.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.426+0000 7f3b790c9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b741ffe50 con 0x7f3b74162440 2023-11-24T08:04:05.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.426+0000 7f3b790c9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7415eca0 con 0x7f3b740d0210 2023-11-24T08:04:05.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f535e209640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5358161650 0x7f53580d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f535e209640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5358161ad0 0x7f53580d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f535e209640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53580cebb0 con 0x7f535815d570 2023-11-24T08:04:05.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f535e209640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53580cdfa0 con 0x7f5358161650 2023-11-24T08:04:05.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f535e209640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53580ce280 con 0x7f5358161ad0 2023-11-24T08:04:05.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7fe4d9ffb640 1 -- 172.21.15.64:0/2593686790 <== mon.1 v2:172.21.15.91:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4e0004b60 con 0x7fe4e40d1f60 2023-11-24T08:04:05.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f535da08640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5358161ad0 0x7f53580d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.429+0000 7f535da08640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5358161ad0 0x7f53580d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34588/0 (socket says 172.21.15.64:34588) 2023-11-24T08:04:05.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.429+0000 7f535da08640 1 -- 172.21.15.64:0/686020224 learned_addr learned my addr 172.21.15.64:0/686020224 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f3b72d9d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b740d0210 0x7f3b741ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f3b72d9d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b740d0210 0x7f3b741ff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54854/0 (socket says 172.21.15.64:54854) 2023-11-24T08:04:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f3b72d9d640 1 -- 172.21.15.64:0/998159238 learned_addr learned my addr 172.21.15.64:0/998159238 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.428+0000 7f3b7359e640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b74162440 0x7f3b740cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f3b72d9d640 1 -- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b74162440 msgr2=0x7f3b740cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f535da08640 1 -- 172.21.15.64:0/686020224 >> v2:172.21.15.64:3300/0 conn(0x7f5358161650 msgr2=0x7f53580d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7fe4d9ffb640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe4c806af10 0x7fe4c806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f535da08640 1 --2- 172.21.15.64:0/686020224 >> v2:172.21.15.64:3300/0 conn(0x7f5358161650 0x7f53580d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f535da08640 1 -- 172.21.15.64:0/686020224 >> v1:172.21.15.64:6789/0 conn(0x7f535815d570 legacy=0x7f5358160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f535da08640 1 -- 172.21.15.64:0/686020224 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53580ce560 con 0x7f5358161ad0 2023-11-24T08:04:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7fe4d9ffb640 1 -- 172.21.15.64:0/2593686790 <== mon.1 v2:172.21.15.91:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4e0088e80 con 0x7fe4e40d1f60 2023-11-24T08:04:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f3b72d9d640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b74162440 0x7f3b740cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f3b72d9d640 1 -- 172.21.15.64:0/998159238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740d05f0 msgr2=0x7f3b74202650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f3b72d9d640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740d05f0 0x7f3b74202650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.430+0000 7f3b72d9d640 1 -- 172.21.15.64:0/998159238 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b68009400 con 0x7f3b740d0210 2023-11-24T08:04:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.431+0000 7f3b72d9d640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b740d0210 0x7f3b741ff910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c007dd0 tx=0x7f3b6c00d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.431+0000 7fe4e93b8640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe4c806af10 0x7fe4c806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.431+0000 7f0711ffb640 1 -- 172.21.15.64:0/1023098340 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0714014980 con 0x7f071c15abe0 2023-11-24T08:04:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.431+0000 7f3b5b7fe640 1 -- 172.21.15.64:0/998159238 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c009e40 con 0x7f3b740d0210 2023-11-24T08:04:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.431+0000 7f3b5b7fe640 1 -- 172.21.15.64:0/998159238 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c00f040 con 0x7f3b740d0210 2023-11-24T08:04:05.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.433+0000 7fe4e93b8640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe4c806af10 0x7fe4c806d3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009ae0 tx=0x7fe4d4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.431+0000 7f3b5b7fe640 1 -- 172.21.15.64:0/998159238 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c015640 con 0x7f3b740d0210 2023-11-24T08:04:05.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.432+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b740b6840 con 0x7f3b740d0210 2023-11-24T08:04:05.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.432+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b740b6e00 con 0x7f3b740d0210 2023-11-24T08:04:05.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.432+0000 7f535da08640 1 --2- 172.21.15.64:0/686020224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161ad0 0x7f53580d1270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f535809c940 tx=0x7f535002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5eff3a2c10c5277 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.433+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b74074770 con 0x7f3b740d0210 2023-11-24T08:04:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.434+0000 7f0711ffb640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006af10 0x7f070006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.434+0000 7f0711ffb640 1 -- 172.21.15.64:0/1023098340 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f071408ab80 con 0x7f071c15abe0 2023-11-24T08:04:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.436+0000 7f3b5b7fe640 1 -- 172.21.15.64:0/998159238 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b6c0157a0 con 0x7f3b740d0210 2023-11-24T08:04:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.436+0000 7f0721126640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006af10 0x7f070006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.437+0000 7f3b5b7fe640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3b6006ae40 0x7f3b6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.438+0000 7f0711ffb640 1 -- 172.21.15.64:0/1023098340 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0714055b80 con 0x7f071c15abe0 2023-11-24T08:04:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.438+0000 7f3b5b7fe640 1 -- 172.21.15.64:0/998159238 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b6c089a10 con 0x7f3b740d0210 2023-11-24T08:04:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.438+0000 7f0721126640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006af10 0x7f070006d3d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0718005880 tx=0x7f0718009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.438+0000 7f3b7259c640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3b6006ae40 0x7f3b6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.439+0000 7f3b7259c640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3b6006ae40 0x7f3b6006d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b68036000 tx=0x7f3b68030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.444+0000 7f534bfff640 1 -- 172.21.15.64:0/686020224 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535002ca80 con 0x7f5358161ad0 2023-11-24T08:04:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.444+0000 7f534bfff640 1 -- 172.21.15.64:0/686020224 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535002cbe0 con 0x7f5358161ad0 2023-11-24T08:04:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.444+0000 7f534bfff640 1 -- 172.21.15.64:0/686020224 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53500343f0 con 0x7f5358161ad0 2023-11-24T08:04:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.445+0000 7f535e209640 1 -- 172.21.15.64:0/686020224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161ad0 msgr2=0x7f53580d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.445+0000 7f535e209640 1 --2- 172.21.15.64:0/686020224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161ad0 0x7f53580d1270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f535809c940 tx=0x7f535002bee0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.446+0000 7f535e209640 1 -- 172.21.15.64:0/686020224 shutdown_connections 2023-11-24T08:04:05.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.446+0000 7fe4d9ffb640 1 -- 172.21.15.64:0/2593686790 <== mon.1 v2:172.21.15.91:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4e0053fc0 con 0x7fe4e40d1f60 2023-11-24T08:04:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.446+0000 7f535e209640 1 --2- 172.21.15.64:0/686020224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161ad0 0x7f53580d1270 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.446+0000 7f535e209640 1 --2- 172.21.15.64:0/686020224 >> v2:172.21.15.64:3300/0 conn(0x7f5358161650 0x7f53580d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.446+0000 7f535e209640 1 -- 172.21.15.64:0/686020224 >> 172.21.15.64:0/686020224 conn(0x7f535809c430 msgr2=0x7f53580d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.446+0000 7f535e209640 1 -- 172.21.15.64:0/686020224 shutdown_connections 2023-11-24T08:04:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.446+0000 7f535e209640 1 -- 172.21.15.64:0/686020224 wait complete. 2023-11-24T08:04:05.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.449+0000 7f3b5b7fe640 1 -- 172.21.15.64:0/998159238 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b6c054b70 con 0x7f3b740d0210 2023-11-24T08:04:05.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.449+0000 7f535e209640 1 Processor -- start 2023-11-24T08:04:05.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.450+0000 7f535e209640 1 -- start start 2023-11-24T08:04:05.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.452+0000 7f535e209640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f535815d570 0x7f535815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.452+0000 7f535e209640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161650 0x7f53580c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.452+0000 7f535e209640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5358161ad0 0x7f53580c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.453+0000 7f535ca06640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161650 0x7f53580c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.453+0000 7f535da08640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5358161ad0 0x7f53580c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.453+0000 7f535da08640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5358161ad0 0x7f53580c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37076/0 (socket says 172.21.15.64:37076) 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.453+0000 7f535da08640 1 -- 172.21.15.64:0/163574530 learned_addr learned my addr 172.21.15.64:0/163574530 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.453+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53580ca3c0 con 0x7f5358161ad0 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.453+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53581fa780 con 0x7f5358161650 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.453+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53581faa60 con 0x7f535815d570 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.455+0000 7f535da08640 1 -- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f535815d570 msgr2=0x7f535815a890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.455+0000 7f535da08640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f535815d570 0x7f535815a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.455+0000 7f535da08640 1 -- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161650 msgr2=0x7f53580c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.455+0000 7f535da08640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161650 0x7f53580c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.455+0000 7f535da08640 1 -- 172.21.15.64:0/163574530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5350009b30 con 0x7f5358161ad0 2023-11-24T08:04:05.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.456+0000 7f535da08640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5358161ad0 0x7f53580c87c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5350036040 tx=0x7f5350004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.457+0000 7f5349ffb640 1 -- 172.21.15.64:0/163574530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350009510 con 0x7f5358161ad0 2023-11-24T08:04:05.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.457+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53581fad40 con 0x7f5358161ad0 2023-11-24T08:04:05.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.457+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53581fb300 con 0x7f5358161ad0 2023-11-24T08:04:05.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.458+0000 7f5349ffb640 1 -- 172.21.15.64:0/163574530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535003e450 con 0x7f5358161ad0 2023-11-24T08:04:05.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.458+0000 7f5349ffb640 1 -- 172.21.15.64:0/163574530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535003e7e0 con 0x7f5358161ad0 2023-11-24T08:04:05.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.459+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5338005250 con 0x7f5358161ad0 2023-11-24T08:04:05.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.464+0000 7f5349ffb640 1 -- 172.21.15.64:0/163574530 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f535002cd50 con 0x7f5358161ad0 2023-11-24T08:04:05.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.465+0000 7f5349ffb640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f533c06ae40 0x7f533c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.465+0000 7f5349ffb640 1 -- 172.21.15.64:0/163574530 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53500ac2e0 con 0x7f5358161ad0 2023-11-24T08:04:05.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.467+0000 7f535d207640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f533c06ae40 0x7f533c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.469+0000 7f5664d89640 1 Processor -- start 2023-11-24T08:04:05.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.469+0000 7f5664d89640 1 -- start start 2023-11-24T08:04:05.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.470+0000 7f535d207640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f533c06ae40 0x7f533c06d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f53540014f0 tx=0x7f5354007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.471+0000 7f5664d89640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f566015b030 0x7f566015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.471+0000 7f5664d89640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56600b39b0 0x7f56600acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f628ec2b640 1 Processor -- start 2023-11-24T08:04:05.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f565e59c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f566015b030 0x7f566015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f628ec2b640 1 -- start start 2023-11-24T08:04:05.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f5664d89640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56600b2e30 con 0x7f5660072090 2023-11-24T08:04:05.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f5664d89640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566015e9c0 con 0x7f56600b39b0 2023-11-24T08:04:05.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f5664d89640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566015b410 con 0x7f566015b030 2023-11-24T08:04:05.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f565e59c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f566015b030 0x7f566015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54870/0 (socket says 172.21.15.64:54870) 2023-11-24T08:04:05.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f565e59c640 1 -- 172.21.15.64:0/3611621969 learned_addr learned my addr 172.21.15.64:0/3611621969 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.472+0000 7f565ed9d640 1 --2- 172.21.15.64:0/3611621969 >> v2:172.21.15.64:3300/0 conn(0x7f56600b39b0 0x7f56600acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.473+0000 7f565e59c640 1 -- 172.21.15.64:0/3611621969 >> v2:172.21.15.64:3300/0 conn(0x7f56600b39b0 msgr2=0x7f56600acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.473+0000 7f565e59c640 1 --2- 172.21.15.64:0/3611621969 >> v2:172.21.15.64:3300/0 conn(0x7f56600b39b0 0x7f56600acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.473+0000 7f565e59c640 1 -- 172.21.15.64:0/3611621969 >> v1:172.21.15.64:6789/0 conn(0x7f5660072090 legacy=0x7f56600b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.474+0000 7f628ec2b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62880d18f0 0x7f62880ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.474+0000 7f628ec2b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62880b6090 con 0x7f628815d6e0 2023-11-24T08:04:05.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.474+0000 7f628ec2b640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62880b6370 con 0x7f62880d1cd0 2023-11-24T08:04:05.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.474+0000 7f628ec2b640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62880b6650 con 0x7f62880d18f0 2023-11-24T08:04:05.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.475+0000 7f628c9c7640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62880d18f0 0x7f62880ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.475+0000 7f628c9c7640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62880d18f0 0x7f62880ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34600/0 (socket says 172.21.15.64:34600) 2023-11-24T08:04:05.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.475+0000 7f628c9c7640 1 -- 172.21.15.64:0/731314468 learned_addr learned my addr 172.21.15.64:0/731314468 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.474+0000 7f5349ffb640 1 -- 172.21.15.64:0/163574530 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53500774c0 con 0x7f5358161ad0 2023-11-24T08:04:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.474+0000 7f565e59c640 1 -- 172.21.15.64:0/3611621969 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600b2220 con 0x7f566015b030 2023-11-24T08:04:05.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.474+0000 7f565ed9d640 1 --2- 172.21.15.64:0/3611621969 >> v2:172.21.15.64:3300/0 conn(0x7f56600b39b0 0x7f56600acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:05.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.476+0000 7f628c9c7640 1 -- 172.21.15.64:0/731314468 >> v1:172.21.15.91:6789/0 conn(0x7f62880d1cd0 legacy=0x7f628815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:05.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.476+0000 7f628c9c7640 1 -- 172.21.15.64:0/731314468 >> v1:172.21.15.64:6789/0 conn(0x7f628815d6e0 legacy=0x7f62880b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.476+0000 7f628c9c7640 1 -- 172.21.15.64:0/731314468 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62880b6930 con 0x7f62880d18f0 2023-11-24T08:04:05.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.476+0000 7f565e59c640 1 --2- 172.21.15.64:0/3611621969 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f566015b030 0x7f566015e480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5658009e50 tx=0x7f565802bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40718e82bcfb782 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.476+0000 7f565d59a640 1 -- 172.21.15.64:0/3611621969 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565802cb20 con 0x7f566015b030 2023-11-24T08:04:05.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.477+0000 7f565d59a640 1 -- 172.21.15.64:0/3611621969 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565802cc80 con 0x7f566015b030 2023-11-24T08:04:05.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.477+0000 7f565d59a640 1 -- 172.21.15.64:0/3611621969 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658034610 con 0x7f566015b030 2023-11-24T08:04:05.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f5664d89640 1 -- 172.21.15.64:0/3611621969 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f566015b030 msgr2=0x7f566015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f5664d89640 1 --2- 172.21.15.64:0/3611621969 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f566015b030 0x7f566015e480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5658009e50 tx=0x7f565802bf80 comp rx=0 tx=0).stop 2023-11-24T08:04:05.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f628c9c7640 1 --2- 172.21.15.64:0/731314468 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62880d18f0 0x7f62880ce770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6280009e50 tx=0x7f628002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47825e1a61d7d176 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f5664d89640 1 -- 172.21.15.64:0/3611621969 shutdown_connections 2023-11-24T08:04:05.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f5664d89640 1 --2- 172.21.15.64:0/3611621969 >> v2:172.21.15.64:3300/0 conn(0x7f56600b39b0 0x7f56600acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f5664d89640 1 --2- 172.21.15.64:0/3611621969 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f566015b030 0x7f566015e480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f5664d89640 1 -- 172.21.15.64:0/3611621969 >> 172.21.15.64:0/3611621969 conn(0x7f56600709f0 msgr2=0x7f56600718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f62877fe640 1 -- 172.21.15.64:0/731314468 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628002ca80 con 0x7f62880d18f0 2023-11-24T08:04:05.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f62877fe640 1 -- 172.21.15.64:0/731314468 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628002cbe0 con 0x7f62880d18f0 2023-11-24T08:04:05.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.478+0000 7f62877fe640 1 -- 172.21.15.64:0/731314468 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800343f0 con 0x7f62880d18f0 2023-11-24T08:04:05.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.479+0000 7f628ec2b640 1 -- 172.21.15.64:0/731314468 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62880d18f0 msgr2=0x7f62880ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.479+0000 7f628ec2b640 1 --2- 172.21.15.64:0/731314468 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62880d18f0 0x7f62880ce770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6280009e50 tx=0x7f628002bee0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.480+0000 7f628ec2b640 1 -- 172.21.15.64:0/731314468 shutdown_connections 2023-11-24T08:04:05.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.480+0000 7f628ec2b640 1 --2- 172.21.15.64:0/731314468 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62880d18f0 0x7f62880ce770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.480+0000 7f628ec2b640 1 -- 172.21.15.64:0/731314468 >> 172.21.15.64:0/731314468 conn(0x7f628809c400 msgr2=0x7f62880d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.480+0000 7f5664d89640 1 -- 172.21.15.64:0/3611621969 shutdown_connections 2023-11-24T08:04:05.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.480+0000 7f628ec2b640 1 -- 172.21.15.64:0/731314468 shutdown_connections 2023-11-24T08:04:05.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.480+0000 7f5664d89640 1 -- 172.21.15.64:0/3611621969 wait complete. 2023-11-24T08:04:05.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.483+0000 7f5664d89640 1 Processor -- start 2023-11-24T08:04:05.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.483+0000 7f5664d89640 1 -- start start 2023-11-24T08:04:05.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.483+0000 7f628ec2b640 1 -- 172.21.15.64:0/731314468 wait complete. 2023-11-24T08:04:05.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.485+0000 7f5664d89640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5660072090 0x7f56600988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.485+0000 7f5664d89640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56600b39b0 0x7f5660144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.485+0000 7f628ec2b640 1 Processor -- start 2023-11-24T08:04:05.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f5664d89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015b030 0x7f5660145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f628ec2b640 1 -- start start 2023-11-24T08:04:05.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f565e59c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5660072090 0x7f56600988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f565ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015b030 0x7f5660145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f565e59c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5660072090 0x7f56600988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:54882/0 (socket says 172.21.15.64:54882) 2023-11-24T08:04:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f565ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015b030 0x7f5660145310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37094/0 (socket says 172.21.15.64:37094) 2023-11-24T08:04:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f565ed9d640 1 -- 172.21.15.64:0/2049624536 learned_addr learned my addr 172.21.15.64:0/2049624536 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5660143510 con 0x7f566015b030 2023-11-24T08:04:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56600b04a0 con 0x7f56600b39b0 2023-11-24T08:04:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.486+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5660142ec0 con 0x7f5660072090 2023-11-24T08:04:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.487+0000 7f565ed9d640 1 -- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5660072090 msgr2=0x7f56600988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.487+0000 7f565ed9d640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5660072090 0x7f56600988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.487+0000 7f565ed9d640 1 -- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56600b39b0 msgr2=0x7f5660144dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T08:04:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.487+0000 7f565ed9d640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56600b39b0 0x7f5660144dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.487+0000 7f565ed9d640 1 -- 172.21.15.64:0/2049624536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658009b30 con 0x7f566015b030 2023-11-24T08:04:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f565ed9d640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015b030 0x7f5660145310 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f565400e7e0 tx=0x7f565400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f628ec2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62880d18f0 0x7f62880ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f564effd640 1 -- 172.21.15.64:0/2049624536 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654004030 con 0x7f566015b030 2023-11-24T08:04:05.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f564effd640 1 -- 172.21.15.64:0/2049624536 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654004190 con 0x7f566015b030 2023-11-24T08:04:05.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f564effd640 1 -- 172.21.15.64:0/2049624536 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654004520 con 0x7f566015b030 2023-11-24T08:04:05.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f628ec2b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62880d1cd0 0x7f62880ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f628ec2b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f628815d6e0 0x7f62881f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.489+0000 7f628ec2b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881f3ef0 con 0x7f62880d18f0 2023-11-24T08:04:05.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.489+0000 7f628ec2b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62880c6db0 con 0x7f628815d6e0 2023-11-24T08:04:05.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.489+0000 7f628ec2b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62881f3650 con 0x7f62880d1cd0 2023-11-24T08:04:05.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.489+0000 7f628c9c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62880d18f0 0x7f62880ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.489+0000 7f628c9c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62880d18f0 0x7f62880ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37106/0 (socket says 172.21.15.64:37106) 2023-11-24T08:04:05.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.489+0000 7f628c9c7640 1 -- 172.21.15.64:0/3523064624 learned_addr learned my addr 172.21.15.64:0/3523064624 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:05.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.489+0000 7f6287fff640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62880d1cd0 0x7f62880ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.489+0000 7f628d1c8640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f628815d6e0 0x7f62881f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56601431a0 con 0x7f566015b030 2023-11-24T08:04:05.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.488+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56600af770 con 0x7f566015b030 2023-11-24T08:04:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.490+0000 7f628c9c7640 1 -- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62880d1cd0 msgr2=0x7f62880ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.490+0000 7f628c9c7640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62880d1cd0 0x7f62880ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.490+0000 7f628c9c7640 1 -- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f628815d6e0 msgr2=0x7f62881f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.490+0000 7f628c9c7640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f628815d6e0 0x7f62881f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.490+0000 7f628c9c7640 1 -- 172.21.15.64:0/3523064624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6280009b30 con 0x7f62880d18f0 2023-11-24T08:04:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.490+0000 7f628c9c7640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62880d18f0 0x7f62880ce3d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6280009e50 tx=0x7f6280004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.491+0000 7f62857fa640 1 -- 172.21.15.64:0/3523064624 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628003e450 con 0x7f62880d18f0 2023-11-24T08:04:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.491+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62881f3930 con 0x7f62880d18f0 2023-11-24T08:04:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.491+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62880c8540 con 0x7f62880d18f0 2023-11-24T08:04:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.493+0000 7f564effd640 1 -- 172.21.15.64:0/2049624536 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f565400c6d0 con 0x7f566015b030 2023-11-24T08:04:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.494+0000 7f564effd640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f564006ae70 0x7f564006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.492+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628804e840 con 0x7f62880d18f0 2023-11-24T08:04:05.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.493+0000 7f62857fa640 1 -- 172.21.15.64:0/3523064624 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628003e5b0 con 0x7f62880d18f0 2023-11-24T08:04:05.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.493+0000 7f62857fa640 1 -- 172.21.15.64:0/3523064624 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628003e980 con 0x7f62880d18f0 2023-11-24T08:04:05.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.494+0000 7f564effd640 1 -- 172.21.15.64:0/2049624536 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5654089540 con 0x7f566015b030 2023-11-24T08:04:05.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.495+0000 7f62857fa640 1 -- 172.21.15.64:0/3523064624 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f628002cd50 con 0x7f62880d18f0 2023-11-24T08:04:05.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.496+0000 7f62857fa640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f626806ae40 0x7f626806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:05.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.497+0000 7f62857fa640 1 -- 172.21.15.64:0/3523064624 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f628003ce30 con 0x7f62880d18f0 2023-11-24T08:04:05.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.497+0000 7f6287fff640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f626806ae40 0x7f626806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.500+0000 7f565e59c640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f564006ae70 0x7f564006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:05.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.500+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5644005250 con 0x7f566015b030 2023-11-24T08:04:05.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.501+0000 7f565e59c640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f564006ae70 0x7f564006d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5658035000 tx=0x7f5658030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.501+0000 7f6287fff640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f626806ae40 0x7f626806d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62880cf1b0 tx=0x7f6278009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:05.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.505+0000 7f62857fa640 1 -- 172.21.15.64:0/3523064624 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6280077310 con 0x7f62880d18f0 2023-11-24T08:04:05.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.513+0000 7f564effd640 1 -- 172.21.15.64:0/2049624536 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5654054550 con 0x7f566015b030 2023-11-24T08:04:05.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.781+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd390009480 con 0x7fd3900529b0 2023-11-24T08:04:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.782+0000 7fd37f7fe640 1 -- 172.21.15.64:0/2578543312 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd390009480 con 0x7fd3900529b0 2023-11-24T08:04:05.784 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:05.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd38406ae90 msgr2=0x7fd38406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd38406ae90 0x7fd38406d350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd398063e60 tx=0x7fd39808af90 comp rx=0 tx=0).stop 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 msgr2=0x7fd390180d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390180d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd38800e7f0 tx=0x7fd38800ecb0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 shutdown_connections 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd38406ae90 0x7fd38406d350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3900529b0 0x7fd390180d30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39004a2e0 0x7fd3900e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 --2- 172.21.15.64:0/2578543312 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd390036170 0x7fd3900e03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.787+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 >> 172.21.15.64:0/2578543312 conn(0x7fd390053770 msgr2=0x7fd3900e82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.788+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 shutdown_connections 2023-11-24T08:04:05.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.788+0000 7fd39d4bb640 1 -- 172.21.15.64:0/2578543312 wait complete. 2023-11-24T08:04:05.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.795+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f93c004e840 con 0x7f93c015e150 2023-11-24T08:04:05.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.796+0000 7f93be7fc640 1 -- 172.21.15.64:0/3131349960 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93b80538a0 con 0x7f93c015e150 2023-11-24T08:04:05.797 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2023-11-24T08:04:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.799+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93ac06ae90 msgr2=0x7f93ac06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.799+0000 7f93c6cea640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93ac06ae90 0x7f93ac06d350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93b4036000 tx=0x7f93b4030000 comp rx=0 tx=0).stop 2023-11-24T08:04:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.799+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c015e150 msgr2=0x7f93c015e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.799+0000 7f93c6cea640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c015e150 0x7f93c015e5d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93c00cfc20 tx=0x7f93b8007720 comp rx=0 tx=0).stop 2023-11-24T08:04:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.800+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 shutdown_connections 2023-11-24T08:04:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.800+0000 7f93c6cea640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93ac06ae90 0x7f93ac06d350 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.800+0000 7f93c6cea640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c015e150 0x7f93c015e5d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.800+0000 7f93c6cea640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c01526a0 0x7f93c0152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.800+0000 7f93c6cea640 1 --2- 172.21.15.64:0/3131349960 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93a80039e0 0x7f93c01f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.800+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 >> 172.21.15.64:0/3131349960 conn(0x7f93c00b0840 msgr2=0x7f93c015bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.801+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 shutdown_connections 2023-11-24T08:04:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.801+0000 7f93c6cea640 1 -- 172.21.15.64:0/3131349960 wait complete. 2023-11-24T08:04:05.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-11-24T08:04:05.818 DEBUG:teuthology.parallel:result is None 2023-11-24T08:04:05.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-11-24T08:04:05.830 DEBUG:teuthology.parallel:result is None 2023-11-24T08:04:05.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.842+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f06fc0057d0 con 0x7f071c15abe0 2023-11-24T08:04:05.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.842+0000 7f0711ffb640 1 -- 172.21.15.64:0/1023098340 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0714018090 con 0x7f071c15abe0 2023-11-24T08:04:05.844 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:05.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.844+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006af10 msgr2=0x7f070006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.844+0000 7f072338a640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006af10 0x7f070006d3d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0718005880 tx=0x7f0718009040 comp rx=0 tx=0).stop 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.844+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c15abe0 msgr2=0x7f071c145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.844+0000 7f072338a640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c15abe0 0x7f071c145200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f071400efa0 tx=0x7f071400c550 comp rx=0 tx=0).stop 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.845+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 shutdown_connections 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.845+0000 7f072338a640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006af10 0x7f070006d3d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.845+0000 7f072338a640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c15abe0 0x7f071c145200 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.845+0000 7f072338a640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f071c159a20 0x7f071c096260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.845+0000 7f072338a640 1 --2- 172.21.15.64:0/1023098340 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f071c071fb0 0x7f071c146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.845+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 >> 172.21.15.64:0/1023098340 conn(0x7f071c0709f0 msgr2=0x7f071c158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.845+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 shutdown_connections 2023-11-24T08:04:05.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.845+0000 7f072338a640 1 -- 172.21.15.64:0/1023098340 wait complete. 2023-11-24T08:04:05.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.853+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5338005500 con 0x7f5358161ad0 2023-11-24T08:04:05.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.854+0000 7f5349ffb640 1 -- 172.21.15.64:0/163574530 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f535007aae0 con 0x7f5358161ad0 2023-11-24T08:04:05.856 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:05.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.856+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f533c06ae40 msgr2=0x7f533c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.856+0000 7f535e209640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f533c06ae40 0x7f533c06d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f53540014f0 tx=0x7f5354007990 comp rx=0 tx=0).stop 2023-11-24T08:04:05.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.856+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5358161ad0 msgr2=0x7f53580c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.856+0000 7f535e209640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5358161ad0 0x7f53580c87c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5350036040 tx=0x7f5350004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:05.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.857+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 shutdown_connections 2023-11-24T08:04:05.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.857+0000 7f535e209640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f533c06ae40 0x7f533c06d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.857+0000 7f535e209640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5358161ad0 0x7f53580c87c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.857+0000 7f535e209640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5358161650 0x7f53580c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.857+0000 7f535e209640 1 --2- 172.21.15.64:0/163574530 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f535815d570 0x7f535815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.857+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 >> 172.21.15.64:0/163574530 conn(0x7f535809c430 msgr2=0x7f535815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.857+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 shutdown_connections 2023-11-24T08:04:05.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.857+0000 7f535e209640 1 -- 172.21.15.64:0/163574530 wait complete. 2023-11-24T08:04:05.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-11-24T08:04:05.864 DEBUG:teuthology.parallel:result is None 2023-11-24T08:04:05.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-11-24T08:04:05.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.873+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3b7415f700 con 0x7f3b740d0210 2023-11-24T08:04:05.876 DEBUG:teuthology.parallel:result is None 2023-11-24T08:04:05.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.874+0000 7f3b5b7fe640 1 -- 172.21.15.64:0/998159238 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b6c01a070 con 0x7f3b740d0210 2023-11-24T08:04:05.876 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:05.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.877+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3b6006ae40 msgr2=0x7f3b6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.877+0000 7f3b790c9640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3b6006ae40 0x7f3b6006d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b68036000 tx=0x7f3b68030000 comp rx=0 tx=0).stop 2023-11-24T08:04:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.877+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b740d0210 msgr2=0x7f3b741ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.877+0000 7f3b790c9640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b740d0210 0x7f3b741ff910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c007dd0 tx=0x7f3b6c00d4a0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.878+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 shutdown_connections 2023-11-24T08:04:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.878+0000 7f3b790c9640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3b6006ae40 0x7f3b6006d300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.878+0000 7f3b790c9640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3b74162440 0x7f3b740cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.878+0000 7f3b790c9640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b740d05f0 0x7f3b74202650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.878+0000 7f3b790c9640 1 --2- 172.21.15.64:0/998159238 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3b740d0210 0x7f3b741ff910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.878+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 >> 172.21.15.64:0/998159238 conn(0x7f3b74071f90 msgr2=0x7f3b7415f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.878+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 shutdown_connections 2023-11-24T08:04:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.878+0000 7f3b790c9640 1 -- 172.21.15.64:0/998159238 wait complete. 2023-11-24T08:04:05.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.881+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe4e4074770 con 0x7fe4e40d1f60 2023-11-24T08:04:05.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.882+0000 7fe4d9ffb640 1 -- 172.21.15.64:0/2593686790 <== mon.1 v2:172.21.15.91:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4e00575e0 con 0x7fe4e40d1f60 2023-11-24T08:04:05.884 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2023-11-24T08:04:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe4c806af10 msgr2=0x7fe4c806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe4c806af10 0x7fe4c806d3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009ae0 tx=0x7fe4d4009190 comp rx=0 tx=0).stop 2023-11-24T08:04:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4e40d1f60 msgr2=0x7fe4e40b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4e40d1f60 0x7fe4e40b3dc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000c430 tx=0x7fe4e000c8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 shutdown_connections 2023-11-24T08:04:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe4c806af10 0x7fe4c806d3d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4e415d4d0 0x7fe4e40b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe4e40d1f60 0x7fe4e40b3dc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 --2- 172.21.15.64:0/2593686790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4e40ce260 0x7fe4e40cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 >> 172.21.15.64:0/2593686790 conn(0x7fe4e4071f90 msgr2=0x7fe4e415c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 shutdown_connections 2023-11-24T08:04:05.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.886+0000 7fe4eb61c640 1 -- 172.21.15.64:0/2593686790 wait complete. 2023-11-24T08:04:05.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-11-24T08:04:05.903 DEBUG:teuthology.parallel:result is None 2023-11-24T08:04:05.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-11-24T08:04:05.908 DEBUG:teuthology.parallel:result is None 2023-11-24T08:04:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.933+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f62880d2390 con 0x7f62880d18f0 2023-11-24T08:04:05.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.936+0000 7f62857fa640 1 -- 172.21.15.64:0/3523064624 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6280043090 con 0x7f62880d18f0 2023-11-24T08:04:05.938 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.937+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f56440050c0 con 0x7f566015b030 2023-11-24T08:04:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.937+0000 7f564effd640 1 -- 172.21.15.64:0/2049624536 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5654017020 con 0x7f566015b030 2023-11-24T08:04:05.940 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2023-11-24T08:04:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f626806ae40 msgr2=0x7f626806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f626806ae40 0x7f626806d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62880cf1b0 tx=0x7f6278009040 comp rx=0 tx=0).stop 2023-11-24T08:04:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62880d18f0 msgr2=0x7f62880ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62880d18f0 0x7f62880ce3d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6280009e50 tx=0x7f6280004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 shutdown_connections 2023-11-24T08:04:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f626806ae40 0x7f626806d300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f628815d6e0 0x7f62881f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62880d1cd0 0x7f62880ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 --2- 172.21.15.64:0/3523064624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62880d18f0 0x7f62880ce3d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 >> 172.21.15.64:0/3523064624 conn(0x7f628809c400 msgr2=0x7f628806c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 shutdown_connections 2023-11-24T08:04:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.939+0000 7f628ec2b640 1 -- 172.21.15.64:0/3523064624 wait complete. 2023-11-24T08:04:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f564006ae70 msgr2=0x7f564006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f564006ae70 0x7f564006d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5658035000 tx=0x7f5658030000 comp rx=0 tx=0).stop 2023-11-24T08:04:05.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015b030 msgr2=0x7f5660145310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:05.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015b030 0x7f5660145310 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f565400e7e0 tx=0x7f565400eca0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 shutdown_connections 2023-11-24T08:04:05.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f564006ae70 0x7f564006d330 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015b030 0x7f5660145310 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56600b39b0 0x7f5660144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 --2- 172.21.15.64:0/2049624536 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5660072090 0x7f56600988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:05.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 >> 172.21.15.64:0/2049624536 conn(0x7f56600709f0 msgr2=0x7f566015c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:05.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 shutdown_connections 2023-11-24T08:04:05.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:05.940+0000 7f5664d89640 1 -- 172.21.15.64:0/2049624536 wait complete. 2023-11-24T08:04:05.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-11-24T08:04:05.967 DEBUG:teuthology.parallel:result is None 2023-11-24T08:04:05.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-11-24T08:04:05.968 DEBUG:teuthology.parallel:result is None 2023-11-24T08:04:05.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T08:04:05.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:04:06.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.222+0000 7fa71d61e640 1 Processor -- start 2023-11-24T08:04:06.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.222+0000 7fa71d61e640 1 -- start start 2023-11-24T08:04:06.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.224+0000 7fa71d61e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180abff0 0x7fa718071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.224+0000 7fa71d61e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71813da30 con 0x7fa7180b39a0 2023-11-24T08:04:06.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.224+0000 7fa71d61e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7180ac560 con 0x7fa71815a100 2023-11-24T08:04:06.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.224+0000 7fa71d61e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180ac840 con 0x7fa7180abff0 2023-11-24T08:04:06.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.224+0000 7fa70e7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa7180b39a0 0x7fa718071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56800/0 (socket says 172.21.15.64:56800) 2023-11-24T08:04:06.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.224+0000 7fa70e7fc640 1 -- 172.21.15.64:0/4181531308 learned_addr learned my addr 172.21.15.64:0/4181531308 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:06.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.224+0000 7fa7177fe640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180abff0 0x7fa718071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.224+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976963149 0 0) 0x7fa71813da30 con 0x7fa7180b39a0 2023-11-24T08:04:06.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa704005730 con 0x7fa7180b39a0 2023-11-24T08:04:06.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091503692 0 0) 0x7fa704005730 con 0x7fa7180b39a0 2023-11-24T08:04:06.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180abff0 msgr2=0x7fa718071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:06.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180abff0 0x7fa718071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 >> v1:172.21.15.91:6789/0 conn(0x7fa71815a100 legacy=0x7fa7180b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718070650 con 0x7fa7180b39a0 2023-11-24T08:04:06.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fa700001670 con 0x7fa7180b39a0 2023-11-24T08:04:06.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 >> v1:172.21.15.64:6789/0 conn(0x7fa7180b39a0 legacy=0x7fa718071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa704005bb0 0x7fa704005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180abff0 0x7fa7040069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa704006f10 0x7fa704007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180ac840 con 0x7fa704005bb0 2023-11-24T08:04:06.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70400dc70 con 0x7fa704006f10 2023-11-24T08:04:06.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70400de30 con 0x7fa7180abff0 2023-11-24T08:04:06.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.225+0000 7fa716ffd640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa704005bb0 0x7fa704005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.226+0000 7fa70e7fc640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180abff0 0x7fa7040069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.226+0000 7fa7177fe640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa704006f10 0x7fa704007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.226+0000 7fa716ffd640 1 -- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180abff0 msgr2=0x7fa7040069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:06.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.226+0000 7fa716ffd640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa7180abff0 0x7fa7040069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.226+0000 7fa716ffd640 1 -- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa704006f10 msgr2=0x7fa704007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:06.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.226+0000 7fa716ffd640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa704006f10 0x7fa704007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.226+0000 7fa716ffd640 1 -- 172.21.15.64:0/4181531308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa704005730 con 0x7fa704005bb0 2023-11-24T08:04:06.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.227+0000 7fa716ffd640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa704005bb0 0x7fa704005fd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa708001e20 tx=0x7fa704007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd6a1c3cad59215 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:06.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.227+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708003b70 con 0x7fa704005bb0 2023-11-24T08:04:06.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.227+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708003cd0 con 0x7fa704005bb0 2023-11-24T08:04:06.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.227+0000 7fa7167fc640 1 -- 172.21.15.64:0/4181531308 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708005310 con 0x7fa704005bb0 2023-11-24T08:04:06.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.228+0000 7fa71d61e640 1 -- 172.21.15.64:0/4181531308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa704005bb0 msgr2=0x7fa704005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.228+0000 7fa71d61e640 1 --2- 172.21.15.64:0/4181531308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa704005bb0 0x7fa704005fd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa708001e20 tx=0x7fa704007a00 comp rx=0 tx=0).stop 2023-11-24T08:04:06.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.228+0000 7fa7177fe640 1 -- 172.21.15.64:0/4181531308 reap_dead start 2023-11-24T08:04:06.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.228+0000 7fa71d61e640 1 -- 172.21.15.64:0/4181531308 shutdown_connections 2023-11-24T08:04:06.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.228+0000 7fa71d61e640 1 -- 172.21.15.64:0/4181531308 >> 172.21.15.64:0/4181531308 conn(0x7fa7180b0d80 msgr2=0x7fa7180b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.228+0000 7fa71d61e640 1 -- 172.21.15.64:0/4181531308 shutdown_connections 2023-11-24T08:04:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.228+0000 7fa71d61e640 1 -- 172.21.15.64:0/4181531308 wait complete. 2023-11-24T08:04:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.229+0000 7fa71d61e640 1 Processor -- start 2023-11-24T08:04:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.230+0000 7fa71d61e640 1 -- start start 2023-11-24T08:04:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.230+0000 7fa71d61e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa704005bb0 0x7fa7181f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.230+0000 7fa71d61e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa71815ce00 0x7fa7180cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa71d61e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7180cbf80 0x7fa7180ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa71d61e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180c8fb0 con 0x7fa7180cbf80 2023-11-24T08:04:06.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa71d61e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180c9290 con 0x7fa71815ce00 2023-11-24T08:04:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa71d61e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180c9570 con 0x7fa704005bb0 2023-11-24T08:04:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7180cbf80 0x7fa7180ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7180cbf80 0x7fa7180ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37126/0 (socket says 172.21.15.64:37126) 2023-11-24T08:04:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 -- 172.21.15.64:0/4162467937 learned_addr learned my addr 172.21.15.64:0/4162467937 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa716ffd640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa704005bb0 0x7fa7181f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa70e7fc640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa71815ce00 0x7fa7180cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 -- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa704005bb0 msgr2=0x7fa7181f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa704005bb0 0x7fa7181f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 -- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa71815ce00 msgr2=0x7fa7180cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa71815ce00 0x7fa7180cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 -- 172.21.15.64:0/4162467937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180c9850 con 0x7fa7180cbf80 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa70e7fc640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa71815ce00 0x7fa7180cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.231+0000 7fa7177fe640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7180cbf80 0x7fa7180ca050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa71815dbb0 tx=0x7fa710010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.232+0000 7fa70ffff640 1 -- 172.21.15.64:0/4162467937 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000e8a0 con 0x7fa7180cbf80 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.232+0000 7fa70ffff640 1 -- 172.21.15.64:0/4162467937 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71000ea20 con 0x7fa7180cbf80 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.232+0000 7fa70ffff640 1 -- 172.21.15.64:0/4162467937 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000edd0 con 0x7fa7180cbf80 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.232+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7180c9b30 con 0x7fa7180cbf80 2023-11-24T08:04:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.232+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa718155ec0 con 0x7fa7180cbf80 2023-11-24T08:04:06.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.233+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7180ced90 con 0x7fa7180cbf80 2023-11-24T08:04:06.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.234+0000 7fa70ffff640 1 -- 172.21.15.64:0/4162467937 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa710020410 con 0x7fa7180cbf80 2023-11-24T08:04:06.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.235+0000 7fa70ffff640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa6f806ae90 0x7fa6f806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.235+0000 7fa70ffff640 1 -- 172.21.15.64:0/4162467937 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa710088e90 con 0x7fa7180cbf80 2023-11-24T08:04:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.235+0000 7fa716ffd640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa6f806ae90 0x7fa6f806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.236+0000 7fa716ffd640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa6f806ae90 0x7fa6f806d350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa708034000 tx=0x7fa708039040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.239+0000 7fa70ffff640 1 -- 172.21.15.64:0/4162467937 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa710053f20 con 0x7fa7180cbf80 2023-11-24T08:04:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.420+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa718070c10 con 0x7fa6f806ae90 2023-11-24T08:04:06.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.441+0000 7fa70ffff640 1 -- 172.21.15.64:0/4162467937 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264310 (secure 0 0 0) 0x7fa708003e40 con 0x7fa6f806ae90 2023-11-24T08:04:06.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:04:06.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa6f806ae90 msgr2=0x7fa6f806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa6f806ae90 0x7fa6f806d350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa708034000 tx=0x7fa708039040 comp rx=0 tx=0).stop 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7180cbf80 msgr2=0x7fa7180ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7180cbf80 0x7fa7180ca050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa71815dbb0 tx=0x7fa710010c20 comp rx=0 tx=0).stop 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 shutdown_connections 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa6f806ae90 0x7fa6f806d350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7180cbf80 0x7fa7180ca050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa71815ce00 0x7fa7180cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 --2- 172.21.15.64:0/4162467937 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa704005bb0 0x7fa7181f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.447+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 >> 172.21.15.64:0/4162467937 conn(0x7fa7180b0d80 msgr2=0x7fa71806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.448+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 shutdown_connections 2023-11-24T08:04:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.448+0000 7fa71d61e640 1 -- 172.21.15.64:0/4162467937 wait complete. 2023-11-24T08:04:06.466 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T08:04:06.014763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003719"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265413+0000","last_change":"2023-11-24T08:03:55.977134+0000","last_active":"2023-11-24T08:03:57.265413+0000","last_peered":"2023-11-24T08:03:57.265413+0000","last_clean":"2023-11-24T08:03:57.265413+0000","last_became_active":"2023-11-24T08:03:55.970402+0000","last_became_peered":"2023-11-24T08:03:55.970402+0000","last_unstale":"2023-11-24T08:03:57.265413+0000","last_undegraded":"2023-11-24T08:03:57.265413+0000","last_fullsized":"2023-11-24T08:03:57.265413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:47.677166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949561+0000","last_change":"2023-11-24T08:03:55.973687+0000","last_active":"2023-11-24T08:03:56.949561+0000","last_peered":"2023-11-24T08:03:56.949561+0000","last_clean":"2023-11-24T08:03:56.949561+0000","last_became_active":"2023-11-24T08:03:55.973369+0000","last_became_peered":"2023-11-24T08:03:55.973369+0000","last_unstale":"2023-11-24T08:03:56.949561+0000","last_undegraded":"2023-11-24T08:03:56.949561+0000","last_fullsized":"2023-11-24T08:03:56.949561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449190+0000","last_change":"2023-11-24T08:03:55.971194+0000","last_active":"2023-11-24T08:03:57.449190+0000","last_peered":"2023-11-24T08:03:57.449190+0000","last_clean":"2023-11-24T08:03:57.449190+0000","last_became_active":"2023-11-24T08:03:55.970751+0000","last_became_peered":"2023-11-24T08:03:55.970751+0000","last_unstale":"2023-11-24T08:03:57.449190+0000","last_undegraded":"2023-11-24T08:03:57.449190+0000","last_fullsized":"2023-11-24T08:03:57.449190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994604+0000","last_change":"2023-11-24T08:03:55.984822+0000","last_active":"2023-11-24T08:03:56.994604+0000","last_peered":"2023-11-24T08:03:56.994604+0000","last_clean":"2023-11-24T08:03:56.994604+0000","last_became_active":"2023-11-24T08:03:55.982969+0000","last_became_peered":"2023-11-24T08:03:55.982969+0000","last_unstale":"2023-11-24T08:03:56.994604+0000","last_undegraded":"2023-11-24T08:03:56.994604+0000","last_fullsized":"2023-11-24T08:03:56.994604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:43.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264555+0000","last_change":"2023-11-24T08:03:55.980587+0000","last_active":"2023-11-24T08:03:57.264555+0000","last_peered":"2023-11-24T08:03:57.264555+0000","last_clean":"2023-11-24T08:03:57.264555+0000","last_became_active":"2023-11-24T08:03:55.980331+0000","last_became_peered":"2023-11-24T08:03:55.980331+0000","last_unstale":"2023-11-24T08:03:57.264555+0000","last_undegraded":"2023-11-24T08:03:57.264555+0000","last_fullsized":"2023-11-24T08:03:57.264555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:16.719789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994145+0000","last_change":"2023-11-24T08:03:55.981917+0000","last_active":"2023-11-24T08:03:56.994145+0000","last_peered":"2023-11-24T08:03:56.994145+0000","last_clean":"2023-11-24T08:03:56.994145+0000","last_became_active":"2023-11-24T08:03:55.981300+0000","last_became_peered":"2023-11-24T08:03:55.981300+0000","last_unstale":"2023-11-24T08:03:56.994145+0000","last_undegraded":"2023-11-24T08:03:56.994145+0000","last_fullsized":"2023-11-24T08:03:56.994145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263992+0000","last_change":"2023-11-24T08:03:55.975783+0000","last_active":"2023-11-24T08:03:57.263992+0000","last_peered":"2023-11-24T08:03:57.263992+0000","last_clean":"2023-11-24T08:03:57.263992+0000","last_became_active":"2023-11-24T08:03:55.975452+0000","last_became_peered":"2023-11-24T08:03:55.975452+0000","last_unstale":"2023-11-24T08:03:57.263992+0000","last_undegraded":"2023-11-24T08:03:57.263992+0000","last_fullsized":"2023-11-24T08:03:57.263992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952790+0000","last_change":"2023-11-24T08:03:55.980610+0000","last_active":"2023-11-24T08:03:56.952790+0000","last_peered":"2023-11-24T08:03:56.952790+0000","last_clean":"2023-11-24T08:03:56.952790+0000","last_became_active":"2023-11-24T08:03:55.980222+0000","last_became_peered":"2023-11-24T08:03:55.980222+0000","last_unstale":"2023-11-24T08:03:56.952790+0000","last_undegraded":"2023-11-24T08:03:56.952790+0000","last_fullsized":"2023-11-24T08:03:56.952790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265494+0000","last_change":"2023-11-24T08:03:55.988607+0000","last_active":"2023-11-24T08:03:57.265494+0000","last_peered":"2023-11-24T08:03:57.265494+0000","last_clean":"2023-11-24T08:03:57.265494+0000","last_became_active":"2023-11-24T08:03:55.988074+0000","last_became_peered":"2023-11-24T08:03:55.988074+0000","last_unstale":"2023-11-24T08:03:57.265494+0000","last_undegraded":"2023-11-24T08:03:57.265494+0000","last_fullsized":"2023-11-24T08:03:57.265494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:08.202382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449121+0000","last_change":"2023-11-24T08:03:55.972034+0000","last_active":"2023-11-24T08:03:57.449121+0000","last_peered":"2023-11-24T08:03:57.449121+0000","last_clean":"2023-11-24T08:03:57.449121+0000","last_became_active":"2023-11-24T08:03:55.971627+0000","last_became_peered":"2023-11-24T08:03:55.971627+0000","last_unstale":"2023-11-24T08:03:57.449121+0000","last_undegraded":"2023-11-24T08:03:57.449121+0000","last_fullsized":"2023-11-24T08:03:57.449121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448738+0000","last_change":"2023-11-24T08:03:55.979788+0000","last_active":"2023-11-24T08:03:57.448738+0000","last_peered":"2023-11-24T08:03:57.448738+0000","last_clean":"2023-11-24T08:03:57.448738+0000","last_became_active":"2023-11-24T08:03:55.979429+0000","last_became_peered":"2023-11-24T08:03:55.979429+0000","last_unstale":"2023-11-24T08:03:57.448738+0000","last_undegraded":"2023-11-24T08:03:57.448738+0000","last_fullsized":"2023-11-24T08:03:57.448738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953475+0000","last_change":"2023-11-24T08:03:55.975330+0000","last_active":"2023-11-24T08:03:56.953475+0000","last_peered":"2023-11-24T08:03:56.953475+0000","last_clean":"2023-11-24T08:03:56.953475+0000","last_became_active":"2023-11-24T08:03:55.974721+0000","last_became_peered":"2023-11-24T08:03:55.974721+0000","last_unstale":"2023-11-24T08:03:56.953475+0000","last_undegraded":"2023-11-24T08:03:56.953475+0000","last_fullsized":"2023-11-24T08:03:56.953475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:35.456457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953384+0000","last_change":"2023-11-24T08:03:55.975039+0000","last_active":"2023-11-24T08:03:56.953384+0000","last_peered":"2023-11-24T08:03:56.953384+0000","last_clean":"2023-11-24T08:03:56.953384+0000","last_became_active":"2023-11-24T08:03:55.974171+0000","last_became_peered":"2023-11-24T08:03:55.974171+0000","last_unstale":"2023-11-24T08:03:56.953384+0000","last_undegraded":"2023-11-24T08:03:56.953384+0000","last_fullsized":"2023-11-24T08:03:56.953384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448417+0000","last_change":"2023-11-24T08:03:55.972138+0000","last_active":"2023-11-24T08:03:57.448417+0000","last_peered":"2023-11-24T08:03:57.448417+0000","last_clean":"2023-11-24T08:03:57.448417+0000","last_became_active":"2023-11-24T08:03:55.968993+0000","last_became_peered":"2023-11-24T08:03:55.968993+0000","last_unstale":"2023-11-24T08:03:57.448417+0000","last_undegraded":"2023-11-24T08:03:57.448417+0000","last_fullsized":"2023-11-24T08:03:57.448417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949358+0000","last_change":"2023-11-24T08:03:55.961573+0000","last_active":"2023-11-24T08:03:56.949358+0000","last_peered":"2023-11-24T08:03:56.949358+0000","last_clean":"2023-11-24T08:03:56.949358+0000","last_became_active":"2023-11-24T08:03:55.961135+0000","last_became_peered":"2023-11-24T08:03:55.961135+0000","last_unstale":"2023-11-24T08:03:56.949358+0000","last_undegraded":"2023-11-24T08:03:56.949358+0000","last_fullsized":"2023-11-24T08:03:56.949358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994390+0000","last_change":"2023-11-24T08:03:55.982805+0000","last_active":"2023-11-24T08:03:56.994390+0000","last_peered":"2023-11-24T08:03:56.994390+0000","last_clean":"2023-11-24T08:03:56.994390+0000","last_became_active":"2023-11-24T08:03:55.980530+0000","last_became_peered":"2023-11-24T08:03:55.980530+0000","last_unstale":"2023-11-24T08:03:56.994390+0000","last_undegraded":"2023-11-24T08:03:56.994390+0000","last_fullsized":"2023-11-24T08:03:56.994390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995521+0000","last_change":"2023-11-24T08:03:55.988078+0000","last_active":"2023-11-24T08:03:56.995521+0000","last_peered":"2023-11-24T08:03:56.995521+0000","last_clean":"2023-11-24T08:03:56.995521+0000","last_became_active":"2023-11-24T08:03:55.987802+0000","last_became_peered":"2023-11-24T08:03:55.987802+0000","last_unstale":"2023-11-24T08:03:56.995521+0000","last_undegraded":"2023-11-24T08:03:56.995521+0000","last_fullsized":"2023-11-24T08:03:56.995521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:19.891560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445715+0000","last_change":"2023-11-24T08:03:55.968360+0000","last_active":"2023-11-24T08:03:57.445715+0000","last_peered":"2023-11-24T08:03:57.445715+0000","last_clean":"2023-11-24T08:03:57.445715+0000","last_became_active":"2023-11-24T08:03:55.968089+0000","last_became_peered":"2023-11-24T08:03:55.968089+0000","last_unstale":"2023-11-24T08:03:57.445715+0000","last_undegraded":"2023-11-24T08:03:57.445715+0000","last_fullsized":"2023-11-24T08:03:57.445715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:04.042778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264933+0000","last_change":"2023-11-24T08:03:55.971957+0000","last_active":"2023-11-24T08:03:57.264933+0000","last_peered":"2023-11-24T08:03:57.264933+0000","last_clean":"2023-11-24T08:03:57.264933+0000","last_became_active":"2023-11-24T08:03:55.971638+0000","last_became_peered":"2023-11-24T08:03:55.971638+0000","last_unstale":"2023-11-24T08:03:57.264933+0000","last_undegraded":"2023-11-24T08:03:57.264933+0000","last_fullsized":"2023-11-24T08:03:57.264933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:01.176228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448754+0000","last_change":"2023-11-24T08:03:55.976704+0000","last_active":"2023-11-24T08:03:57.448754+0000","last_peered":"2023-11-24T08:03:57.448754+0000","last_clean":"2023-11-24T08:03:57.448754+0000","last_became_active":"2023-11-24T08:03:55.976410+0000","last_became_peered":"2023-11-24T08:03:55.976410+0000","last_unstale":"2023-11-24T08:03:57.448754+0000","last_undegraded":"2023-11-24T08:03:57.448754+0000","last_fullsized":"2023-11-24T08:03:57.448754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448488+0000","last_change":"2023-11-24T08:03:55.985384+0000","last_active":"2023-11-24T08:03:57.448488+0000","last_peered":"2023-11-24T08:03:57.448488+0000","last_clean":"2023-11-24T08:03:57.448488+0000","last_became_active":"2023-11-24T08:03:55.985155+0000","last_became_peered":"2023-11-24T08:03:55.985155+0000","last_unstale":"2023-11-24T08:03:57.448488+0000","last_undegraded":"2023-11-24T08:03:57.448488+0000","last_fullsized":"2023-11-24T08:03:57.448488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994409+0000","last_change":"2023-11-24T08:03:55.982133+0000","last_active":"2023-11-24T08:03:56.994409+0000","last_peered":"2023-11-24T08:03:56.994409+0000","last_clean":"2023-11-24T08:03:56.994409+0000","last_became_active":"2023-11-24T08:03:55.981629+0000","last_became_peered":"2023-11-24T08:03:55.981629+0000","last_unstale":"2023-11-24T08:03:56.994409+0000","last_undegraded":"2023-11-24T08:03:56.994409+0000","last_fullsized":"2023-11-24T08:03:56.994409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952961+0000","last_change":"2023-11-24T08:03:55.963916+0000","last_active":"2023-11-24T08:03:56.952961+0000","last_peered":"2023-11-24T08:03:56.952961+0000","last_clean":"2023-11-24T08:03:56.952961+0000","last_became_active":"2023-11-24T08:03:55.958460+0000","last_became_peered":"2023-11-24T08:03:55.958460+0000","last_unstale":"2023-11-24T08:03:56.952961+0000","last_undegraded":"2023-11-24T08:03:56.952961+0000","last_fullsized":"2023-11-24T08:03:56.952961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994284+0000","last_change":"2023-11-24T08:03:55.980845+0000","last_active":"2023-11-24T08:03:56.994284+0000","last_peered":"2023-11-24T08:03:56.994284+0000","last_clean":"2023-11-24T08:03:56.994284+0000","last_became_active":"2023-11-24T08:03:55.979729+0000","last_became_peered":"2023-11-24T08:03:55.979729+0000","last_unstale":"2023-11-24T08:03:56.994284+0000","last_undegraded":"2023-11-24T08:03:56.994284+0000","last_fullsized":"2023-11-24T08:03:56.994284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953148+0000","last_change":"2023-11-24T08:03:55.956896+0000","last_active":"2023-11-24T08:03:56.953148+0000","last_peered":"2023-11-24T08:03:56.953148+0000","last_clean":"2023-11-24T08:03:56.953148+0000","last_became_active":"2023-11-24T08:03:55.956452+0000","last_became_peered":"2023-11-24T08:03:55.956452+0000","last_unstale":"2023-11-24T08:03:56.953148+0000","last_undegraded":"2023-11-24T08:03:56.953148+0000","last_fullsized":"2023-11-24T08:03:56.953148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449134+0000","last_change":"2023-11-24T08:03:55.968031+0000","last_active":"2023-11-24T08:03:57.449134+0000","last_peered":"2023-11-24T08:03:57.449134+0000","last_clean":"2023-11-24T08:03:57.449134+0000","last_became_active":"2023-11-24T08:03:55.967733+0000","last_became_peered":"2023-11-24T08:03:55.967733+0000","last_unstale":"2023-11-24T08:03:57.449134+0000","last_undegraded":"2023-11-24T08:03:57.449134+0000","last_fullsized":"2023-11-24T08:03:57.449134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949553+0000","last_change":"2023-11-24T08:03:55.971172+0000","last_active":"2023-11-24T08:03:56.949553+0000","last_peered":"2023-11-24T08:03:56.949553+0000","last_clean":"2023-11-24T08:03:56.949553+0000","last_became_active":"2023-11-24T08:03:55.970829+0000","last_became_peered":"2023-11-24T08:03:55.970829+0000","last_unstale":"2023-11-24T08:03:56.949553+0000","last_undegraded":"2023-11-24T08:03:56.949553+0000","last_fullsized":"2023-11-24T08:03:56.949553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953446+0000","last_change":"2023-11-24T08:03:55.965485+0000","last_active":"2023-11-24T08:03:56.953446+0000","last_peered":"2023-11-24T08:03:56.953446+0000","last_clean":"2023-11-24T08:03:56.953446+0000","last_became_active":"2023-11-24T08:03:55.965220+0000","last_became_peered":"2023-11-24T08:03:55.965220+0000","last_unstale":"2023-11-24T08:03:56.953446+0000","last_undegraded":"2023-11-24T08:03:56.953446+0000","last_fullsized":"2023-11-24T08:03:56.953446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:43.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448732+0000","last_change":"2023-11-24T08:03:55.978834+0000","last_active":"2023-11-24T08:03:57.448732+0000","last_peered":"2023-11-24T08:03:57.448732+0000","last_clean":"2023-11-24T08:03:57.448732+0000","last_became_active":"2023-11-24T08:03:55.978604+0000","last_became_peered":"2023-11-24T08:03:55.978604+0000","last_unstale":"2023-11-24T08:03:57.448732+0000","last_undegraded":"2023-11-24T08:03:57.448732+0000","last_fullsized":"2023-11-24T08:03:57.448732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994520+0000","last_change":"2023-11-24T08:03:55.982758+0000","last_active":"2023-11-24T08:03:56.994520+0000","last_peered":"2023-11-24T08:03:56.994520+0000","last_clean":"2023-11-24T08:03:56.994520+0000","last_became_active":"2023-11-24T08:03:55.978909+0000","last_became_peered":"2023-11-24T08:03:55.978909+0000","last_unstale":"2023-11-24T08:03:56.994520+0000","last_undegraded":"2023-11-24T08:03:56.994520+0000","last_fullsized":"2023-11-24T08:03:56.994520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:55.841729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445961+0000","last_change":"2023-11-24T08:03:55.972047+0000","last_active":"2023-11-24T08:03:57.445961+0000","last_peered":"2023-11-24T08:03:57.445961+0000","last_clean":"2023-11-24T08:03:57.445961+0000","last_became_active":"2023-11-24T08:03:55.971770+0000","last_became_peered":"2023-11-24T08:03:55.971770+0000","last_unstale":"2023-11-24T08:03:57.445961+0000","last_undegraded":"2023-11-24T08:03:57.445961+0000","last_fullsized":"2023-11-24T08:03:57.445961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:35.456457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448439+0000","last_change":"2023-11-24T08:03:55.974157+0000","last_active":"2023-11-24T08:03:57.448439+0000","last_peered":"2023-11-24T08:03:57.448439+0000","last_clean":"2023-11-24T08:03:57.448439+0000","last_became_active":"2023-11-24T08:03:55.973864+0000","last_became_peered":"2023-11-24T08:03:55.973864+0000","last_unstale":"2023-11-24T08:03:57.448439+0000","last_undegraded":"2023-11-24T08:03:57.448439+0000","last_fullsized":"2023-11-24T08:03:57.448439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952874+0000","last_change":"2023-11-24T08:03:55.959866+0000","last_active":"2023-11-24T08:03:56.952874+0000","last_peered":"2023-11-24T08:03:56.952874+0000","last_clean":"2023-11-24T08:03:56.952874+0000","last_became_active":"2023-11-24T08:03:55.959194+0000","last_became_peered":"2023-11-24T08:03:55.959194+0000","last_unstale":"2023-11-24T08:03:56.952874+0000","last_undegraded":"2023-11-24T08:03:56.952874+0000","last_fullsized":"2023-11-24T08:03:56.952874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:13.909369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953570+0000","last_change":"2023-11-24T08:03:55.243163+0000","last_active":"2023-11-24T08:03:56.953570+0000","last_peered":"2023-11-24T08:03:56.953570+0000","last_clean":"2023-11-24T08:03:56.953570+0000","last_became_active":"2023-11-24T08:03:55.242828+0000","last_became_peered":"2023-11-24T08:03:55.242828+0000","last_unstale":"2023-11-24T08:03:56.953570+0000","last_undegraded":"2023-11-24T08:03:56.953570+0000","last_fullsized":"2023-11-24T08:03:56.953570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448964+0000","last_change":"2023-11-24T08:03:55.976360+0000","last_active":"2023-11-24T08:03:57.448964+0000","last_peered":"2023-11-24T08:03:57.448964+0000","last_clean":"2023-11-24T08:03:57.448964+0000","last_became_active":"2023-11-24T08:03:55.975680+0000","last_became_peered":"2023-11-24T08:03:55.975680+0000","last_unstale":"2023-11-24T08:03:57.448964+0000","last_undegraded":"2023-11-24T08:03:57.448964+0000","last_fullsized":"2023-11-24T08:03:57.448964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448633+0000","last_change":"2023-11-24T08:03:55.250034+0000","last_active":"2023-11-24T08:03:57.448633+0000","last_peered":"2023-11-24T08:03:57.448633+0000","last_clean":"2023-11-24T08:03:57.448633+0000","last_became_active":"2023-11-24T08:03:55.247004+0000","last_became_peered":"2023-11-24T08:03:55.247004+0000","last_unstale":"2023-11-24T08:03:57.448633+0000","last_undegraded":"2023-11-24T08:03:57.448633+0000","last_fullsized":"2023-11-24T08:03:57.448633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:58.276728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952866+0000","last_change":"2023-11-24T08:03:55.249873+0000","last_active":"2023-11-24T08:03:56.952866+0000","last_peered":"2023-11-24T08:03:56.952866+0000","last_clean":"2023-11-24T08:03:56.952866+0000","last_became_active":"2023-11-24T08:03:55.249613+0000","last_became_peered":"2023-11-24T08:03:55.249613+0000","last_unstale":"2023-11-24T08:03:56.952866+0000","last_undegraded":"2023-11-24T08:03:56.952866+0000","last_fullsized":"2023-11-24T08:03:56.952866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952860+0000","last_change":"2023-11-24T08:03:55.975878+0000","last_active":"2023-11-24T08:03:56.952860+0000","last_peered":"2023-11-24T08:03:56.952860+0000","last_clean":"2023-11-24T08:03:56.952860+0000","last_became_active":"2023-11-24T08:03:55.974080+0000","last_became_peered":"2023-11-24T08:03:55.974080+0000","last_unstale":"2023-11-24T08:03:56.952860+0000","last_undegraded":"2023-11-24T08:03:56.952860+0000","last_fullsized":"2023-11-24T08:03:56.952860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994534+0000","last_change":"2023-11-24T08:03:55.249828+0000","last_active":"2023-11-24T08:03:56.994534+0000","last_peered":"2023-11-24T08:03:56.994534+0000","last_clean":"2023-11-24T08:03:56.994534+0000","last_became_active":"2023-11-24T08:03:55.246927+0000","last_became_peered":"2023-11-24T08:03:55.246927+0000","last_unstale":"2023-11-24T08:03:56.994534+0000","last_undegraded":"2023-11-24T08:03:56.994534+0000","last_fullsized":"2023-11-24T08:03:56.994534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949045+0000","last_change":"2023-11-24T08:03:55.971036+0000","last_active":"2023-11-24T08:03:56.949045+0000","last_peered":"2023-11-24T08:03:56.949045+0000","last_clean":"2023-11-24T08:03:56.949045+0000","last_became_active":"2023-11-24T08:03:55.970127+0000","last_became_peered":"2023-11-24T08:03:55.970127+0000","last_unstale":"2023-11-24T08:03:56.949045+0000","last_undegraded":"2023-11-24T08:03:56.949045+0000","last_fullsized":"2023-11-24T08:03:56.949045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448936+0000","last_change":"2023-11-24T08:03:55.253888+0000","last_active":"2023-11-24T08:03:57.448936+0000","last_peered":"2023-11-24T08:03:57.448936+0000","last_clean":"2023-11-24T08:03:57.448936+0000","last_became_active":"2023-11-24T08:03:55.253583+0000","last_became_peered":"2023-11-24T08:03:55.253583+0000","last_unstale":"2023-11-24T08:03:57.448936+0000","last_undegraded":"2023-11-24T08:03:57.448936+0000","last_fullsized":"2023-11-24T08:03:57.448936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:01.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995133+0000","last_change":"2023-11-24T08:03:55.986660+0000","last_active":"2023-11-24T08:03:56.995133+0000","last_peered":"2023-11-24T08:03:56.995133+0000","last_clean":"2023-11-24T08:03:56.995133+0000","last_became_active":"2023-11-24T08:03:55.986351+0000","last_became_peered":"2023-11-24T08:03:55.986351+0000","last_unstale":"2023-11-24T08:03:56.995133+0000","last_undegraded":"2023-11-24T08:03:56.995133+0000","last_fullsized":"2023-11-24T08:03:56.995133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:16.719789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995284+0000","last_change":"2023-11-24T08:03:55.256523+0000","last_active":"2023-11-24T08:03:56.995284+0000","last_peered":"2023-11-24T08:03:56.995284+0000","last_clean":"2023-11-24T08:03:56.995284+0000","last_became_active":"2023-11-24T08:03:55.256208+0000","last_became_peered":"2023-11-24T08:03:55.256208+0000","last_unstale":"2023-11-24T08:03:56.995284+0000","last_undegraded":"2023-11-24T08:03:56.995284+0000","last_fullsized":"2023-11-24T08:03:56.995284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449400+0000","last_change":"2023-11-24T08:03:55.980794+0000","last_active":"2023-11-24T08:03:57.449400+0000","last_peered":"2023-11-24T08:03:57.449400+0000","last_clean":"2023-11-24T08:03:57.449400+0000","last_became_active":"2023-11-24T08:03:55.980463+0000","last_became_peered":"2023-11-24T08:03:55.980463+0000","last_unstale":"2023-11-24T08:03:57.449400+0000","last_undegraded":"2023-11-24T08:03:57.449400+0000","last_fullsized":"2023-11-24T08:03:57.449400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949486+0000","last_change":"2023-11-24T08:03:55.244567+0000","last_active":"2023-11-24T08:03:56.949486+0000","last_peered":"2023-11-24T08:03:56.949486+0000","last_clean":"2023-11-24T08:03:56.949486+0000","last_became_active":"2023-11-24T08:03:55.244286+0000","last_became_peered":"2023-11-24T08:03:55.244286+0000","last_unstale":"2023-11-24T08:03:56.949486+0000","last_undegraded":"2023-11-24T08:03:56.949486+0000","last_fullsized":"2023-11-24T08:03:56.949486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.896540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953786+0000","last_change":"2023-11-24T08:03:55.982726+0000","last_active":"2023-11-24T08:03:56.953786+0000","last_peered":"2023-11-24T08:03:56.953786+0000","last_clean":"2023-11-24T08:03:56.953786+0000","last_became_active":"2023-11-24T08:03:55.982437+0000","last_became_peered":"2023-11-24T08:03:55.982437+0000","last_unstale":"2023-11-24T08:03:56.953786+0000","last_undegraded":"2023-11-24T08:03:56.953786+0000","last_fullsized":"2023-11-24T08:03:56.953786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:59.289556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448900+0000","last_change":"2023-11-24T08:03:55.252214+0000","last_active":"2023-11-24T08:03:57.448900+0000","last_peered":"2023-11-24T08:03:57.448900+0000","last_clean":"2023-11-24T08:03:57.448900+0000","last_became_active":"2023-11-24T08:03:55.251878+0000","last_became_peered":"2023-11-24T08:03:55.251878+0000","last_unstale":"2023-11-24T08:03:57.448900+0000","last_undegraded":"2023-11-24T08:03:57.448900+0000","last_fullsized":"2023-11-24T08:03:57.448900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:59.543627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264565+0000","last_change":"2023-11-24T08:03:55.981213+0000","last_active":"2023-11-24T08:03:57.264565+0000","last_peered":"2023-11-24T08:03:57.264565+0000","last_clean":"2023-11-24T08:03:57.264565+0000","last_became_active":"2023-11-24T08:03:55.979791+0000","last_became_peered":"2023-11-24T08:03:55.979791+0000","last_unstale":"2023-11-24T08:03:57.264565+0000","last_undegraded":"2023-11-24T08:03:57.264565+0000","last_fullsized":"2023-11-24T08:03:57.264565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:03.856484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448918+0000","last_change":"2023-11-24T08:03:55.242184+0000","last_active":"2023-11-24T08:03:57.448918+0000","last_peered":"2023-11-24T08:03:57.448918+0000","last_clean":"2023-11-24T08:03:57.448918+0000","last_became_active":"2023-11-24T08:03:55.241829+0000","last_became_peered":"2023-11-24T08:03:55.241829+0000","last_unstale":"2023-11-24T08:03:57.448918+0000","last_undegraded":"2023-11-24T08:03:57.448918+0000","last_fullsized":"2023-11-24T08:03:57.448918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448374+0000","last_change":"2023-11-24T08:03:55.979303+0000","last_active":"2023-11-24T08:03:57.448374+0000","last_peered":"2023-11-24T08:03:57.448374+0000","last_clean":"2023-11-24T08:03:57.448374+0000","last_became_active":"2023-11-24T08:03:55.979044+0000","last_became_peered":"2023-11-24T08:03:55.979044+0000","last_unstale":"2023-11-24T08:03:57.448374+0000","last_undegraded":"2023-11-24T08:03:57.448374+0000","last_fullsized":"2023-11-24T08:03:57.448374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994583+0000","last_change":"2023-11-24T08:03:55.247945+0000","last_active":"2023-11-24T08:03:56.994583+0000","last_peered":"2023-11-24T08:03:56.994583+0000","last_clean":"2023-11-24T08:03:56.994583+0000","last_became_active":"2023-11-24T08:03:55.247662+0000","last_became_peered":"2023-11-24T08:03:55.247662+0000","last_unstale":"2023-11-24T08:03:56.994583+0000","last_undegraded":"2023-11-24T08:03:56.994583+0000","last_fullsized":"2023-11-24T08:03:56.994583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:47.114189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264673+0000","last_change":"2023-11-24T08:03:55.985619+0000","last_active":"2023-11-24T08:03:57.264673+0000","last_peered":"2023-11-24T08:03:57.264673+0000","last_clean":"2023-11-24T08:03:57.264673+0000","last_became_active":"2023-11-24T08:03:55.985179+0000","last_became_peered":"2023-11-24T08:03:55.985179+0000","last_unstale":"2023-11-24T08:03:57.264673+0000","last_undegraded":"2023-11-24T08:03:57.264673+0000","last_fullsized":"2023-11-24T08:03:57.264673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:19.891560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449005+0000","last_change":"2023-11-24T08:03:55.247678+0000","last_active":"2023-11-24T08:03:57.449005+0000","last_peered":"2023-11-24T08:03:57.449005+0000","last_clean":"2023-11-24T08:03:57.449005+0000","last_became_active":"2023-11-24T08:03:55.247366+0000","last_became_peered":"2023-11-24T08:03:55.247366+0000","last_unstale":"2023-11-24T08:03:57.449005+0000","last_undegraded":"2023-11-24T08:03:57.449005+0000","last_fullsized":"2023-11-24T08:03:57.449005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:21.976953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448482+0000","last_change":"2023-11-24T08:03:55.973556+0000","last_active":"2023-11-24T08:03:57.448482+0000","last_peered":"2023-11-24T08:03:57.448482+0000","last_clean":"2023-11-24T08:03:57.448482+0000","last_became_active":"2023-11-24T08:03:55.971826+0000","last_became_peered":"2023-11-24T08:03:55.971826+0000","last_unstale":"2023-11-24T08:03:57.448482+0000","last_undegraded":"2023-11-24T08:03:57.448482+0000","last_fullsized":"2023-11-24T08:03:57.448482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445426+0000","last_change":"2023-11-24T08:03:55.240904+0000","last_active":"2023-11-24T08:03:57.445426+0000","last_peered":"2023-11-24T08:03:57.445426+0000","last_clean":"2023-11-24T08:03:57.445426+0000","last_became_active":"2023-11-24T08:03:55.240317+0000","last_became_peered":"2023-11-24T08:03:55.240317+0000","last_unstale":"2023-11-24T08:03:57.445426+0000","last_undegraded":"2023-11-24T08:03:57.445426+0000","last_fullsized":"2023-11-24T08:03:57.445426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:25.132329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264211+0000","last_change":"2023-11-24T08:03:55.979396+0000","last_active":"2023-11-24T08:03:57.264211+0000","last_peered":"2023-11-24T08:03:57.264211+0000","last_clean":"2023-11-24T08:03:57.264211+0000","last_became_active":"2023-11-24T08:03:55.979048+0000","last_became_peered":"2023-11-24T08:03:55.979048+0000","last_unstale":"2023-11-24T08:03:57.264211+0000","last_undegraded":"2023-11-24T08:03:57.264211+0000","last_fullsized":"2023-11-24T08:03:57.264211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:14.268136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953570+0000","last_change":"2023-11-24T08:03:55.241439+0000","last_active":"2023-11-24T08:03:56.953570+0000","last_peered":"2023-11-24T08:03:56.953570+0000","last_clean":"2023-11-24T08:03:56.953570+0000","last_became_active":"2023-11-24T08:03:55.239231+0000","last_became_peered":"2023-11-24T08:03:55.239231+0000","last_unstale":"2023-11-24T08:03:56.953570+0000","last_undegraded":"2023-11-24T08:03:56.953570+0000","last_fullsized":"2023-11-24T08:03:56.953570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:25.132329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265081+0000","last_change":"2023-11-24T08:03:55.987779+0000","last_active":"2023-11-24T08:03:57.265081+0000","last_peered":"2023-11-24T08:03:57.265081+0000","last_clean":"2023-11-24T08:03:57.265081+0000","last_became_active":"2023-11-24T08:03:55.987443+0000","last_became_peered":"2023-11-24T08:03:55.987443+0000","last_unstale":"2023-11-24T08:03:57.265081+0000","last_undegraded":"2023-11-24T08:03:57.265081+0000","last_fullsized":"2023-11-24T08:03:57.265081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:18.291474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445818+0000","last_change":"2023-11-24T08:03:55.240840+0000","last_active":"2023-11-24T08:03:57.445818+0000","last_peered":"2023-11-24T08:03:57.445818+0000","last_clean":"2023-11-24T08:03:57.445818+0000","last_became_active":"2023-11-24T08:03:55.239508+0000","last_became_peered":"2023-11-24T08:03:55.239508+0000","last_unstale":"2023-11-24T08:03:57.445818+0000","last_undegraded":"2023-11-24T08:03:57.445818+0000","last_fullsized":"2023-11-24T08:03:57.445818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.896540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265617+0000","last_change":"2023-11-24T08:03:55.987770+0000","last_active":"2023-11-24T08:03:57.265617+0000","last_peered":"2023-11-24T08:03:57.265617+0000","last_clean":"2023-11-24T08:03:57.265617+0000","last_became_active":"2023-11-24T08:03:55.987481+0000","last_became_peered":"2023-11-24T08:03:55.987481+0000","last_unstale":"2023-11-24T08:03:57.265617+0000","last_undegraded":"2023-11-24T08:03:57.265617+0000","last_fullsized":"2023-11-24T08:03:57.265617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:36.996687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949593+0000","last_change":"2023-11-24T08:03:55.243684+0000","last_active":"2023-11-24T08:03:56.949593+0000","last_peered":"2023-11-24T08:03:56.949593+0000","last_clean":"2023-11-24T08:03:56.949593+0000","last_became_active":"2023-11-24T08:03:55.242363+0000","last_became_peered":"2023-11-24T08:03:55.242363+0000","last_unstale":"2023-11-24T08:03:56.949593+0000","last_undegraded":"2023-11-24T08:03:56.949593+0000","last_fullsized":"2023-11-24T08:03:56.949593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:34.443628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995391+0000","last_change":"2023-11-24T08:03:55.985203+0000","last_active":"2023-11-24T08:03:56.995391+0000","last_peered":"2023-11-24T08:03:56.995391+0000","last_clean":"2023-11-24T08:03:56.995391+0000","last_became_active":"2023-11-24T08:03:55.979146+0000","last_became_peered":"2023-11-24T08:03:55.979146+0000","last_unstale":"2023-11-24T08:03:56.995391+0000","last_undegraded":"2023-11-24T08:03:56.995391+0000","last_fullsized":"2023-11-24T08:03:56.995391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:47.350513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445560+0000","last_change":"2023-11-24T08:03:55.252157+0000","last_active":"2023-11-24T08:03:57.445560+0000","last_peered":"2023-11-24T08:03:57.445560+0000","last_clean":"2023-11-24T08:03:57.445560+0000","last_became_active":"2023-11-24T08:03:55.251772+0000","last_became_peered":"2023-11-24T08:03:55.251772+0000","last_unstale":"2023-11-24T08:03:57.445560+0000","last_undegraded":"2023-11-24T08:03:57.445560+0000","last_fullsized":"2023-11-24T08:03:57.445560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:01.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264581+0000","last_change":"2023-11-24T08:03:55.981873+0000","last_active":"2023-11-24T08:03:57.264581+0000","last_peered":"2023-11-24T08:03:57.264581+0000","last_clean":"2023-11-24T08:03:57.264581+0000","last_became_active":"2023-11-24T08:03:55.981562+0000","last_became_peered":"2023-11-24T08:03:55.981562+0000","last_unstale":"2023-11-24T08:03:57.264581+0000","last_undegraded":"2023-11-24T08:03:57.264581+0000","last_fullsized":"2023-11-24T08:03:57.264581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:47.350513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T08:03:55.953376+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:03:55.953376+0000","last_peered":"2023-11-24T08:03:55.953376+0000","last_clean":"2023-11-24T08:03:55.953376+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:03:55.953376+0000","last_undegraded":"2023-11-24T08:03:55.953376+0000","last_fullsized":"2023-11-24T08:03:55.953376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994403+0000","last_change":"2023-11-24T08:03:55.244940+0000","last_active":"2023-11-24T08:03:56.994403+0000","last_peered":"2023-11-24T08:03:56.994403+0000","last_clean":"2023-11-24T08:03:56.994403+0000","last_became_active":"2023-11-24T08:03:55.243981+0000","last_became_peered":"2023-11-24T08:03:55.243981+0000","last_unstale":"2023-11-24T08:03:56.994403+0000","last_undegraded":"2023-11-24T08:03:56.994403+0000","last_fullsized":"2023-11-24T08:03:56.994403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:30.740161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953049+0000","last_change":"2023-11-24T08:03:55.969277+0000","last_active":"2023-11-24T08:03:56.953049+0000","last_peered":"2023-11-24T08:03:56.953049+0000","last_clean":"2023-11-24T08:03:56.953049+0000","last_became_active":"2023-11-24T08:03:55.968980+0000","last_became_peered":"2023-11-24T08:03:55.968980+0000","last_unstale":"2023-11-24T08:03:56.953049+0000","last_undegraded":"2023-11-24T08:03:56.953049+0000","last_fullsized":"2023-11-24T08:03:56.953049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263854+0000","last_change":"2023-11-24T08:03:55.255880+0000","last_active":"2023-11-24T08:03:57.263854+0000","last_peered":"2023-11-24T08:03:57.263854+0000","last_clean":"2023-11-24T08:03:57.263854+0000","last_became_active":"2023-11-24T08:03:55.255584+0000","last_became_peered":"2023-11-24T08:03:55.255584+0000","last_unstale":"2023-11-24T08:03:57.263854+0000","last_undegraded":"2023-11-24T08:03:57.263854+0000","last_fullsized":"2023-11-24T08:03:57.263854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.458202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448244+0000","last_change":"2023-11-24T08:03:55.972919+0000","last_active":"2023-11-24T08:03:57.448244+0000","last_peered":"2023-11-24T08:03:57.448244+0000","last_clean":"2023-11-24T08:03:57.448244+0000","last_became_active":"2023-11-24T08:03:55.972754+0000","last_became_peered":"2023-11-24T08:03:55.972754+0000","last_unstale":"2023-11-24T08:03:57.448244+0000","last_undegraded":"2023-11-24T08:03:57.448244+0000","last_fullsized":"2023-11-24T08:03:57.448244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:01.646788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445758+0000","last_change":"2023-11-24T08:03:55.237610+0000","last_active":"2023-11-24T08:03:57.445758+0000","last_peered":"2023-11-24T08:03:57.445758+0000","last_clean":"2023-11-24T08:03:57.445758+0000","last_became_active":"2023-11-24T08:03:55.237273+0000","last_became_peered":"2023-11-24T08:03:55.237273+0000","last_unstale":"2023-11-24T08:03:57.445758+0000","last_undegraded":"2023-11-24T08:03:57.445758+0000","last_fullsized":"2023-11-24T08:03:57.445758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:24.389964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949322+0000","last_change":"2023-11-24T08:03:55.965491+0000","last_active":"2023-11-24T08:03:56.949322+0000","last_peered":"2023-11-24T08:03:56.949322+0000","last_clean":"2023-11-24T08:03:56.949322+0000","last_became_active":"2023-11-24T08:03:55.965148+0000","last_became_peered":"2023-11-24T08:03:55.965148+0000","last_unstale":"2023-11-24T08:03:56.949322+0000","last_undegraded":"2023-11-24T08:03:56.949322+0000","last_fullsized":"2023-11-24T08:03:56.949322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994656+0000","last_change":"2023-11-24T08:03:55.246876+0000","last_active":"2023-11-24T08:03:56.994656+0000","last_peered":"2023-11-24T08:03:56.994656+0000","last_clean":"2023-11-24T08:03:56.994656+0000","last_became_active":"2023-11-24T08:03:55.243530+0000","last_became_peered":"2023-11-24T08:03:55.243530+0000","last_unstale":"2023-11-24T08:03:56.994656+0000","last_undegraded":"2023-11-24T08:03:56.994656+0000","last_fullsized":"2023-11-24T08:03:56.994656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:27.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953302+0000","last_change":"2023-11-24T08:03:55.975512+0000","last_active":"2023-11-24T08:03:56.953302+0000","last_peered":"2023-11-24T08:03:56.953302+0000","last_clean":"2023-11-24T08:03:56.953302+0000","last_became_active":"2023-11-24T08:03:55.975099+0000","last_became_peered":"2023-11-24T08:03:55.975099+0000","last_unstale":"2023-11-24T08:03:56.953302+0000","last_undegraded":"2023-11-24T08:03:56.953302+0000","last_fullsized":"2023-11-24T08:03:56.953302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:25.402792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949417+0000","last_change":"2023-11-24T08:03:55.236722+0000","last_active":"2023-11-24T08:03:56.949417+0000","last_peered":"2023-11-24T08:03:56.949417+0000","last_clean":"2023-11-24T08:03:56.949417+0000","last_became_active":"2023-11-24T08:03:55.236383+0000","last_became_peered":"2023-11-24T08:03:55.236383+0000","last_unstale":"2023-11-24T08:03:56.949417+0000","last_undegraded":"2023-11-24T08:03:56.949417+0000","last_fullsized":"2023-11-24T08:03:56.949417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995298+0000","last_change":"2023-11-24T08:03:55.978747+0000","last_active":"2023-11-24T08:03:56.995298+0000","last_peered":"2023-11-24T08:03:56.995298+0000","last_clean":"2023-11-24T08:03:56.995298+0000","last_became_active":"2023-11-24T08:03:55.978454+0000","last_became_peered":"2023-11-24T08:03:55.978454+0000","last_unstale":"2023-11-24T08:03:56.995298+0000","last_undegraded":"2023-11-24T08:03:56.995298+0000","last_fullsized":"2023-11-24T08:03:56.995298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:03.856484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264892+0000","last_change":"2023-11-24T08:03:55.255119+0000","last_active":"2023-11-24T08:03:57.264892+0000","last_peered":"2023-11-24T08:03:57.264892+0000","last_clean":"2023-11-24T08:03:57.264892+0000","last_became_active":"2023-11-24T08:03:55.254773+0000","last_became_peered":"2023-11-24T08:03:55.254773+0000","last_unstale":"2023-11-24T08:03:57.264892+0000","last_undegraded":"2023-11-24T08:03:57.264892+0000","last_fullsized":"2023-11-24T08:03:57.264892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:27.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949334+0000","last_change":"2023-11-24T08:03:55.969522+0000","last_active":"2023-11-24T08:03:56.949334+0000","last_peered":"2023-11-24T08:03:56.949334+0000","last_clean":"2023-11-24T08:03:56.949334+0000","last_became_active":"2023-11-24T08:03:55.968807+0000","last_became_peered":"2023-11-24T08:03:55.968807+0000","last_unstale":"2023-11-24T08:03:56.949334+0000","last_undegraded":"2023-11-24T08:03:56.949334+0000","last_fullsized":"2023-11-24T08:03:56.949334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:59.052521+0000","last_change":"2023-11-24T08:03:55.250820+0000","last_active":"2023-11-24T08:03:59.052521+0000","last_peered":"2023-11-24T08:03:59.052521+0000","last_clean":"2023-11-24T08:03:59.052521+0000","last_became_active":"2023-11-24T08:03:55.250479+0000","last_became_peered":"2023-11-24T08:03:55.250479+0000","last_unstale":"2023-11-24T08:03:59.052521+0000","last_undegraded":"2023-11-24T08:03:59.052521+0000","last_fullsized":"2023-11-24T08:03:59.052521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953296+0000","last_change":"2023-11-24T08:03:55.970065+0000","last_active":"2023-11-24T08:03:56.953296+0000","last_peered":"2023-11-24T08:03:56.953296+0000","last_clean":"2023-11-24T08:03:56.953296+0000","last_became_active":"2023-11-24T08:03:55.969811+0000","last_became_peered":"2023-11-24T08:03:55.969811+0000","last_unstale":"2023-11-24T08:03:56.953296+0000","last_undegraded":"2023-11-24T08:03:56.953296+0000","last_fullsized":"2023-11-24T08:03:56.953296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449201+0000","last_change":"2023-11-24T08:03:55.256073+0000","last_active":"2023-11-24T08:03:57.449201+0000","last_peered":"2023-11-24T08:03:57.449201+0000","last_clean":"2023-11-24T08:03:57.449201+0000","last_became_active":"2023-11-24T08:03:55.255797+0000","last_became_peered":"2023-11-24T08:03:55.255797+0000","last_unstale":"2023-11-24T08:03:57.449201+0000","last_undegraded":"2023-11-24T08:03:57.449201+0000","last_fullsized":"2023-11-24T08:03:57.449201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:24.389964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449472+0000","last_change":"2023-11-24T08:03:55.971989+0000","last_active":"2023-11-24T08:03:57.449472+0000","last_peered":"2023-11-24T08:03:57.449472+0000","last_clean":"2023-11-24T08:03:57.449472+0000","last_became_active":"2023-11-24T08:03:55.971690+0000","last_became_peered":"2023-11-24T08:03:55.971690+0000","last_unstale":"2023-11-24T08:03:57.449472+0000","last_undegraded":"2023-11-24T08:03:57.449472+0000","last_fullsized":"2023-11-24T08:03:57.449472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263829+0000","last_change":"2023-11-24T08:03:55.252987+0000","last_active":"2023-11-24T08:03:57.263829+0000","last_peered":"2023-11-24T08:03:57.263829+0000","last_clean":"2023-11-24T08:03:57.263829+0000","last_became_active":"2023-11-24T08:03:55.252708+0000","last_became_peered":"2023-11-24T08:03:55.252708+0000","last_unstale":"2023-11-24T08:03:57.263829+0000","last_undegraded":"2023-11-24T08:03:57.263829+0000","last_fullsized":"2023-11-24T08:03:57.263829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:34.443628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953362+0000","last_change":"2023-11-24T08:03:55.970570+0000","last_active":"2023-11-24T08:03:56.953362+0000","last_peered":"2023-11-24T08:03:56.953362+0000","last_clean":"2023-11-24T08:03:56.953362+0000","last_became_active":"2023-11-24T08:03:55.969759+0000","last_became_peered":"2023-11-24T08:03:55.969759+0000","last_unstale":"2023-11-24T08:03:56.953362+0000","last_undegraded":"2023-11-24T08:03:56.953362+0000","last_fullsized":"2023-11-24T08:03:56.953362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:04.042778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949401+0000","last_change":"2023-11-24T08:03:55.243962+0000","last_active":"2023-11-24T08:03:56.949401+0000","last_peered":"2023-11-24T08:03:56.949401+0000","last_clean":"2023-11-24T08:03:56.949401+0000","last_became_active":"2023-11-24T08:03:55.243548+0000","last_became_peered":"2023-11-24T08:03:55.243548+0000","last_unstale":"2023-11-24T08:03:56.949401+0000","last_undegraded":"2023-11-24T08:03:56.949401+0000","last_fullsized":"2023-11-24T08:03:56.949401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:59.543627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952949+0000","last_change":"2023-11-24T08:03:55.960318+0000","last_active":"2023-11-24T08:03:56.952949+0000","last_peered":"2023-11-24T08:03:56.952949+0000","last_clean":"2023-11-24T08:03:56.952949+0000","last_became_active":"2023-11-24T08:03:55.957342+0000","last_became_peered":"2023-11-24T08:03:55.957342+0000","last_unstale":"2023-11-24T08:03:56.952949+0000","last_undegraded":"2023-11-24T08:03:56.952949+0000","last_fullsized":"2023-11-24T08:03:56.952949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949250+0000","last_change":"2023-11-24T08:03:55.234991+0000","last_active":"2023-11-24T08:03:56.949250+0000","last_peered":"2023-11-24T08:03:56.949250+0000","last_clean":"2023-11-24T08:03:56.949250+0000","last_became_active":"2023-11-24T08:03:55.234480+0000","last_became_peered":"2023-11-24T08:03:55.234480+0000","last_unstale":"2023-11-24T08:03:56.949250+0000","last_undegraded":"2023-11-24T08:03:56.949250+0000","last_fullsized":"2023-11-24T08:03:56.949250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953403+0000","last_change":"2023-11-24T08:03:55.974916+0000","last_active":"2023-11-24T08:03:56.953403+0000","last_peered":"2023-11-24T08:03:56.953403+0000","last_clean":"2023-11-24T08:03:56.953403+0000","last_became_active":"2023-11-24T08:03:55.974533+0000","last_became_peered":"2023-11-24T08:03:55.974533+0000","last_unstale":"2023-11-24T08:03:56.953403+0000","last_undegraded":"2023-11-24T08:03:56.953403+0000","last_fullsized":"2023-11-24T08:03:56.953403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:55.841729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264803+0000","last_change":"2023-11-24T08:03:55.258218+0000","last_active":"2023-11-24T08:03:57.264803+0000","last_peered":"2023-11-24T08:03:57.264803+0000","last_clean":"2023-11-24T08:03:57.264803+0000","last_became_active":"2023-11-24T08:03:55.257904+0000","last_became_peered":"2023-11-24T08:03:55.257904+0000","last_unstale":"2023-11-24T08:03:57.264803+0000","last_undegraded":"2023-11-24T08:03:57.264803+0000","last_fullsized":"2023-11-24T08:03:57.264803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:42.858025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448404+0000","last_change":"2023-11-24T08:03:55.965022+0000","last_active":"2023-11-24T08:03:57.448404+0000","last_peered":"2023-11-24T08:03:57.448404+0000","last_clean":"2023-11-24T08:03:57.448404+0000","last_became_active":"2023-11-24T08:03:55.964652+0000","last_became_peered":"2023-11-24T08:03:55.964652+0000","last_unstale":"2023-11-24T08:03:57.448404+0000","last_undegraded":"2023-11-24T08:03:57.448404+0000","last_fullsized":"2023-11-24T08:03:57.448404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994904+0000","last_change":"2023-11-24T08:03:55.980852+0000","last_active":"2023-11-24T08:03:56.994904+0000","last_peered":"2023-11-24T08:03:56.994904+0000","last_clean":"2023-11-24T08:03:56.994904+0000","last_became_active":"2023-11-24T08:03:55.980467+0000","last_became_peered":"2023-11-24T08:03:55.980467+0000","last_unstale":"2023-11-24T08:03:56.994904+0000","last_undegraded":"2023-11-24T08:03:56.994904+0000","last_fullsized":"2023-11-24T08:03:56.994904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264433+0000","last_change":"2023-11-24T08:03:55.254837+0000","last_active":"2023-11-24T08:03:57.264433+0000","last_peered":"2023-11-24T08:03:57.264433+0000","last_clean":"2023-11-24T08:03:57.264433+0000","last_became_active":"2023-11-24T08:03:55.254593+0000","last_became_peered":"2023-11-24T08:03:55.254593+0000","last_unstale":"2023-11-24T08:03:57.264433+0000","last_undegraded":"2023-11-24T08:03:57.264433+0000","last_fullsized":"2023-11-24T08:03:57.264433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:47.114189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994917+0000","last_change":"2023-11-24T08:03:55.985005+0000","last_active":"2023-11-24T08:03:56.994917+0000","last_peered":"2023-11-24T08:03:56.994917+0000","last_clean":"2023-11-24T08:03:56.994917+0000","last_became_active":"2023-11-24T08:03:55.984533+0000","last_became_peered":"2023-11-24T08:03:55.984533+0000","last_unstale":"2023-11-24T08:03:56.994917+0000","last_undegraded":"2023-11-24T08:03:56.994917+0000","last_fullsized":"2023-11-24T08:03:56.994917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:14.268136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264754+0000","last_change":"2023-11-24T08:03:55.257482+0000","last_active":"2023-11-24T08:03:57.264754+0000","last_peered":"2023-11-24T08:03:57.264754+0000","last_clean":"2023-11-24T08:03:57.264754+0000","last_became_active":"2023-11-24T08:03:55.257267+0000","last_became_peered":"2023-11-24T08:03:55.257267+0000","last_unstale":"2023-11-24T08:03:57.264754+0000","last_undegraded":"2023-11-24T08:03:57.264754+0000","last_fullsized":"2023-11-24T08:03:57.264754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449374+0000","last_change":"2023-11-24T08:03:55.981373+0000","last_active":"2023-11-24T08:03:57.449374+0000","last_peered":"2023-11-24T08:03:57.449374+0000","last_clean":"2023-11-24T08:03:57.449374+0000","last_became_active":"2023-11-24T08:03:55.980504+0000","last_became_peered":"2023-11-24T08:03:55.980504+0000","last_unstale":"2023-11-24T08:03:57.449374+0000","last_undegraded":"2023-11-24T08:03:57.449374+0000","last_fullsized":"2023-11-24T08:03:57.449374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953650+0000","last_change":"2023-11-24T08:03:55.240395+0000","last_active":"2023-11-24T08:03:56.953650+0000","last_peered":"2023-11-24T08:03:56.953650+0000","last_clean":"2023-11-24T08:03:56.953650+0000","last_became_active":"2023-11-24T08:03:55.240021+0000","last_became_peered":"2023-11-24T08:03:55.240021+0000","last_unstale":"2023-11-24T08:03:56.953650+0000","last_undegraded":"2023-11-24T08:03:56.953650+0000","last_fullsized":"2023-11-24T08:03:56.953650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953580+0000","last_change":"2023-11-24T08:03:55.971086+0000","last_active":"2023-11-24T08:03:56.953580+0000","last_peered":"2023-11-24T08:03:56.953580+0000","last_clean":"2023-11-24T08:03:56.953580+0000","last_became_active":"2023-11-24T08:03:55.970765+0000","last_became_peered":"2023-11-24T08:03:55.970765+0000","last_unstale":"2023-11-24T08:03:56.953580+0000","last_undegraded":"2023-11-24T08:03:56.953580+0000","last_fullsized":"2023-11-24T08:03:56.953580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449518+0000","last_change":"2023-11-24T08:03:55.252498+0000","last_active":"2023-11-24T08:03:57.449518+0000","last_peered":"2023-11-24T08:03:57.449518+0000","last_clean":"2023-11-24T08:03:57.449518+0000","last_became_active":"2023-11-24T08:03:55.252172+0000","last_became_peered":"2023-11-24T08:03:55.252172+0000","last_unstale":"2023-11-24T08:03:57.449518+0000","last_undegraded":"2023-11-24T08:03:57.449518+0000","last_fullsized":"2023-11-24T08:03:57.449518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.458202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.466}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8979999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.798}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2480000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.516}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.181}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.786}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7519999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4009999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.468}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1970000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0539999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1829999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.302}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.101}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:04:06.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:04:06.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.732+0000 7f56e0c2e640 1 Processor -- start 2023-11-24T08:04:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.733+0000 7f56e0c2e640 1 -- start start 2023-11-24T08:04:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.734+0000 7f56e0c2e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56dc15d5b0 0x7f56dc0d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.734+0000 7f56e0c2e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56dc0cf0c0 con 0x7f56dc0d14a0 2023-11-24T08:04:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.734+0000 7f56e0c2e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56dc0b01b0 con 0x7f56dc0b1650 2023-11-24T08:04:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.734+0000 7f56e0c2e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56dc0d05c0 con 0x7f56dc15d5b0 2023-11-24T08:04:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.735+0000 7f56d3fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f56dc0d14a0 0x7f56dc0cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56804/0 (socket says 172.21.15.64:56804) 2023-11-24T08:04:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.735+0000 7f56d3fff640 1 -- 172.21.15.64:0/655212921 learned_addr learned my addr 172.21.15.64:0/655212921 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.735+0000 7f56db7fe640 1 --2- 172.21.15.64:0/655212921 >> v2:172.21.15.64:3300/0 conn(0x7f56dc15d5b0 0x7f56dc0d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.735+0000 7f56db7fe640 1 -- 172.21.15.64:0/655212921 >> v1:172.21.15.91:6789/0 conn(0x7f56dc0b1650 legacy=0x7f56dc0cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.735+0000 7f56db7fe640 1 -- 172.21.15.64:0/655212921 >> v1:172.21.15.64:6789/0 conn(0x7f56dc0d14a0 legacy=0x7f56dc0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.735+0000 7f56db7fe640 1 -- 172.21.15.64:0/655212921 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc15d990 con 0x7f56dc15d5b0 2023-11-24T08:04:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.735+0000 7f56daffd640 1 -- 172.21.15.64:0/655212921 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491744470 0 0) 0x7f56dc0cf0c0 con 0x7f56dc0d14a0 2023-11-24T08:04:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.736+0000 7f56db7fe640 1 --2- 172.21.15.64:0/655212921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc15d5b0 0x7f56dc0d3990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0097a0 tx=0x7f56cc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e6ccc28d818293 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56daffd640 1 -- 172.21.15.64:0/655212921 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc030610 con 0x7f56dc15d5b0 2023-11-24T08:04:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56daffd640 1 -- 172.21.15.64:0/655212921 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc030770 con 0x7f56dc15d5b0 2023-11-24T08:04:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56daffd640 1 -- 172.21.15.64:0/655212921 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc030b00 con 0x7f56dc15d5b0 2023-11-24T08:04:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56e0c2e640 1 -- 172.21.15.64:0/655212921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc15d5b0 msgr2=0x7f56dc0d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56e0c2e640 1 --2- 172.21.15.64:0/655212921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc15d5b0 0x7f56dc0d3990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0097a0 tx=0x7f56cc02b960 comp rx=0 tx=0).stop 2023-11-24T08:04:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56e0c2e640 1 -- 172.21.15.64:0/655212921 shutdown_connections 2023-11-24T08:04:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56e0c2e640 1 --2- 172.21.15.64:0/655212921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc15d5b0 0x7f56dc0d3990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56e0c2e640 1 -- 172.21.15.64:0/655212921 >> 172.21.15.64:0/655212921 conn(0x7f56dc09c3b0 msgr2=0x7f56dc0d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56e0c2e640 1 -- 172.21.15.64:0/655212921 shutdown_connections 2023-11-24T08:04:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.737+0000 7f56e0c2e640 1 -- 172.21.15.64:0/655212921 wait complete. 2023-11-24T08:04:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.739+0000 7f56e0c2e640 1 Processor -- start 2023-11-24T08:04:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.739+0000 7f56e0c2e640 1 -- start start 2023-11-24T08:04:06.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.740+0000 7f56e0c2e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56dc0b1650 0x7f56dc0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.740+0000 7f56e0c2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc0d14a0 0x7f56dc14d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.740+0000 7f56e0c2e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f56dc15d5b0 0x7f56dc0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.740+0000 7f56e0c2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0c7150 con 0x7f56dc0d14a0 2023-11-24T08:04:06.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.740+0000 7f56e0c2e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0b64d0 con 0x7f56dc0b1650 2023-11-24T08:04:06.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.740+0000 7f56e0c2e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0c97c0 con 0x7f56dc15d5b0 2023-11-24T08:04:06.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.740+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc0d14a0 0x7f56dc14d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc0d14a0 0x7f56dc14d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37152/0 (socket says 172.21.15.64:37152) 2023-11-24T08:04:06.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56d3fff640 1 -- 172.21.15.64:0/1551558291 learned_addr learned my addr 172.21.15.64:0/1551558291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:06.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56db7fe640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56dc0b1650 0x7f56dc0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56d3fff640 1 -- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f56dc15d5b0 msgr2=0x7f56dc0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56dbfff640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f56dc15d5b0 0x7f56dc0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56d3fff640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f56dc15d5b0 0x7f56dc0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56d3fff640 1 -- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56dc0b1650 msgr2=0x7f56dc0c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56d3fff640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56dc0b1650 0x7f56dc0c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56d3fff640 1 -- 172.21.15.64:0/1551558291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc009400 con 0x7f56dc0d14a0 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56db7fe640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56dc0b1650 0x7f56dc0c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56dbfff640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f56dc15d5b0 0x7f56dc0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.741+0000 7f56d3fff640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc0d14a0 0x7f56dc14d5b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56c400c3f0 tx=0x7f56c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:06.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.742+0000 7f56d8ff9640 1 -- 172.21.15.64:0/1551558291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40074e0 con 0x7f56dc0d14a0 2023-11-24T08:04:06.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.742+0000 7f56d8ff9640 1 -- 172.21.15.64:0/1551558291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4007640 con 0x7f56dc0d14a0 2023-11-24T08:04:06.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.742+0000 7f56d8ff9640 1 -- 172.21.15.64:0/1551558291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004a00 con 0x7f56dc0d14a0 2023-11-24T08:04:06.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.742+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc0c9aa0 con 0x7f56dc0d14a0 2023-11-24T08:04:06.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.742+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc0b6160 con 0x7f56dc0d14a0 2023-11-24T08:04:06.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.743+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56dc0cedc0 con 0x7f56dc0d14a0 2023-11-24T08:04:06.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.744+0000 7f56d8ff9640 1 -- 172.21.15.64:0/1551558291 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56c4004b60 con 0x7f56dc0d14a0 2023-11-24T08:04:06.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.745+0000 7f56d8ff9640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56bc06ae40 0x7f56bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:06.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.745+0000 7f56d8ff9640 1 -- 172.21.15.64:0/1551558291 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56c4089470 con 0x7f56dc0d14a0 2023-11-24T08:04:06.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.745+0000 7f56db7fe640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56bc06ae40 0x7f56bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:06.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.746+0000 7f56db7fe640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56bc06ae40 0x7f56bc06d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56cc036000 tx=0x7f56cc03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:06.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.749+0000 7f56d8ff9640 1 -- 172.21.15.64:0/1551558291 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56dc0cedc0 con 0x7f56dc0d14a0 2023-11-24T08:04:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.931+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56dc0afe40 con 0x7f56bc06ae40 2023-11-24T08:04:06.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.951+0000 7f56d8ff9640 1 -- 172.21.15.64:0/1551558291 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264310 (secure 0 0 0) 0x7f56cc03f040 con 0x7f56bc06ae40 2023-11-24T08:04:06.956 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:04:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.956+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56bc06ae40 msgr2=0x7f56bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.956+0000 7f56e0c2e640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56bc06ae40 0x7f56bc06d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56cc036000 tx=0x7f56cc03b040 comp rx=0 tx=0).stop 2023-11-24T08:04:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.956+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc0d14a0 msgr2=0x7f56dc14d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.956+0000 7f56e0c2e640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc0d14a0 0x7f56dc14d5b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56c400c3f0 tx=0x7f56c400c8b0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.956+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 shutdown_connections 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.956+0000 7f56e0c2e640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56bc06ae40 0x7f56bc06d300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.956+0000 7f56e0c2e640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f56dc15d5b0 0x7f56dc0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.957+0000 7f56e0c2e640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56dc0d14a0 0x7f56dc14d5b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.957+0000 7f56e0c2e640 1 --2- 172.21.15.64:0/1551558291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f56dc0b1650 0x7f56dc0c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.957+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 >> 172.21.15.64:0/1551558291 conn(0x7f56dc09c3b0 msgr2=0x7f56dc0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.957+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 shutdown_connections 2023-11-24T08:04:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:06.957+0000 7f56e0c2e640 1 -- 172.21.15.64:0/1551558291 wait complete. 2023-11-24T08:04:06.976 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T08:04:06.014763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003719"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265413+0000","last_change":"2023-11-24T08:03:55.977134+0000","last_active":"2023-11-24T08:03:57.265413+0000","last_peered":"2023-11-24T08:03:57.265413+0000","last_clean":"2023-11-24T08:03:57.265413+0000","last_became_active":"2023-11-24T08:03:55.970402+0000","last_became_peered":"2023-11-24T08:03:55.970402+0000","last_unstale":"2023-11-24T08:03:57.265413+0000","last_undegraded":"2023-11-24T08:03:57.265413+0000","last_fullsized":"2023-11-24T08:03:57.265413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:47.677166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949561+0000","last_change":"2023-11-24T08:03:55.973687+0000","last_active":"2023-11-24T08:03:56.949561+0000","last_peered":"2023-11-24T08:03:56.949561+0000","last_clean":"2023-11-24T08:03:56.949561+0000","last_became_active":"2023-11-24T08:03:55.973369+0000","last_became_peered":"2023-11-24T08:03:55.973369+0000","last_unstale":"2023-11-24T08:03:56.949561+0000","last_undegraded":"2023-11-24T08:03:56.949561+0000","last_fullsized":"2023-11-24T08:03:56.949561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449190+0000","last_change":"2023-11-24T08:03:55.971194+0000","last_active":"2023-11-24T08:03:57.449190+0000","last_peered":"2023-11-24T08:03:57.449190+0000","last_clean":"2023-11-24T08:03:57.449190+0000","last_became_active":"2023-11-24T08:03:55.970751+0000","last_became_peered":"2023-11-24T08:03:55.970751+0000","last_unstale":"2023-11-24T08:03:57.449190+0000","last_undegraded":"2023-11-24T08:03:57.449190+0000","last_fullsized":"2023-11-24T08:03:57.449190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994604+0000","last_change":"2023-11-24T08:03:55.984822+0000","last_active":"2023-11-24T08:03:56.994604+0000","last_peered":"2023-11-24T08:03:56.994604+0000","last_clean":"2023-11-24T08:03:56.994604+0000","last_became_active":"2023-11-24T08:03:55.982969+0000","last_became_peered":"2023-11-24T08:03:55.982969+0000","last_unstale":"2023-11-24T08:03:56.994604+0000","last_undegraded":"2023-11-24T08:03:56.994604+0000","last_fullsized":"2023-11-24T08:03:56.994604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:43.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264555+0000","last_change":"2023-11-24T08:03:55.980587+0000","last_active":"2023-11-24T08:03:57.264555+0000","last_peered":"2023-11-24T08:03:57.264555+0000","last_clean":"2023-11-24T08:03:57.264555+0000","last_became_active":"2023-11-24T08:03:55.980331+0000","last_became_peered":"2023-11-24T08:03:55.980331+0000","last_unstale":"2023-11-24T08:03:57.264555+0000","last_undegraded":"2023-11-24T08:03:57.264555+0000","last_fullsized":"2023-11-24T08:03:57.264555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:16.719789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994145+0000","last_change":"2023-11-24T08:03:55.981917+0000","last_active":"2023-11-24T08:03:56.994145+0000","last_peered":"2023-11-24T08:03:56.994145+0000","last_clean":"2023-11-24T08:03:56.994145+0000","last_became_active":"2023-11-24T08:03:55.981300+0000","last_became_peered":"2023-11-24T08:03:55.981300+0000","last_unstale":"2023-11-24T08:03:56.994145+0000","last_undegraded":"2023-11-24T08:03:56.994145+0000","last_fullsized":"2023-11-24T08:03:56.994145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263992+0000","last_change":"2023-11-24T08:03:55.975783+0000","last_active":"2023-11-24T08:03:57.263992+0000","last_peered":"2023-11-24T08:03:57.263992+0000","last_clean":"2023-11-24T08:03:57.263992+0000","last_became_active":"2023-11-24T08:03:55.975452+0000","last_became_peered":"2023-11-24T08:03:55.975452+0000","last_unstale":"2023-11-24T08:03:57.263992+0000","last_undegraded":"2023-11-24T08:03:57.263992+0000","last_fullsized":"2023-11-24T08:03:57.263992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952790+0000","last_change":"2023-11-24T08:03:55.980610+0000","last_active":"2023-11-24T08:03:56.952790+0000","last_peered":"2023-11-24T08:03:56.952790+0000","last_clean":"2023-11-24T08:03:56.952790+0000","last_became_active":"2023-11-24T08:03:55.980222+0000","last_became_peered":"2023-11-24T08:03:55.980222+0000","last_unstale":"2023-11-24T08:03:56.952790+0000","last_undegraded":"2023-11-24T08:03:56.952790+0000","last_fullsized":"2023-11-24T08:03:56.952790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265494+0000","last_change":"2023-11-24T08:03:55.988607+0000","last_active":"2023-11-24T08:03:57.265494+0000","last_peered":"2023-11-24T08:03:57.265494+0000","last_clean":"2023-11-24T08:03:57.265494+0000","last_became_active":"2023-11-24T08:03:55.988074+0000","last_became_peered":"2023-11-24T08:03:55.988074+0000","last_unstale":"2023-11-24T08:03:57.265494+0000","last_undegraded":"2023-11-24T08:03:57.265494+0000","last_fullsized":"2023-11-24T08:03:57.265494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:08.202382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449121+0000","last_change":"2023-11-24T08:03:55.972034+0000","last_active":"2023-11-24T08:03:57.449121+0000","last_peered":"2023-11-24T08:03:57.449121+0000","last_clean":"2023-11-24T08:03:57.449121+0000","last_became_active":"2023-11-24T08:03:55.971627+0000","last_became_peered":"2023-11-24T08:03:55.971627+0000","last_unstale":"2023-11-24T08:03:57.449121+0000","last_undegraded":"2023-11-24T08:03:57.449121+0000","last_fullsized":"2023-11-24T08:03:57.449121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448738+0000","last_change":"2023-11-24T08:03:55.979788+0000","last_active":"2023-11-24T08:03:57.448738+0000","last_peered":"2023-11-24T08:03:57.448738+0000","last_clean":"2023-11-24T08:03:57.448738+0000","last_became_active":"2023-11-24T08:03:55.979429+0000","last_became_peered":"2023-11-24T08:03:55.979429+0000","last_unstale":"2023-11-24T08:03:57.448738+0000","last_undegraded":"2023-11-24T08:03:57.448738+0000","last_fullsized":"2023-11-24T08:03:57.448738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953475+0000","last_change":"2023-11-24T08:03:55.975330+0000","last_active":"2023-11-24T08:03:56.953475+0000","last_peered":"2023-11-24T08:03:56.953475+0000","last_clean":"2023-11-24T08:03:56.953475+0000","last_became_active":"2023-11-24T08:03:55.974721+0000","last_became_peered":"2023-11-24T08:03:55.974721+0000","last_unstale":"2023-11-24T08:03:56.953475+0000","last_undegraded":"2023-11-24T08:03:56.953475+0000","last_fullsized":"2023-11-24T08:03:56.953475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:35.456457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953384+0000","last_change":"2023-11-24T08:03:55.975039+0000","last_active":"2023-11-24T08:03:56.953384+0000","last_peered":"2023-11-24T08:03:56.953384+0000","last_clean":"2023-11-24T08:03:56.953384+0000","last_became_active":"2023-11-24T08:03:55.974171+0000","last_became_peered":"2023-11-24T08:03:55.974171+0000","last_unstale":"2023-11-24T08:03:56.953384+0000","last_undegraded":"2023-11-24T08:03:56.953384+0000","last_fullsized":"2023-11-24T08:03:56.953384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448417+0000","last_change":"2023-11-24T08:03:55.972138+0000","last_active":"2023-11-24T08:03:57.448417+0000","last_peered":"2023-11-24T08:03:57.448417+0000","last_clean":"2023-11-24T08:03:57.448417+0000","last_became_active":"2023-11-24T08:03:55.968993+0000","last_became_peered":"2023-11-24T08:03:55.968993+0000","last_unstale":"2023-11-24T08:03:57.448417+0000","last_undegraded":"2023-11-24T08:03:57.448417+0000","last_fullsized":"2023-11-24T08:03:57.448417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949358+0000","last_change":"2023-11-24T08:03:55.961573+0000","last_active":"2023-11-24T08:03:56.949358+0000","last_peered":"2023-11-24T08:03:56.949358+0000","last_clean":"2023-11-24T08:03:56.949358+0000","last_became_active":"2023-11-24T08:03:55.961135+0000","last_became_peered":"2023-11-24T08:03:55.961135+0000","last_unstale":"2023-11-24T08:03:56.949358+0000","last_undegraded":"2023-11-24T08:03:56.949358+0000","last_fullsized":"2023-11-24T08:03:56.949358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994390+0000","last_change":"2023-11-24T08:03:55.982805+0000","last_active":"2023-11-24T08:03:56.994390+0000","last_peered":"2023-11-24T08:03:56.994390+0000","last_clean":"2023-11-24T08:03:56.994390+0000","last_became_active":"2023-11-24T08:03:55.980530+0000","last_became_peered":"2023-11-24T08:03:55.980530+0000","last_unstale":"2023-11-24T08:03:56.994390+0000","last_undegraded":"2023-11-24T08:03:56.994390+0000","last_fullsized":"2023-11-24T08:03:56.994390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995521+0000","last_change":"2023-11-24T08:03:55.988078+0000","last_active":"2023-11-24T08:03:56.995521+0000","last_peered":"2023-11-24T08:03:56.995521+0000","last_clean":"2023-11-24T08:03:56.995521+0000","last_became_active":"2023-11-24T08:03:55.987802+0000","last_became_peered":"2023-11-24T08:03:55.987802+0000","last_unstale":"2023-11-24T08:03:56.995521+0000","last_undegraded":"2023-11-24T08:03:56.995521+0000","last_fullsized":"2023-11-24T08:03:56.995521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:19.891560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445715+0000","last_change":"2023-11-24T08:03:55.968360+0000","last_active":"2023-11-24T08:03:57.445715+0000","last_peered":"2023-11-24T08:03:57.445715+0000","last_clean":"2023-11-24T08:03:57.445715+0000","last_became_active":"2023-11-24T08:03:55.968089+0000","last_became_peered":"2023-11-24T08:03:55.968089+0000","last_unstale":"2023-11-24T08:03:57.445715+0000","last_undegraded":"2023-11-24T08:03:57.445715+0000","last_fullsized":"2023-11-24T08:03:57.445715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:04.042778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264933+0000","last_change":"2023-11-24T08:03:55.971957+0000","last_active":"2023-11-24T08:03:57.264933+0000","last_peered":"2023-11-24T08:03:57.264933+0000","last_clean":"2023-11-24T08:03:57.264933+0000","last_became_active":"2023-11-24T08:03:55.971638+0000","last_became_peered":"2023-11-24T08:03:55.971638+0000","last_unstale":"2023-11-24T08:03:57.264933+0000","last_undegraded":"2023-11-24T08:03:57.264933+0000","last_fullsized":"2023-11-24T08:03:57.264933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:01.176228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448754+0000","last_change":"2023-11-24T08:03:55.976704+0000","last_active":"2023-11-24T08:03:57.448754+0000","last_peered":"2023-11-24T08:03:57.448754+0000","last_clean":"2023-11-24T08:03:57.448754+0000","last_became_active":"2023-11-24T08:03:55.976410+0000","last_became_peered":"2023-11-24T08:03:55.976410+0000","last_unstale":"2023-11-24T08:03:57.448754+0000","last_undegraded":"2023-11-24T08:03:57.448754+0000","last_fullsized":"2023-11-24T08:03:57.448754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448488+0000","last_change":"2023-11-24T08:03:55.985384+0000","last_active":"2023-11-24T08:03:57.448488+0000","last_peered":"2023-11-24T08:03:57.448488+0000","last_clean":"2023-11-24T08:03:57.448488+0000","last_became_active":"2023-11-24T08:03:55.985155+0000","last_became_peered":"2023-11-24T08:03:55.985155+0000","last_unstale":"2023-11-24T08:03:57.448488+0000","last_undegraded":"2023-11-24T08:03:57.448488+0000","last_fullsized":"2023-11-24T08:03:57.448488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994409+0000","last_change":"2023-11-24T08:03:55.982133+0000","last_active":"2023-11-24T08:03:56.994409+0000","last_peered":"2023-11-24T08:03:56.994409+0000","last_clean":"2023-11-24T08:03:56.994409+0000","last_became_active":"2023-11-24T08:03:55.981629+0000","last_became_peered":"2023-11-24T08:03:55.981629+0000","last_unstale":"2023-11-24T08:03:56.994409+0000","last_undegraded":"2023-11-24T08:03:56.994409+0000","last_fullsized":"2023-11-24T08:03:56.994409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952961+0000","last_change":"2023-11-24T08:03:55.963916+0000","last_active":"2023-11-24T08:03:56.952961+0000","last_peered":"2023-11-24T08:03:56.952961+0000","last_clean":"2023-11-24T08:03:56.952961+0000","last_became_active":"2023-11-24T08:03:55.958460+0000","last_became_peered":"2023-11-24T08:03:55.958460+0000","last_unstale":"2023-11-24T08:03:56.952961+0000","last_undegraded":"2023-11-24T08:03:56.952961+0000","last_fullsized":"2023-11-24T08:03:56.952961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994284+0000","last_change":"2023-11-24T08:03:55.980845+0000","last_active":"2023-11-24T08:03:56.994284+0000","last_peered":"2023-11-24T08:03:56.994284+0000","last_clean":"2023-11-24T08:03:56.994284+0000","last_became_active":"2023-11-24T08:03:55.979729+0000","last_became_peered":"2023-11-24T08:03:55.979729+0000","last_unstale":"2023-11-24T08:03:56.994284+0000","last_undegraded":"2023-11-24T08:03:56.994284+0000","last_fullsized":"2023-11-24T08:03:56.994284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953148+0000","last_change":"2023-11-24T08:03:55.956896+0000","last_active":"2023-11-24T08:03:56.953148+0000","last_peered":"2023-11-24T08:03:56.953148+0000","last_clean":"2023-11-24T08:03:56.953148+0000","last_became_active":"2023-11-24T08:03:55.956452+0000","last_became_peered":"2023-11-24T08:03:55.956452+0000","last_unstale":"2023-11-24T08:03:56.953148+0000","last_undegraded":"2023-11-24T08:03:56.953148+0000","last_fullsized":"2023-11-24T08:03:56.953148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449134+0000","last_change":"2023-11-24T08:03:55.968031+0000","last_active":"2023-11-24T08:03:57.449134+0000","last_peered":"2023-11-24T08:03:57.449134+0000","last_clean":"2023-11-24T08:03:57.449134+0000","last_became_active":"2023-11-24T08:03:55.967733+0000","last_became_peered":"2023-11-24T08:03:55.967733+0000","last_unstale":"2023-11-24T08:03:57.449134+0000","last_undegraded":"2023-11-24T08:03:57.449134+0000","last_fullsized":"2023-11-24T08:03:57.449134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949553+0000","last_change":"2023-11-24T08:03:55.971172+0000","last_active":"2023-11-24T08:03:56.949553+0000","last_peered":"2023-11-24T08:03:56.949553+0000","last_clean":"2023-11-24T08:03:56.949553+0000","last_became_active":"2023-11-24T08:03:55.970829+0000","last_became_peered":"2023-11-24T08:03:55.970829+0000","last_unstale":"2023-11-24T08:03:56.949553+0000","last_undegraded":"2023-11-24T08:03:56.949553+0000","last_fullsized":"2023-11-24T08:03:56.949553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953446+0000","last_change":"2023-11-24T08:03:55.965485+0000","last_active":"2023-11-24T08:03:56.953446+0000","last_peered":"2023-11-24T08:03:56.953446+0000","last_clean":"2023-11-24T08:03:56.953446+0000","last_became_active":"2023-11-24T08:03:55.965220+0000","last_became_peered":"2023-11-24T08:03:55.965220+0000","last_unstale":"2023-11-24T08:03:56.953446+0000","last_undegraded":"2023-11-24T08:03:56.953446+0000","last_fullsized":"2023-11-24T08:03:56.953446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:43.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448732+0000","last_change":"2023-11-24T08:03:55.978834+0000","last_active":"2023-11-24T08:03:57.448732+0000","last_peered":"2023-11-24T08:03:57.448732+0000","last_clean":"2023-11-24T08:03:57.448732+0000","last_became_active":"2023-11-24T08:03:55.978604+0000","last_became_peered":"2023-11-24T08:03:55.978604+0000","last_unstale":"2023-11-24T08:03:57.448732+0000","last_undegraded":"2023-11-24T08:03:57.448732+0000","last_fullsized":"2023-11-24T08:03:57.448732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994520+0000","last_change":"2023-11-24T08:03:55.982758+0000","last_active":"2023-11-24T08:03:56.994520+0000","last_peered":"2023-11-24T08:03:56.994520+0000","last_clean":"2023-11-24T08:03:56.994520+0000","last_became_active":"2023-11-24T08:03:55.978909+0000","last_became_peered":"2023-11-24T08:03:55.978909+0000","last_unstale":"2023-11-24T08:03:56.994520+0000","last_undegraded":"2023-11-24T08:03:56.994520+0000","last_fullsized":"2023-11-24T08:03:56.994520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:55.841729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445961+0000","last_change":"2023-11-24T08:03:55.972047+0000","last_active":"2023-11-24T08:03:57.445961+0000","last_peered":"2023-11-24T08:03:57.445961+0000","last_clean":"2023-11-24T08:03:57.445961+0000","last_became_active":"2023-11-24T08:03:55.971770+0000","last_became_peered":"2023-11-24T08:03:55.971770+0000","last_unstale":"2023-11-24T08:03:57.445961+0000","last_undegraded":"2023-11-24T08:03:57.445961+0000","last_fullsized":"2023-11-24T08:03:57.445961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:35.456457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448439+0000","last_change":"2023-11-24T08:03:55.974157+0000","last_active":"2023-11-24T08:03:57.448439+0000","last_peered":"2023-11-24T08:03:57.448439+0000","last_clean":"2023-11-24T08:03:57.448439+0000","last_became_active":"2023-11-24T08:03:55.973864+0000","last_became_peered":"2023-11-24T08:03:55.973864+0000","last_unstale":"2023-11-24T08:03:57.448439+0000","last_undegraded":"2023-11-24T08:03:57.448439+0000","last_fullsized":"2023-11-24T08:03:57.448439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952874+0000","last_change":"2023-11-24T08:03:55.959866+0000","last_active":"2023-11-24T08:03:56.952874+0000","last_peered":"2023-11-24T08:03:56.952874+0000","last_clean":"2023-11-24T08:03:56.952874+0000","last_became_active":"2023-11-24T08:03:55.959194+0000","last_became_peered":"2023-11-24T08:03:55.959194+0000","last_unstale":"2023-11-24T08:03:56.952874+0000","last_undegraded":"2023-11-24T08:03:56.952874+0000","last_fullsized":"2023-11-24T08:03:56.952874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:13.909369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953570+0000","last_change":"2023-11-24T08:03:55.243163+0000","last_active":"2023-11-24T08:03:56.953570+0000","last_peered":"2023-11-24T08:03:56.953570+0000","last_clean":"2023-11-24T08:03:56.953570+0000","last_became_active":"2023-11-24T08:03:55.242828+0000","last_became_peered":"2023-11-24T08:03:55.242828+0000","last_unstale":"2023-11-24T08:03:56.953570+0000","last_undegraded":"2023-11-24T08:03:56.953570+0000","last_fullsized":"2023-11-24T08:03:56.953570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448964+0000","last_change":"2023-11-24T08:03:55.976360+0000","last_active":"2023-11-24T08:03:57.448964+0000","last_peered":"2023-11-24T08:03:57.448964+0000","last_clean":"2023-11-24T08:03:57.448964+0000","last_became_active":"2023-11-24T08:03:55.975680+0000","last_became_peered":"2023-11-24T08:03:55.975680+0000","last_unstale":"2023-11-24T08:03:57.448964+0000","last_undegraded":"2023-11-24T08:03:57.448964+0000","last_fullsized":"2023-11-24T08:03:57.448964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:43.870854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448633+0000","last_change":"2023-11-24T08:03:55.250034+0000","last_active":"2023-11-24T08:03:57.448633+0000","last_peered":"2023-11-24T08:03:57.448633+0000","last_clean":"2023-11-24T08:03:57.448633+0000","last_became_active":"2023-11-24T08:03:55.247004+0000","last_became_peered":"2023-11-24T08:03:55.247004+0000","last_unstale":"2023-11-24T08:03:57.448633+0000","last_undegraded":"2023-11-24T08:03:57.448633+0000","last_fullsized":"2023-11-24T08:03:57.448633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:58.276728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952866+0000","last_change":"2023-11-24T08:03:55.249873+0000","last_active":"2023-11-24T08:03:56.952866+0000","last_peered":"2023-11-24T08:03:56.952866+0000","last_clean":"2023-11-24T08:03:56.952866+0000","last_became_active":"2023-11-24T08:03:55.249613+0000","last_became_peered":"2023-11-24T08:03:55.249613+0000","last_unstale":"2023-11-24T08:03:56.952866+0000","last_undegraded":"2023-11-24T08:03:56.952866+0000","last_fullsized":"2023-11-24T08:03:56.952866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952860+0000","last_change":"2023-11-24T08:03:55.975878+0000","last_active":"2023-11-24T08:03:56.952860+0000","last_peered":"2023-11-24T08:03:56.952860+0000","last_clean":"2023-11-24T08:03:56.952860+0000","last_became_active":"2023-11-24T08:03:55.974080+0000","last_became_peered":"2023-11-24T08:03:55.974080+0000","last_unstale":"2023-11-24T08:03:56.952860+0000","last_undegraded":"2023-11-24T08:03:56.952860+0000","last_fullsized":"2023-11-24T08:03:56.952860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994534+0000","last_change":"2023-11-24T08:03:55.249828+0000","last_active":"2023-11-24T08:03:56.994534+0000","last_peered":"2023-11-24T08:03:56.994534+0000","last_clean":"2023-11-24T08:03:56.994534+0000","last_became_active":"2023-11-24T08:03:55.246927+0000","last_became_peered":"2023-11-24T08:03:55.246927+0000","last_unstale":"2023-11-24T08:03:56.994534+0000","last_undegraded":"2023-11-24T08:03:56.994534+0000","last_fullsized":"2023-11-24T08:03:56.994534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949045+0000","last_change":"2023-11-24T08:03:55.971036+0000","last_active":"2023-11-24T08:03:56.949045+0000","last_peered":"2023-11-24T08:03:56.949045+0000","last_clean":"2023-11-24T08:03:56.949045+0000","last_became_active":"2023-11-24T08:03:55.970127+0000","last_became_peered":"2023-11-24T08:03:55.970127+0000","last_unstale":"2023-11-24T08:03:56.949045+0000","last_undegraded":"2023-11-24T08:03:56.949045+0000","last_fullsized":"2023-11-24T08:03:56.949045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448936+0000","last_change":"2023-11-24T08:03:55.253888+0000","last_active":"2023-11-24T08:03:57.448936+0000","last_peered":"2023-11-24T08:03:57.448936+0000","last_clean":"2023-11-24T08:03:57.448936+0000","last_became_active":"2023-11-24T08:03:55.253583+0000","last_became_peered":"2023-11-24T08:03:55.253583+0000","last_unstale":"2023-11-24T08:03:57.448936+0000","last_undegraded":"2023-11-24T08:03:57.448936+0000","last_fullsized":"2023-11-24T08:03:57.448936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:01.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995133+0000","last_change":"2023-11-24T08:03:55.986660+0000","last_active":"2023-11-24T08:03:56.995133+0000","last_peered":"2023-11-24T08:03:56.995133+0000","last_clean":"2023-11-24T08:03:56.995133+0000","last_became_active":"2023-11-24T08:03:55.986351+0000","last_became_peered":"2023-11-24T08:03:55.986351+0000","last_unstale":"2023-11-24T08:03:56.995133+0000","last_undegraded":"2023-11-24T08:03:56.995133+0000","last_fullsized":"2023-11-24T08:03:56.995133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:16.719789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995284+0000","last_change":"2023-11-24T08:03:55.256523+0000","last_active":"2023-11-24T08:03:56.995284+0000","last_peered":"2023-11-24T08:03:56.995284+0000","last_clean":"2023-11-24T08:03:56.995284+0000","last_became_active":"2023-11-24T08:03:55.256208+0000","last_became_peered":"2023-11-24T08:03:55.256208+0000","last_unstale":"2023-11-24T08:03:56.995284+0000","last_undegraded":"2023-11-24T08:03:56.995284+0000","last_fullsized":"2023-11-24T08:03:56.995284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449400+0000","last_change":"2023-11-24T08:03:55.980794+0000","last_active":"2023-11-24T08:03:57.449400+0000","last_peered":"2023-11-24T08:03:57.449400+0000","last_clean":"2023-11-24T08:03:57.449400+0000","last_became_active":"2023-11-24T08:03:55.980463+0000","last_became_peered":"2023-11-24T08:03:55.980463+0000","last_unstale":"2023-11-24T08:03:57.449400+0000","last_undegraded":"2023-11-24T08:03:57.449400+0000","last_fullsized":"2023-11-24T08:03:57.449400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949486+0000","last_change":"2023-11-24T08:03:55.244567+0000","last_active":"2023-11-24T08:03:56.949486+0000","last_peered":"2023-11-24T08:03:56.949486+0000","last_clean":"2023-11-24T08:03:56.949486+0000","last_became_active":"2023-11-24T08:03:55.244286+0000","last_became_peered":"2023-11-24T08:03:55.244286+0000","last_unstale":"2023-11-24T08:03:56.949486+0000","last_undegraded":"2023-11-24T08:03:56.949486+0000","last_fullsized":"2023-11-24T08:03:56.949486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.896540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953786+0000","last_change":"2023-11-24T08:03:55.982726+0000","last_active":"2023-11-24T08:03:56.953786+0000","last_peered":"2023-11-24T08:03:56.953786+0000","last_clean":"2023-11-24T08:03:56.953786+0000","last_became_active":"2023-11-24T08:03:55.982437+0000","last_became_peered":"2023-11-24T08:03:55.982437+0000","last_unstale":"2023-11-24T08:03:56.953786+0000","last_undegraded":"2023-11-24T08:03:56.953786+0000","last_fullsized":"2023-11-24T08:03:56.953786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:59.289556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448900+0000","last_change":"2023-11-24T08:03:55.252214+0000","last_active":"2023-11-24T08:03:57.448900+0000","last_peered":"2023-11-24T08:03:57.448900+0000","last_clean":"2023-11-24T08:03:57.448900+0000","last_became_active":"2023-11-24T08:03:55.251878+0000","last_became_peered":"2023-11-24T08:03:55.251878+0000","last_unstale":"2023-11-24T08:03:57.448900+0000","last_undegraded":"2023-11-24T08:03:57.448900+0000","last_fullsized":"2023-11-24T08:03:57.448900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:59.543627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264565+0000","last_change":"2023-11-24T08:03:55.981213+0000","last_active":"2023-11-24T08:03:57.264565+0000","last_peered":"2023-11-24T08:03:57.264565+0000","last_clean":"2023-11-24T08:03:57.264565+0000","last_became_active":"2023-11-24T08:03:55.979791+0000","last_became_peered":"2023-11-24T08:03:55.979791+0000","last_unstale":"2023-11-24T08:03:57.264565+0000","last_undegraded":"2023-11-24T08:03:57.264565+0000","last_fullsized":"2023-11-24T08:03:57.264565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:03.856484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448918+0000","last_change":"2023-11-24T08:03:55.242184+0000","last_active":"2023-11-24T08:03:57.448918+0000","last_peered":"2023-11-24T08:03:57.448918+0000","last_clean":"2023-11-24T08:03:57.448918+0000","last_became_active":"2023-11-24T08:03:55.241829+0000","last_became_peered":"2023-11-24T08:03:55.241829+0000","last_unstale":"2023-11-24T08:03:57.448918+0000","last_undegraded":"2023-11-24T08:03:57.448918+0000","last_fullsized":"2023-11-24T08:03:57.448918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448374+0000","last_change":"2023-11-24T08:03:55.979303+0000","last_active":"2023-11-24T08:03:57.448374+0000","last_peered":"2023-11-24T08:03:57.448374+0000","last_clean":"2023-11-24T08:03:57.448374+0000","last_became_active":"2023-11-24T08:03:55.979044+0000","last_became_peered":"2023-11-24T08:03:55.979044+0000","last_unstale":"2023-11-24T08:03:57.448374+0000","last_undegraded":"2023-11-24T08:03:57.448374+0000","last_fullsized":"2023-11-24T08:03:57.448374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994583+0000","last_change":"2023-11-24T08:03:55.247945+0000","last_active":"2023-11-24T08:03:56.994583+0000","last_peered":"2023-11-24T08:03:56.994583+0000","last_clean":"2023-11-24T08:03:56.994583+0000","last_became_active":"2023-11-24T08:03:55.247662+0000","last_became_peered":"2023-11-24T08:03:55.247662+0000","last_unstale":"2023-11-24T08:03:56.994583+0000","last_undegraded":"2023-11-24T08:03:56.994583+0000","last_fullsized":"2023-11-24T08:03:56.994583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:47.114189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264673+0000","last_change":"2023-11-24T08:03:55.985619+0000","last_active":"2023-11-24T08:03:57.264673+0000","last_peered":"2023-11-24T08:03:57.264673+0000","last_clean":"2023-11-24T08:03:57.264673+0000","last_became_active":"2023-11-24T08:03:55.985179+0000","last_became_peered":"2023-11-24T08:03:55.985179+0000","last_unstale":"2023-11-24T08:03:57.264673+0000","last_undegraded":"2023-11-24T08:03:57.264673+0000","last_fullsized":"2023-11-24T08:03:57.264673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:19.891560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449005+0000","last_change":"2023-11-24T08:03:55.247678+0000","last_active":"2023-11-24T08:03:57.449005+0000","last_peered":"2023-11-24T08:03:57.449005+0000","last_clean":"2023-11-24T08:03:57.449005+0000","last_became_active":"2023-11-24T08:03:55.247366+0000","last_became_peered":"2023-11-24T08:03:55.247366+0000","last_unstale":"2023-11-24T08:03:57.449005+0000","last_undegraded":"2023-11-24T08:03:57.449005+0000","last_fullsized":"2023-11-24T08:03:57.449005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:21.976953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448482+0000","last_change":"2023-11-24T08:03:55.973556+0000","last_active":"2023-11-24T08:03:57.448482+0000","last_peered":"2023-11-24T08:03:57.448482+0000","last_clean":"2023-11-24T08:03:57.448482+0000","last_became_active":"2023-11-24T08:03:55.971826+0000","last_became_peered":"2023-11-24T08:03:55.971826+0000","last_unstale":"2023-11-24T08:03:57.448482+0000","last_undegraded":"2023-11-24T08:03:57.448482+0000","last_fullsized":"2023-11-24T08:03:57.448482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445426+0000","last_change":"2023-11-24T08:03:55.240904+0000","last_active":"2023-11-24T08:03:57.445426+0000","last_peered":"2023-11-24T08:03:57.445426+0000","last_clean":"2023-11-24T08:03:57.445426+0000","last_became_active":"2023-11-24T08:03:55.240317+0000","last_became_peered":"2023-11-24T08:03:55.240317+0000","last_unstale":"2023-11-24T08:03:57.445426+0000","last_undegraded":"2023-11-24T08:03:57.445426+0000","last_fullsized":"2023-11-24T08:03:57.445426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:25.132329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264211+0000","last_change":"2023-11-24T08:03:55.979396+0000","last_active":"2023-11-24T08:03:57.264211+0000","last_peered":"2023-11-24T08:03:57.264211+0000","last_clean":"2023-11-24T08:03:57.264211+0000","last_became_active":"2023-11-24T08:03:55.979048+0000","last_became_peered":"2023-11-24T08:03:55.979048+0000","last_unstale":"2023-11-24T08:03:57.264211+0000","last_undegraded":"2023-11-24T08:03:57.264211+0000","last_fullsized":"2023-11-24T08:03:57.264211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:14.268136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953570+0000","last_change":"2023-11-24T08:03:55.241439+0000","last_active":"2023-11-24T08:03:56.953570+0000","last_peered":"2023-11-24T08:03:56.953570+0000","last_clean":"2023-11-24T08:03:56.953570+0000","last_became_active":"2023-11-24T08:03:55.239231+0000","last_became_peered":"2023-11-24T08:03:55.239231+0000","last_unstale":"2023-11-24T08:03:56.953570+0000","last_undegraded":"2023-11-24T08:03:56.953570+0000","last_fullsized":"2023-11-24T08:03:56.953570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:25.132329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265081+0000","last_change":"2023-11-24T08:03:55.987779+0000","last_active":"2023-11-24T08:03:57.265081+0000","last_peered":"2023-11-24T08:03:57.265081+0000","last_clean":"2023-11-24T08:03:57.265081+0000","last_became_active":"2023-11-24T08:03:55.987443+0000","last_became_peered":"2023-11-24T08:03:55.987443+0000","last_unstale":"2023-11-24T08:03:57.265081+0000","last_undegraded":"2023-11-24T08:03:57.265081+0000","last_fullsized":"2023-11-24T08:03:57.265081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:18.291474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445818+0000","last_change":"2023-11-24T08:03:55.240840+0000","last_active":"2023-11-24T08:03:57.445818+0000","last_peered":"2023-11-24T08:03:57.445818+0000","last_clean":"2023-11-24T08:03:57.445818+0000","last_became_active":"2023-11-24T08:03:55.239508+0000","last_became_peered":"2023-11-24T08:03:55.239508+0000","last_unstale":"2023-11-24T08:03:57.445818+0000","last_undegraded":"2023-11-24T08:03:57.445818+0000","last_fullsized":"2023-11-24T08:03:57.445818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.896540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.265617+0000","last_change":"2023-11-24T08:03:55.987770+0000","last_active":"2023-11-24T08:03:57.265617+0000","last_peered":"2023-11-24T08:03:57.265617+0000","last_clean":"2023-11-24T08:03:57.265617+0000","last_became_active":"2023-11-24T08:03:55.987481+0000","last_became_peered":"2023-11-24T08:03:55.987481+0000","last_unstale":"2023-11-24T08:03:57.265617+0000","last_undegraded":"2023-11-24T08:03:57.265617+0000","last_fullsized":"2023-11-24T08:03:57.265617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:36.996687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949593+0000","last_change":"2023-11-24T08:03:55.243684+0000","last_active":"2023-11-24T08:03:56.949593+0000","last_peered":"2023-11-24T08:03:56.949593+0000","last_clean":"2023-11-24T08:03:56.949593+0000","last_became_active":"2023-11-24T08:03:55.242363+0000","last_became_peered":"2023-11-24T08:03:55.242363+0000","last_unstale":"2023-11-24T08:03:56.949593+0000","last_undegraded":"2023-11-24T08:03:56.949593+0000","last_fullsized":"2023-11-24T08:03:56.949593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:34.443628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995391+0000","last_change":"2023-11-24T08:03:55.985203+0000","last_active":"2023-11-24T08:03:56.995391+0000","last_peered":"2023-11-24T08:03:56.995391+0000","last_clean":"2023-11-24T08:03:56.995391+0000","last_became_active":"2023-11-24T08:03:55.979146+0000","last_became_peered":"2023-11-24T08:03:55.979146+0000","last_unstale":"2023-11-24T08:03:56.995391+0000","last_undegraded":"2023-11-24T08:03:56.995391+0000","last_fullsized":"2023-11-24T08:03:56.995391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:47.350513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445560+0000","last_change":"2023-11-24T08:03:55.252157+0000","last_active":"2023-11-24T08:03:57.445560+0000","last_peered":"2023-11-24T08:03:57.445560+0000","last_clean":"2023-11-24T08:03:57.445560+0000","last_became_active":"2023-11-24T08:03:55.251772+0000","last_became_peered":"2023-11-24T08:03:55.251772+0000","last_unstale":"2023-11-24T08:03:57.445560+0000","last_undegraded":"2023-11-24T08:03:57.445560+0000","last_fullsized":"2023-11-24T08:03:57.445560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:01.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264581+0000","last_change":"2023-11-24T08:03:55.981873+0000","last_active":"2023-11-24T08:03:57.264581+0000","last_peered":"2023-11-24T08:03:57.264581+0000","last_clean":"2023-11-24T08:03:57.264581+0000","last_became_active":"2023-11-24T08:03:55.981562+0000","last_became_peered":"2023-11-24T08:03:55.981562+0000","last_unstale":"2023-11-24T08:03:57.264581+0000","last_undegraded":"2023-11-24T08:03:57.264581+0000","last_fullsized":"2023-11-24T08:03:57.264581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:47.350513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T08:03:55.953376+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:03:55.953376+0000","last_peered":"2023-11-24T08:03:55.953376+0000","last_clean":"2023-11-24T08:03:55.953376+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:03:55.953376+0000","last_undegraded":"2023-11-24T08:03:55.953376+0000","last_fullsized":"2023-11-24T08:03:55.953376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994403+0000","last_change":"2023-11-24T08:03:55.244940+0000","last_active":"2023-11-24T08:03:56.994403+0000","last_peered":"2023-11-24T08:03:56.994403+0000","last_clean":"2023-11-24T08:03:56.994403+0000","last_became_active":"2023-11-24T08:03:55.243981+0000","last_became_peered":"2023-11-24T08:03:55.243981+0000","last_unstale":"2023-11-24T08:03:56.994403+0000","last_undegraded":"2023-11-24T08:03:56.994403+0000","last_fullsized":"2023-11-24T08:03:56.994403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:30.740161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953049+0000","last_change":"2023-11-24T08:03:55.969277+0000","last_active":"2023-11-24T08:03:56.953049+0000","last_peered":"2023-11-24T08:03:56.953049+0000","last_clean":"2023-11-24T08:03:56.953049+0000","last_became_active":"2023-11-24T08:03:55.968980+0000","last_became_peered":"2023-11-24T08:03:55.968980+0000","last_unstale":"2023-11-24T08:03:56.953049+0000","last_undegraded":"2023-11-24T08:03:56.953049+0000","last_fullsized":"2023-11-24T08:03:56.953049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:39.832606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263854+0000","last_change":"2023-11-24T08:03:55.255880+0000","last_active":"2023-11-24T08:03:57.263854+0000","last_peered":"2023-11-24T08:03:57.263854+0000","last_clean":"2023-11-24T08:03:57.263854+0000","last_became_active":"2023-11-24T08:03:55.255584+0000","last_became_peered":"2023-11-24T08:03:55.255584+0000","last_unstale":"2023-11-24T08:03:57.263854+0000","last_undegraded":"2023-11-24T08:03:57.263854+0000","last_fullsized":"2023-11-24T08:03:57.263854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.458202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448244+0000","last_change":"2023-11-24T08:03:55.972919+0000","last_active":"2023-11-24T08:03:57.448244+0000","last_peered":"2023-11-24T08:03:57.448244+0000","last_clean":"2023-11-24T08:03:57.448244+0000","last_became_active":"2023-11-24T08:03:55.972754+0000","last_became_peered":"2023-11-24T08:03:55.972754+0000","last_unstale":"2023-11-24T08:03:57.448244+0000","last_undegraded":"2023-11-24T08:03:57.448244+0000","last_fullsized":"2023-11-24T08:03:57.448244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:01.646788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.445758+0000","last_change":"2023-11-24T08:03:55.237610+0000","last_active":"2023-11-24T08:03:57.445758+0000","last_peered":"2023-11-24T08:03:57.445758+0000","last_clean":"2023-11-24T08:03:57.445758+0000","last_became_active":"2023-11-24T08:03:55.237273+0000","last_became_peered":"2023-11-24T08:03:55.237273+0000","last_unstale":"2023-11-24T08:03:57.445758+0000","last_undegraded":"2023-11-24T08:03:57.445758+0000","last_fullsized":"2023-11-24T08:03:57.445758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:24.389964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949322+0000","last_change":"2023-11-24T08:03:55.965491+0000","last_active":"2023-11-24T08:03:56.949322+0000","last_peered":"2023-11-24T08:03:56.949322+0000","last_clean":"2023-11-24T08:03:56.949322+0000","last_became_active":"2023-11-24T08:03:55.965148+0000","last_became_peered":"2023-11-24T08:03:55.965148+0000","last_unstale":"2023-11-24T08:03:56.949322+0000","last_undegraded":"2023-11-24T08:03:56.949322+0000","last_fullsized":"2023-11-24T08:03:56.949322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:28.241727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994656+0000","last_change":"2023-11-24T08:03:55.246876+0000","last_active":"2023-11-24T08:03:56.994656+0000","last_peered":"2023-11-24T08:03:56.994656+0000","last_clean":"2023-11-24T08:03:56.994656+0000","last_became_active":"2023-11-24T08:03:55.243530+0000","last_became_peered":"2023-11-24T08:03:55.243530+0000","last_unstale":"2023-11-24T08:03:56.994656+0000","last_undegraded":"2023-11-24T08:03:56.994656+0000","last_fullsized":"2023-11-24T08:03:56.994656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:27.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953302+0000","last_change":"2023-11-24T08:03:55.975512+0000","last_active":"2023-11-24T08:03:56.953302+0000","last_peered":"2023-11-24T08:03:56.953302+0000","last_clean":"2023-11-24T08:03:56.953302+0000","last_became_active":"2023-11-24T08:03:55.975099+0000","last_became_peered":"2023-11-24T08:03:55.975099+0000","last_unstale":"2023-11-24T08:03:56.953302+0000","last_undegraded":"2023-11-24T08:03:56.953302+0000","last_fullsized":"2023-11-24T08:03:56.953302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:25.402792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949417+0000","last_change":"2023-11-24T08:03:55.236722+0000","last_active":"2023-11-24T08:03:56.949417+0000","last_peered":"2023-11-24T08:03:56.949417+0000","last_clean":"2023-11-24T08:03:56.949417+0000","last_became_active":"2023-11-24T08:03:55.236383+0000","last_became_peered":"2023-11-24T08:03:55.236383+0000","last_unstale":"2023-11-24T08:03:56.949417+0000","last_undegraded":"2023-11-24T08:03:56.949417+0000","last_fullsized":"2023-11-24T08:03:56.949417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.995298+0000","last_change":"2023-11-24T08:03:55.978747+0000","last_active":"2023-11-24T08:03:56.995298+0000","last_peered":"2023-11-24T08:03:56.995298+0000","last_clean":"2023-11-24T08:03:56.995298+0000","last_became_active":"2023-11-24T08:03:55.978454+0000","last_became_peered":"2023-11-24T08:03:55.978454+0000","last_unstale":"2023-11-24T08:03:56.995298+0000","last_undegraded":"2023-11-24T08:03:56.995298+0000","last_fullsized":"2023-11-24T08:03:56.995298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:03.856484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264892+0000","last_change":"2023-11-24T08:03:55.255119+0000","last_active":"2023-11-24T08:03:57.264892+0000","last_peered":"2023-11-24T08:03:57.264892+0000","last_clean":"2023-11-24T08:03:57.264892+0000","last_became_active":"2023-11-24T08:03:55.254773+0000","last_became_peered":"2023-11-24T08:03:55.254773+0000","last_unstale":"2023-11-24T08:03:57.264892+0000","last_undegraded":"2023-11-24T08:03:57.264892+0000","last_fullsized":"2023-11-24T08:03:57.264892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:27.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949334+0000","last_change":"2023-11-24T08:03:55.969522+0000","last_active":"2023-11-24T08:03:56.949334+0000","last_peered":"2023-11-24T08:03:56.949334+0000","last_clean":"2023-11-24T08:03:56.949334+0000","last_became_active":"2023-11-24T08:03:55.968807+0000","last_became_peered":"2023-11-24T08:03:55.968807+0000","last_unstale":"2023-11-24T08:03:56.949334+0000","last_undegraded":"2023-11-24T08:03:56.949334+0000","last_fullsized":"2023-11-24T08:03:56.949334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:07.768487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:59.052521+0000","last_change":"2023-11-24T08:03:55.250820+0000","last_active":"2023-11-24T08:03:59.052521+0000","last_peered":"2023-11-24T08:03:59.052521+0000","last_clean":"2023-11-24T08:03:59.052521+0000","last_became_active":"2023-11-24T08:03:55.250479+0000","last_became_peered":"2023-11-24T08:03:55.250479+0000","last_unstale":"2023-11-24T08:03:59.052521+0000","last_undegraded":"2023-11-24T08:03:59.052521+0000","last_fullsized":"2023-11-24T08:03:59.052521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953296+0000","last_change":"2023-11-24T08:03:55.970065+0000","last_active":"2023-11-24T08:03:56.953296+0000","last_peered":"2023-11-24T08:03:56.953296+0000","last_clean":"2023-11-24T08:03:56.953296+0000","last_became_active":"2023-11-24T08:03:55.969811+0000","last_became_peered":"2023-11-24T08:03:55.969811+0000","last_unstale":"2023-11-24T08:03:56.953296+0000","last_undegraded":"2023-11-24T08:03:56.953296+0000","last_fullsized":"2023-11-24T08:03:56.953296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:14.117187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449201+0000","last_change":"2023-11-24T08:03:55.256073+0000","last_active":"2023-11-24T08:03:57.449201+0000","last_peered":"2023-11-24T08:03:57.449201+0000","last_clean":"2023-11-24T08:03:57.449201+0000","last_became_active":"2023-11-24T08:03:55.255797+0000","last_became_peered":"2023-11-24T08:03:55.255797+0000","last_unstale":"2023-11-24T08:03:57.449201+0000","last_undegraded":"2023-11-24T08:03:57.449201+0000","last_fullsized":"2023-11-24T08:03:57.449201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:24.389964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449472+0000","last_change":"2023-11-24T08:03:55.971989+0000","last_active":"2023-11-24T08:03:57.449472+0000","last_peered":"2023-11-24T08:03:57.449472+0000","last_clean":"2023-11-24T08:03:57.449472+0000","last_became_active":"2023-11-24T08:03:55.971690+0000","last_became_peered":"2023-11-24T08:03:55.971690+0000","last_unstale":"2023-11-24T08:03:57.449472+0000","last_undegraded":"2023-11-24T08:03:57.449472+0000","last_fullsized":"2023-11-24T08:03:57.449472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:20.046916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.263829+0000","last_change":"2023-11-24T08:03:55.252987+0000","last_active":"2023-11-24T08:03:57.263829+0000","last_peered":"2023-11-24T08:03:57.263829+0000","last_clean":"2023-11-24T08:03:57.263829+0000","last_became_active":"2023-11-24T08:03:55.252708+0000","last_became_peered":"2023-11-24T08:03:55.252708+0000","last_unstale":"2023-11-24T08:03:57.263829+0000","last_undegraded":"2023-11-24T08:03:57.263829+0000","last_fullsized":"2023-11-24T08:03:57.263829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:34.443628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953362+0000","last_change":"2023-11-24T08:03:55.970570+0000","last_active":"2023-11-24T08:03:56.953362+0000","last_peered":"2023-11-24T08:03:56.953362+0000","last_clean":"2023-11-24T08:03:56.953362+0000","last_became_active":"2023-11-24T08:03:55.969759+0000","last_became_peered":"2023-11-24T08:03:55.969759+0000","last_unstale":"2023-11-24T08:03:56.953362+0000","last_undegraded":"2023-11-24T08:03:56.953362+0000","last_fullsized":"2023-11-24T08:03:56.953362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:04.042778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949401+0000","last_change":"2023-11-24T08:03:55.243962+0000","last_active":"2023-11-24T08:03:56.949401+0000","last_peered":"2023-11-24T08:03:56.949401+0000","last_clean":"2023-11-24T08:03:56.949401+0000","last_became_active":"2023-11-24T08:03:55.243548+0000","last_became_peered":"2023-11-24T08:03:55.243548+0000","last_unstale":"2023-11-24T08:03:56.949401+0000","last_undegraded":"2023-11-24T08:03:56.949401+0000","last_fullsized":"2023-11-24T08:03:56.949401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:59.543627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.952949+0000","last_change":"2023-11-24T08:03:55.960318+0000","last_active":"2023-11-24T08:03:56.952949+0000","last_peered":"2023-11-24T08:03:56.952949+0000","last_clean":"2023-11-24T08:03:56.952949+0000","last_became_active":"2023-11-24T08:03:55.957342+0000","last_became_peered":"2023-11-24T08:03:55.957342+0000","last_unstale":"2023-11-24T08:03:56.952949+0000","last_undegraded":"2023-11-24T08:03:56.952949+0000","last_fullsized":"2023-11-24T08:03:56.952949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:31.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.949250+0000","last_change":"2023-11-24T08:03:55.234991+0000","last_active":"2023-11-24T08:03:56.949250+0000","last_peered":"2023-11-24T08:03:56.949250+0000","last_clean":"2023-11-24T08:03:56.949250+0000","last_became_active":"2023-11-24T08:03:55.234480+0000","last_became_peered":"2023-11-24T08:03:55.234480+0000","last_unstale":"2023-11-24T08:03:56.949250+0000","last_undegraded":"2023-11-24T08:03:56.949250+0000","last_fullsized":"2023-11-24T08:03:56.949250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:06.198310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953403+0000","last_change":"2023-11-24T08:03:55.974916+0000","last_active":"2023-11-24T08:03:56.953403+0000","last_peered":"2023-11-24T08:03:56.953403+0000","last_clean":"2023-11-24T08:03:56.953403+0000","last_became_active":"2023-11-24T08:03:55.974533+0000","last_became_peered":"2023-11-24T08:03:55.974533+0000","last_unstale":"2023-11-24T08:03:56.953403+0000","last_undegraded":"2023-11-24T08:03:56.953403+0000","last_fullsized":"2023-11-24T08:03:56.953403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:55.841729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264803+0000","last_change":"2023-11-24T08:03:55.258218+0000","last_active":"2023-11-24T08:03:57.264803+0000","last_peered":"2023-11-24T08:03:57.264803+0000","last_clean":"2023-11-24T08:03:57.264803+0000","last_became_active":"2023-11-24T08:03:55.257904+0000","last_became_peered":"2023-11-24T08:03:55.257904+0000","last_unstale":"2023-11-24T08:03:57.264803+0000","last_undegraded":"2023-11-24T08:03:57.264803+0000","last_fullsized":"2023-11-24T08:03:57.264803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:42.858025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.448404+0000","last_change":"2023-11-24T08:03:55.965022+0000","last_active":"2023-11-24T08:03:57.448404+0000","last_peered":"2023-11-24T08:03:57.448404+0000","last_clean":"2023-11-24T08:03:57.448404+0000","last_became_active":"2023-11-24T08:03:55.964652+0000","last_became_peered":"2023-11-24T08:03:55.964652+0000","last_unstale":"2023-11-24T08:03:57.448404+0000","last_undegraded":"2023-11-24T08:03:57.448404+0000","last_fullsized":"2023-11-24T08:03:57.448404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:48.127017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994904+0000","last_change":"2023-11-24T08:03:55.980852+0000","last_active":"2023-11-24T08:03:56.994904+0000","last_peered":"2023-11-24T08:03:56.994904+0000","last_clean":"2023-11-24T08:03:56.994904+0000","last_became_active":"2023-11-24T08:03:55.980467+0000","last_became_peered":"2023-11-24T08:03:55.980467+0000","last_unstale":"2023-11-24T08:03:56.994904+0000","last_undegraded":"2023-11-24T08:03:56.994904+0000","last_fullsized":"2023-11-24T08:03:56.994904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264433+0000","last_change":"2023-11-24T08:03:55.254837+0000","last_active":"2023-11-24T08:03:57.264433+0000","last_peered":"2023-11-24T08:03:57.264433+0000","last_clean":"2023-11-24T08:03:57.264433+0000","last_became_active":"2023-11-24T08:03:55.254593+0000","last_became_peered":"2023-11-24T08:03:55.254593+0000","last_unstale":"2023-11-24T08:03:57.264433+0000","last_undegraded":"2023-11-24T08:03:57.264433+0000","last_fullsized":"2023-11-24T08:03:57.264433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:47.114189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.994917+0000","last_change":"2023-11-24T08:03:55.985005+0000","last_active":"2023-11-24T08:03:56.994917+0000","last_peered":"2023-11-24T08:03:56.994917+0000","last_clean":"2023-11-24T08:03:56.994917+0000","last_became_active":"2023-11-24T08:03:55.984533+0000","last_became_peered":"2023-11-24T08:03:55.984533+0000","last_unstale":"2023-11-24T08:03:56.994917+0000","last_undegraded":"2023-11-24T08:03:56.994917+0000","last_fullsized":"2023-11-24T08:03:56.994917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:14.268136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.264754+0000","last_change":"2023-11-24T08:03:55.257482+0000","last_active":"2023-11-24T08:03:57.264754+0000","last_peered":"2023-11-24T08:03:57.264754+0000","last_clean":"2023-11-24T08:03:57.264754+0000","last_became_active":"2023-11-24T08:03:55.257267+0000","last_became_peered":"2023-11-24T08:03:55.257267+0000","last_unstale":"2023-11-24T08:03:57.264754+0000","last_undegraded":"2023-11-24T08:03:57.264754+0000","last_fullsized":"2023-11-24T08:03:57.264754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:40.782425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449374+0000","last_change":"2023-11-24T08:03:55.981373+0000","last_active":"2023-11-24T08:03:57.449374+0000","last_peered":"2023-11-24T08:03:57.449374+0000","last_clean":"2023-11-24T08:03:57.449374+0000","last_became_active":"2023-11-24T08:03:55.980504+0000","last_became_peered":"2023-11-24T08:03:55.980504+0000","last_unstale":"2023-11-24T08:03:57.449374+0000","last_undegraded":"2023-11-24T08:03:57.449374+0000","last_fullsized":"2023-11-24T08:03:57.449374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:19.149318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953650+0000","last_change":"2023-11-24T08:03:55.240395+0000","last_active":"2023-11-24T08:03:56.953650+0000","last_peered":"2023-11-24T08:03:56.953650+0000","last_clean":"2023-11-24T08:03:56.953650+0000","last_became_active":"2023-11-24T08:03:55.240021+0000","last_became_peered":"2023-11-24T08:03:55.240021+0000","last_unstale":"2023-11-24T08:03:56.953650+0000","last_undegraded":"2023-11-24T08:03:56.953650+0000","last_fullsized":"2023-11-24T08:03:56.953650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:00.633960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:56.953580+0000","last_change":"2023-11-24T08:03:55.971086+0000","last_active":"2023-11-24T08:03:56.953580+0000","last_peered":"2023-11-24T08:03:56.953580+0000","last_clean":"2023-11-24T08:03:56.953580+0000","last_became_active":"2023-11-24T08:03:55.970765+0000","last_became_peered":"2023-11-24T08:03:55.970765+0000","last_unstale":"2023-11-24T08:03:56.953580+0000","last_undegraded":"2023-11-24T08:03:56.953580+0000","last_fullsized":"2023-11-24T08:03:56.953580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:55.204176+0000","last_clean_scrub_stamp":"2023-11-24T08:03:55.204176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:08.636374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T08:03:57.449518+0000","last_change":"2023-11-24T08:03:55.252498+0000","last_active":"2023-11-24T08:03:57.449518+0000","last_peered":"2023-11-24T08:03:57.449518+0000","last_clean":"2023-11-24T08:03:57.449518+0000","last_became_active":"2023-11-24T08:03:55.252172+0000","last_became_peered":"2023-11-24T08:03:55.252172+0000","last_unstale":"2023-11-24T08:03:57.449518+0000","last_undegraded":"2023-11-24T08:03:57.449518+0000","last_fullsized":"2023-11-24T08:03:57.449518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:54.191347+0000","last_clean_scrub_stamp":"2023-11-24T08:03:54.191347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.458202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.466}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8979999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.798}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2480000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.516}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.181}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.786}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7519999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4009999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.468}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1970000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0539999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1829999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.302}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.101}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:04:06.984 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T08:04:06.984 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T08:04:06.984 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T08:04:06.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T08:04:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.238+0000 7fbff71b3640 1 Processor -- start 2023-11-24T08:04:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.239+0000 7fbff71b3640 1 -- start start 2023-11-24T08:04:07.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.240+0000 7fbff71b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbff013dfd0 0x7fbff015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.240+0000 7fbff71b3640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff00ce7c0 con 0x7fbff00cf4d0 2023-11-24T08:04:07.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.241+0000 7fbff71b3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff0140900 con 0x7fbff00cf050 2023-11-24T08:04:07.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.241+0000 7fbff71b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff006cf50 con 0x7fbff013dfd0 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.241+0000 7fbff69b2640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbff00cf4d0 0x7fbff006c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43086/0 (socket says 172.21.15.64:43086) 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.241+0000 7fbff69b2640 1 -- 172.21.15.64:0/641625350 learned_addr learned my addr 172.21.15.64:0/641625350 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.241+0000 7fbff61b1640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbff013dfd0 0x7fbff015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.241+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683951617 0 0) 0x7fbff00ce7c0 con 0x7fbff00cf4d0 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfdc003520 con 0x7fbff00cf4d0 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408834600 0 0) 0x7fbfdc003520 con 0x7fbff00cf4d0 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbff013dfd0 msgr2=0x7fbff015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbff013dfd0 0x7fbff015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 >> v1:172.21.15.91:6789/0 conn(0x7fbff00cf050 legacy=0x7fbff006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff015fa70 con 0x7fbff00cf4d0 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fbfe80019f0 con 0x7fbff00cf4d0 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 >> v1:172.21.15.64:6789/0 conn(0x7fbff00cf4d0 legacy=0x7fbff006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbff013dfd0 0x7fbfdc003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfdc003f50 0x7fbfdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfdc004d00 0x7fbfdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff006cf50 con 0x7fbfdc004d00 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc00ba60 con 0x7fbfdc003f50 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc00bc20 con 0x7fbff013dfd0 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.242+0000 7fbff69b2640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfdc004d00 0x7fbfdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.243+0000 7fbff59b0640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfdc003f50 0x7fbfdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.243+0000 7fbff61b1640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbff013dfd0 0x7fbfdc003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.243+0000 7fbff69b2640 1 -- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbff013dfd0 msgr2=0x7fbfdc003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.243+0000 7fbff69b2640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbff013dfd0 0x7fbfdc003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.243+0000 7fbff69b2640 1 -- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfdc003f50 msgr2=0x7fbfdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.243+0000 7fbff69b2640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfdc003f50 0x7fbfdc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.243+0000 7fbff69b2640 1 -- 172.21.15.64:0/641625350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc003520 con 0x7fbfdc004d00 2023-11-24T08:04:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.243+0000 7fbff59b0640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbfdc003f50 0x7fbfdc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.244+0000 7fbff69b2640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfdc004d00 0x7fbfdc005180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800a820 tx=0x7fbfdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d991fa6e981023 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:07.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.244+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80072d0 con 0x7fbfdc004d00 2023-11-24T08:04:07.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.244+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe802f070 con 0x7fbfdc004d00 2023-11-24T08:04:07.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.244+0000 7fbff51af640 1 -- 172.21.15.64:0/641625350 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8007690 con 0x7fbfdc004d00 2023-11-24T08:04:07.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.245+0000 7fbff71b3640 1 -- 172.21.15.64:0/641625350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfdc004d00 msgr2=0x7fbfdc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.245+0000 7fbff71b3640 1 --2- 172.21.15.64:0/641625350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfdc004d00 0x7fbfdc005180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800a820 tx=0x7fbfdc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.245+0000 7fbff69b2640 1 -- 172.21.15.64:0/641625350 reap_dead start 2023-11-24T08:04:07.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.245+0000 7fbff71b3640 1 -- 172.21.15.64:0/641625350 shutdown_connections 2023-11-24T08:04:07.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.245+0000 7fbff71b3640 1 -- 172.21.15.64:0/641625350 >> 172.21.15.64:0/641625350 conn(0x7fbff006a860 msgr2=0x7fbff006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:07.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.245+0000 7fbff71b3640 1 -- 172.21.15.64:0/641625350 shutdown_connections 2023-11-24T08:04:07.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.245+0000 7fbff71b3640 1 -- 172.21.15.64:0/641625350 wait complete. 2023-11-24T08:04:07.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.247+0000 7fbff71b3640 1 Processor -- start 2023-11-24T08:04:07.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.247+0000 7fbff71b3640 1 -- start start 2023-11-24T08:04:07.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff71b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbfdc004d00 0x7fbff0157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff71b3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff01583d0 0x7fbff01545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff71b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff015dab0 0x7fbff015df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff71b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff015e470 con 0x7fbff015dab0 2023-11-24T08:04:07.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff71b3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff0156610 con 0x7fbff01583d0 2023-11-24T08:04:07.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff71b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff0154b20 con 0x7fbfdc004d00 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff69b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff015dab0 0x7fbff015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff69b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff015dab0 0x7fbff015df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42316/0 (socket says 172.21.15.64:42316) 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff69b2640 1 -- 172.21.15.64:0/1153680516 learned_addr learned my addr 172.21.15.64:0/1153680516 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.248+0000 7fbff59b0640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff01583d0 0x7fbff01545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbff61b1640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbfdc004d00 0x7fbff0157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbff69b2640 1 -- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbfdc004d00 msgr2=0x7fbff0157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbff69b2640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbfdc004d00 0x7fbff0157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbff69b2640 1 -- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff01583d0 msgr2=0x7fbff01545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbff69b2640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff01583d0 0x7fbff01545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbff69b2640 1 -- 172.21.15.64:0/1153680516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff01588d0 con 0x7fbff015dab0 2023-11-24T08:04:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbff69b2640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff015dab0 0x7fbff015df30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfe802cb50 tx=0x7fbff0140b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbfe6ffd640 1 -- 172.21.15.64:0/1153680516 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80074e0 con 0x7fbff015dab0 2023-11-24T08:04:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbfe6ffd640 1 -- 172.21.15.64:0/1153680516 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe802ce90 con 0x7fbff015dab0 2023-11-24T08:04:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbfe6ffd640 1 -- 172.21.15.64:0/1153680516 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8007840 con 0x7fbff015dab0 2023-11-24T08:04:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.249+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff0156860 con 0x7fbff015dab0 2023-11-24T08:04:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.250+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff0156e20 con 0x7fbff015dab0 2023-11-24T08:04:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.251+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff004e840 con 0x7fbff015dab0 2023-11-24T08:04:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.252+0000 7fbfe6ffd640 1 -- 172.21.15.64:0/1153680516 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbfe8031030 con 0x7fbff015dab0 2023-11-24T08:04:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.253+0000 7fbfe6ffd640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbfd406ae40 0x7fbfd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.253+0000 7fbfe6ffd640 1 -- 172.21.15.64:0/1153680516 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfe80ab1f0 con 0x7fbff015dab0 2023-11-24T08:04:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.254+0000 7fbff61b1640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbfd406ae40 0x7fbfd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.255+0000 7fbff61b1640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbfd406ae40 0x7fbfd406d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009a10 tx=0x7fbfec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.257+0000 7fbfe6ffd640 1 -- 172.21.15.64:0/1153680516 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbfe8076200 con 0x7fbff015dab0 2023-11-24T08:04:07.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.494+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbff00ce9a0 con 0x7fbff015dab0 2023-11-24T08:04:07.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.495+0000 7fbfe6ffd640 1 -- 172.21.15.64:0/1153680516 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbfe8001f90 con 0x7fbff015dab0 2023-11-24T08:04:07.497 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:07.497 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.497+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbfd406ae40 msgr2=0x7fbfd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.497+0000 7fbff71b3640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbfd406ae40 0x7fbfd406d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009a10 tx=0x7fbfec008040 comp rx=0 tx=0).stop 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.497+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff015dab0 msgr2=0x7fbff015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.497+0000 7fbff71b3640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff015dab0 0x7fbff015df30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfe802cb50 tx=0x7fbff0140b50 comp rx=0 tx=0).stop 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.498+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 shutdown_connections 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.498+0000 7fbff71b3640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbfd406ae40 0x7fbfd406d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.498+0000 7fbff71b3640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbff015dab0 0x7fbff015df30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.498+0000 7fbff71b3640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbff01583d0 0x7fbff01545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.498+0000 7fbff71b3640 1 --2- 172.21.15.64:0/1153680516 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbfdc004d00 0x7fbff0157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.498+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 >> 172.21.15.64:0/1153680516 conn(0x7fbff006a860 msgr2=0x7fbff015ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.498+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 shutdown_connections 2023-11-24T08:04:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.498+0000 7fbff71b3640 1 -- 172.21.15.64:0/1153680516 wait complete. 2023-11-24T08:04:07.520 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T08:04:07.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:07.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.741+0000 7f8ba6def640 1 Processor -- start 2023-11-24T08:04:07.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.742+0000 7f8ba6def640 1 -- start start 2023-11-24T08:04:07.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.743+0000 7f8ba6def640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ba00afd30 0x7f8ba00cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.743+0000 7f8ba6def640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 0x7f8ba00b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.743+0000 7f8ba6def640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba00b2b30 con 0x7f8ba0160b60 2023-11-24T08:04:07.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.743+0000 7f8ba6def640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba00d0460 con 0x7f8ba00afd30 2023-11-24T08:04:07.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.743+0000 7f8ba6def640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba0160f40 con 0x7f8ba00cffd0 2023-11-24T08:04:07.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.744+0000 7f8ba55ec640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ba00afd30 0x7f8ba00cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.744+0000 7f8ba55ec640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ba00afd30 0x7f8ba00cfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:35140/0 (socket says 172.21.15.64:35140) 2023-11-24T08:04:07.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.744+0000 7f8ba55ec640 1 -- 172.21.15.64:0/4133750620 learned_addr learned my addr 172.21.15.64:0/4133750620 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:07.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.744+0000 7f8ba65ee640 1 --2- 172.21.15.64:0/4133750620 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 0x7f8ba00b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.745+0000 7f8ba55ec640 1 -- 172.21.15.64:0/4133750620 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 msgr2=0x7f8ba00b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:07.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.745+0000 7f8ba55ec640 1 --2- 172.21.15.64:0/4133750620 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 0x7f8ba00b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.745+0000 7f8ba55ec640 1 -- 172.21.15.64:0/4133750620 >> v1:172.21.15.91:6789/0 conn(0x7f8ba0160b60 legacy=0x7f8ba00d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.745+0000 7f8ba55ec640 1 -- 172.21.15.64:0/4133750620 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba00b2e10 con 0x7f8ba00afd30 2023-11-24T08:04:07.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.746+0000 7f8ba55ec640 1 --2- 172.21.15.64:0/4133750620 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 0x7f8ba00cfa90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009870 tx=0x7f8b9402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e03b07b909984a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:07.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.747+0000 7f8ba4deb640 1 -- 172.21.15.64:0/4133750620 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94031640 con 0x7f8ba00afd30 2023-11-24T08:04:07.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.747+0000 7f8ba4deb640 1 -- 172.21.15.64:0/4133750620 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b940317a0 con 0x7f8ba00afd30 2023-11-24T08:04:07.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.747+0000 7f8ba4deb640 1 -- 172.21.15.64:0/4133750620 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94031b30 con 0x7f8ba00afd30 2023-11-24T08:04:07.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.747+0000 7f8ba6def640 1 -- 172.21.15.64:0/4133750620 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 msgr2=0x7f8ba00cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.747+0000 7f8ba6def640 1 --2- 172.21.15.64:0/4133750620 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 0x7f8ba00cfa90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009870 tx=0x7f8b9402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.748+0000 7f8ba6def640 1 -- 172.21.15.64:0/4133750620 shutdown_connections 2023-11-24T08:04:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.748+0000 7f8ba6def640 1 --2- 172.21.15.64:0/4133750620 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 0x7f8ba00b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.748+0000 7f8ba6def640 1 --2- 172.21.15.64:0/4133750620 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 0x7f8ba00cfa90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.748+0000 7f8ba6def640 1 -- 172.21.15.64:0/4133750620 >> 172.21.15.64:0/4133750620 conn(0x7f8ba009c3d0 msgr2=0x7f8ba00d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.748+0000 7f8ba6def640 1 -- 172.21.15.64:0/4133750620 shutdown_connections 2023-11-24T08:04:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.748+0000 7f8ba6def640 1 -- 172.21.15.64:0/4133750620 wait complete. 2023-11-24T08:04:07.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.749+0000 7f8ba6def640 1 Processor -- start 2023-11-24T08:04:07.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.750+0000 7f8ba6def640 1 -- start start 2023-11-24T08:04:07.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.750+0000 7f8ba6def640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 0x7f8ba00ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.750+0000 7f8ba6def640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 0x7f8ba00ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.750+0000 7f8ba6def640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ba0160b60 0x7f8ba0153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.750+0000 7f8ba6def640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00cc090 con 0x7f8ba0160b60 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.750+0000 7f8ba6def640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00cadc0 con 0x7f8ba00afd30 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.750+0000 7f8ba6def640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00caff0 con 0x7f8ba00cffd0 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba65ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ba0160b60 0x7f8ba0153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba65ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ba0160b60 0x7f8ba0153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42326/0 (socket says 172.21.15.64:42326) 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba65ee640 1 -- 172.21.15.64:0/1117859419 learned_addr learned my addr 172.21.15.64:0/1117859419 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba5ded640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 0x7f8ba00ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba55ec640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 0x7f8ba00ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba65ee640 1 -- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 msgr2=0x7f8ba00ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba65ee640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 0x7f8ba00ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba65ee640 1 -- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 msgr2=0x7f8ba00ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba65ee640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 0x7f8ba00ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba65ee640 1 -- 172.21.15.64:0/1117859419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b940094d0 con 0x7f8ba0160b60 2023-11-24T08:04:07.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.751+0000 7f8ba5ded640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 0x7f8ba00ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:07.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.752+0000 7f8ba65ee640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ba0160b60 0x7f8ba0153db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800efc0 tx=0x7f8b9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:07.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.752+0000 7f8b9e7fc640 1 -- 172.21.15.64:0/1117859419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98014480 con 0x7f8ba0160b60 2023-11-24T08:04:07.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.752+0000 7f8b9e7fc640 1 -- 172.21.15.64:0/1117859419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9800f040 con 0x7f8ba0160b60 2023-11-24T08:04:07.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.752+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba00cb2d0 con 0x7f8ba0160b60 2023-11-24T08:04:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.752+0000 7f8b9e7fc640 1 -- 172.21.15.64:0/1117859419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98014820 con 0x7f8ba0160b60 2023-11-24T08:04:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.752+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba00cb890 con 0x7f8ba0160b60 2023-11-24T08:04:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.753+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba004e840 con 0x7f8ba0160b60 2023-11-24T08:04:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.754+0000 7f8b9e7fc640 1 -- 172.21.15.64:0/1117859419 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b98014980 con 0x7f8ba0160b60 2023-11-24T08:04:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.755+0000 7f8b9e7fc640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b8c06ae70 0x7f8b8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.755+0000 7f8b9e7fc640 1 -- 172.21.15.64:0/1117859419 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b9808a500 con 0x7f8ba0160b60 2023-11-24T08:04:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.756+0000 7f8ba5ded640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b8c06ae70 0x7f8b8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.757+0000 7f8ba5ded640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b8c06ae70 0x7f8b8c06d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b90007fd0 tx=0x7f8b90009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:07.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.760+0000 7f8b9e7fc640 1 -- 172.21.15.64:0/1117859419 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b98055510 con 0x7f8ba0160b60 2023-11-24T08:04:07.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.960+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ba00d3e90 con 0x7f8ba0160b60 2023-11-24T08:04:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.963+0000 7f8b9e7fc640 1 -- 172.21.15.64:0/1117859419 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f8b98058b30 con 0x7f8ba0160b60 2023-11-24T08:04:07.965 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:07.965 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:07.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b8c06ae70 msgr2=0x7f8b8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b8c06ae70 0x7f8b8c06d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b90007fd0 tx=0x7f8b90009040 comp rx=0 tx=0).stop 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ba0160b60 msgr2=0x7f8ba0153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ba0160b60 0x7f8ba0153db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800efc0 tx=0x7f8b9800c490 comp rx=0 tx=0).stop 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 shutdown_connections 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b8c06ae70 0x7f8b8c06d330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ba0160b60 0x7f8ba0153db0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ba00cffd0 0x7f8ba00ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 --2- 172.21.15.64:0/1117859419 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ba00afd30 0x7f8ba00ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.965+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 >> 172.21.15.64:0/1117859419 conn(0x7f8ba009c3d0 msgr2=0x7f8ba015f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.966+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 shutdown_connections 2023-11-24T08:04:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:07.966+0000 7f8ba6def640 1 -- 172.21.15.64:0/1117859419 wait complete. 2023-11-24T08:04:07.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:07.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:08.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.214+0000 7f2d30a32640 1 Processor -- start 2023-11-24T08:04:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.215+0000 7f2d30a32640 1 -- start start 2023-11-24T08:04:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.216+0000 7f2d30a32640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d2c15d3b0 0x7f2d2c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.216+0000 7f2d30a32640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.216+0000 7f2d30a32640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c0c6bf0 con 0x7f2d2c0d0fa0 2023-11-24T08:04:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.216+0000 7f2d30a32640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c0d1430 con 0x7f2d2c15d3b0 2023-11-24T08:04:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.216+0000 7f2d30a32640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0cedc0 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.217+0000 7f2d2b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d2c15d3b0 0x7f2d2c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.217+0000 7f2d2b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d2c15d3b0 0x7f2d2c0ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42338/0 (socket says 172.21.15.64:42338) 2023-11-24T08:04:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.217+0000 7f2d2b7fe640 1 -- 172.21.15.64:0/70586652 learned_addr learned my addr 172.21.15.64:0/70586652 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.217+0000 7f2d2bfff640 1 --2- 172.21.15.64:0/70586652 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.217+0000 7f2d2b7fe640 1 -- 172.21.15.64:0/70586652 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c0d1d90 msgr2=0x7f2d2c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.217+0000 7f2d2b7fe640 1 --2- 172.21.15.64:0/70586652 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.217+0000 7f2d2b7fe640 1 -- 172.21.15.64:0/70586652 >> v1:172.21.15.91:6789/0 conn(0x7f2d2c0d0fa0 legacy=0x7f2d2c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.217+0000 7f2d2b7fe640 1 -- 172.21.15.64:0/70586652 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c0c6e40 con 0x7f2d2c15d3b0 2023-11-24T08:04:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.218+0000 7f2d2b7fe640 1 --2- 172.21.15.64:0/70586652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c15d3b0 0x7f2d2c0ce6c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009870 tx=0x7f2d1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239362e3b435a80d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.219+0000 7f2d2affd640 1 -- 172.21.15.64:0/70586652 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c031640 con 0x7f2d2c15d3b0 2023-11-24T08:04:08.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.219+0000 7f2d2affd640 1 -- 172.21.15.64:0/70586652 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c0317a0 con 0x7f2d2c15d3b0 2023-11-24T08:04:08.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.219+0000 7f2d2affd640 1 -- 172.21.15.64:0/70586652 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c031b30 con 0x7f2d2c15d3b0 2023-11-24T08:04:08.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.219+0000 7f2d30a32640 1 -- 172.21.15.64:0/70586652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c15d3b0 msgr2=0x7f2d2c0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.219+0000 7f2d30a32640 1 --2- 172.21.15.64:0/70586652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c15d3b0 0x7f2d2c0ce6c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009870 tx=0x7f2d1c02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.220+0000 7f2d30a32640 1 -- 172.21.15.64:0/70586652 shutdown_connections 2023-11-24T08:04:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.220+0000 7f2d30a32640 1 --2- 172.21.15.64:0/70586652 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.220+0000 7f2d30a32640 1 --2- 172.21.15.64:0/70586652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c15d3b0 0x7f2d2c0ce6c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.220+0000 7f2d30a32640 1 -- 172.21.15.64:0/70586652 >> 172.21.15.64:0/70586652 conn(0x7f2d2c0d30a0 msgr2=0x7f2d2c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.220+0000 7f2d30a32640 1 -- 172.21.15.64:0/70586652 shutdown_connections 2023-11-24T08:04:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.220+0000 7f2d30a32640 1 -- 172.21.15.64:0/70586652 wait complete. 2023-11-24T08:04:08.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.221+0000 7f2d30a32640 1 Processor -- start 2023-11-24T08:04:08.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.222+0000 7f2d30a32640 1 -- start start 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d30a32640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d2c0d0fa0 0x7f2d2c152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d30a32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d30a32640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c15d3b0 0x7f2d2c0b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d30a32640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0a0420 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d30a32640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c09fe60 con 0x7f2d2c0d0fa0 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d30a32640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0a0140 con 0x7f2d2c15d3b0 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42340/0 (socket says 172.21.15.64:42340) 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d23fff640 1 -- 172.21.15.64:0/2896212796 learned_addr learned my addr 172.21.15.64:0/2896212796 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d2b7fe640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d2c0d0fa0 0x7f2d2c152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d2bfff640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c15d3b0 0x7f2d2c0b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d23fff640 1 -- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c15d3b0 msgr2=0x7f2d2c0b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d23fff640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c15d3b0 0x7f2d2c0b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d23fff640 1 -- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d2c0d0fa0 msgr2=0x7f2d2c152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d23fff640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d2c0d0fa0 0x7f2d2c152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d23fff640 1 -- 172.21.15.64:0/2896212796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c0094d0 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.223+0000 7f2d2b7fe640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d2c0d0fa0 0x7f2d2c152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.224+0000 7f2d23fff640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b6e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c450 tx=0x7f2d1400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.224+0000 7f2d28ff9640 1 -- 172.21.15.64:0/2896212796 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d140052d0 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.224+0000 7f2d28ff9640 1 -- 172.21.15.64:0/2896212796 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d14005430 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.224+0000 7f2d28ff9640 1 -- 172.21.15.64:0/2896212796 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14012870 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.224+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c0ca8e0 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.224+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c152940 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.225+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c0ca740 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.226+0000 7f2d28ff9640 1 -- 172.21.15.64:0/2896212796 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d140129d0 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.227+0000 7f2d28ff9640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d0c06ae40 0x7f2d0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.227+0000 7f2d28ff9640 1 -- 172.21.15.64:0/2896212796 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d14089280 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.228+0000 7f2d2b7fe640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d0c06ae40 0x7f2d0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.229+0000 7f2d2b7fe640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d0c06ae40 0x7f2d0c06d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c036000 tx=0x7f2d1c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:08.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.232+0000 7f2d28ff9640 1 -- 172.21.15.64:0/2896212796 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d2c0ca740 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.425+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d2c0ca740 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.426+0000 7f2d28ff9640 1 -- 172.21.15.64:0/2896212796 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d2c0ca740 con 0x7f2d2c0d1d90 2023-11-24T08:04:08.428 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d0c06ae40 msgr2=0x7f2d0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d0c06ae40 0x7f2d0c06d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c036000 tx=0x7f2d1c03c040 comp rx=0 tx=0).stop 2023-11-24T08:04:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0d1d90 msgr2=0x7f2d2c0b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b6e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c450 tx=0x7f2d1400c910 comp rx=0 tx=0).stop 2023-11-24T08:04:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 shutdown_connections 2023-11-24T08:04:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2d0c06ae40 0x7f2d0c06d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2d2c15d3b0 0x7f2d2c0b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0d1d90 0x7f2d2c0b6e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 --2- 172.21.15.64:0/2896212796 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2d2c0d0fa0 0x7f2d2c152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.429+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 >> 172.21.15.64:0/2896212796 conn(0x7f2d2c0d30a0 msgr2=0x7f2d2c0d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.430+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 shutdown_connections 2023-11-24T08:04:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.430+0000 7f2d30a32640 1 -- 172.21.15.64:0/2896212796 wait complete. 2023-11-24T08:04:08.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.675+0000 7f2a8e2c6640 1 Processor -- start 2023-11-24T08:04:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.675+0000 7f2a8e2c6640 1 -- start start 2023-11-24T08:04:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a8e2c6640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a8815d350 0x7f2a880d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a8e2c6640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2a8815d730 0x7f2a880d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a8e2c6640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a880d1d80 con 0x7f2a880ceb30 2023-11-24T08:04:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a8e2c6640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a880d1fb0 con 0x7f2a8815d730 2023-11-24T08:04:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a8e2c6640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880ce050 con 0x7f2a8815d350 2023-11-24T08:04:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a8c863640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2a880ceb30 0x7f2a880cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43096/0 (socket says 172.21.15.64:43096) 2023-11-24T08:04:08.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a8c863640 1 -- 172.21.15.64:0/1226824775 learned_addr learned my addr 172.21.15.64:0/1226824775 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:08.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a87fff640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a8815d350 0x7f2a880d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a877fe640 1 --2- 172.21.15.64:0/1226824775 >> v2:172.21.15.91:3300/0 conn(0x7f2a8815d730 0x7f2a880d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.677+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896837121 0 0) 0x7f2a880d1d80 con 0x7f2a880ceb30 2023-11-24T08:04:08.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a70003520 con 0x7f2a880ceb30 2023-11-24T08:04:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1726428880 0 0) 0x7f2a70003520 con 0x7f2a880ceb30 2023-11-24T08:04:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a8815d350 msgr2=0x7f2a880d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a8815d350 0x7f2a880d3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 >> v2:172.21.15.91:3300/0 conn(0x7f2a8815d730 msgr2=0x7f2a880d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 --2- 172.21.15.64:0/1226824775 >> v2:172.21.15.91:3300/0 conn(0x7f2a8815d730 0x7f2a880d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880ce330 con 0x7f2a880ceb30 2023-11-24T08:04:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2a80001670 con 0x7f2a880ceb30 2023-11-24T08:04:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 >> v1:172.21.15.64:6789/0 conn(0x7f2a880ceb30 legacy=0x7f2a880cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a8815d350 0x7f2a70003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a70003ef0 0x7f2a700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880d1fb0 con 0x7f2a70004d00 2023-11-24T08:04:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880ce050 con 0x7f2a70003ef0 2023-11-24T08:04:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7000b980 con 0x7f2a8815d350 2023-11-24T08:04:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.678+0000 7f2a8c863640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.679+0000 7f2a877fe640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a70003ef0 0x7f2a700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.679+0000 7f2a87fff640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a8815d350 0x7f2a70003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.679+0000 7f2a8c863640 1 -- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a8815d350 msgr2=0x7f2a70003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.679+0000 7f2a8c863640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a8815d350 0x7f2a70003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.679+0000 7f2a8c863640 1 -- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a70003ef0 msgr2=0x7f2a700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.679+0000 7f2a8c863640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a70003ef0 0x7f2a700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.679+0000 7f2a8c863640 1 -- 172.21.15.64:0/1226824775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70003520 con 0x7f2a70004d00 2023-11-24T08:04:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.680+0000 7f2a8c863640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a70005180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c240 tx=0x7f2a700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34faeeb2b8af5263 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.680+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8002cae0 con 0x7f2a70004d00 2023-11-24T08:04:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.680+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8002cc40 con 0x7f2a70004d00 2023-11-24T08:04:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.680+0000 7f2a86ffd640 1 -- 172.21.15.64:0/1226824775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80005af0 con 0x7f2a70004d00 2023-11-24T08:04:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.681+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/1226824775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 msgr2=0x7f2a70005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.681+0000 7f2a8e2c6640 1 --2- 172.21.15.64:0/1226824775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a70005180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c240 tx=0x7f2a700057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.681+0000 7f2a8c863640 1 -- 172.21.15.64:0/1226824775 reap_dead start 2023-11-24T08:04:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.681+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/1226824775 shutdown_connections 2023-11-24T08:04:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.681+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/1226824775 >> 172.21.15.64:0/1226824775 conn(0x7f2a8809c5c0 msgr2=0x7f2a880d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.681+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/1226824775 shutdown_connections 2023-11-24T08:04:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.681+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/1226824775 wait complete. 2023-11-24T08:04:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.683+0000 7f2a8e2c6640 1 Processor -- start 2023-11-24T08:04:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.683+0000 7f2a8e2c6640 1 -- start start 2023-11-24T08:04:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a8e2c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a88149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a8e2c6640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a880cc2e0 0x7f2a88146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a8e2c6640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a880ca330 0x7f2a880ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a8e2c6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880cacf0 con 0x7f2a70004d00 2023-11-24T08:04:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a8e2c6640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880cd660 con 0x7f2a880ca330 2023-11-24T08:04:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a8e2c6640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880ccae0 con 0x7f2a880cc2e0 2023-11-24T08:04:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a88149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a88149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42364/0 (socket says 172.21.15.64:42364) 2023-11-24T08:04:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.684+0000 7f2a87fff640 1 -- 172.21.15.64:0/3554630211 learned_addr learned my addr 172.21.15.64:0/3554630211 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a87fff640 1 -- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a880cc2e0 msgr2=0x7f2a88146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a87fff640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a880cc2e0 0x7f2a88146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a87fff640 1 -- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a880ca330 msgr2=0x7f2a880ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a87fff640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a880ca330 0x7f2a880ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a87fff640 1 -- 172.21.15.64:0/3554630211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880ccdc0 con 0x7f2a70004d00 2023-11-24T08:04:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a87fff640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a88149710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007f30 tx=0x7f2a78011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a84ff9640 1 -- 172.21.15.64:0/3554630211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800e030 con 0x7f2a70004d00 2023-11-24T08:04:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a84ff9640 1 -- 172.21.15.64:0/3554630211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a78016470 con 0x7f2a70004d00 2023-11-24T08:04:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a84ff9640 1 -- 172.21.15.64:0/3554630211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78016820 con 0x7f2a70004d00 2023-11-24T08:04:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a880cd0a0 con 0x7f2a70004d00 2023-11-24T08:04:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.685+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a880cc760 con 0x7f2a70004d00 2023-11-24T08:04:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.687+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a880a1230 con 0x7f2a70004d00 2023-11-24T08:04:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.687+0000 7f2a84ff9640 1 -- 172.21.15.64:0/3554630211 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a78016980 con 0x7f2a70004d00 2023-11-24T08:04:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.688+0000 7f2a84ff9640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7406aec0 0x7f2a7406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.688+0000 7f2a84ff9640 1 -- 172.21.15.64:0/3554630211 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a7800f070 con 0x7f2a70004d00 2023-11-24T08:04:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.689+0000 7f2a877fe640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7406aec0 0x7f2a7406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.690+0000 7f2a877fe640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7406aec0 0x7f2a7406d380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a8815a030 tx=0x7f2a7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:08.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.693+0000 7f2a84ff9640 1 -- 172.21.15.64:0/3554630211 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a780555e0 con 0x7f2a70004d00 2023-11-24T08:04:08.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.888+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a8804e920 con 0x7f2a70004d00 2023-11-24T08:04:08.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.889+0000 7f2a84ff9640 1 -- 172.21.15.64:0/3554630211 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a78058c00 con 0x7f2a70004d00 2023-11-24T08:04:08.892 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:08.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7406aec0 msgr2=0x7f2a7406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7406aec0 0x7f2a7406d380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a8815a030 tx=0x7f2a7c008040 comp rx=0 tx=0).stop 2023-11-24T08:04:08.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 msgr2=0x7f2a88149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:08.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a88149710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007f30 tx=0x7f2a78011660 comp rx=0 tx=0).stop 2023-11-24T08:04:08.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 shutdown_connections 2023-11-24T08:04:08.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7406aec0 0x7f2a7406d380 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a880ca330 0x7f2a880ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a880cc2e0 0x7f2a88146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 --2- 172.21.15.64:0/3554630211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a70004d00 0x7f2a88149710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:08.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.892+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 >> 172.21.15.64:0/3554630211 conn(0x7f2a8809c5c0 msgr2=0x7f2a880b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:08.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.893+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 shutdown_connections 2023-11-24T08:04:08.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:08.893+0000 7f2a8e2c6640 1 -- 172.21.15.64:0/3554630211 wait complete. 2023-11-24T08:04:08.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:09.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.130+0000 7f857ffff640 1 Processor -- start 2023-11-24T08:04:09.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.130+0000 7f857ffff640 1 -- start start 2023-11-24T08:04:09.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.132+0000 7f857ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85800c81f0 0x7f8580168400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.132+0000 7f857ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85800ce980 con 0x7f85800d3ba0 2023-11-24T08:04:09.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.132+0000 7f857ffff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85800cf1b0 con 0x7f858015d5c0 2023-11-24T08:04:09.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.132+0000 7f857ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85800d1da0 con 0x7f85800c81f0 2023-11-24T08:04:09.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.132+0000 7f857e7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f85800d3ba0 0x7f85800d1690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43100/0 (socket says 172.21.15.64:43100) 2023-11-24T08:04:09.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.132+0000 7f857f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85800c81f0 0x7f8580168400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.132+0000 7f857e7fc640 1 -- 172.21.15.64:0/2370642203 learned_addr learned my addr 172.21.15.64:0/2370642203 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:09.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.133+0000 7f857f7fe640 1 -- 172.21.15.64:0/2370642203 >> v1:172.21.15.91:6789/0 conn(0x7f858015d5c0 legacy=0x7f85800d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.133+0000 7f857f7fe640 1 -- 172.21.15.64:0/2370642203 >> v1:172.21.15.64:6789/0 conn(0x7f85800d3ba0 legacy=0x7f85800d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.133+0000 7f857f7fe640 1 -- 172.21.15.64:0/2370642203 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800c6d00 con 0x7f85800c81f0 2023-11-24T08:04:09.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.134+0000 7f857f7fe640 1 --2- 172.21.15.64:0/2370642203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800c81f0 0x7f8580168400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85780097a0 tx=0x7f857802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1b7dea9b6216da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:09.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857dffb640 1 -- 172.21.15.64:0/2370642203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578031830 con 0x7f85800c81f0 2023-11-24T08:04:09.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857dffb640 1 -- 172.21.15.64:0/2370642203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578031990 con 0x7f85800c81f0 2023-11-24T08:04:09.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857dffb640 1 -- 172.21.15.64:0/2370642203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578031d20 con 0x7f85800c81f0 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857ffff640 1 -- 172.21.15.64:0/2370642203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800c81f0 msgr2=0x7f8580168400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857ffff640 1 --2- 172.21.15.64:0/2370642203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800c81f0 0x7f8580168400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85780097a0 tx=0x7f857802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857ffff640 1 -- 172.21.15.64:0/2370642203 shutdown_connections 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857ffff640 1 --2- 172.21.15.64:0/2370642203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800c81f0 0x7f8580168400 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857ffff640 1 -- 172.21.15.64:0/2370642203 >> 172.21.15.64:0/2370642203 conn(0x7f85800d3080 msgr2=0x7f85800d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857ffff640 1 -- 172.21.15.64:0/2370642203 shutdown_connections 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.135+0000 7f857ffff640 1 -- 172.21.15.64:0/2370642203 wait complete. 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.137+0000 7f857ffff640 1 Processor -- start 2023-11-24T08:04:09.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.137+0000 7f857ffff640 1 -- start start 2023-11-24T08:04:09.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.138+0000 7f857ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85800c81f0 0x7f85801f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.138+0000 7f857ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800d3ba0 0x7f85800cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f858015d5c0 0x7f858006b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85801595b0 con 0x7f85800d3ba0 2023-11-24T08:04:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8580158650 con 0x7f858015d5c0 2023-11-24T08:04:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8580158930 con 0x7f85800c81f0 2023-11-24T08:04:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800d3ba0 0x7f85800cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800d3ba0 0x7f85800cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42384/0 (socket says 172.21.15.64:42384) 2023-11-24T08:04:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857e7fc640 1 -- 172.21.15.64:0/1110835771 learned_addr learned my addr 172.21.15.64:0/1110835771 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857effd640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85800c81f0 0x7f85801f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857f7fe640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f858015d5c0 0x7f858006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857e7fc640 1 -- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85800c81f0 msgr2=0x7f85801f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857e7fc640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85800c81f0 0x7f85801f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857e7fc640 1 -- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f858015d5c0 msgr2=0x7f858006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857e7fc640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f858015d5c0 0x7f858006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.139+0000 7f857e7fc640 1 -- 172.21.15.64:0/1110835771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8578009400 con 0x7f85800d3ba0 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.140+0000 7f857e7fc640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800d3ba0 0x7f85800cc7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f856800c4a0 tx=0x7f856800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.140+0000 7f85777fe640 1 -- 172.21.15.64:0/1110835771 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568005ce0 con 0x7f85800d3ba0 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.140+0000 7f85777fe640 1 -- 172.21.15.64:0/1110835771 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568005e40 con 0x7f85800d3ba0 2023-11-24T08:04:09.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.140+0000 7f85777fe640 1 -- 172.21.15.64:0/1110835771 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568007270 con 0x7f85800d3ba0 2023-11-24T08:04:09.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.140+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8580158c10 con 0x7f85800d3ba0 2023-11-24T08:04:09.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.140+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85801591d0 con 0x7f85800d3ba0 2023-11-24T08:04:09.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.141+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858015c340 con 0x7f85800d3ba0 2023-11-24T08:04:09.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.142+0000 7f85777fe640 1 -- 172.21.15.64:0/1110835771 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8568007490 con 0x7f85800d3ba0 2023-11-24T08:04:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.143+0000 7f85777fe640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f856006ae40 0x7f856006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.143+0000 7f85777fe640 1 -- 172.21.15.64:0/1110835771 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8568088b70 con 0x7f85800d3ba0 2023-11-24T08:04:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.144+0000 7f857effd640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f856006ae40 0x7f856006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.145+0000 7f857effd640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f856006ae40 0x7f856006d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f858006c530 tx=0x7f8570009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:09.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.147+0000 7f85777fe640 1 -- 172.21.15.64:0/1110835771 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8568053c30 con 0x7f85800d3ba0 2023-11-24T08:04:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.340+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f858004e840 con 0x7f85800d3ba0 2023-11-24T08:04:09.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.341+0000 7f85777fe640 1 -- 172.21.15.64:0/1110835771 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8568017020 con 0x7f85800d3ba0 2023-11-24T08:04:09.342 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:09.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f856006ae40 msgr2=0x7f856006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f856006ae40 0x7f856006d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f858006c530 tx=0x7f8570009040 comp rx=0 tx=0).stop 2023-11-24T08:04:09.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800d3ba0 msgr2=0x7f85800cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800d3ba0 0x7f85800cc7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f856800c4a0 tx=0x7f856800c960 comp rx=0 tx=0).stop 2023-11-24T08:04:09.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 shutdown_connections 2023-11-24T08:04:09.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f856006ae40 0x7f856006d300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f858015d5c0 0x7f858006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85800d3ba0 0x7f85800cc7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 --2- 172.21.15.64:0/1110835771 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85800c81f0 0x7f85801f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.343+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 >> 172.21.15.64:0/1110835771 conn(0x7f85800d3080 msgr2=0x7f85800d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:09.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.344+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 shutdown_connections 2023-11-24T08:04:09.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.344+0000 7f857ffff640 1 -- 172.21.15.64:0/1110835771 wait complete. 2023-11-24T08:04:09.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:04:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.589+0000 7f84b6421640 1 Processor -- start 2023-11-24T08:04:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.589+0000 7f84b6421640 1 -- start start 2023-11-24T08:04:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.591+0000 7f84b6421640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f84b00cf760 0x7f84b015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.591+0000 7f84b6421640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84b0143950 0x7f84b015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.591+0000 7f84b6421640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b0071700 con 0x7f84b0143570 2023-11-24T08:04:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.591+0000 7f84b6421640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b015af30 con 0x7f84b0143950 2023-11-24T08:04:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.591+0000 7f84b6421640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b015b410 con 0x7f84b00cf760 2023-11-24T08:04:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.591+0000 7f84b541f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f84b0143570 0x7f84b00cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43102/0 (socket says 172.21.15.64:43102) 2023-11-24T08:04:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.591+0000 7f84b541f640 1 -- 172.21.15.64:0/3319819249 learned_addr learned my addr 172.21.15.64:0/3319819249 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.591+0000 7f84b5c20640 1 --2- 172.21.15.64:0/3319819249 >> v2:172.21.15.64:3300/0 conn(0x7f84b0143950 0x7f84b015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.592+0000 7f84b4c1e640 1 --2- 172.21.15.64:0/3319819249 >> v2:172.21.15.91:3300/0 conn(0x7f84b00cf760 0x7f84b015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.592+0000 7f84a7fff640 1 -- 172.21.15.64:0/3319819249 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196155679 0 0) 0x7f84ac001110 con 0x7f84b0143570 2023-11-24T08:04:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.592+0000 7f84a7fff640 1 -- 172.21.15.64:0/3319819249 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f849c003520 con 0x7f84b0143570 2023-11-24T08:04:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.592+0000 7f84b5c20640 1 -- 172.21.15.64:0/3319819249 >> v2:172.21.15.91:3300/0 conn(0x7f84b00cf760 msgr2=0x7f84b015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.592+0000 7f84b5c20640 1 --2- 172.21.15.64:0/3319819249 >> v2:172.21.15.91:3300/0 conn(0x7f84b00cf760 0x7f84b015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.592+0000 7f84b5c20640 1 -- 172.21.15.64:0/3319819249 >> v1:172.21.15.64:6789/0 conn(0x7f84b0143570 legacy=0x7f84b00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.592+0000 7f84b5c20640 1 -- 172.21.15.64:0/3319819249 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b015b130 con 0x7f84b0143950 2023-11-24T08:04:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.593+0000 7f84b5c20640 1 --2- 172.21.15.64:0/3319819249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015a9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84b00cbe10 tx=0x7f84a802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7160769522c3c0d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.593+0000 7f84a7fff640 1 -- 172.21.15.64:0/3319819249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a802cbf0 con 0x7f84b0143950 2023-11-24T08:04:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.593+0000 7f84a7fff640 1 -- 172.21.15.64:0/3319819249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a802cd50 con 0x7f84b0143950 2023-11-24T08:04:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.593+0000 7f84a7fff640 1 -- 172.21.15.64:0/3319819249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8034660 con 0x7f84b0143950 2023-11-24T08:04:09.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.594+0000 7f84b6421640 1 -- 172.21.15.64:0/3319819249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 msgr2=0x7f84b015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.594+0000 7f84b6421640 1 --2- 172.21.15.64:0/3319819249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015a9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84b00cbe10 tx=0x7f84a802be40 comp rx=0 tx=0).stop 2023-11-24T08:04:09.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.594+0000 7f84b6421640 1 -- 172.21.15.64:0/3319819249 shutdown_connections 2023-11-24T08:04:09.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.594+0000 7f84b6421640 1 --2- 172.21.15.64:0/3319819249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015a9f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.594+0000 7f84b6421640 1 --2- 172.21.15.64:0/3319819249 >> v2:172.21.15.91:3300/0 conn(0x7f84b00cf760 0x7f84b015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.594+0000 7f84b6421640 1 -- 172.21.15.64:0/3319819249 >> 172.21.15.64:0/3319819249 conn(0x7f84b006a860 msgr2=0x7f84b006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.594+0000 7f84b6421640 1 -- 172.21.15.64:0/3319819249 shutdown_connections 2023-11-24T08:04:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.594+0000 7f84b6421640 1 -- 172.21.15.64:0/3319819249 wait complete. 2023-11-24T08:04:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.596+0000 7f84b6421640 1 Processor -- start 2023-11-24T08:04:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.596+0000 7f84b6421640 1 -- start start 2023-11-24T08:04:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.597+0000 7f84b6421640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84b00cf760 0x7f84b01f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.597+0000 7f84b6421640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84b0143570 0x7f84b0131130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.597+0000 7f84b6421640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.597+0000 7f84b6421640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01f2a20 con 0x7f84b0143950 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.597+0000 7f84b6421640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0131670 con 0x7f84b00cf760 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.597+0000 7f84b6421640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0131940 con 0x7f84b0143570 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42398/0 (socket says 172.21.15.64:42398) 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 -- 172.21.15.64:0/4253480035 learned_addr learned my addr 172.21.15.64:0/4253480035 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b4c1e640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84b0143570 0x7f84b0131130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b541f640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84b00cf760 0x7f84b01f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 -- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84b0143570 msgr2=0x7f84b0131130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84b0143570 0x7f84b0131130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 -- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84b00cf760 msgr2=0x7f84b01f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84b00cf760 0x7f84b01f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 -- 172.21.15.64:0/4253480035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a8009a70 con 0x7f84b0143950 2023-11-24T08:04:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.598+0000 7f84b5c20640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015c400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009e80 tx=0x7f84a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.599+0000 7f84a5ffb640 1 -- 172.21.15.64:0/4253480035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8040650 con 0x7f84b0143950 2023-11-24T08:04:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.599+0000 7f84a5ffb640 1 -- 172.21.15.64:0/4253480035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a80407b0 con 0x7f84b0143950 2023-11-24T08:04:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.599+0000 7f84a5ffb640 1 -- 172.21.15.64:0/4253480035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8040b60 con 0x7f84b0143950 2023-11-24T08:04:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.599+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b0131c20 con 0x7f84b0143950 2023-11-24T08:04:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.599+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b01f2ce0 con 0x7f84b0143950 2023-11-24T08:04:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.600+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b004e840 con 0x7f84b0143950 2023-11-24T08:04:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.601+0000 7f84a5ffb640 1 -- 172.21.15.64:0/4253480035 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84a8040cc0 con 0x7f84b0143950 2023-11-24T08:04:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.602+0000 7f84a5ffb640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f849406ae40 0x7f849406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.603+0000 7f84a5ffb640 1 -- 172.21.15.64:0/4253480035 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a80ac460 con 0x7f84b0143950 2023-11-24T08:04:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.603+0000 7f84b541f640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f849406ae40 0x7f849406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.604+0000 7f84b541f640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f849406ae40 0x7f849406d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f84ac007fd0 tx=0x7f84ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.607+0000 7f84a5ffb640 1 -- 172.21.15.64:0/4253480035 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84a8077470 con 0x7f84b0143950 2023-11-24T08:04:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.839+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84b00b3910 con 0x7f84b0143950 2023-11-24T08:04:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.841+0000 7f84a5ffb640 1 -- 172.21.15.64:0/4253480035 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f84a8009450 con 0x7f84b0143950 2023-11-24T08:04:09.843 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:09.843 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:03:55.699656+0000","modified":"2023-11-24T08:03:58.759467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:04:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T08:04:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.843+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f849406ae40 msgr2=0x7f849406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.843+0000 7f84b6421640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f849406ae40 0x7f849406d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f84ac007fd0 tx=0x7f84ac009040 comp rx=0 tx=0).stop 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.843+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 msgr2=0x7f84b015c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.843+0000 7f84b6421640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015c400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009e80 tx=0x7f84a8004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.844+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 shutdown_connections 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.844+0000 7f84b6421640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f849406ae40 0x7f849406d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.844+0000 7f84b6421640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b0143950 0x7f84b015c400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.844+0000 7f84b6421640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84b0143570 0x7f84b0131130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.844+0000 7f84b6421640 1 --2- 172.21.15.64:0/4253480035 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84b00cf760 0x7f84b01f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.844+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 >> 172.21.15.64:0/4253480035 conn(0x7f84b006a860 msgr2=0x7f84b00cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.844+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 shutdown_connections 2023-11-24T08:04:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:09.844+0000 7f84b6421640 1 -- 172.21.15.64:0/4253480035 wait complete. 2023-11-24T08:04:09.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:10.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.080+0000 7fd391aad640 1 Processor -- start 2023-11-24T08:04:10.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.080+0000 7fd391aad640 1 -- start start 2023-11-24T08:04:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.082+0000 7fd391aad640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0d14a0 0x7fd38c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.082+0000 7fd391aad640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd38c0b0640 0x7fd38c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.082+0000 7fd391aad640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0cf0c0 con 0x7fd38c15d5b0 2023-11-24T08:04:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.082+0000 7fd391aad640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0b01b0 con 0x7fd38c0b0640 2023-11-24T08:04:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.082+0000 7fd391aad640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c15d990 con 0x7fd38c0d14a0 2023-11-24T08:04:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.082+0000 7fd390aab640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd38c15d5b0 0x7fd38c0d3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43114/0 (socket says 172.21.15.64:43114) 2023-11-24T08:04:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.082+0000 7fd390aab640 1 -- 172.21.15.64:0/3126629591 learned_addr learned my addr 172.21.15.64:0/3126629591 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd3912ac640 1 --2- 172.21.15.64:0/3126629591 >> v2:172.21.15.91:3300/0 conn(0x7fd38c0b0640 0x7fd38c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38bfff640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0d14a0 0x7fd38c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809357434 0 0) 0x7fd38c0cf0c0 con 0x7fd38c15d5b0 2023-11-24T08:04:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd37c003520 con 0x7fd38c15d5b0 2023-11-24T08:04:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 505393863 0 0) 0x7fd37c003520 con 0x7fd38c15d5b0 2023-11-24T08:04:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0d14a0 msgr2=0x7fd38c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0d14a0 0x7fd38c0d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 >> v2:172.21.15.91:3300/0 conn(0x7fd38c0b0640 msgr2=0x7fd38c0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 --2- 172.21.15.64:0/3126629591 >> v2:172.21.15.91:3300/0 conn(0x7fd38c0b0640 0x7fd38c0b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0d1e40 con 0x7fd38c15d5b0 2023-11-24T08:04:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fd380001670 con 0x7fd38c15d5b0 2023-11-24T08:04:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 >> v1:172.21.15.64:6789/0 conn(0x7fd38c15d5b0 legacy=0x7fd38c0d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd37c0039e0 0x7fd37c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0d14a0 0x7fd37c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd37c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b01b0 con 0x7fd37c004d00 2023-11-24T08:04:10.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c15d990 con 0x7fd37c0039e0 2023-11-24T08:04:10.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.083+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c00b980 con 0x7fd38c0d14a0 2023-11-24T08:04:10.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.084+0000 7fd3912ac640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd37c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.084+0000 7fd390aab640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd37c0039e0 0x7fd37c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.084+0000 7fd38bfff640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0d14a0 0x7fd37c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.084+0000 7fd3912ac640 1 -- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0d14a0 msgr2=0x7fd37c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.084+0000 7fd3912ac640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0d14a0 0x7fd37c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.084+0000 7fd3912ac640 1 -- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd37c0039e0 msgr2=0x7fd37c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.084+0000 7fd3912ac640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd37c0039e0 0x7fd37c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.084+0000 7fd3912ac640 1 -- 172.21.15.64:0/3126629591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c003520 con 0x7fd37c004d00 2023-11-24T08:04:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.085+0000 7fd3912ac640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd37c005180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd384009fc0 tx=0x7fd37c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cbc56f60f2e9dd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.085+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38402cc60 con 0x7fd37c004d00 2023-11-24T08:04:10.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.085+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38400f040 con 0x7fd37c004d00 2023-11-24T08:04:10.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.085+0000 7fd38b7fe640 1 -- 172.21.15.64:0/3126629591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384034440 con 0x7fd37c004d00 2023-11-24T08:04:10.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.086+0000 7fd391aad640 1 -- 172.21.15.64:0/3126629591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 msgr2=0x7fd37c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.086+0000 7fd391aad640 1 --2- 172.21.15.64:0/3126629591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd37c005180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd384009fc0 tx=0x7fd37c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.086+0000 7fd3912ac640 1 -- 172.21.15.64:0/3126629591 reap_dead start 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.086+0000 7fd391aad640 1 -- 172.21.15.64:0/3126629591 shutdown_connections 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.086+0000 7fd391aad640 1 -- 172.21.15.64:0/3126629591 >> 172.21.15.64:0/3126629591 conn(0x7fd38c09c3b0 msgr2=0x7fd38c0d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.086+0000 7fd391aad640 1 -- 172.21.15.64:0/3126629591 shutdown_connections 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.086+0000 7fd391aad640 1 -- 172.21.15.64:0/3126629591 wait complete. 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.088+0000 7fd391aad640 1 Processor -- start 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.088+0000 7fd391aad640 1 -- start start 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd391aad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd38c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd391aad640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0b64a0 0x7fd38c1f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd391aad640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd38c0b84a0 0x7fd38c0b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd391aad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c158780 con 0x7fd37c004d00 2023-11-24T08:04:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd391aad640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b69a0 con 0x7fd38c0b84a0 2023-11-24T08:04:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd391aad640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b6c80 con 0x7fd38c0b64a0 2023-11-24T08:04:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd390aab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd38c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd390aab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd38c0b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42426/0 (socket says 172.21.15.64:42426) 2023-11-24T08:04:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.089+0000 7fd390aab640 1 -- 172.21.15.64:0/339302694 learned_addr learned my addr 172.21.15.64:0/339302694 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd390aab640 1 -- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0b64a0 msgr2=0x7fd38c1f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd390aab640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0b64a0 0x7fd38c1f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd390aab640 1 -- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd38c0b84a0 msgr2=0x7fd38c0b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd390aab640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd38c0b84a0 0x7fd38c0b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd390aab640 1 -- 172.21.15.64:0/339302694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c158410 con 0x7fd37c004d00 2023-11-24T08:04:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd390aab640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd38c0b5c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd38000e780 tx=0x7fd38000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd3897fa640 1 -- 172.21.15.64:0/339302694 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38000ee90 con 0x7fd37c004d00 2023-11-24T08:04:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd3897fa640 1 -- 172.21.15.64:0/339302694 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38000c870 con 0x7fd37c004d00 2023-11-24T08:04:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd3897fa640 1 -- 172.21.15.64:0/339302694 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38000cc20 con 0x7fd37c004d00 2023-11-24T08:04:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c0b58e0 con 0x7fd37c004d00 2023-11-24T08:04:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.090+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1f28e0 con 0x7fd37c004d00 2023-11-24T08:04:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.091+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c04e840 con 0x7fd37c004d00 2023-11-24T08:04:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.092+0000 7fd3897fa640 1 -- 172.21.15.64:0/339302694 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd38001d460 con 0x7fd37c004d00 2023-11-24T08:04:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.093+0000 7fd3897fa640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd37006ae70 0x7fd37006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.093+0000 7fd3897fa640 1 -- 172.21.15.64:0/339302694 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd380089520 con 0x7fd37c004d00 2023-11-24T08:04:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.094+0000 7fd38bfff640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd37006ae70 0x7fd37006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.095+0000 7fd38bfff640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd37006ae70 0x7fd37006d330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0b53b0 tx=0x7fd378008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.097+0000 7fd3897fa640 1 -- 172.21.15.64:0/339302694 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3800545d0 con 0x7fd37c004d00 2023-11-24T08:04:10.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.283+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd38c0678d0 con 0x7fd37c004d00 2023-11-24T08:04:10.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.286+0000 7fd3897fa640 1 -- 172.21.15.64:0/339302694 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fd380057bf0 con 0x7fd37c004d00 2023-11-24T08:04:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:10.288 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:10.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.288+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd37006ae70 msgr2=0x7fd37006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.288+0000 7fd391aad640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd37006ae70 0x7fd37006d330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0b53b0 tx=0x7fd378008040 comp rx=0 tx=0).stop 2023-11-24T08:04:10.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.288+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 msgr2=0x7fd38c0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.288+0000 7fd391aad640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd38c0b5c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd38000e780 tx=0x7fd38000ec40 comp rx=0 tx=0).stop 2023-11-24T08:04:10.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.288+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 shutdown_connections 2023-11-24T08:04:10.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.288+0000 7fd391aad640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd37006ae70 0x7fd37006d330 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.288+0000 7fd391aad640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd38c0b84a0 0x7fd38c0b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.288+0000 7fd391aad640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd38c0b64a0 0x7fd38c1f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.289+0000 7fd391aad640 1 --2- 172.21.15.64:0/339302694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37c004d00 0x7fd38c0b5c90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.289+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 >> 172.21.15.64:0/339302694 conn(0x7fd38c09c3b0 msgr2=0x7fd38c06c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:10.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.289+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 shutdown_connections 2023-11-24T08:04:10.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.289+0000 7fd391aad640 1 -- 172.21.15.64:0/339302694 wait complete. 2023-11-24T08:04:10.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:10.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:10.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.528+0000 7f0a8997a640 1 Processor -- start 2023-11-24T08:04:10.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.529+0000 7f0a8997a640 1 -- start start 2023-11-24T08:04:10.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a8997a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a840d1d90 0x7f0a840ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a8997a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0a840d0fa0 0x7f0a840b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a8997a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840c6bf0 con 0x7f0a8415d3b0 2023-11-24T08:04:10.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a8997a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a840d13f0 con 0x7f0a840d1d90 2023-11-24T08:04:10.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a8997a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a840cedc0 con 0x7f0a840d0fa0 2023-11-24T08:04:10.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a88978640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0a8415d3b0 0x7f0a840d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43126/0 (socket says 172.21.15.64:43126) 2023-11-24T08:04:10.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a88978640 1 -- 172.21.15.64:0/3988498710 learned_addr learned my addr 172.21.15.64:0/3988498710 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:10.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/3988498710 >> v2:172.21.15.64:3300/0 conn(0x7f0a840d1d90 0x7f0a840ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a89179640 1 --2- 172.21.15.64:0/3988498710 >> v2:172.21.15.91:3300/0 conn(0x7f0a840d0fa0 0x7f0a840b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.530+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3988498710 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385133879 0 0) 0x7f0a840c6bf0 con 0x7f0a8415d3b0 2023-11-24T08:04:10.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.531+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3988498710 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c003520 con 0x7f0a8415d3b0 2023-11-24T08:04:10.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.531+0000 7f0a7bfff640 1 -- 172.21.15.64:0/3988498710 >> v2:172.21.15.91:3300/0 conn(0x7f0a840d0fa0 msgr2=0x7f0a840b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:10.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.531+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/3988498710 >> v2:172.21.15.91:3300/0 conn(0x7f0a840d0fa0 0x7f0a840b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.531+0000 7f0a7bfff640 1 -- 172.21.15.64:0/3988498710 >> v1:172.21.15.64:6789/0 conn(0x7f0a8415d3b0 legacy=0x7f0a840d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.531+0000 7f0a7bfff640 1 -- 172.21.15.64:0/3988498710 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840c6e40 con 0x7f0a840d1d90 2023-11-24T08:04:10.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.532+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/3988498710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a840d1d90 0x7f0a840ce6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0a84072b10 tx=0x7f0a8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32d3e39d1efa7a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.532+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3988498710 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002e070 con 0x7f0a840d1d90 2023-11-24T08:04:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.532+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3988498710 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a800315a0 con 0x7f0a840d1d90 2023-11-24T08:04:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.532+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3988498710 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80031930 con 0x7f0a840d1d90 2023-11-24T08:04:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.533+0000 7f0a8997a640 1 -- 172.21.15.64:0/3988498710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a840d1d90 msgr2=0x7f0a840ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.533+0000 7f0a8997a640 1 --2- 172.21.15.64:0/3988498710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a840d1d90 0x7f0a840ce6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0a84072b10 tx=0x7f0a8002b840 comp rx=0 tx=0).stop 2023-11-24T08:04:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.533+0000 7f0a8997a640 1 -- 172.21.15.64:0/3988498710 shutdown_connections 2023-11-24T08:04:10.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.533+0000 7f0a8997a640 1 --2- 172.21.15.64:0/3988498710 >> v2:172.21.15.91:3300/0 conn(0x7f0a840d0fa0 0x7f0a840b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.533+0000 7f0a8997a640 1 --2- 172.21.15.64:0/3988498710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a840d1d90 0x7f0a840ce6c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.533+0000 7f0a8997a640 1 -- 172.21.15.64:0/3988498710 >> 172.21.15.64:0/3988498710 conn(0x7f0a840d30a0 msgr2=0x7f0a840d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:10.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.533+0000 7f0a8997a640 1 -- 172.21.15.64:0/3988498710 shutdown_connections 2023-11-24T08:04:10.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.533+0000 7f0a8997a640 1 -- 172.21.15.64:0/3988498710 wait complete. 2023-11-24T08:04:10.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.535+0000 7f0a8997a640 1 Processor -- start 2023-11-24T08:04:10.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.535+0000 7f0a8997a640 1 -- start start 2023-11-24T08:04:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.536+0000 7f0a8997a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a840d0fa0 0x7f0a84151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.536+0000 7f0a8997a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a840d1d90 0x7f0a840b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.536+0000 7f0a8997a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8415d3b0 0x7f0a840caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.536+0000 7f0a8997a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841523f0 con 0x7f0a8415d3b0 2023-11-24T08:04:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.536+0000 7f0a8997a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840b6e50 con 0x7f0a840d0fa0 2023-11-24T08:04:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.536+0000 7f0a8997a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840b7130 con 0x7f0a840d1d90 2023-11-24T08:04:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8415d3b0 0x7f0a840caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8415d3b0 0x7f0a840caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42454/0 (socket says 172.21.15.64:42454) 2023-11-24T08:04:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 -- 172.21.15.64:0/2765897035 learned_addr learned my addr 172.21.15.64:0/2765897035 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a840d1d90 0x7f0a840b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a88978640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a840d0fa0 0x7f0a84151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 -- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a840d1d90 msgr2=0x7f0a840b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a840d1d90 0x7f0a840b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 -- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a840d0fa0 msgr2=0x7f0a84151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a840d0fa0 0x7f0a84151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 -- 172.21.15.64:0/2765897035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80009400 con 0x7f0a8415d3b0 2023-11-24T08:04:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a840d1d90 0x7f0a840b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.537+0000 7f0a89179640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8415d3b0 0x7f0a840caa40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c007e40 tx=0x7f0a7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.538+0000 7f0a797fa640 1 -- 172.21.15.64:0/2765897035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c014620 con 0x7f0a8415d3b0 2023-11-24T08:04:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.538+0000 7f0a797fa640 1 -- 172.21.15.64:0/2765897035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c00ee90 con 0x7f0a8415d3b0 2023-11-24T08:04:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.538+0000 7f0a797fa640 1 -- 172.21.15.64:0/2765897035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0149e0 con 0x7f0a8415d3b0 2023-11-24T08:04:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.538+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a840b7410 con 0x7f0a8415d3b0 2023-11-24T08:04:10.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.538+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a840b79d0 con 0x7f0a8415d3b0 2023-11-24T08:04:10.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.539+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a840b7bc0 con 0x7f0a8415d3b0 2023-11-24T08:04:10.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.540+0000 7f0a797fa640 1 -- 172.21.15.64:0/2765897035 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a7c014b40 con 0x7f0a8415d3b0 2023-11-24T08:04:10.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.541+0000 7f0a797fa640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a7006ae40 0x7f0a7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.541+0000 7f0a797fa640 1 -- 172.21.15.64:0/2765897035 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a7c08a0d0 con 0x7f0a8415d3b0 2023-11-24T08:04:10.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.541+0000 7f0a88978640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a7006ae40 0x7f0a7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.542+0000 7f0a88978640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a7006ae40 0x7f0a7006d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a740011e0 tx=0x7f0a74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.545+0000 7f0a797fa640 1 -- 172.21.15.64:0/2765897035 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a7c0550e0 con 0x7f0a8415d3b0 2023-11-24T08:04:10.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.730+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a8404e840 con 0x7f0a8415d3b0 2023-11-24T08:04:10.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.731+0000 7f0a797fa640 1 -- 172.21.15.64:0/2765897035 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a7c058700 con 0x7f0a8415d3b0 2023-11-24T08:04:10.733 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.733+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a7006ae40 msgr2=0x7f0a7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.733+0000 7f0a8997a640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a7006ae40 0x7f0a7006d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a740011e0 tx=0x7f0a74009040 comp rx=0 tx=0).stop 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.733+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8415d3b0 msgr2=0x7f0a840caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.733+0000 7f0a8997a640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8415d3b0 0x7f0a840caa40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c007e40 tx=0x7f0a7c00e4b0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.734+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 shutdown_connections 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.734+0000 7f0a8997a640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a7006ae40 0x7f0a7006d300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.734+0000 7f0a8997a640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a8415d3b0 0x7f0a840caa40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.734+0000 7f0a8997a640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0a840d1d90 0x7f0a840b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.734+0000 7f0a8997a640 1 --2- 172.21.15.64:0/2765897035 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0a840d0fa0 0x7f0a84151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.734+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 >> 172.21.15.64:0/2765897035 conn(0x7f0a840d30a0 msgr2=0x7f0a840c8600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.734+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 shutdown_connections 2023-11-24T08:04:10.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.734+0000 7f0a8997a640 1 -- 172.21.15.64:0/2765897035 wait complete. 2023-11-24T08:04:10.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.965+0000 7f26b91ba640 1 Processor -- start 2023-11-24T08:04:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.965+0000 7f26b91ba640 1 -- start start 2023-11-24T08:04:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.966+0000 7f26b91ba640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f26b41618d0 0x7f26b4070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.967+0000 7f26b91ba640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40b3290 con 0x7f26b4071310 2023-11-24T08:04:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.967+0000 7f26b91ba640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b415e350 con 0x7f26b40b2eb0 2023-11-24T08:04:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.967+0000 7f26b91ba640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b4161d10 con 0x7f26b41618d0 2023-11-24T08:04:10.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.967+0000 7f26b89b9640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f26b4071310 0x7f26b406a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43136/0 (socket says 172.21.15.64:43136) 2023-11-24T08:04:10.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.967+0000 7f26b89b9640 1 -- 172.21.15.64:0/1024842003 learned_addr learned my addr 172.21.15.64:0/1024842003 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:10.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.967+0000 7f26b37fe640 1 --2- 172.21.15.64:0/1024842003 >> v2:172.21.15.91:3300/0 conn(0x7f26b41618d0 0x7f26b4070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.967+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401200409 0 0) 0x7f26b40b3290 con 0x7f26b4071310 2023-11-24T08:04:10.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26a0003520 con 0x7f26b4071310 2023-11-24T08:04:10.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527642921 0 0) 0x7f26a0003520 con 0x7f26b4071310 2023-11-24T08:04:10.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 >> v2:172.21.15.91:3300/0 conn(0x7f26b41618d0 msgr2=0x7f26b4070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:10.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 --2- 172.21.15.64:0/1024842003 >> v2:172.21.15.91:3300/0 conn(0x7f26b41618d0 0x7f26b4070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 >> v1:172.21.15.91:6789/0 conn(0x7f26b40b2eb0 legacy=0x7f26b415d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b415e630 con 0x7f26b4071310 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f26ac001670 con 0x7f26b4071310 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 >> v1:172.21.15.64:6789/0 conn(0x7f26b4071310 legacy=0x7f26b406a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26a0003a30 0x7f26a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a00051d0 0x7f26a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4161d10 con 0x7f26a00047c0 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a000c100 con 0x7f26a0003a30 2023-11-24T08:04:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a000c2c0 con 0x7f26a00051d0 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.968+0000 7f26b37fe640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b3fff640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26a0003a30 0x7f26a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b89b9640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a00051d0 0x7f26a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b37fe640 1 -- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a00051d0 msgr2=0x7f26a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b37fe640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a00051d0 0x7f26a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b37fe640 1 -- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26a0003a30 msgr2=0x7f26a0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b37fe640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26a0003a30 0x7f26a0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b37fe640 1 -- 172.21.15.64:0/1024842003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0003520 con 0x7f26a00047c0 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b89b9640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26a00051d0 0x7f26a000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.969+0000 7f26b89b9640 1 -- 172.21.15.64:0/1024842003 reap_dead start 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.970+0000 7f26b37fe640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26a0004c40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f269c009ab0 tx=0x7f26a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4f6b02deba33d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.970+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c005990 con 0x7f26a00047c0 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.970+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c005af0 con 0x7f26a00047c0 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.970+0000 7f26b2ffd640 1 -- 172.21.15.64:0/1024842003 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c005ea0 con 0x7f26a00047c0 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.971+0000 7f26b91ba640 1 -- 172.21.15.64:0/1024842003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 msgr2=0x7f26a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.971+0000 7f26b91ba640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26a0004c40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f269c009ab0 tx=0x7f26a00057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.971+0000 7f26b91ba640 1 -- 172.21.15.64:0/1024842003 shutdown_connections 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.971+0000 7f26b91ba640 1 --2- 172.21.15.64:0/1024842003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26a0004c40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.971+0000 7f26b91ba640 1 -- 172.21.15.64:0/1024842003 >> 172.21.15.64:0/1024842003 conn(0x7f26b40b15c0 msgr2=0x7f26b40b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.971+0000 7f26b91ba640 1 -- 172.21.15.64:0/1024842003 shutdown_connections 2023-11-24T08:04:10.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.971+0000 7f26b91ba640 1 -- 172.21.15.64:0/1024842003 wait complete. 2023-11-24T08:04:10.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.973+0000 7f26b91ba640 1 Processor -- start 2023-11-24T08:04:10.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.973+0000 7f26b91ba640 1 -- start start 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b91ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26b414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b91ba640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26b414ab70 0x7f26b414c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b91ba640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26b414c660 0x7f26b40b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b91ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b414b080 con 0x7f26a00047c0 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b91ba640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b414b360 con 0x7f26b414ab70 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b91ba640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b414b640 con 0x7f26b414c660 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26b414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26b414a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42466/0 (socket says 172.21.15.64:42466) 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b3fff640 1 -- 172.21.15.64:0/517917397 learned_addr learned my addr 172.21.15.64:0/517917397 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b37fe640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26b414ab70 0x7f26b414c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b89b9640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26b414c660 0x7f26b40b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.974+0000 7f26b3fff640 1 -- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26b414c660 msgr2=0x7f26b40b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b3fff640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26b414c660 0x7f26b40b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b3fff640 1 -- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26b414ab70 msgr2=0x7f26b414c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b3fff640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26b414ab70 0x7f26b414c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b3fff640 1 -- 172.21.15.64:0/517917397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b414b920 con 0x7f26a00047c0 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b37fe640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26b414ab70 0x7f26b414c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b3fff640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26b414a630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26a400e8c0 tx=0x7f26a400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b0ff9640 1 -- 172.21.15.64:0/517917397 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4003b80 con 0x7f26a00047c0 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b0ff9640 1 -- 172.21.15.64:0/517917397 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a4003ce0 con 0x7f26a00047c0 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b0ff9640 1 -- 172.21.15.64:0/517917397 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40052f0 con 0x7f26a00047c0 2023-11-24T08:04:10.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b414bc00 con 0x7f26a00047c0 2023-11-24T08:04:10.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.975+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b40b07d0 con 0x7f26a00047c0 2023-11-24T08:04:10.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.977+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b4071780 con 0x7f26a00047c0 2023-11-24T08:04:10.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.978+0000 7f26b0ff9640 1 -- 172.21.15.64:0/517917397 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26a4005450 con 0x7f26a00047c0 2023-11-24T08:04:10.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.978+0000 7f26b0ff9640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f269406ae40 0x7f269406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:10.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.979+0000 7f26b0ff9640 1 -- 172.21.15.64:0/517917397 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26a4089790 con 0x7f26a00047c0 2023-11-24T08:04:10.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.979+0000 7f26b37fe640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f269406ae40 0x7f269406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:10.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.980+0000 7f26b37fe640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f269406ae40 0x7f269406d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f269c035000 tx=0x7f269c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:10.983+0000 7f26b0ff9640 1 -- 172.21.15.64:0/517917397 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26a4054830 con 0x7f26a00047c0 2023-11-24T08:04:11.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.177+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26b404e840 con 0x7f26a00047c0 2023-11-24T08:04:11.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.178+0000 7f26b0ff9640 1 -- 172.21.15.64:0/517917397 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26a401a310 con 0x7f26a00047c0 2023-11-24T08:04:11.180 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f269406ae40 msgr2=0x7f269406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f269406ae40 0x7f269406d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f269c035000 tx=0x7f269c030000 comp rx=0 tx=0).stop 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 msgr2=0x7f26b414a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26b414a630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26a400e8c0 tx=0x7f26a400ed80 comp rx=0 tx=0).stop 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 shutdown_connections 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f269406ae40 0x7f269406d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f26b414c660 0x7f26b40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f26b414ab70 0x7f26b414c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 --2- 172.21.15.64:0/517917397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a00047c0 0x7f26b414a630 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.181+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 >> 172.21.15.64:0/517917397 conn(0x7f26b40b15c0 msgr2=0x7f26b406ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.182+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 shutdown_connections 2023-11-24T08:04:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.182+0000 7f26b91ba640 1 -- 172.21.15.64:0/517917397 wait complete. 2023-11-24T08:04:11.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:11.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.427+0000 7f55dffff640 1 Processor -- start 2023-11-24T08:04:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.428+0000 7f55dffff640 1 -- start start 2023-11-24T08:04:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.429+0000 7f55dffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f55e015d3b0 0x7f55e00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.429+0000 7f55dffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f55e00d0fa0 0x7f55e00d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.429+0000 7f55dffff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e00c6bf0 con 0x7f55e00d1bc0 2023-11-24T08:04:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.429+0000 7f55dffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e00d1430 con 0x7f55e015d3b0 2023-11-24T08:04:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.429+0000 7f55dffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e00d1fa0 con 0x7f55e00d0fa0 2023-11-24T08:04:11.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55deffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f55e015d3b0 0x7f55e00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55deffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f55e015d3b0 0x7f55e00ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42468/0 (socket says 172.21.15.64:42468) 2023-11-24T08:04:11.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55deffd640 1 -- 172.21.15.64:0/1342009391 learned_addr learned my addr 172.21.15.64:0/1342009391 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55de7fc640 1 --2- 172.21.15.64:0/1342009391 >> v2:172.21.15.91:3300/0 conn(0x7f55e00d0fa0 0x7f55e00d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55deffd640 1 -- 172.21.15.64:0/1342009391 >> v2:172.21.15.91:3300/0 conn(0x7f55e00d0fa0 msgr2=0x7f55e00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55deffd640 1 --2- 172.21.15.64:0/1342009391 >> v2:172.21.15.91:3300/0 conn(0x7f55e00d0fa0 0x7f55e00d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55deffd640 1 -- 172.21.15.64:0/1342009391 >> v1:172.21.15.91:6789/0 conn(0x7f55e00d1bc0 legacy=0x7f55e00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55deffd640 1 -- 172.21.15.64:0/1342009391 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00cedc0 con 0x7f55e015d3b0 2023-11-24T08:04:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.430+0000 7f55de7fc640 1 --2- 172.21.15.64:0/1342009391 >> v2:172.21.15.91:3300/0 conn(0x7f55e00d0fa0 0x7f55e00d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:11.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.432+0000 7f55deffd640 1 --2- 172.21.15.64:0/1342009391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e015d3b0 0x7f55e00ce6c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f55e008c4a0 tx=0x7f55d002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44161198f9a2c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:11.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.432+0000 7f55ddffb640 1 -- 172.21.15.64:0/1342009391 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d002c9f0 con 0x7f55e015d3b0 2023-11-24T08:04:11.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.432+0000 7f55ddffb640 1 -- 172.21.15.64:0/1342009391 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d002cb50 con 0x7f55e015d3b0 2023-11-24T08:04:11.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.432+0000 7f55ddffb640 1 -- 172.21.15.64:0/1342009391 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00343f0 con 0x7f55e015d3b0 2023-11-24T08:04:11.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.433+0000 7f55dffff640 1 -- 172.21.15.64:0/1342009391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e015d3b0 msgr2=0x7f55e00ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.433+0000 7f55dffff640 1 --2- 172.21.15.64:0/1342009391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e015d3b0 0x7f55e00ce6c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f55e008c4a0 tx=0x7f55d002be50 comp rx=0 tx=0).stop 2023-11-24T08:04:11.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.433+0000 7f55dffff640 1 -- 172.21.15.64:0/1342009391 shutdown_connections 2023-11-24T08:04:11.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.433+0000 7f55dffff640 1 --2- 172.21.15.64:0/1342009391 >> v2:172.21.15.91:3300/0 conn(0x7f55e00d0fa0 0x7f55e00d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.433+0000 7f55dffff640 1 --2- 172.21.15.64:0/1342009391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e015d3b0 0x7f55e00ce6c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.433+0000 7f55dffff640 1 -- 172.21.15.64:0/1342009391 >> 172.21.15.64:0/1342009391 conn(0x7f55e00d30a0 msgr2=0x7f55e00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:11.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.433+0000 7f55dffff640 1 -- 172.21.15.64:0/1342009391 shutdown_connections 2023-11-24T08:04:11.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.433+0000 7f55dffff640 1 -- 172.21.15.64:0/1342009391 wait complete. 2023-11-24T08:04:11.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.435+0000 7f55dffff640 1 Processor -- start 2023-11-24T08:04:11.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.435+0000 7f55dffff640 1 -- start start 2023-11-24T08:04:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55dffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e00d0fa0 0x7f55e0148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55dffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55e00d1bc0 0x7f55e015a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55dffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55e015d3b0 0x7f55e0157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55dffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00cb1c0 con 0x7f55e00d0fa0 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55dffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00cba20 con 0x7f55e00d1bc0 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55dffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00c6960 con 0x7f55e015d3b0 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55deffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e00d0fa0 0x7f55e0148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55deffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e00d0fa0 0x7f55e0148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42470/0 (socket says 172.21.15.64:42470) 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.436+0000 7f55deffd640 1 -- 172.21.15.64:0/4076532875 learned_addr learned my addr 172.21.15.64:0/4076532875 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55de7fc640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55e00d1bc0 0x7f55e015a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55deffd640 1 -- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55e015d3b0 msgr2=0x7f55e0157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55df7fe640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55e015d3b0 0x7f55e0157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55deffd640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55e015d3b0 0x7f55e0157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55deffd640 1 -- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55e00d1bc0 msgr2=0x7f55e015a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55deffd640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55e00d1bc0 0x7f55e015a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55deffd640 1 -- 172.21.15.64:0/4076532875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d0009b30 con 0x7f55e00d0fa0 2023-11-24T08:04:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55deffd640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e00d0fa0 0x7f55e0148110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f55d0000f80 tx=0x7f55d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:11.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55d77fe640 1 -- 172.21.15.64:0/4076532875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0009510 con 0x7f55e00d0fa0 2023-11-24T08:04:11.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55d77fe640 1 -- 172.21.15.64:0/4076532875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d003e450 con 0x7f55e00d0fa0 2023-11-24T08:04:11.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55d77fe640 1 -- 172.21.15.64:0/4076532875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d003e800 con 0x7f55e00d0fa0 2023-11-24T08:04:11.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.437+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e015af90 con 0x7f55e00d0fa0 2023-11-24T08:04:11.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.438+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e00cb6b0 con 0x7f55e00d0fa0 2023-11-24T08:04:11.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.439+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e004e840 con 0x7f55e00d0fa0 2023-11-24T08:04:11.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.440+0000 7f55d77fe640 1 -- 172.21.15.64:0/4076532875 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f55d003e960 con 0x7f55e00d0fa0 2023-11-24T08:04:11.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.441+0000 7f55d77fe640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f55c006ae70 0x7f55c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.441+0000 7f55d77fe640 1 -- 172.21.15.64:0/4076532875 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55d00ac190 con 0x7f55e00d0fa0 2023-11-24T08:04:11.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.442+0000 7f55de7fc640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f55c006ae70 0x7f55c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.443+0000 7f55de7fc640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f55c006ae70 0x7f55c006d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55e00c8960 tx=0x7f55c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:11.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.446+0000 7f55d77fe640 1 -- 172.21.15.64:0/4076532875 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55d00771a0 con 0x7f55e00d0fa0 2023-11-24T08:04:11.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.638+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55e00c86b0 con 0x7f55e00d0fa0 2023-11-24T08:04:11.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.639+0000 7f55d77fe640 1 -- 172.21.15.64:0/4076532875 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55d007a7c0 con 0x7f55e00d0fa0 2023-11-24T08:04:11.641 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:11.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f55c006ae70 msgr2=0x7f55c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f55c006ae70 0x7f55c006d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55e00c8960 tx=0x7f55c8008040 comp rx=0 tx=0).stop 2023-11-24T08:04:11.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e00d0fa0 msgr2=0x7f55e0148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e00d0fa0 0x7f55e0148110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f55d0000f80 tx=0x7f55d0004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:11.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 shutdown_connections 2023-11-24T08:04:11.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f55c006ae70 0x7f55c006d330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55e015d3b0 0x7f55e0157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f55e00d1bc0 0x7f55e015a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 --2- 172.21.15.64:0/4076532875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e00d0fa0 0x7f55e0148110 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 >> 172.21.15.64:0/4076532875 conn(0x7f55e00d30a0 msgr2=0x7f55e00d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:11.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.641+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 shutdown_connections 2023-11-24T08:04:11.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.642+0000 7f55dffff640 1 -- 172.21.15.64:0/4076532875 wait complete. 2023-11-24T08:04:11.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:04:11.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.878+0000 7fc28e067640 1 Processor -- start 2023-11-24T08:04:11.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.878+0000 7fc28e067640 1 -- start start 2023-11-24T08:04:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.880+0000 7fc28e067640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc28813dfd0 0x7fc28815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.880+0000 7fc28e067640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc2880cf050 0x7fc2880cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.880+0000 7fc28e067640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880ce7c0 con 0x7fc28806c130 2023-11-24T08:04:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.880+0000 7fc28e067640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2880cfa10 con 0x7fc2880cf050 2023-11-24T08:04:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.880+0000 7fc28e067640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc288140900 con 0x7fc28813dfd0 2023-11-24T08:04:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.881+0000 7fc28d065640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc28813dfd0 0x7fc28815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.881+0000 7fc28c864640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc2880cf050 0x7fc2880cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.881+0000 7fc28d065640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc28813dfd0 0x7fc28815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:60758/0 (socket says 172.21.15.64:60758) 2023-11-24T08:04:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.881+0000 7fc28d065640 1 -- 172.21.15.64:0/2354168383 learned_addr learned my addr 172.21.15.64:0/2354168383 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.881+0000 7fc28d065640 1 -- 172.21.15.64:0/2354168383 >> v2:172.21.15.91:3300/0 conn(0x7fc2880cf050 msgr2=0x7fc2880cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.881+0000 7fc28d065640 1 --2- 172.21.15.64:0/2354168383 >> v2:172.21.15.91:3300/0 conn(0x7fc2880cf050 0x7fc2880cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.881+0000 7fc28d065640 1 -- 172.21.15.64:0/2354168383 >> v1:172.21.15.91:6789/0 conn(0x7fc28806c130 legacy=0x7fc28806c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.881+0000 7fc28d065640 1 -- 172.21.15.64:0/2354168383 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28815fa70 con 0x7fc28813dfd0 2023-11-24T08:04:11.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.883+0000 7fc28d065640 1 --2- 172.21.15.64:0/2354168383 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc28813dfd0 0x7fc28815be70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc284009870 tx=0x7fc28402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=721321d1c3d7f9f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:11.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.883+0000 7fc27ffff640 1 -- 172.21.15.64:0/2354168383 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284030840 con 0x7fc28813dfd0 2023-11-24T08:04:11.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.883+0000 7fc27ffff640 1 -- 172.21.15.64:0/2354168383 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2840309a0 con 0x7fc28813dfd0 2023-11-24T08:04:11.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.883+0000 7fc27ffff640 1 -- 172.21.15.64:0/2354168383 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284030d30 con 0x7fc28813dfd0 2023-11-24T08:04:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.884+0000 7fc28e067640 1 -- 172.21.15.64:0/2354168383 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc28813dfd0 msgr2=0x7fc28815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.884+0000 7fc28e067640 1 --2- 172.21.15.64:0/2354168383 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc28813dfd0 0x7fc28815be70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc284009870 tx=0x7fc28402b980 comp rx=0 tx=0).stop 2023-11-24T08:04:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.884+0000 7fc28e067640 1 -- 172.21.15.64:0/2354168383 shutdown_connections 2023-11-24T08:04:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.884+0000 7fc28e067640 1 --2- 172.21.15.64:0/2354168383 >> v2:172.21.15.91:3300/0 conn(0x7fc2880cf050 0x7fc2880cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.884+0000 7fc28e067640 1 --2- 172.21.15.64:0/2354168383 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc28813dfd0 0x7fc28815be70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.884+0000 7fc28e067640 1 -- 172.21.15.64:0/2354168383 >> 172.21.15.64:0/2354168383 conn(0x7fc28806a860 msgr2=0x7fc28806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.884+0000 7fc28e067640 1 -- 172.21.15.64:0/2354168383 shutdown_connections 2023-11-24T08:04:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.884+0000 7fc28e067640 1 -- 172.21.15.64:0/2354168383 wait complete. 2023-11-24T08:04:11.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.887+0000 7fc28e067640 1 Processor -- start 2023-11-24T08:04:11.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.887+0000 7fc28e067640 1 -- start start 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28e067640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28806c130 0x7fc2881f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28e067640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2880cf050 0x7fc2881579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28e067640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc28813dfd0 0x7fc2880b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28e067640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288071130 con 0x7fc28806c130 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28e067640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288071410 con 0x7fc28813dfd0 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28e067640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880716f0 con 0x7fc2880cf050 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28d065640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28806c130 0x7fc2881f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28d065640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28806c130 0x7fc2881f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42484/0 (socket says 172.21.15.64:42484) 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.888+0000 7fc28d065640 1 -- 172.21.15.64:0/2108876538 learned_addr learned my addr 172.21.15.64:0/2108876538 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.889+0000 7fc28c864640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2880cf050 0x7fc2881579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.889+0000 7fc28d065640 1 -- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2880cf050 msgr2=0x7fc2881579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.889+0000 7fc28d866640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc28813dfd0 0x7fc2880b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.889+0000 7fc28d065640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2880cf050 0x7fc2881579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.889+0000 7fc28d065640 1 -- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc28813dfd0 msgr2=0x7fc2880b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:11.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.889+0000 7fc28d065640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc28813dfd0 0x7fc2880b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:11.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.889+0000 7fc28d065640 1 -- 172.21.15.64:0/2108876538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2840094d0 con 0x7fc28806c130 2023-11-24T08:04:11.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.889+0000 7fc28d065640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28806c130 0x7fc2881f6060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc28402b9c0 tx=0x7fc2840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.890+0000 7fc27dffb640 1 -- 172.21.15.64:0/2108876538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28403e070 con 0x7fc28806c130 2023-11-24T08:04:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.890+0000 7fc27dffb640 1 -- 172.21.15.64:0/2108876538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc284039a10 con 0x7fc28806c130 2023-11-24T08:04:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.890+0000 7fc27dffb640 1 -- 172.21.15.64:0/2108876538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284039de0 con 0x7fc28806c130 2023-11-24T08:04:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.890+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2880719d0 con 0x7fc28806c130 2023-11-24T08:04:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.890+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc288070dc0 con 0x7fc28806c130 2023-11-24T08:04:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.891+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288071ef0 con 0x7fc28806c130 2023-11-24T08:04:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.892+0000 7fc27dffb640 1 -- 172.21.15.64:0/2108876538 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc28404b400 con 0x7fc28806c130 2023-11-24T08:04:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.893+0000 7fc27dffb640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc26c06ae40 0x7fc26c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.893+0000 7fc27dffb640 1 -- 172.21.15.64:0/2108876538 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2840abb50 con 0x7fc28806c130 2023-11-24T08:04:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.893+0000 7fc28c864640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc26c06ae40 0x7fc26c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:11.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.894+0000 7fc28c864640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc26c06ae40 0x7fc26c06d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc288133340 tx=0x7fc278008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:11.897+0000 7fc27dffb640 1 -- 172.21.15.64:0/2108876538 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc288071ef0 con 0x7fc28806c130 2023-11-24T08:04:12.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.130+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc288071ef0 con 0x7fc28806c130 2023-11-24T08:04:12.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.132+0000 7fc27dffb640 1 -- 172.21.15.64:0/2108876538 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fc288071ef0 con 0x7fc28806c130 2023-11-24T08:04:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:12.134 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:03:55.699656+0000","modified":"2023-11-24T08:03:58.759467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:04:12.134 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc26c06ae40 msgr2=0x7fc26c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc26c06ae40 0x7fc26c06d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc288133340 tx=0x7fc278008040 comp rx=0 tx=0).stop 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28806c130 msgr2=0x7fc2881f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28806c130 0x7fc2881f6060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc28402b9c0 tx=0x7fc2840046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 shutdown_connections 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc26c06ae40 0x7fc26c06d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc28813dfd0 0x7fc2880b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc2880cf050 0x7fc2881579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 --2- 172.21.15.64:0/2108876538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28806c130 0x7fc2881f6060 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.134+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 >> 172.21.15.64:0/2108876538 conn(0x7fc28806a860 msgr2=0x7fc28813fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:12.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.135+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 shutdown_connections 2023-11-24T08:04:12.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.135+0000 7fc28e067640 1 -- 172.21.15.64:0/2108876538 wait complete. 2023-11-24T08:04:12.157 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:12.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.376+0000 7fc084023640 1 Processor -- start 2023-11-24T08:04:12.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.376+0000 7fc084023640 1 -- start start 2023-11-24T08:04:12.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc084023640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc07c0abff0 0x7fc07c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc084023640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07c15c360 con 0x7fc07c06d520 2023-11-24T08:04:12.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc084023640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07c15ba30 con 0x7fc07c0ad720 2023-11-24T08:04:12.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc084023640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c0b41a0 con 0x7fc07c0abff0 2023-11-24T08:04:12.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc081dbf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc07c06d520 0x7fc07c06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43144/0 (socket says 172.21.15.64:43144) 2023-11-24T08:04:12.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc081dbf640 1 -- 172.21.15.64:0/1960507198 learned_addr learned my addr 172.21.15.64:0/1960507198 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667642913 0 0) 0x7fc07c15c360 con 0x7fc07c06d520 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc068003520 con 0x7fc07c06d520 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.378+0000 7fc0825c0640 1 --2- 172.21.15.64:0/1960507198 >> v2:172.21.15.91:3300/0 conn(0x7fc07c0abff0 0x7fc07c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 306673852 0 0) 0x7fc068003520 con 0x7fc07c06d520 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 >> v2:172.21.15.91:3300/0 conn(0x7fc07c0abff0 msgr2=0x7fc07c0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 --2- 172.21.15.64:0/1960507198 >> v2:172.21.15.91:3300/0 conn(0x7fc07c0abff0 0x7fc07c0b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 >> v1:172.21.15.91:6789/0 conn(0x7fc07c0ad720 legacy=0x7fc07c0b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c15bd10 con 0x7fc07c06d520 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fc078001670 con 0x7fc07c06d520 2023-11-24T08:04:12.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 >> v1:172.21.15.64:6789/0 conn(0x7fc07c06d520 legacy=0x7fc07c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680039c0 0x7fc068003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680047c0 0x7fc068004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0680051d0 0x7fc06800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06800c0d0 con 0x7fc0680039c0 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06800c240 con 0x7fc0680047c0 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06800c400 con 0x7fc0680051d0 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc081dbf640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680039c0 0x7fc068003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc0815be640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680047c0 0x7fc068004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.379+0000 7fc0825c0640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0680051d0 0x7fc06800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.380+0000 7fc081dbf640 1 -- 172.21.15.64:0/1960507198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0680051d0 msgr2=0x7fc06800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.380+0000 7fc081dbf640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0680051d0 0x7fc06800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.380+0000 7fc081dbf640 1 -- 172.21.15.64:0/1960507198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680047c0 msgr2=0x7fc068004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:12.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.380+0000 7fc081dbf640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680047c0 0x7fc068004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.380+0000 7fc081dbf640 1 -- 172.21.15.64:0/1960507198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068003520 con 0x7fc0680039c0 2023-11-24T08:04:12.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.380+0000 7fc0825c0640 1 -- 172.21.15.64:0/1960507198 reap_dead start 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.381+0000 7fc081dbf640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680039c0 0x7fc068003dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c160 tx=0x7fc0680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4b216675d4689 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.381+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078005ad0 con 0x7fc0680039c0 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.381+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc078005c30 con 0x7fc0680039c0 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.381+0000 7fc080dbd640 1 -- 172.21.15.64:0/1960507198 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07802c790 con 0x7fc0680039c0 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.382+0000 7fc084023640 1 -- 172.21.15.64:0/1960507198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680039c0 msgr2=0x7fc068003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.382+0000 7fc084023640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680039c0 0x7fc068003dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c160 tx=0x7fc0680057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.382+0000 7fc084023640 1 -- 172.21.15.64:0/1960507198 shutdown_connections 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.382+0000 7fc084023640 1 --2- 172.21.15.64:0/1960507198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680039c0 0x7fc068003dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.382+0000 7fc084023640 1 -- 172.21.15.64:0/1960507198 >> 172.21.15.64:0/1960507198 conn(0x7fc07c0b0d80 msgr2=0x7fc07c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.382+0000 7fc084023640 1 -- 172.21.15.64:0/1960507198 shutdown_connections 2023-11-24T08:04:12.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.382+0000 7fc084023640 1 -- 172.21.15.64:0/1960507198 wait complete. 2023-11-24T08:04:12.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.383+0000 7fc084023640 1 Processor -- start 2023-11-24T08:04:12.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.384+0000 7fc084023640 1 -- start start 2023-11-24T08:04:12.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.384+0000 7fc084023640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680039c0 0x7fc07c1f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc084023640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c09ca90 0x7fc07c154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc084023640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c156b90 0x7fc07c157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc084023640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c157560 con 0x7fc07c09ca90 2023-11-24T08:04:12.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc084023640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0cd450 con 0x7fc0680039c0 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc084023640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0ce2b0 con 0x7fc07c156b90 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c09ca90 0x7fc07c154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c09ca90 0x7fc07c154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42500/0 (socket says 172.21.15.64:42500) 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 -- 172.21.15.64:0/1596988759 learned_addr learned my addr 172.21.15.64:0/1596988759 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc081dbf640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680039c0 0x7fc07c1f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0825c0640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c156b90 0x7fc07c157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 -- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c156b90 msgr2=0x7fc07c157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c156b90 0x7fc07c157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 -- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680039c0 msgr2=0x7fc07c1f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680039c0 0x7fc07c1f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 -- 172.21.15.64:0/1596988759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c1f1fe0 con 0x7fc07c09ca90 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc081dbf640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680039c0 0x7fc07c1f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:12.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.385+0000 7fc0815be640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c09ca90 0x7fc07c154f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00e480 tx=0x7fc06c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:12.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.386+0000 7fc0727fc640 1 -- 172.21.15.64:0/1596988759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0135f0 con 0x7fc07c09ca90 2023-11-24T08:04:12.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.386+0000 7fc0727fc640 1 -- 172.21.15.64:0/1596988759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c013770 con 0x7fc07c09ca90 2023-11-24T08:04:12.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.386+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c155480 con 0x7fc07c09ca90 2023-11-24T08:04:12.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.386+0000 7fc0727fc640 1 -- 172.21.15.64:0/1596988759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c013b20 con 0x7fc07c09ca90 2023-11-24T08:04:12.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.386+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c0cd980 con 0x7fc07c09ca90 2023-11-24T08:04:12.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.387+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c0ae6d0 con 0x7fc07c09ca90 2023-11-24T08:04:12.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.388+0000 7fc0727fc640 1 -- 172.21.15.64:0/1596988759 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc06c013c80 con 0x7fc07c09ca90 2023-11-24T08:04:12.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.389+0000 7fc0727fc640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc06006ae40 0x7fc06006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.389+0000 7fc0727fc640 1 -- 172.21.15.64:0/1596988759 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc06c089790 con 0x7fc07c09ca90 2023-11-24T08:04:12.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.390+0000 7fc081dbf640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc06006ae40 0x7fc06006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.391+0000 7fc081dbf640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc06006ae40 0x7fc06006d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc078034000 tx=0x7fc078009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:12.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.393+0000 7fc0727fc640 1 -- 172.21.15.64:0/1596988759 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc06c0548d0 con 0x7fc07c09ca90 2023-11-24T08:04:12.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.589+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc07c0ce420 con 0x7fc07c09ca90 2023-11-24T08:04:12.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.591+0000 7fc0727fc640 1 -- 172.21.15.64:0/1596988759 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fc06c057ef0 con 0x7fc07c09ca90 2023-11-24T08:04:12.593 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:12.593 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc06006ae40 msgr2=0x7fc06006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc06006ae40 0x7fc06006d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc078034000 tx=0x7fc078009030 comp rx=0 tx=0).stop 2023-11-24T08:04:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c09ca90 msgr2=0x7fc07c154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c09ca90 0x7fc07c154f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00e480 tx=0x7fc06c00e940 comp rx=0 tx=0).stop 2023-11-24T08:04:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 shutdown_connections 2023-11-24T08:04:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc06006ae40 0x7fc06006d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc07c156b90 0x7fc07c157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c09ca90 0x7fc07c154f40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 --2- 172.21.15.64:0/1596988759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc0680039c0 0x7fc07c1f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 >> 172.21.15.64:0/1596988759 conn(0x7fc07c0b0d80 msgr2=0x7fc07c0af730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 shutdown_connections 2023-11-24T08:04:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.594+0000 7fc084023640 1 -- 172.21.15.64:0/1596988759 wait complete. 2023-11-24T08:04:12.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:04:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.835+0000 7f51127fa640 1 Processor -- start 2023-11-24T08:04:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.835+0000 7f51127fa640 1 -- start start 2023-11-24T08:04:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f51127fa640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f510c0cf050 0x7f510c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f51127fa640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f510c0cd7b0 0x7f510c0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f51127fa640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510c15fa70 con 0x7f510c13dfd0 2023-11-24T08:04:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f51127fa640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c0cfa40 con 0x7f510c0cf050 2023-11-24T08:04:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f51127fa640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c15c580 con 0x7f510c0cd7b0 2023-11-24T08:04:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f5110ff7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f510c0cf050 0x7f510c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f51117f8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f510c13dfd0 0x7f510c15be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43152/0 (socket says 172.21.15.64:43152) 2023-11-24T08:04:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f51117f8640 1 -- 172.21.15.64:0/691423648 learned_addr learned my addr 172.21.15.64:0/691423648 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.837+0000 7f5111ff9640 1 --2- 172.21.15.64:0/691423648 >> v2:172.21.15.91:3300/0 conn(0x7f510c0cd7b0 0x7f510c0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.838+0000 7f5110ff7640 1 -- 172.21.15.64:0/691423648 >> v2:172.21.15.91:3300/0 conn(0x7f510c0cd7b0 msgr2=0x7f510c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.838+0000 7f5110ff7640 1 --2- 172.21.15.64:0/691423648 >> v2:172.21.15.91:3300/0 conn(0x7f510c0cd7b0 0x7f510c0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.838+0000 7f5110ff7640 1 -- 172.21.15.64:0/691423648 >> v1:172.21.15.64:6789/0 conn(0x7f510c13dfd0 legacy=0x7f510c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.838+0000 7f5110ff7640 1 -- 172.21.15.64:0/691423648 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c140900 con 0x7f510c0cf050 2023-11-24T08:04:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.838+0000 7f5103fff640 1 -- 172.21.15.64:0/691423648 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835539759 0 0) 0x7f510c15fa70 con 0x7f510c13dfd0 2023-11-24T08:04:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.839+0000 7f5110ff7640 1 --2- 172.21.15.64:0/691423648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c0cf050 0x7f510c0cf500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f510c1416f0 tx=0x7f50fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9594791837779f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.839+0000 7f5103fff640 1 -- 172.21.15.64:0/691423648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc031830 con 0x7f510c0cf050 2023-11-24T08:04:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.839+0000 7f5103fff640 1 -- 172.21.15.64:0/691423648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc031990 con 0x7f510c0cf050 2023-11-24T08:04:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.839+0000 7f5103fff640 1 -- 172.21.15.64:0/691423648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc031d20 con 0x7f510c0cf050 2023-11-24T08:04:12.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.840+0000 7f51127fa640 1 -- 172.21.15.64:0/691423648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c0cf050 msgr2=0x7f510c0cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.840+0000 7f51127fa640 1 --2- 172.21.15.64:0/691423648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c0cf050 0x7f510c0cf500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f510c1416f0 tx=0x7f50fc02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.840+0000 7f51127fa640 1 -- 172.21.15.64:0/691423648 shutdown_connections 2023-11-24T08:04:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.840+0000 7f51127fa640 1 --2- 172.21.15.64:0/691423648 >> v2:172.21.15.91:3300/0 conn(0x7f510c0cd7b0 0x7f510c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.840+0000 7f51127fa640 1 --2- 172.21.15.64:0/691423648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c0cf050 0x7f510c0cf500 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.840+0000 7f51127fa640 1 -- 172.21.15.64:0/691423648 >> 172.21.15.64:0/691423648 conn(0x7f510c06a860 msgr2=0x7f510c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.840+0000 7f51127fa640 1 -- 172.21.15.64:0/691423648 shutdown_connections 2023-11-24T08:04:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.840+0000 7f51127fa640 1 -- 172.21.15.64:0/691423648 wait complete. 2023-11-24T08:04:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.842+0000 7f51127fa640 1 Processor -- start 2023-11-24T08:04:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.842+0000 7f51127fa640 1 -- start start 2023-11-24T08:04:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f51127fa640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c0cd7b0 0x7f510c14def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f51127fa640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c0cf050 0x7f510c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f51127fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c13dfd0 0x7f510c12d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f51127fa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c159000 con 0x7f510c13dfd0 2023-11-24T08:04:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f51127fa640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c1592e0 con 0x7f510c0cd7b0 2023-11-24T08:04:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f51127fa640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c1595c0 con 0x7f510c0cf050 2023-11-24T08:04:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f5111ff9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c13dfd0 0x7f510c12d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f5111ff9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c13dfd0 0x7f510c12d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42524/0 (socket says 172.21.15.64:42524) 2023-11-24T08:04:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.843+0000 7f5111ff9640 1 -- 172.21.15.64:0/1068235804 learned_addr learned my addr 172.21.15.64:0/1068235804 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f51117f8640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c0cd7b0 0x7f510c14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5110ff7640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c0cf050 0x7f510c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5111ff9640 1 -- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c0cf050 msgr2=0x7f510c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5111ff9640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c0cf050 0x7f510c15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5111ff9640 1 -- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c0cd7b0 msgr2=0x7f510c14def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5111ff9640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c0cd7b0 0x7f510c14def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5111ff9640 1 -- 172.21.15.64:0/1068235804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc009400 con 0x7f510c13dfd0 2023-11-24T08:04:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5111ff9640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c13dfd0 0x7f510c12d230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5104007e40 tx=0x7f510400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5101ffb640 1 -- 172.21.15.64:0/1068235804 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104014620 con 0x7f510c13dfd0 2023-11-24T08:04:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5101ffb640 1 -- 172.21.15.64:0/1068235804 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510400ee90 con 0x7f510c13dfd0 2023-11-24T08:04:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f5101ffb640 1 -- 172.21.15.64:0/1068235804 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51040149e0 con 0x7f510c13dfd0 2023-11-24T08:04:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.844+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c1598a0 con 0x7f510c13dfd0 2023-11-24T08:04:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.845+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c15a5e0 con 0x7f510c13dfd0 2023-11-24T08:04:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.846+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c159dc0 con 0x7f510c13dfd0 2023-11-24T08:04:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.847+0000 7f5101ffb640 1 -- 172.21.15.64:0/1068235804 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5104014b40 con 0x7f510c13dfd0 2023-11-24T08:04:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.848+0000 7f5101ffb640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f50f006ae70 0x7f50f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.848+0000 7f5101ffb640 1 -- 172.21.15.64:0/1068235804 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f510408a060 con 0x7f510c13dfd0 2023-11-24T08:04:12.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.848+0000 7f51117f8640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f50f006ae70 0x7f50f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:12.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.849+0000 7f51117f8640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f50f006ae70 0x7f50f006d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5108000f60 tx=0x7f5108007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:12.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:12.852+0000 7f5101ffb640 1 -- 172.21.15.64:0/1068235804 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51040bce90 con 0x7f510c13dfd0 2023-11-24T08:04:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.085+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f510c04e840 con 0x7f510c13dfd0 2023-11-24T08:04:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.086+0000 7f5101ffb640 1 -- 172.21.15.64:0/1068235804 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f510c159dc0 con 0x7f510c13dfd0 2023-11-24T08:04:13.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:13.089 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:03:55.699656+0000","modified":"2023-11-24T08:03:58.759467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:04:13.090 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T08:04:13.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f50f006ae70 msgr2=0x7f50f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f50f006ae70 0x7f50f006d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5108000f60 tx=0x7f5108007c00 comp rx=0 tx=0).stop 2023-11-24T08:04:13.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c13dfd0 msgr2=0x7f510c12d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c13dfd0 0x7f510c12d230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5104007e40 tx=0x7f510400e4b0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 shutdown_connections 2023-11-24T08:04:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f50f006ae70 0x7f50f006d330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c13dfd0 0x7f510c12d230 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c0cf050 0x7f510c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 --2- 172.21.15.64:0/1068235804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c0cd7b0 0x7f510c14def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 >> 172.21.15.64:0/1068235804 conn(0x7f510c06a860 msgr2=0x7f510c15e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 shutdown_connections 2023-11-24T08:04:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.089+0000 7f51127fa640 1 -- 172.21.15.64:0/1068235804 wait complete. 2023-11-24T08:04:13.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:03:55.699656+0000', 'modified': '2023-11-24T08:03:58.759467+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.91:6837/3953784254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.91:6835', 'nonce': 3953784254}, {'type': 'v1', 'addr': '172.21.15.91:6837', 'nonce': 3953784254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:04:13.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:04:13.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-24T08:04:13.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.749+0000 7f650a516640 1 Processor -- start 2023-11-24T08:04:13.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.749+0000 7f650a516640 1 -- start start 2023-11-24T08:04:13.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.750+0000 7f650a516640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65040cea20 0x7f65040d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.750+0000 7f650a516640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f650406e440 0x7f650406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.750+0000 7f650a516640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040cb960 con 0x7f650415b370 2023-11-24T08:04:13.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.750+0000 7f650a516640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650415b750 con 0x7f650406e440 2023-11-24T08:04:13.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.750+0000 7f650a516640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040cbb90 con 0x7f65040cea20 2023-11-24T08:04:13.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.751+0000 7f6509d15640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f650406e440 0x7f650406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.751+0000 7f6509d15640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f650406e440 0x7f650406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42526/0 (socket says 172.21.15.64:42526) 2023-11-24T08:04:13.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.751+0000 7f6509d15640 1 -- 172.21.15.64:0/4122703058 learned_addr learned my addr 172.21.15.64:0/4122703058 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:13.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.751+0000 7f6508d13640 1 --2- 172.21.15.64:0/4122703058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65040cea20 0x7f65040d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.751+0000 7f6509d15640 1 -- 172.21.15.64:0/4122703058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65040cea20 msgr2=0x7f65040d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.751+0000 7f6509d15640 1 --2- 172.21.15.64:0/4122703058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65040cea20 0x7f65040d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.751+0000 7f6509d15640 1 -- 172.21.15.64:0/4122703058 >> v1:172.21.15.91:6789/0 conn(0x7f650415b370 legacy=0x7f65040cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.751+0000 7f6509d15640 1 -- 172.21.15.64:0/4122703058 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040cf200 con 0x7f650406e440 2023-11-24T08:04:13.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.752+0000 7f6509d15640 1 --2- 172.21.15.64:0/4122703058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f650406e440 0x7f650406e8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009870 tx=0x7f64fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b4ce794bee40ba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f6503fff640 1 -- 172.21.15.64:0/4122703058 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc031640 con 0x7f650406e440 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f6503fff640 1 -- 172.21.15.64:0/4122703058 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc0317a0 con 0x7f650406e440 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f6503fff640 1 -- 172.21.15.64:0/4122703058 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc031b30 con 0x7f650406e440 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f650a516640 1 -- 172.21.15.64:0/4122703058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f650406e440 msgr2=0x7f650406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f650a516640 1 --2- 172.21.15.64:0/4122703058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f650406e440 0x7f650406e8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009870 tx=0x7f64fc02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f650a516640 1 -- 172.21.15.64:0/4122703058 shutdown_connections 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f650a516640 1 --2- 172.21.15.64:0/4122703058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f650406e440 0x7f650406e8f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f650a516640 1 --2- 172.21.15.64:0/4122703058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65040cea20 0x7f65040d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.753+0000 7f650a516640 1 -- 172.21.15.64:0/4122703058 >> 172.21.15.64:0/4122703058 conn(0x7f65040d0e50 msgr2=0x7f65040d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:13.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.754+0000 7f650a516640 1 -- 172.21.15.64:0/4122703058 shutdown_connections 2023-11-24T08:04:13.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.754+0000 7f650a516640 1 -- 172.21.15.64:0/4122703058 wait complete. 2023-11-24T08:04:13.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.755+0000 7f650a516640 1 Processor -- start 2023-11-24T08:04:13.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.756+0000 7f650a516640 1 -- start start 2023-11-24T08:04:13.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.756+0000 7f650a516640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f650406e440 0x7f65040c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f650a516640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040cea20 0x7f6504158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f650a516640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f650415b370 0x7f650406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f650a516640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504159440 con 0x7f65040cea20 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f650a516640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b4b90 con 0x7f650415b370 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f650a516640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b3ca0 con 0x7f650406e440 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6508d13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040cea20 0x7f6504158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6508d13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040cea20 0x7f6504158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42540/0 (socket says 172.21.15.64:42540) 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6508d13640 1 -- 172.21.15.64:0/2454316577 learned_addr learned my addr 172.21.15.64:0/2454316577 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6509514640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f650406e440 0x7f65040c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6509d15640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f650415b370 0x7f650406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6508d13640 1 -- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f650406e440 msgr2=0x7f65040c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6508d13640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f650406e440 0x7f65040c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6508d13640 1 -- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f650415b370 msgr2=0x7f650406c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6508d13640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f650415b370 0x7f650406c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.757+0000 7f6508d13640 1 -- 172.21.15.64:0/2454316577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0094d0 con 0x7f65040cea20 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.758+0000 7f6508d13640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040cea20 0x7f6504158f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d620 tx=0x7f64f800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.758+0000 7f6501ffb640 1 -- 172.21.15.64:0/2454316577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f800b620 con 0x7f65040cea20 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.758+0000 7f6501ffb640 1 -- 172.21.15.64:0/2454316577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f800b780 con 0x7f65040cea20 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.758+0000 7f6501ffb640 1 -- 172.21.15.64:0/2454316577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f800bb30 con 0x7f65040cea20 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.758+0000 7f650a516640 1 -- 172.21.15.64:0/2454316577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65040b3f80 con 0x7f65040cea20 2023-11-24T08:04:13.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.758+0000 7f650a516640 1 -- 172.21.15.64:0/2454316577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65040b4540 con 0x7f65040cea20 2023-11-24T08:04:13.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.760+0000 7f6501ffb640 1 -- 172.21.15.64:0/2454316577 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64f800bc90 con 0x7f65040cea20 2023-11-24T08:04:13.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.761+0000 7f6501ffb640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f64f006ae40 0x7f64f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.761+0000 7f6501ffb640 1 -- 172.21.15.64:0/2454316577 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64f8089af0 con 0x7f65040cea20 2023-11-24T08:04:13.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.762+0000 7f6509514640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f64f006ae40 0x7f64f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.762+0000 7f650e3e8740 1 Processor -- start 2023-11-24T08:04:13.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.762+0000 7f650e3e8740 1 -- start start 2023-11-24T08:04:13.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.763+0000 7f6509514640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f64f006ae40 0x7f64f006d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f64f4005520 tx=0x7f64f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:13.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.763+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d04948b070 0x55d04948b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.763+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d049485d70 0x55d04948a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.763+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.763+0000 7f650e3e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0493e9880 con 0x55d04948b070 2023-11-24T08:04:13.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.763+0000 7f650e3e8740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0493f8ae0 con 0x55d04948a7b0 2023-11-24T08:04:13.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.763+0000 7f650e3e8740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0493f61d0 con 0x55d049485d70 2023-11-24T08:04:13.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.763+0000 7f6509514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d04948b070 0x55d04948b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d04948b070 0x55d04948b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42552/0 (socket says 172.21.15.64:42552) 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509514640 1 -- 172.21.15.64:0/2161343367 learned_addr learned my addr 172.21.15.64:0/2161343367 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6508d13640 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d049485d70 0x55d04948a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509514640 1 -- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d049485d70 msgr2=0x55d04948a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509514640 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d049485d70 0x55d04948a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509d15640 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509514640 1 -- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 msgr2=0x55d0494ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509514640 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509514640 1 -- 172.21.15.64:0/2161343367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0493b9860 con 0x55d04948b070 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509d15640 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.764+0000 7f6509514640 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d04948b070 0x55d04948b450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f64f400eb40 tx=0x7f64f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0821363f5893daf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:13.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f64eaffd640 1 -- 172.21.15.64:0/2161343367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4015c80 con 0x55d04948b070 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f64eaffd640 1 -- 172.21.15.64:0/2161343367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4015de0 con 0x55d04948b070 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f64eaffd640 1 -- 172.21.15.64:0/2161343367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4020600 con 0x55d04948b070 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 -- 172.21.15.64:0/2161343367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d04948b070 msgr2=0x55d04948b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d04948b070 0x55d04948b450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f64f400eb40 tx=0x7f64f4011000 comp rx=0 tx=0).stop 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 -- 172.21.15.64:0/2161343367 shutdown_connections 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d049485d70 0x55d04948a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2161343367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d04948b070 0x55d04948b450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 -- 172.21.15.64:0/2161343367 >> 172.21.15.64:0/2161343367 conn(0x55d04944cc70 msgr2=0x55d0494d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 -- 172.21.15.64:0/2161343367 shutdown_connections 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.765+0000 7f650e3e8740 1 -- 172.21.15.64:0/2161343367 wait complete. 2023-11-24T08:04:13.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.766+0000 7f650e3e8740 1 -- start start 2023-11-24T08:04:13.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.766+0000 7f650e3e8740 1 Processor -- start 2023-11-24T08:04:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.767+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.767+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948a7b0 0x55d0494d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.767+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948b070 0x55d0494d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.767+0000 7f650e3e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0493f1ac0 con 0x55d049485d70 2023-11-24T08:04:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.767+0000 7f650e3e8740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04938e2f0 con 0x55d04948b070 2023-11-24T08:04:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.767+0000 7f650e3e8740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0493abc70 con 0x55d04948a7b0 2023-11-24T08:04:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.767+0000 7f650e3e8740 10 client.0 _ll_register_callbacks cb 0x55d0493b03b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:04:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42554/0 (socket says 172.21.15.64:42554) 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 -- 172.21.15.64:0/594777527 learned_addr learned my addr 172.21.15.64:0/594777527 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509d15640 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948b070 0x55d0494d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6508d13640 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948a7b0 0x55d0494d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 -- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948a7b0 msgr2=0x55d0494d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948a7b0 0x55d0494d2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 -- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948b070 msgr2=0x55d0494d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948b070 0x55d0494d2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 -- 172.21.15.64:0/594777527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f400f040 con 0x55d049485d70 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f6509514640 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494cf4a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64f400eae0 tx=0x7f64f4003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8a16cf479a3c44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.768+0000 7f64dffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:04:13.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f64dffff640 1 -- 172.21.15.64:0/594777527 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4003e70 con 0x55d049485d70 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f64dffff640 1 -- 172.21.15.64:0/594777527 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64f402b650 con 0x55d049485d70 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 1 -- 172.21.15.64:0/594777527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0493eaaa0 con 0x55d049485d70 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f64dffff640 1 -- 172.21.15.64:0/594777527 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f402b810 con 0x55d049485d70 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 10 client.4662 fetch_fsmap learned FSMap version 8 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 1 -- 172.21.15.64:0/594777527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d0493f0140 con 0x55d049485d70 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f64dffff640 1 -- 172.21.15.64:0/594777527 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d0493eaaa0 con 0x55d049485d70 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f64dffff640 1 -- 172.21.15.64:0/594777527 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f64f4032060 con 0x55d049485d70 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 10 client.4662 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 20 client.4662 populate_metadata read hostname 'smithi064' 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] conn(0x55d0494a99e0 0x55d0494edd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 4 client.4662 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:04:13.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.769+0000 7f650e3e8740 1 -- 172.21.15.64:0/594777527 --> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0493ea7c0 con 0x55d0494a99e0 2023-11-24T08:04:13.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.770+0000 7f6508d13640 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] conn(0x55d0494a99e0 0x55d0494edd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.771+0000 7f6508d13640 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] conn(0x55d0494a99e0 0x55d0494edd70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914e1c2778feba82 server_cookie=e4484d98fc9f7586 in_seq=0 out_seq=0 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.771+0000 7f64dffff640 10 client.4662 ms_handle_connect on v2:172.21.15.91:6835/3953784254 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f64dffff640 1 -- 172.21.15.64:0/594777527 <== mds.0 v2:172.21.15.91:6835/3953784254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0493ea7c0 con 0x55d0494a99e0 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f64dffff640 10 client.4662 handle_command_reply: tid=0 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 client.4662 shutdown 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 -- 172.21.15.64:0/594777527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 msgr2=0x55d0494cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494cf4a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64f400eae0 tx=0x7f64f4003a70 comp rx=0 tx=0).stop 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 -- 172.21.15.64:0/594777527 shutdown_connections 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] conn(0x55d0494a99e0 0x55d0494edd70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948b070 0x55d0494d2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948a7b0 0x55d0494d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 --2- 172.21.15.64:0/594777527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494cf4a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f64dffff640 0 client.4662 ms_handle_reset on v2:172.21.15.91:6835/3953784254 2023-11-24T08:04:13.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 -- 172.21.15.64:0/594777527 >> 172.21.15.64:0/594777527 conn(0x55d04944cc70 msgr2=0x55d0494d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:13.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 -- 172.21.15.64:0/594777527 shutdown_connections 2023-11-24T08:04:13.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 1 -- 172.21.15.64:0/594777527 wait complete. 2023-11-24T08:04:13.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.773+0000 7f650e3e8740 20 client.4662 trim_cache size 0 max 16384 2023-11-24T08:04:13.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.791+0000 7f650e3e8740 1 Processor -- start 2023-11-24T08:04:13.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.791+0000 7f650e3e8740 1 -- start start 2023-11-24T08:04:13.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 0x55d0494f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f650e3e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d049154f20 con 0x55d049485d70 2023-11-24T08:04:13.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f650e3e8740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0494d1410 con 0x55d04948a7b0 2023-11-24T08:04:13.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f650e3e8740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d049481930 con 0x55d04948b070 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f6509514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f6509514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494d21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42560/0 (socket says 172.21.15.64:42560) 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f6509514640 1 -- 172.21.15.64:0/2765336429 learned_addr learned my addr 172.21.15.64:0/2765336429 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f6508d13640 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f6509d15640 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 0x55d0494f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.792+0000 7f6509514640 1 -- 172.21.15.64:0/2765336429 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 msgr2=0x55d0494f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.793+0000 7f6509514640 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 0x55d0494f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.793+0000 7f6509514640 1 -- 172.21.15.64:0/2765336429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 msgr2=0x55d0494f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.793+0000 7f6509514640 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.793+0000 7f6509514640 1 -- 172.21.15.64:0/2765336429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d04938db80 con 0x55d049485d70 2023-11-24T08:04:13.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.793+0000 7f6509514640 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494d21d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f64f400e9e0 tx=0x7f64f402b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca76528512a599f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:13.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.793+0000 7f64eb7fe640 1 -- 172.21.15.64:0/2765336429 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f402bd60 con 0x55d049485d70 2023-11-24T08:04:13.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.793+0000 7f64eb7fe640 1 -- 172.21.15.64:0/2765336429 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4019070 con 0x55d049485d70 2023-11-24T08:04:13.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.793+0000 7f64eb7fe640 1 -- 172.21.15.64:0/2765336429 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4020400 con 0x55d049485d70 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 -- 172.21.15.64:0/2765336429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 msgr2=0x55d0494d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494d21d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f64f400e9e0 tx=0x7f64f402b000 comp rx=0 tx=0).stop 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 -- 172.21.15.64:0/2765336429 shutdown_connections 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 0x55d0494f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2765336429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d0494d21d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 -- 172.21.15.64:0/2765336429 >> 172.21.15.64:0/2765336429 conn(0x55d04944cc70 msgr2=0x55d04948b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 -- 172.21.15.64:0/2765336429 shutdown_connections 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.794+0000 7f650e3e8740 1 -- 172.21.15.64:0/2765336429 wait complete. 2023-11-24T08:04:13.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.795+0000 7f650e3e8740 1 -- start start 2023-11-24T08:04:13.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.795+0000 7f650e3e8740 1 Processor -- start 2023-11-24T08:04:13.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d04953a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f650e3e8740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 0x55d0494f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f650e3e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0494d0410 con 0x55d049485d70 2023-11-24T08:04:13.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f650e3e8740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0494b1780 con 0x55d04948a7b0 2023-11-24T08:04:13.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f650e3e8740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d04953c070 con 0x55d04948b070 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f650e3e8740 10 client.0 _ll_register_callbacks cb 0x55d0494d5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f6509514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d04953a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f6509514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d04953a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42570/0 (socket says 172.21.15.64:42570) 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f6509514640 1 -- 172.21.15.64:0/2459709869 learned_addr learned my addr 172.21.15.64:0/2459709869 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.796+0000 7f6508d13640 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f6509d15640 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 0x55d0494f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f6509514640 1 -- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 msgr2=0x55d0494f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f6509514640 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 0x55d0494f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f6509514640 1 -- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 msgr2=0x55d0494f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f6509514640 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f6509514640 1 -- 172.21.15.64:0/2459709869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0493f0140 con 0x55d049485d70 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f6508d13640 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:13.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f6509514640 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d04953a380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f64f402bfd0 tx=0x7f64f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864728113a149d31 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f64dffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f64dffff640 1 -- 172.21.15.64:0/2459709869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4020610 con 0x55d049485d70 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f64dffff640 1 -- 172.21.15.64:0/2459709869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64f4046ae0 con 0x55d049485d70 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.797+0000 7f650e3e8740 1 -- 172.21.15.64:0/2459709869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64f4015de0 con 0x55d049485d70 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f64dffff640 1 -- 172.21.15.64:0/2459709869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f4046ca0 con 0x55d049485d70 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f650e3e8740 10 client.4668 fetch_fsmap learned FSMap version 8 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f650e3e8740 1 -- 172.21.15.64:0/2459709869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d0493b3170 con 0x55d049485d70 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f64dffff640 1 -- 172.21.15.64:0/2459709869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4015de0 con 0x55d049485d70 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f64dffff640 1 -- 172.21.15.64:0/2459709869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f64f4020770 con 0x55d049485d70 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f650e3e8740 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T08:04:13.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f650e3e8740 10 client.4668 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T08:04:13.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f650e3e8740 20 client.4668 populate_metadata read hostname 'smithi064' 2023-11-24T08:04:13.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] conn(0x55d0494aac00 0x55d0494aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:13.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f650e3e8740 4 client.4668 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:04:13.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.798+0000 7f650e3e8740 1 -- 172.21.15.64:0/2459709869 --> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d0490f26c0 con 0x55d0494aac00 2023-11-24T08:04:13.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.799+0000 7f6508d13640 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] conn(0x55d0494aac00 0x55d0494aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:13.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.799+0000 7f6508d13640 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] conn(0x55d0494aac00 0x55d0494aafe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16aadb76fbb1ae4d server_cookie=ee3b972e5b9a4572 in_seq=0 out_seq=0 2023-11-24T08:04:13.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.799+0000 7f64dffff640 10 client.4668 ms_handle_connect on v2:172.21.15.91:6835/3953784254 2023-11-24T08:04:13.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f64dffff640 1 -- 172.21.15.64:0/2459709869 <== mds.0 v2:172.21.15.91:6835/3953784254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55d0490f26c0 con 0x55d0494aac00 2023-11-24T08:04:13.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f64dffff640 10 client.4668 handle_command_reply: tid=0 2023-11-24T08:04:13.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 client.4668 shutdown 2023-11-24T08:04:13.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 -- 172.21.15.64:0/2459709869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 msgr2=0x55d04953a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d04953a380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f64f402bfd0 tx=0x7f64f4014000 comp rx=0 tx=0).stop 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 -- 172.21.15.64:0/2459709869 shutdown_connections 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:6835/3953784254,v1:172.21.15.91:6837/3953784254] conn(0x55d0494aac00 0x55d0494aafe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55d04948b070 0x55d0494f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55d04948a7b0 0x55d0494f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 --2- 172.21.15.64:0/2459709869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d049485d70 0x55d04953a380 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f64dffff640 0 client.4668 ms_handle_reset on v2:172.21.15.91:6835/3953784254 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 -- 172.21.15.64:0/2459709869 >> 172.21.15.64:0/2459709869 conn(0x55d04944cc70 msgr2=0x55d0494d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 -- 172.21.15.64:0/2459709869 shutdown_connections 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.800+0000 7f650e3e8740 1 -- 172.21.15.64:0/2459709869 wait complete. 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650e3e8740 20 client.4668 trim_cache size 0 max 16384 2023-11-24T08:04:13.805 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918", 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4290, 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 17.559701234999999, 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 8, 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 20, 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 20, 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 23.252346157000002 2023-11-24T08:04:13.806 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:04:13.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 -- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f64f006ae40 msgr2=0x7f64f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f64f006ae40 0x7f64f006d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f64f4005520 tx=0x7f64f4009040 comp rx=0 tx=0).stop 2023-11-24T08:04:13.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 -- 172.21.15.64:0/2454316577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040cea20 msgr2=0x7f6504158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:13.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040cea20 0x7f6504158f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d620 tx=0x7f64f800dae0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 -- 172.21.15.64:0/2454316577 shutdown_connections 2023-11-24T08:04:13.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f64f006ae40 0x7f64f006d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f650415b370 0x7f650406c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65040cea20 0x7f6504158f00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 --2- 172.21.15.64:0/2454316577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f650406e440 0x7f65040c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:13.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 -- 172.21.15.64:0/2454316577 >> 172.21.15.64:0/2454316577 conn(0x7f65040d0e50 msgr2=0x7f65040b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:13.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 -- 172.21.15.64:0/2454316577 shutdown_connections 2023-11-24T08:04:13.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:13.801+0000 7f650a516640 1 -- 172.21.15.64:0/2454316577 wait complete. 2023-11-24T08:04:13.820 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-24T08:04:13.843 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-24T08:04:13.843 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T08:04:13.844 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-24T08:04:13.844 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T08:04:13.844 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:13.844 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-24T08:04:13.844 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T08:04:13.844 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T08:04:13.844 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-24T08:04:13.844 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T08:04:13.844 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:13.845 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-24T08:04:13.845 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T08:04:13.845 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T08:04:13.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:13.845 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:04:13.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:13.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-11-24T08:04:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2023-11-24T08:04:14.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:04:14.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:14.004 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:04:14.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:14.068 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link delete ceph-brx 2023-11-24T08:04:14.170 INFO:teuthology.orchestra.run.smithi091.stderr:Cannot find device "ceph-brx" 2023-11-24T08:04:14.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:04:14.174 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-24T08:04:14.174 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:04:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.404+0000 7f7d23fff640 1 Processor -- start 2023-11-24T08:04:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.404+0000 7f7d23fff640 1 -- start start 2023-11-24T08:04:14.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.405+0000 7f7d23fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d2415d5b0 0x7f7d240d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.405+0000 7f7d23fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7d240b1650 0x7f7d240d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.405+0000 7f7d23fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d240cf0c0 con 0x7f7d240d14a0 2023-11-24T08:04:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.405+0000 7f7d23fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d240b01b0 con 0x7f7d2415d5b0 2023-11-24T08:04:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.405+0000 7f7d23fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2415d990 con 0x7f7d240b1650 2023-11-24T08:04:14.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.406+0000 7f7d22ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d2415d5b0 0x7f7d240d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:14.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.406+0000 7f7d22ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d2415d5b0 0x7f7d240d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42576/0 (socket says 172.21.15.64:42576) 2023-11-24T08:04:14.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.406+0000 7f7d22ffd640 1 -- 172.21.15.64:0/2519588484 learned_addr learned my addr 172.21.15.64:0/2519588484 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:14.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.406+0000 7f7d237fe640 1 --2- 172.21.15.64:0/2519588484 >> v2:172.21.15.91:3300/0 conn(0x7f7d240b1650 0x7f7d240d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.406+0000 7f7d22ffd640 1 -- 172.21.15.64:0/2519588484 >> v2:172.21.15.91:3300/0 conn(0x7f7d240b1650 msgr2=0x7f7d240d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.407+0000 7f7d22ffd640 1 --2- 172.21.15.64:0/2519588484 >> v2:172.21.15.91:3300/0 conn(0x7f7d240b1650 0x7f7d240d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.407+0000 7f7d22ffd640 1 -- 172.21.15.64:0/2519588484 >> v1:172.21.15.91:6789/0 conn(0x7f7d240d14a0 legacy=0x7f7d240cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:14.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.407+0000 7f7d22ffd640 1 -- 172.21.15.64:0/2519588484 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d240d1e40 con 0x7f7d2415d5b0 2023-11-24T08:04:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.408+0000 7f7d22ffd640 1 --2- 172.21.15.64:0/2519588484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240d3990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d2408bc30 tx=0x7f7d1402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64425e0bf151028e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.408+0000 7f7d21ffb640 1 -- 172.21.15.64:0/2519588484 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1402ca80 con 0x7f7d2415d5b0 2023-11-24T08:04:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.409+0000 7f7d21ffb640 1 -- 172.21.15.64:0/2519588484 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1402cbe0 con 0x7f7d2415d5b0 2023-11-24T08:04:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.409+0000 7f7d21ffb640 1 -- 172.21.15.64:0/2519588484 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d140343f0 con 0x7f7d2415d5b0 2023-11-24T08:04:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.409+0000 7f7d23fff640 1 -- 172.21.15.64:0/2519588484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 msgr2=0x7f7d240d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.409+0000 7f7d23fff640 1 --2- 172.21.15.64:0/2519588484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240d3990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d2408bc30 tx=0x7f7d1402bee0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.409+0000 7f7d23fff640 1 -- 172.21.15.64:0/2519588484 shutdown_connections 2023-11-24T08:04:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.409+0000 7f7d23fff640 1 --2- 172.21.15.64:0/2519588484 >> v2:172.21.15.91:3300/0 conn(0x7f7d240b1650 0x7f7d240d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.409+0000 7f7d23fff640 1 --2- 172.21.15.64:0/2519588484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240d3990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.409+0000 7f7d23fff640 1 -- 172.21.15.64:0/2519588484 >> 172.21.15.64:0/2519588484 conn(0x7f7d2409c3b0 msgr2=0x7f7d240d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.410+0000 7f7d23fff640 1 -- 172.21.15.64:0/2519588484 shutdown_connections 2023-11-24T08:04:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.410+0000 7f7d23fff640 1 -- 172.21.15.64:0/2519588484 wait complete. 2023-11-24T08:04:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.412+0000 7f7d23fff640 1 Processor -- start 2023-11-24T08:04:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.412+0000 7f7d23fff640 1 -- start start 2023-11-24T08:04:14.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.413+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d240b1650 0x7f7d240b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.413+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d240d14a0 0x7f7d240c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.413+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.413+0000 7f7d23fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240c9b00 con 0x7f7d2415d5b0 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.413+0000 7f7d23fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240ca680 con 0x7f7d240d14a0 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.413+0000 7f7d23fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240b6950 con 0x7f7d240b1650 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42588/0 (socket says 172.21.15.64:42588) 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d237fe640 1 -- 172.21.15.64:0/4262538912 learned_addr learned my addr 172.21.15.64:0/4262538912 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d227fc640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d240d14a0 0x7f7d240c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d22ffd640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d240b1650 0x7f7d240b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d237fe640 1 -- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d240b1650 msgr2=0x7f7d240b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d237fe640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d240b1650 0x7f7d240b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d237fe640 1 -- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d240d14a0 msgr2=0x7f7d240c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d237fe640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d240d14a0 0x7f7d240c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.414+0000 7f7d237fe640 1 -- 172.21.15.64:0/4262538912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14009b30 con 0x7f7d2415d5b0 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.415+0000 7f7d237fe640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240ca140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00c540 tx=0x7f7d1c00ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.415+0000 7f7d1b7fe640 1 -- 172.21.15.64:0/4262538912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004220 con 0x7f7d2415d5b0 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.415+0000 7f7d1b7fe640 1 -- 172.21.15.64:0/4262538912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c004380 con 0x7f7d2415d5b0 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.415+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d240ca890 con 0x7f7d2415d5b0 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.415+0000 7f7d1b7fe640 1 -- 172.21.15.64:0/4262538912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c003850 con 0x7f7d2415d5b0 2023-11-24T08:04:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.415+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d240cae50 con 0x7f7d2415d5b0 2023-11-24T08:04:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.416+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d240b6b10 con 0x7f7d2415d5b0 2023-11-24T08:04:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.417+0000 7f7d1b7fe640 1 -- 172.21.15.64:0/4262538912 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d1c0039b0 con 0x7f7d2415d5b0 2023-11-24T08:04:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.418+0000 7f7d1b7fe640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d0406ae40 0x7f7d0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.418+0000 7f7d1b7fe640 1 -- 172.21.15.64:0/4262538912 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d1c0890d0 con 0x7f7d2415d5b0 2023-11-24T08:04:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.419+0000 7f7d22ffd640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d0406ae40 0x7f7d0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.420+0000 7f7d22ffd640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d0406ae40 0x7f7d0406d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7d14035000 tx=0x7f7d14030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.423+0000 7f7d1b7fe640 1 -- 172.21.15.64:0/4262538912 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d240b6b10 con 0x7f7d2415d5b0 2023-11-24T08:04:14.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:04:14.495 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:04:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.654+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7d240b6b10 con 0x7f7d2415d5b0 2023-11-24T08:04:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.655+0000 7f7d1b7fe640 1 -- 172.21.15.64:0/4262538912 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7d240b6b10 con 0x7f7d2415d5b0 2023-11-24T08:04:14.657 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d0406ae40 msgr2=0x7f7d0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d0406ae40 0x7f7d0406d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7d14035000 tx=0x7f7d14030000 comp rx=0 tx=0).stop 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 msgr2=0x7f7d240ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240ca140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00c540 tx=0x7f7d1c00ca00 comp rx=0 tx=0).stop 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 shutdown_connections 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d0406ae40 0x7f7d0406d300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d2415d5b0 0x7f7d240ca140 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d240d14a0 0x7f7d240c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 --2- 172.21.15.64:0/4262538912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d240b1650 0x7f7d240b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 >> 172.21.15.64:0/4262538912 conn(0x7f7d2409c3b0 msgr2=0x7f7d240cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 shutdown_connections 2023-11-24T08:04:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:14.658+0000 7f7d23fff640 1 -- 172.21.15.64:0/4262538912 wait complete. 2023-11-24T08:04:14.679 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:04:14.679 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:04:14.679 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T08:04:14.679 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:04:14.679 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:14.680 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:04:14.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:04:14.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:04:14.680 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:04:14.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:14.680 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42328sec preferred_lft 42328sec 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T08:04:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:14.738 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T08:04:14.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:14.739 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:14.739 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:14.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2023-11-24T08:04:14.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2023-11-24T08:04:14.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2023-11-24T08:04:14.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T08:04:14.739 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:15.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:15.226 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:04:15.310 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T08:04:15.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:15.313 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T08:04:15.369 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T08:04:15.369 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:04:15.369 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:04:15.370 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:04:15.370 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:04:15.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:15.370 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:15.370 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:15.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:04:15.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:04:15.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:04:15.371 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:16.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:16.168 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:04:16.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:16.233 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-11-24T08:04:16.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:16.292 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:16.292 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:16.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:16.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T08:04:16.292 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:16.446 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T08:04:16.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:16.447 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:16.447 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:16.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T08:04:16.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T08:04:16.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T08:04:16.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T08:04:16.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T08:04:16.447 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:16.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:16.766 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:16.766 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:16.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-11-24T08:04:16.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T08:04:16.766 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:16.936 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T08:04:16.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:04:16.937 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:16.989 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:04:16.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:04:16.989 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:17.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:04:17.137 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:17.190 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:04:17.190 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:04:17.190 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:04:17.191 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:04:17.191 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:04:17.191 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:04:17.191 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:04:17.192 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:04:17.193 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:04:17.194 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:17.194 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:17.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:17.195 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:04:17.249 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:04:17.249 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T08:04:17.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:04:17.338 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:04:17.388 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:04:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:04:17.390 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:04:17.390 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:04:17.390 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:04:17.390 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:04:17.390 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:17.390 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:17.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:17.391 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:04:17.461 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-11-24T08:04:17.459+0000 7f51ac4bb480 -1 init, newargv = 0x55a52e699a70 newargc=15 2023-11-24T08:04:17.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[96582]: starting ceph client 2023-11-24T08:04:17.474 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[96582]: starting fuse 2023-11-24T08:04:18.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:04:18.650 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:18.714 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:04:18.714 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:04:18.714 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:04:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:04:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:04:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:04:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:04:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:04:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:04:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:04:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:18.716 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:18.717 INFO:teuthology.orchestra.run.smithi064.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:18.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:18.719 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:04:18.774 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T08:04:18.774 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T08:04:18.775 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T08:04:18.776 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:04:18.777 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:04:18.777 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-11-24T08:04:18.777 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T08:04:18.959 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.96582.asok 2023-11-24T08:04:18.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:04:18.977 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.96582.asok 2023-11-24T08:04:18.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:18.978 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.96582.asok status 2023-11-24T08:04:19.230 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:04:19.230 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T08:04:19.230 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:04:19.230 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:04:19.230 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-11-24T08:04:19.230 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T08:04:19.230 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T08:04:19.230 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "96582", 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4677, 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4677 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:04:19.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2255891976 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2255891976 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4677 192.168.0.1:0/2255891976", 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2255891976", 2023-11-24T08:04:19.232 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T08:04:19.233 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 8, 2023-11-24T08:04:19.233 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 20, 2023-11-24T08:04:19.233 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:04:19.233 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T08:04:19.233 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T08:04:19.233 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:04:19.247 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:04:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.470+0000 7f20ce747640 1 Processor -- start 2023-11-24T08:04:19.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.470+0000 7f20ce747640 1 -- start start 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.471+0000 7f20ce747640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c815d5b0 0x7f20c80d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.471+0000 7f20ce747640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f20c80d14a0 0x7f20c80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ce747640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c80cf0c0 con 0x7f20c80cf7a0 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ce747640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c80b01b0 con 0x7f20c80d14a0 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ce747640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c815d990 con 0x7f20c815d5b0 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ccf44640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f20c80d14a0 0x7f20c80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ccf44640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f20c80d14a0 0x7f20c80d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57246/0 (socket says 172.21.15.64:57246) 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ccf44640 1 -- 172.21.15.64:0/1923134127 learned_addr learned my addr 172.21.15.64:0/1923134127 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20cd745640 1 --2- 172.21.15.64:0/1923134127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c815d5b0 0x7f20c80d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ccf44640 1 -- 172.21.15.64:0/1923134127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c815d5b0 msgr2=0x7f20c80d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ccf44640 1 --2- 172.21.15.64:0/1923134127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c815d5b0 0x7f20c80d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.472+0000 7f20ccf44640 1 -- 172.21.15.64:0/1923134127 >> v1:172.21.15.91:6789/0 conn(0x7f20c80cf7a0 legacy=0x7f20c80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.473+0000 7f20ccf44640 1 -- 172.21.15.64:0/1923134127 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80d1e40 con 0x7f20c80d14a0 2023-11-24T08:04:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.474+0000 7f20ccf44640 1 --2- 172.21.15.64:0/1923134127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c80d14a0 0x7f20c80d1900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009870 tx=0x7f20b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d014e572d7c37b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.474+0000 7f20bffff640 1 -- 172.21.15.64:0/1923134127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8031640 con 0x7f20c80d14a0 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.474+0000 7f20bffff640 1 -- 172.21.15.64:0/1923134127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b80317a0 con 0x7f20c80d14a0 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.474+0000 7f20bffff640 1 -- 172.21.15.64:0/1923134127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8031b30 con 0x7f20c80d14a0 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.474+0000 7f20ce747640 1 -- 172.21.15.64:0/1923134127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c80d14a0 msgr2=0x7f20c80d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.474+0000 7f20ce747640 1 --2- 172.21.15.64:0/1923134127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c80d14a0 0x7f20c80d1900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009870 tx=0x7f20b802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.475+0000 7f20ce747640 1 -- 172.21.15.64:0/1923134127 shutdown_connections 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.475+0000 7f20ce747640 1 --2- 172.21.15.64:0/1923134127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c80d14a0 0x7f20c80d1900 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.475+0000 7f20ce747640 1 --2- 172.21.15.64:0/1923134127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c815d5b0 0x7f20c80d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.475+0000 7f20ce747640 1 -- 172.21.15.64:0/1923134127 >> 172.21.15.64:0/1923134127 conn(0x7f20c809c3b0 msgr2=0x7f20c80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.475+0000 7f20ce747640 1 -- 172.21.15.64:0/1923134127 shutdown_connections 2023-11-24T08:04:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.475+0000 7f20ce747640 1 -- 172.21.15.64:0/1923134127 wait complete. 2023-11-24T08:04:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.476+0000 7f20ce747640 1 Processor -- start 2023-11-24T08:04:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.477+0000 7f20ce747640 1 -- start start 2023-11-24T08:04:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20ce747640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20c80cf7a0 0x7f20c81604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20ce747640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c80d14a0 0x7f20c814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20ce747640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c815d5b0 0x7f20c80c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20ce747640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c81609f0 con 0x7f20c815d5b0 2023-11-24T08:04:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20ce747640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c814ea90 con 0x7f20c80cf7a0 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20ce747640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80b86a0 con 0x7f20c80d14a0 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cdf46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c815d5b0 0x7f20c80c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cdf46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c815d5b0 0x7f20c80c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57250/0 (socket says 172.21.15.64:57250) 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cdf46640 1 -- 172.21.15.64:0/2887163799 learned_addr learned my addr 172.21.15.64:0/2887163799 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20ccf44640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c80d14a0 0x7f20c814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cd745640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20c80cf7a0 0x7f20c81604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cdf46640 1 -- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c80d14a0 msgr2=0x7f20c814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cdf46640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c80d14a0 0x7f20c814e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cdf46640 1 -- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20c80cf7a0 msgr2=0x7f20c81604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cdf46640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20c80cf7a0 0x7f20c81604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.478+0000 7f20cdf46640 1 -- 172.21.15.64:0/2887163799 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80094d0 con 0x7f20c815d5b0 2023-11-24T08:04:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.479+0000 7f20cdf46640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c815d5b0 0x7f20c80c6ba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20c000b410 tx=0x7f20c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:19.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.479+0000 7f20bdffb640 1 -- 172.21.15.64:0/2887163799 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0004020 con 0x7f20c815d5b0 2023-11-24T08:04:19.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.479+0000 7f20bdffb640 1 -- 172.21.15.64:0/2887163799 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c0004180 con 0x7f20c815d5b0 2023-11-24T08:04:19.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.479+0000 7f20bdffb640 1 -- 172.21.15.64:0/2887163799 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0011630 con 0x7f20c815d5b0 2023-11-24T08:04:19.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.479+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c80b8980 con 0x7f20c815d5b0 2023-11-24T08:04:19.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.479+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c80b5f20 con 0x7f20c815d5b0 2023-11-24T08:04:19.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.481+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c804e840 con 0x7f20c815d5b0 2023-11-24T08:04:19.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.482+0000 7f20bdffb640 1 -- 172.21.15.64:0/2887163799 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20c0011790 con 0x7f20c815d5b0 2023-11-24T08:04:19.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.483+0000 7f20bdffb640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f20ac06ae40 0x7f20ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:19.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.483+0000 7f20bdffb640 1 -- 172.21.15.64:0/2887163799 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20c00892d0 con 0x7f20c815d5b0 2023-11-24T08:04:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.483+0000 7f20cd745640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f20ac06ae40 0x7f20ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.484+0000 7f20cd745640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f20ac06ae40 0x7f20ac06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009a60 tx=0x7f20c4003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:19.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.488+0000 7f20bdffb640 1 -- 172.21.15.64:0/2887163799 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20c0054370 con 0x7f20c815d5b0 2023-11-24T08:04:19.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.721+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f20c80678d0 con 0x7f20c815d5b0 2023-11-24T08:04:19.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.722+0000 7f20bdffb640 1 -- 172.21.15.64:0/2887163799 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f20c0057990 con 0x7f20c815d5b0 2023-11-24T08:04:19.723 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:04:19.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f20ac06ae40 msgr2=0x7f20ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:19.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f20ac06ae40 0x7f20ac06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009a60 tx=0x7f20c4003990 comp rx=0 tx=0).stop 2023-11-24T08:04:19.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c815d5b0 msgr2=0x7f20c80c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:19.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c815d5b0 0x7f20c80c6ba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20c000b410 tx=0x7f20c000b8d0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 shutdown_connections 2023-11-24T08:04:19.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f20ac06ae40 0x7f20ac06d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c815d5b0 0x7f20c80c6ba0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f20c80d14a0 0x7f20c814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.724+0000 7f20ce747640 1 --2- 172.21.15.64:0/2887163799 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f20c80cf7a0 0x7f20c81604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:19.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.725+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 >> 172.21.15.64:0/2887163799 conn(0x7f20c809c3b0 msgr2=0x7f20c806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:19.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.725+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 shutdown_connections 2023-11-24T08:04:19.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:19.725+0000 7f20ce747640 1 -- 172.21.15.64:0/2887163799 wait complete. 2023-11-24T08:04:19.746 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:04:19.747 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:04:19.747 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T08:04:19.747 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T08:04:19.747 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T08:04:19.747 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:04:19.747 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:04:19.747 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:04:19.747 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T08:04:19.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:19.748 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T08:04:19.803 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:04:19.803 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:04:19.803 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:04:19.803 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:19.803 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:04:19.803 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:04:19.804 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42323sec preferred_lft 42323sec 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3c83:ffff:fee3:836f/64 scope link 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::54d4:76ff:fe60:984a/64 scope link 2023-11-24T08:04:19.805 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:19.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:19.807 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:04:19.903 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T08:04:19.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:19.907 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T08:04:19.969 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T08:04:19.969 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:04:19.969 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:04:19.969 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:04:19.969 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:04:19.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:19.970 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:19.971 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:19.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:04:19.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:04:19.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:04:19.971 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:20.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:20.165 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:04:20.220 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:04:20.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:20.221 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-11-24T08:04:20.278 INFO:teuthology.orchestra.run.smithi064.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T08:04:20.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:20.280 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:20.281 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:20.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:04:20.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T08:04:20.281 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:20.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:20.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T08:04:20.534 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:04:20.535 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:04:20.535 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:04:20.535 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:20.535 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:04:20.535 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:20.535 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:04:20.535 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 2a:52:23:e5:53:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T08:04:20.536 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T08:04:20.536 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:20.536 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::2852:23ff:fee5:53bd/64 scope link 2023-11-24T08:04:20.536 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:20.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:20.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T08:04:20.642 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:04:20.642 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:04:20.642 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:04:20.642 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:20.642 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:04:20.642 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:20.642 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:04:20.642 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 2a:52:23:e5:53:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T08:04:20.643 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T08:04:20.643 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:20.643 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::2852:23ff:fee5:53bd/64 scope link 2023-11-24T08:04:20.643 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:20.647 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T08:04:20.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:20.647 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:20.648 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:20.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T08:04:20.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T08:04:20.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T08:04:20.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T08:04:20.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T08:04:20.648 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:20.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:20.920 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:04:20.920 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:04:20.921 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.1 up 2023-11-24T08:04:20.921 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T08:04:20.921 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:04:21.095 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-24T08:04:21.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:04:21.095 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T08:04:21.149 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T08:04:21.150 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:04:21.150 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T08:04:21.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:04:21.296 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:21.357 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:04:21.358 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:04:21.358 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:04:21.358 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:04:21.358 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:04:21.358 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:04:21.359 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:04:21.359 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:04:21.359 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:04:21.359 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:04:21.359 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:04:21.359 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:21.360 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:04:21.360 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:04:21.360 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:04:21.360 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:04:21.360 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:21.360 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:04:21.360 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:21.360 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:21.361 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:21.361 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:21.361 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:04:21.361 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:04:21.361 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:04:21.361 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:04:21.361 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:04:21.362 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:04:21.362 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:04:21.362 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:21.362 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:21.362 INFO:teuthology.orchestra.run.smithi064.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:21.363 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:04:21.363 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:04:21.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:21.364 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:04:21.426 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T08:04:21.426 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-11-24T08:04:21.426 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-24T08:04:21.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:04:21.516 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:21.560 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:04:21.560 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:04:21.560 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:04:21.561 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:04:21.562 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:04:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:04:21.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:21.564 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:04:21.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:2023-11-24T08:04:21.637+0000 7f3e5a845480 -1 init, newargv = 0x55a24a74fa30 newargc=15 2023-11-24T08:04:21.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[97128]: starting ceph client 2023-11-24T08:04:21.650 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[97128]: starting fuse 2023-11-24T08:04:21.655 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T08:04:21.655 INFO:teuthology.orchestra.run.smithi064.stdout:42 2023-11-24T08:04:21.655 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-24T08:04:21.656 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T08:04:21.657 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:04:21.658 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:04:21.658 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.2")) 2023-11-24T08:04:21.658 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T08:04:21.871 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.2.97128.asok 2023-11-24T08:04:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:04:21.890 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.97128.asok 2023-11-24T08:04:21.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:21.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.97128.asok status 2023-11-24T08:04:22.145 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:04:22.145 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T08:04:22.145 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:04:22.145 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "2", 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "97128", 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4689, 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T08:04:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4689 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 761736373 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:04:22.147 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 761736373 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4689 192.168.0.2:0/761736373", 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.2:0/761736373", 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 8, 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 20, 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T08:04:22.148 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:04:22.161 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:04:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.383+0000 7f8b1759e640 1 Processor -- start 2023-11-24T08:04:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.383+0000 7f8b1759e640 1 -- start start 2023-11-24T08:04:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.385+0000 7f8b1759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b18143950 0x7f8b18164e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.385+0000 7f8b1759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b18071700 con 0x7f8b180cf760 2023-11-24T08:04:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.385+0000 7f8b1759e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1815b410 con 0x7f8b18143570 2023-11-24T08:04:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.385+0000 7f8b1759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1815b130 con 0x7f8b18143950 2023-11-24T08:04:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.385+0000 7f8b16d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b18143950 0x7f8b18164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.385+0000 7f8b0ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8b180cf760 0x7f8b1815a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54682/0 (socket says 172.21.15.64:54682) 2023-11-24T08:04:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.385+0000 7f8b0ffff640 1 -- 172.21.15.64:0/265886810 learned_addr learned my addr 172.21.15.64:0/265886810 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.386+0000 7f8b16d9d640 1 -- 172.21.15.64:0/265886810 >> v1:172.21.15.91:6789/0 conn(0x7f8b18143570 legacy=0x7f8b180cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.386+0000 7f8b16d9d640 1 -- 172.21.15.64:0/265886810 >> v1:172.21.15.64:6789/0 conn(0x7f8b180cf760 legacy=0x7f8b1815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.386+0000 7f8b16d9d640 1 -- 172.21.15.64:0/265886810 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1815dbd0 con 0x7f8b18143950 2023-11-24T08:04:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.387+0000 7f8b16d9d640 1 --2- 172.21.15.64:0/265886810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143950 0x7f8b18164e30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009d90 tx=0x7f8b1002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e472dcbad02af9e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.387+0000 7f8b15d9b640 1 -- 172.21.15.64:0/265886810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1002ede0 con 0x7f8b18143950 2023-11-24T08:04:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.387+0000 7f8b15d9b640 1 -- 172.21.15.64:0/265886810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10036440 con 0x7f8b18143950 2023-11-24T08:04:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.387+0000 7f8b15d9b640 1 -- 172.21.15.64:0/265886810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b100367d0 con 0x7f8b18143950 2023-11-24T08:04:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.388+0000 7f8b1759e640 1 -- 172.21.15.64:0/265886810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143950 msgr2=0x7f8b18164e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.388+0000 7f8b1759e640 1 --2- 172.21.15.64:0/265886810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143950 0x7f8b18164e30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009d90 tx=0x7f8b1002e030 comp rx=0 tx=0).stop 2023-11-24T08:04:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.388+0000 7f8b1759e640 1 -- 172.21.15.64:0/265886810 shutdown_connections 2023-11-24T08:04:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.388+0000 7f8b1759e640 1 --2- 172.21.15.64:0/265886810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143950 0x7f8b18164e30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.388+0000 7f8b1759e640 1 -- 172.21.15.64:0/265886810 >> 172.21.15.64:0/265886810 conn(0x7f8b1806a860 msgr2=0x7f8b1806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.388+0000 7f8b1759e640 1 -- 172.21.15.64:0/265886810 shutdown_connections 2023-11-24T08:04:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.388+0000 7f8b1759e640 1 -- 172.21.15.64:0/265886810 wait complete. 2023-11-24T08:04:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.390+0000 7f8b1759e640 1 Processor -- start 2023-11-24T08:04:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.390+0000 7f8b1759e640 1 -- start start 2023-11-24T08:04:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b1759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b180cf760 0x7f8b1815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b1759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143570 0x7f8b18136980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b1759e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b18143950 0x7f8b1813bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b1759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180ce8e0 con 0x7f8b18143570 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b1759e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1813b200 con 0x7f8b18143950 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b1759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1813b4e0 con 0x7f8b180cf760 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143570 0x7f8b18136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143570 0x7f8b18136980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57262/0 (socket says 172.21.15.64:57262) 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.391+0000 7f8b0ffff640 1 -- 172.21.15.64:0/4159700927 learned_addr learned my addr 172.21.15.64:0/4159700927 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b16d9d640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b18143950 0x7f8b1813bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b1659c640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b180cf760 0x7f8b1815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b0ffff640 1 -- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b180cf760 msgr2=0x7f8b1815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:22.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b0ffff640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b180cf760 0x7f8b1815c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b0ffff640 1 -- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b18143950 msgr2=0x7f8b1813bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b0ffff640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b18143950 0x7f8b1813bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b0ffff640 1 -- 172.21.15.64:0/4159700927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b10009a70 con 0x7f8b18143570 2023-11-24T08:04:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b0ffff640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143570 0x7f8b18136980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c500 tx=0x7f8b0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.392+0000 7f8b0effd640 1 -- 172.21.15.64:0/4159700927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00005ce0 con 0x7f8b18143570 2023-11-24T08:04:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.393+0000 7f8b0effd640 1 -- 172.21.15.64:0/4159700927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b00005e40 con 0x7f8b18143570 2023-11-24T08:04:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.393+0000 7f8b0effd640 1 -- 172.21.15.64:0/4159700927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000072f0 con 0x7f8b18143570 2023-11-24T08:04:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.393+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1813b7c0 con 0x7f8b18143570 2023-11-24T08:04:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.393+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b181598c0 con 0x7f8b18143570 2023-11-24T08:04:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.394+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1813bce0 con 0x7f8b18143570 2023-11-24T08:04:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.395+0000 7f8b0effd640 1 -- 172.21.15.64:0/4159700927 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b00007510 con 0x7f8b18143570 2023-11-24T08:04:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.396+0000 7f8b0effd640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8af806ae40 0x7f8af806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.396+0000 7f8b0effd640 1 -- 172.21.15.64:0/4159700927 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b000888e0 con 0x7f8b18143570 2023-11-24T08:04:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.396+0000 7f8b1659c640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8af806ae40 0x7f8af806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.397+0000 7f8b1659c640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8af806ae40 0x7f8af806d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8b08004ea0 tx=0x7f8b08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:22.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.400+0000 7f8b0effd640 1 -- 172.21.15.64:0/4159700927 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b1813bce0 con 0x7f8b18143570 2023-11-24T08:04:22.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.633+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b18071700 con 0x7f8b18143570 2023-11-24T08:04:22.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.633+0000 7f8b0effd640 1 -- 172.21.15.64:0/4159700927 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b18071700 con 0x7f8b18143570 2023-11-24T08:04:22.635 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:04:22.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8af806ae40 msgr2=0x7f8af806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:22.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8af806ae40 0x7f8af806d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8b08004ea0 tx=0x7f8b08009040 comp rx=0 tx=0).stop 2023-11-24T08:04:22.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143570 msgr2=0x7f8b18136980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:22.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143570 0x7f8b18136980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c500 tx=0x7f8b0000c9c0 comp rx=0 tx=0).stop 2023-11-24T08:04:22.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 shutdown_connections 2023-11-24T08:04:22.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8af806ae40 0x7f8af806d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:22.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b18143950 0x7f8b1813bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:22.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b18143570 0x7f8b18136980 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:22.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 --2- 172.21.15.64:0/4159700927 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b180cf760 0x7f8b1815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:22.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 >> 172.21.15.64:0/4159700927 conn(0x7f8b1806a860 msgr2=0x7f8b180cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:22.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 shutdown_connections 2023-11-24T08:04:22.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:22.636+0000 7f8b1759e640 1 -- 172.21.15.64:0/4159700927 wait complete. 2023-11-24T08:04:22.658 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:04:22.658 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:04:22.658 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2023-11-24T08:04:22.658 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:04:22.659 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:22.659 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:04:22.659 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:04:22.659 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:04:22.659 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:04:22.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:22.659 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2023-11-24T08:04:22.720 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2023-11-24T08:04:22.721 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42310sec preferred_lft 42310sec 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link 2023-11-24T08:04:22.722 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:22.723 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T08:04:22.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:22.724 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:22.724 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:22.724 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link add name ceph-brx type bridge 2023-11-24T08:04:22.724 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip addr flush dev ceph-brx 2023-11-24T08:04:22.724 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set ceph-brx up 2023-11-24T08:04:22.724 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T08:04:22.724 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:23.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:23.201 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:04:23.279 INFO:teuthology.orchestra.run.smithi091.stdout:1 2023-11-24T08:04:23.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:23.283 DEBUG:teuthology.orchestra.run.smithi091:> route 2023-11-24T08:04:23.340 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2023-11-24T08:04:23.340 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:04:23.340 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:04:23.340 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:04:23.341 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:04:23.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:23.341 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:23.342 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:23.342 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:04:23.342 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:04:23.342 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:04:23.343 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:24.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:24.091 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:04:24.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:24.149 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list-id 2023-11-24T08:04:24.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:24.201 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:24.201 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:24.201 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:24.201 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T08:04:24.202 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:24.343 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T08:04:24.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:24.344 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:24.344 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:24.344 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T08:04:24.344 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T08:04:24.344 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T08:04:24.344 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T08:04:24.344 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T08:04:24.344 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:24.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:24.618 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:24.618 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:24.618 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2023-11-24T08:04:24.619 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T08:04:24.619 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:24.776 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T08:04:24.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:04:24.776 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:24.821 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:04:24.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:04:24.822 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:24.918 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:04:25.017 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:25.078 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:04:25.078 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:04:25.079 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:04:25.079 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:04:25.080 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:04:25.080 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:04:25.080 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:04:25.080 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:04:25.080 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:04:25.081 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:04:25.081 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:04:25.081 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:25.081 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:04:25.081 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:04:25.081 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:04:25.082 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:04:25.082 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:25.082 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:04:25.082 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:25.082 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:25.083 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:25.083 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:25.083 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:04:25.083 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:04:25.084 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:04:25.084 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:04:25.084 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:04:25.084 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:04:25.085 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:04:25.085 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:25.085 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:25.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:25.086 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:04:25.133 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:04:25.133 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T08:04:25.182 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:04:25.260 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:25.301 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:04:25.302 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:25.303 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:04:25.304 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:25.305 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:25.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:25.306 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:04:25.383 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:2023-11-24T08:04:25.381+0000 7f6846e6a480 -1 init, newargv = 0x563c212a4970 newargc=15 2023-11-24T08:04:25.383 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[94579]: starting ceph client 2023-11-24T08:04:25.392 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[94579]: starting fuse 2023-11-24T08:04:26.492 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:04:26.600 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:26.656 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:26.657 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:04:26.658 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:04:26.659 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:04:26.659 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:04:26.659 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:04:26.659 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:04:26.659 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:04:26.659 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:26.659 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:26.659 INFO:teuthology.orchestra.run.smithi091.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:26.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:26.660 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:04:26.709 INFO:teuthology.orchestra.run.smithi091.stdout:41 2023-11-24T08:04:26.709 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T08:04:26.709 DEBUG:teuthology.orchestra.run.smithi091:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:04:26.709 DEBUG:teuthology.orchestra.run.smithi091:> import glob 2023-11-24T08:04:26.709 DEBUG:teuthology.orchestra.run.smithi091:> import re 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> import os 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> def _find_admin_socket(client_name): 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> files = glob.glob(asok_path) 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> # Given a non-glob path, it better be there 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> if "*" not in asok_path: 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> assert(len(files) == 1) 2023-11-24T08:04:26.710 DEBUG:teuthology.orchestra.run.smithi091:> return files[0] 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> for f in files: 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> contents = proc_f.read() 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> if mountpoint in contents: 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> return f 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> print(_find_admin_socket("client.1")) 2023-11-24T08:04:26.711 DEBUG:teuthology.orchestra.run.smithi091:> ' 2023-11-24T08:04:26.885 INFO:teuthology.orchestra.run.smithi091.stdout:/var/run/ceph/ceph-client.1.94579.asok 2023-11-24T08:04:26.889 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:04:26.902 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94579.asok 2023-11-24T08:04:26.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:26.903 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94579.asok status 2023-11-24T08:04:27.141 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": { 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "entity_id": "1", 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "hostname": "smithi091", 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "pid": "94579", 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "root": "/" 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:04:27.142 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_count": 0, 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_pinned_count": 0, 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 4460, 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "inst": { 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "name": { 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "client", 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "num": 4460 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 2231953502 2023-11-24T08:04:27.143 INFO:teuthology.orchestra.run.smithi091.stdout: } 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 2231953502 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "inst_str": "client.4460 192.168.0.1:0/2231953502", 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "addr_str": "192.168.0.1:0/2231953502", 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "inode_count": 1, 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_epoch": 8, 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch": 20, 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:04:27.144 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklisted": false, 2023-11-24T08:04:27.145 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs" 2023-11-24T08:04:27.145 INFO:teuthology.orchestra.run.smithi091.stdout:} 2023-11-24T08:04:27.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:04:27.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.398+0000 7f01dfa90640 1 Processor -- start 2023-11-24T08:04:27.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.398+0000 7f01dfa90640 1 -- start start 2023-11-24T08:04:27.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.400+0000 7f01dfa90640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01d806dc30 0x7f01d80ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:27.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.400+0000 7f01dfa90640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80ad510 con 0x7f01d80719c0 2023-11-24T08:04:27.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.400+0000 7f01dfa90640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80acc50 con 0x7f01d8161930 2023-11-24T08:04:27.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.400+0000 7f01dfa90640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8071e60 con 0x7f01d806dc30 2023-11-24T08:04:27.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.400+0000 7f01dd02b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f01d80719c0 0x7f01d806d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37732/0 (socket says 172.21.15.64:37732) 2023-11-24T08:04:27.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.400+0000 7f01dd02b640 1 -- 172.21.15.64:0/202783099 learned_addr learned my addr 172.21.15.64:0/202783099 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:27.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.400+0000 7f01de02d640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01d806dc30 0x7f01d80ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:27.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396494125 0 0) 0x7f01d80ad510 con 0x7f01d80719c0 2023-11-24T08:04:27.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01c4003520 con 0x7f01d80719c0 2023-11-24T08:04:27.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3093414125 0 0) 0x7f01c4003520 con 0x7f01d80719c0 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01d806dc30 msgr2=0x7f01d80ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01d806dc30 0x7f01d80ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 >> v1:172.21.15.91:6789/0 conn(0x7f01d8161930 legacy=0x7f01d80ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80acec0 con 0x7f01d80719c0 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f01cc001670 con 0x7f01d80719c0 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 >> v1:172.21.15.64:6789/0 conn(0x7f01d80719c0 legacy=0x7f01d806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01c40039a0 0x7f01c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01d806dc30 0x7f01c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01c4004d00 0x7f01c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8071e60 con 0x7f01c4004d00 2023-11-24T08:04:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c400ba90 con 0x7f01c40039a0 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c400bc50 con 0x7f01d806dc30 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.401+0000 7f01de02d640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01c4004d00 0x7f01c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.402+0000 7f01dd82c640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01c40039a0 0x7f01c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.402+0000 7f01dd02b640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01d806dc30 0x7f01c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.402+0000 7f01de02d640 1 -- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01d806dc30 msgr2=0x7f01c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.402+0000 7f01de02d640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01d806dc30 0x7f01c40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.402+0000 7f01de02d640 1 -- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01c40039a0 msgr2=0x7f01c4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.402+0000 7f01de02d640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01c40039a0 0x7f01c4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.402+0000 7f01de02d640 1 -- 172.21.15.64:0/202783099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c4003520 con 0x7f01c4004d00 2023-11-24T08:04:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.402+0000 7f01dd82c640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01c40039a0 0x7f01c4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.403+0000 7f01de02d640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01c4004d00 0x7f01c4005180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f01d8166670 tx=0x7f01c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bd1ce58276219e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.403+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d002cb90 con 0x7f01c4004d00 2023-11-24T08:04:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.403+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d000f040 con 0x7f01c4004d00 2023-11-24T08:04:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.403+0000 7f01dc82a640 1 -- 172.21.15.64:0/202783099 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0034440 con 0x7f01c4004d00 2023-11-24T08:04:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.404+0000 7f01dfa90640 1 -- 172.21.15.64:0/202783099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01c4004d00 msgr2=0x7f01c4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.404+0000 7f01dfa90640 1 --2- 172.21.15.64:0/202783099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01c4004d00 0x7f01c4005180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f01d8166670 tx=0x7f01c40057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.404+0000 7f01de02d640 1 -- 172.21.15.64:0/202783099 reap_dead start 2023-11-24T08:04:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.404+0000 7f01dfa90640 1 -- 172.21.15.64:0/202783099 shutdown_connections 2023-11-24T08:04:27.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.404+0000 7f01dfa90640 1 -- 172.21.15.64:0/202783099 >> 172.21.15.64:0/202783099 conn(0x7f01d80b0d80 msgr2=0x7f01d80b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:27.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.404+0000 7f01dfa90640 1 -- 172.21.15.64:0/202783099 shutdown_connections 2023-11-24T08:04:27.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.404+0000 7f01dfa90640 1 -- 172.21.15.64:0/202783099 wait complete. 2023-11-24T08:04:27.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.406+0000 7f01dfa90640 1 Processor -- start 2023-11-24T08:04:27.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.406+0000 7f01dfa90640 1 -- start start 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.407+0000 7f01dfa90640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01c4004d00 0x7f01d814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.407+0000 7f01dfa90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d81fa860 0x7f01d81f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.407+0000 7f01dfa90640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01d80af820 0x7f01d80afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.407+0000 7f01dfa90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80b01e0 con 0x7f01d81fa860 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.407+0000 7f01dfa90640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d814efd0 con 0x7f01d80af820 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.407+0000 7f01dfa90640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d81fade0 con 0x7f01c4004d00 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d81fa860 0x7f01d81f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d81fa860 0x7f01d81f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51282/0 (socket says 172.21.15.64:51282) 2023-11-24T08:04:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 -- 172.21.15.64:0/166592577 learned_addr learned my addr 172.21.15.64:0/166592577 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 -- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01c4004d00 msgr2=0x7f01d814ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01c4004d00 0x7f01d814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 -- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01d80af820 msgr2=0x7f01d80afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01d80af820 0x7f01d80afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 -- 172.21.15.64:0/166592577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d81fb0c0 con 0x7f01d81fa860 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.408+0000 7f01dd02b640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d81fa860 0x7f01d81f6bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00e830 tx=0x7f01cc00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.409+0000 7f01ca7fc640 1 -- 172.21.15.64:0/166592577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc00c7f0 con 0x7f01d81fa860 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.409+0000 7f01ca7fc640 1 -- 172.21.15.64:0/166592577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc00c950 con 0x7f01d81fa860 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.409+0000 7f01ca7fc640 1 -- 172.21.15.64:0/166592577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc00cd00 con 0x7f01d81fa860 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.409+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d81fb3a0 con 0x7f01d81fa860 2023-11-24T08:04:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.409+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d814bff0 con 0x7f01d81fa860 2023-11-24T08:04:27.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.410+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d8074770 con 0x7f01d81fa860 2023-11-24T08:04:27.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.411+0000 7f01ca7fc640 1 -- 172.21.15.64:0/166592577 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01cc01d460 con 0x7f01d81fa860 2023-11-24T08:04:27.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.412+0000 7f01ca7fc640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f01bc06ae70 0x7f01bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:27.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.412+0000 7f01ca7fc640 1 -- 172.21.15.64:0/166592577 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01cc089670 con 0x7f01d81fa860 2023-11-24T08:04:27.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.412+0000 7f01dd82c640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f01bc06ae70 0x7f01bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:27.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.414+0000 7f01dd82c640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f01bc06ae70 0x7f01bc06d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01d4007fd0 tx=0x7f01d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:27.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.417+0000 7f01ca7fc640 1 -- 172.21.15.64:0/166592577 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01cc054680 con 0x7f01d81fa860 2023-11-24T08:04:27.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.655+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01d806e290 con 0x7f01d81fa860 2023-11-24T08:04:27.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.656+0000 7f01ca7fc640 1 -- 172.21.15.64:0/166592577 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f01cc00ce60 con 0x7f01d81fa860 2023-11-24T08:04:27.658 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:04:27.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f01bc06ae70 msgr2=0x7f01bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:27.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f01bc06ae70 0x7f01bc06d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01d4007fd0 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2023-11-24T08:04:27.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d81fa860 msgr2=0x7f01d81f6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d81fa860 0x7f01d81f6bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00e830 tx=0x7f01cc00ecf0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 shutdown_connections 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f01bc06ae70 0x7f01bc06d330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01d80af820 0x7f01d80afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d81fa860 0x7f01d81f6bd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 --2- 172.21.15.64:0/166592577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f01c4004d00 0x7f01d814ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 >> 172.21.15.64:0/166592577 conn(0x7f01d80b0d80 msgr2=0x7f01d8164050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.659+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 shutdown_connections 2023-11-24T08:04:27.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:27.660+0000 7f01dfa90640 1 -- 172.21.15.64:0/166592577 wait complete. 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:04:27.682 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T08:04:27.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:27.683 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2023-11-24T08:04:27.743 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:27.744 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42305sec preferred_lft 42305sec 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::5073:3fff:fef1:71bb/64 scope link 2023-11-24T08:04:27.745 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:27.746 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:04:27.746 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:27.746 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::2497:37ff:fe0c:852a/64 scope link 2023-11-24T08:04:27.746 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:27.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:27.747 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:04:27.838 INFO:teuthology.orchestra.run.smithi091.stdout:1 2023-11-24T08:04:27.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:27.843 DEBUG:teuthology.orchestra.run.smithi091:> route 2023-11-24T08:04:27.897 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2023-11-24T08:04:27.897 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:04:27.897 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:04:27.897 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:04:27.898 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:04:27.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:27.898 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:27.898 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:27.898 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:04:27.898 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:04:27.898 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:04:27.898 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:28.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:28.074 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:04:28.124 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:04:28.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:28.126 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list-id 2023-11-24T08:04:28.177 INFO:teuthology.orchestra.run.smithi091.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T08:04:28.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:28.179 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:28.179 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:28.179 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:04:28.179 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T08:04:28.179 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:28.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:28.322 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T08:04:28.414 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:04:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:04:28.414 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether be:e0:e4:8b:54:6d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::bce0:e4ff:fe8b:546d/64 scope link 2023-11-24T08:04:28.415 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:28.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:28.418 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T08:04:28.514 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:04:28.514 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:04:28.515 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:04:28.515 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:28.515 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:04:28.515 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:28.515 INFO:teuthology.orchestra.run.smithi091.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:04:28.515 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether be:e0:e4:8b:54:6d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T08:04:28.515 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T08:04:28.515 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:28.516 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::bce0:e4ff:fe8b:546d/64 scope link 2023-11-24T08:04:28.516 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:04:28.518 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T08:04:28.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:28.519 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:28.519 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:28.519 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T08:04:28.519 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T08:04:28.519 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T08:04:28.519 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T08:04:28.520 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T08:04:28.520 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:28.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:28.772 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:04:28.772 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:04:28.773 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.1 up 2023-11-24T08:04:28.773 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T08:04:28.773 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:04:28.936 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-24T08:04:28.936 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:04:28.936 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T08:04:28.984 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T08:04:28.984 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:04:28.984 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T08:04:29.078 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:04:29.170 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:29.223 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:04:29.223 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:04:29.223 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:04:29.223 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:04:29.223 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:04:29.224 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:04:29.224 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:04:29.224 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:04:29.224 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:04:29.224 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:04:29.225 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:04:29.225 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:29.225 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:04:29.225 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:04:29.225 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:04:29.225 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:04:29.226 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:29.226 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:04:29.226 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:29.226 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:29.226 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:29.227 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:29.227 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:04:29.227 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:04:29.227 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:04:29.227 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:04:29.228 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:04:29.228 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:04:29.228 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:04:29.228 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:29.229 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:29.229 INFO:teuthology.orchestra.run.smithi091.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:29.230 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:04:29.230 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:04:29.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:29.231 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:04:29.284 INFO:teuthology.orchestra.run.smithi091.stdout:41 2023-11-24T08:04:29.284 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-24T08:04:29.284 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-24T08:04:29.289 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:04:29.372 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:04:29.414 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:04:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:04:29.416 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:29.417 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:04:29.418 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:04:29.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:29.419 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:04:29.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi091.stderr:2023-11-24T08:04:29.474+0000 7f9078c3e480 -1 init, newargv = 0x55f4c9ae4970 newargc=15 2023-11-24T08:04:29.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi091.stderr:ceph-fuse[95099]: starting ceph client 2023-11-24T08:04:29.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi091.stderr:ceph-fuse[95099]: starting fuse 2023-11-24T08:04:29.504 INFO:teuthology.orchestra.run.smithi091.stdout:41 2023-11-24T08:04:29.504 INFO:teuthology.orchestra.run.smithi091.stdout:42 2023-11-24T08:04:29.504 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2023-11-24T08:04:29.505 DEBUG:teuthology.orchestra.run.smithi091:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:04:29.505 DEBUG:teuthology.orchestra.run.smithi091:> import glob 2023-11-24T08:04:29.505 DEBUG:teuthology.orchestra.run.smithi091:> import re 2023-11-24T08:04:29.505 DEBUG:teuthology.orchestra.run.smithi091:> import os 2023-11-24T08:04:29.505 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2023-11-24T08:04:29.505 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:04:29.505 DEBUG:teuthology.orchestra.run.smithi091:> def _find_admin_socket(client_name): 2023-11-24T08:04:29.506 DEBUG:teuthology.orchestra.run.smithi091:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-24T08:04:29.506 DEBUG:teuthology.orchestra.run.smithi091:> files = glob.glob(asok_path) 2023-11-24T08:04:29.506 DEBUG:teuthology.orchestra.run.smithi091:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-24T08:04:29.506 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:04:29.506 DEBUG:teuthology.orchestra.run.smithi091:> # Given a non-glob path, it better be there 2023-11-24T08:04:29.506 DEBUG:teuthology.orchestra.run.smithi091:> if "*" not in asok_path: 2023-11-24T08:04:29.506 DEBUG:teuthology.orchestra.run.smithi091:> assert(len(files) == 1) 2023-11-24T08:04:29.506 DEBUG:teuthology.orchestra.run.smithi091:> return files[0] 2023-11-24T08:04:29.507 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:04:29.507 DEBUG:teuthology.orchestra.run.smithi091:> for f in files: 2023-11-24T08:04:29.507 DEBUG:teuthology.orchestra.run.smithi091:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:04:29.507 DEBUG:teuthology.orchestra.run.smithi091:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:04:29.507 DEBUG:teuthology.orchestra.run.smithi091:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:04:29.507 DEBUG:teuthology.orchestra.run.smithi091:> contents = proc_f.read() 2023-11-24T08:04:29.507 DEBUG:teuthology.orchestra.run.smithi091:> if mountpoint in contents: 2023-11-24T08:04:29.507 DEBUG:teuthology.orchestra.run.smithi091:> return f 2023-11-24T08:04:29.508 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:04:29.508 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:04:29.508 DEBUG:teuthology.orchestra.run.smithi091:> print(_find_admin_socket("client.3")) 2023-11-24T08:04:29.508 DEBUG:teuthology.orchestra.run.smithi091:> ' 2023-11-24T08:04:29.711 INFO:teuthology.orchestra.run.smithi091.stdout:/var/run/ceph/ceph-client.3.95099.asok 2023-11-24T08:04:29.715 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:04:29.728 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.95099.asok 2023-11-24T08:04:29.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:29.728 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.95099.asok status 2023-11-24T08:04:29.971 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": { 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: "entity_id": "3", 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: "hostname": "smithi091", 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: "pid": "95099", 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: "root": "/" 2023-11-24T08:04:29.972 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_count": 0, 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_pinned_count": 0, 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 4591, 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: "inst": { 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: "name": { 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "client", 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: "num": 4591 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:04:29.973 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:04:29.974 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:04:29.974 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.2:0", 2023-11-24T08:04:29.974 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 3286193359 2023-11-24T08:04:29.974 INFO:teuthology.orchestra.run.smithi091.stdout: } 2023-11-24T08:04:29.974 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:04:29.975 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:04:29.975 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:04:29.976 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.2:0", 2023-11-24T08:04:29.976 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 3286193359 2023-11-24T08:04:29.976 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:04:29.977 INFO:teuthology.orchestra.run.smithi091.stdout: "inst_str": "client.4591 192.168.0.2:0/3286193359", 2023-11-24T08:04:29.977 INFO:teuthology.orchestra.run.smithi091.stdout: "addr_str": "192.168.0.2:0/3286193359", 2023-11-24T08:04:29.977 INFO:teuthology.orchestra.run.smithi091.stdout: "inode_count": 1, 2023-11-24T08:04:29.978 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_epoch": 8, 2023-11-24T08:04:29.978 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch": 20, 2023-11-24T08:04:29.978 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:04:29.978 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklisted": false, 2023-11-24T08:04:29.979 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs" 2023-11-24T08:04:29.979 INFO:teuthology.orchestra.run.smithi091.stdout:} 2023-11-24T08:04:29.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:29.987 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T08:04:29.987 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:30.054 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T08:04:30.054 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:30.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:30.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:30.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:30.161 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T08:04:30.161 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-24T08:04:30.220 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T08:04:30.221 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-24T08:04:30.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:30.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T08:04:30.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:30.324 DEBUG:teuthology.orchestra.run.smithi091:> stat --file-system '--printf=%T 2023-11-24T08:04:30.324 DEBUG:teuthology.orchestra.run.smithi091:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:30.385 INFO:teuthology.orchestra.run.smithi091.stdout:fuseblk 2023-11-24T08:04:30.386 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:30.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:30.386 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:30.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:30.482 DEBUG:teuthology.orchestra.run.smithi091:> stat --file-system '--printf=%T 2023-11-24T08:04:30.482 DEBUG:teuthology.orchestra.run.smithi091:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-24T08:04:30.538 INFO:teuthology.orchestra.run.smithi091.stdout:fuseblk 2023-11-24T08:04:30.539 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-24T08:04:30.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:30.539 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T08:04:30.644 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T08:04:30.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.891+0000 7f33eb15c640 1 Processor -- start 2023-11-24T08:04:30.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.891+0000 7f33eb15c640 1 -- start start 2023-11-24T08:04:30.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.893+0000 7f33eb15c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e4157770 0x7f33e406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:30.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.893+0000 7f33eb15c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33e406d080 0x7f33e415d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:30.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.893+0000 7f33eb15c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e4161670 con 0x7f33e415ddb0 2023-11-24T08:04:30.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.893+0000 7f33eb15c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e406d500 con 0x7f33e406d080 2023-11-24T08:04:30.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.893+0000 7f33eb15c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4071730 con 0x7f33e4157770 2023-11-24T08:04:30.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.893+0000 7f33e9959640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33e406d080 0x7f33e415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:30.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.894+0000 7f33ea95b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f33e415ddb0 0x7f33e40ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37748/0 (socket says 172.21.15.64:37748) 2023-11-24T08:04:30.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.894+0000 7f33ea95b640 1 -- 172.21.15.64:0/3668608612 learned_addr learned my addr 172.21.15.64:0/3668608612 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:30.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.894+0000 7f33e9959640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33e406d080 0x7f33e415d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51288/0 (socket says 172.21.15.64:51288) 2023-11-24T08:04:30.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.894+0000 7f33ea15a640 1 --2- 172.21.15.64:0/3668608612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e4157770 0x7f33e406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:30.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.894+0000 7f33e9959640 1 -- 172.21.15.64:0/3668608612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e4157770 msgr2=0x7f33e406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:30.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.894+0000 7f33e9959640 1 --2- 172.21.15.64:0/3668608612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e4157770 0x7f33e406cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:30.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.894+0000 7f33e9959640 1 -- 172.21.15.64:0/3668608612 >> v1:172.21.15.64:6789/0 conn(0x7f33e415ddb0 legacy=0x7f33e40ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:30.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.894+0000 7f33e9959640 1 -- 172.21.15.64:0/3668608612 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e415e610 con 0x7f33e406d080 2023-11-24T08:04:30.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.896+0000 7f33e9959640 1 --2- 172.21.15.64:0/3668608612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e415d870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f33d40097a0 tx=0x7f33d402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffa831d6f836813 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:30.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.896+0000 7f33e9158640 1 -- 172.21.15.64:0/3668608612 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d402d040 con 0x7f33e406d080 2023-11-24T08:04:30.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.896+0000 7f33e9158640 1 -- 172.21.15.64:0/3668608612 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d4031810 con 0x7f33e406d080 2023-11-24T08:04:30.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.896+0000 7f33e9158640 1 -- 172.21.15.64:0/3668608612 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4031ba0 con 0x7f33e406d080 2023-11-24T08:04:30.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.897+0000 7f33eb15c640 1 -- 172.21.15.64:0/3668608612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 msgr2=0x7f33e415d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:30.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.897+0000 7f33eb15c640 1 --2- 172.21.15.64:0/3668608612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e415d870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f33d40097a0 tx=0x7f33d402b8b0 comp rx=0 tx=0).stop 2023-11-24T08:04:30.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.897+0000 7f33eb15c640 1 -- 172.21.15.64:0/3668608612 shutdown_connections 2023-11-24T08:04:30.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.897+0000 7f33eb15c640 1 --2- 172.21.15.64:0/3668608612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e415d870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:30.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.897+0000 7f33eb15c640 1 --2- 172.21.15.64:0/3668608612 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e4157770 0x7f33e406cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:30.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.897+0000 7f33eb15c640 1 -- 172.21.15.64:0/3668608612 >> 172.21.15.64:0/3668608612 conn(0x7f33e406a860 msgr2=0x7f33e406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:30.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.897+0000 7f33eb15c640 1 -- 172.21.15.64:0/3668608612 shutdown_connections 2023-11-24T08:04:30.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.897+0000 7f33eb15c640 1 -- 172.21.15.64:0/3668608612 wait complete. 2023-11-24T08:04:30.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.900+0000 7f33eb15c640 1 Processor -- start 2023-11-24T08:04:30.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.900+0000 7f33eb15c640 1 -- start start 2023-11-24T08:04:30.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33eb15c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e4154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:30.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33eb15c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4157770 0x7f33e4158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:30.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33eb15c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e415ddb0 0x7f33e41593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:30.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33eb15c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4155110 con 0x7f33e406d080 2023-11-24T08:04:30.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33eb15c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4157df0 con 0x7f33e4157770 2023-11-24T08:04:30.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33eb15c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41f38e0 con 0x7f33e415ddb0 2023-11-24T08:04:30.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33ea15a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e4154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:30.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33ea15a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e4154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51304/0 (socket says 172.21.15.64:51304) 2023-11-24T08:04:30.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.901+0000 7f33ea15a640 1 -- 172.21.15.64:0/2695762760 learned_addr learned my addr 172.21.15.64:0/2695762760 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:30.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.902+0000 7f33e9959640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4157770 0x7f33e4158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:30.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.902+0000 7f33ea15a640 1 -- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e415ddb0 msgr2=0x7f33e41593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:30.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.902+0000 7f33ea15a640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e415ddb0 0x7f33e41593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:30.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.902+0000 7f33ea15a640 1 -- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4157770 msgr2=0x7f33e4158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:30.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.902+0000 7f33ea15a640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4157770 0x7f33e4158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:30.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.902+0000 7f33ea15a640 1 -- 172.21.15.64:0/2695762760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4009400 con 0x7f33e406d080 2023-11-24T08:04:30.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.902+0000 7f33ea15a640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e4154bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f33e000d800 tx=0x7f33e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:30.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.902+0000 7f33daffd640 1 -- 172.21.15.64:0/2695762760 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e000b6c0 con 0x7f33e406d080 2023-11-24T08:04:30.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.903+0000 7f33daffd640 1 -- 172.21.15.64:0/2695762760 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e000b820 con 0x7f33e406d080 2023-11-24T08:04:30.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.903+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e4158040 con 0x7f33e406d080 2023-11-24T08:04:30.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.903+0000 7f33daffd640 1 -- 172.21.15.64:0/2695762760 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e000bbb0 con 0x7f33e406d080 2023-11-24T08:04:30.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.903+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e4158600 con 0x7f33e406d080 2023-11-24T08:04:30.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.904+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e4158790 con 0x7f33e406d080 2023-11-24T08:04:30.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.905+0000 7f33daffd640 1 -- 172.21.15.64:0/2695762760 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33e000bd10 con 0x7f33e406d080 2023-11-24T08:04:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.906+0000 7f33daffd640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f33c806ae40 0x7f33c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.906+0000 7f33daffd640 1 -- 172.21.15.64:0/2695762760 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33e0089c70 con 0x7f33e406d080 2023-11-24T08:04:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.907+0000 7f33e9959640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f33c806ae40 0x7f33c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:30.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.908+0000 7f33e9959640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f33c806ae40 0x7f33c806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33d4036000 tx=0x7f33d403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:30.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:30.911+0000 7f33daffd640 1 -- 172.21.15.64:0/2695762760 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33e0054c80 con 0x7f33e406d080 2023-11-24T08:04:31.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.111+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33e404e840 con 0x7f33e406d080 2023-11-24T08:04:31.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.114+0000 7f33daffd640 1 -- 172.21.15.64:0/2695762760 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f33e00582a0 con 0x7f33e406d080 2023-11-24T08:04:31.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:31.117 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:31.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.117+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f33c806ae40 msgr2=0x7f33c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.117+0000 7f33eb15c640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f33c806ae40 0x7f33c806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33d4036000 tx=0x7f33d403c040 comp rx=0 tx=0).stop 2023-11-24T08:04:31.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.117+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 msgr2=0x7f33e4154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.117+0000 7f33eb15c640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e4154bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f33e000d800 tx=0x7f33e000dcc0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.118+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 shutdown_connections 2023-11-24T08:04:31.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.118+0000 7f33eb15c640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f33c806ae40 0x7f33c806d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.118+0000 7f33eb15c640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f33e415ddb0 0x7f33e41593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.118+0000 7f33eb15c640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f33e4157770 0x7f33e4158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.118+0000 7f33eb15c640 1 --2- 172.21.15.64:0/2695762760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e406d080 0x7f33e4154bd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.118+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 >> 172.21.15.64:0/2695762760 conn(0x7f33e406a860 msgr2=0x7f33e40af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:31.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.118+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 shutdown_connections 2023-11-24T08:04:31.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.118+0000 7f33eb15c640 1 -- 172.21.15.64:0/2695762760 wait complete. 2023-11-24T08:04:31.140 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:31.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.368+0000 7f7bdeee1640 1 Processor -- start 2023-11-24T08:04:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.368+0000 7f7bdeee1640 1 -- start start 2023-11-24T08:04:31.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.370+0000 7f7bdeee1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 0x7f7bd815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.370+0000 7f7bdeee1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd815f930 con 0x7f7bd80b35a0 2023-11-24T08:04:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.370+0000 7f7bdeee1640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd80b3b10 con 0x7f7bd815f0a0 2023-11-24T08:04:31.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.370+0000 7f7bdeee1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80b43b0 con 0x7f7bd8144770 2023-11-24T08:04:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.370+0000 7f7bdd6de640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7bd80b35a0 0x7f7bd806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37758/0 (socket says 172.21.15.64:37758) 2023-11-24T08:04:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.370+0000 7f7bdd6de640 1 -- 172.21.15.64:0/2213162382 learned_addr learned my addr 172.21.15.64:0/2213162382 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.370+0000 7f7bddedf640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 0x7f7bd815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306848245 0 0) 0x7f7bd815f930 con 0x7f7bd80b35a0 2023-11-24T08:04:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc4003520 con 0x7f7bd80b35a0 2023-11-24T08:04:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1821336532 0 0) 0x7f7bc4003520 con 0x7f7bd80b35a0 2023-11-24T08:04:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 msgr2=0x7f7bd815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 0x7f7bd815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 >> v1:172.21.15.91:6789/0 conn(0x7f7bd815f0a0 legacy=0x7f7bd80ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd80b3df0 con 0x7f7bd80b35a0 2023-11-24T08:04:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f7bc8001670 con 0x7f7bd80b35a0 2023-11-24T08:04:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 >> v1:172.21.15.64:6789/0 conn(0x7f7bd80b35a0 legacy=0x7f7bd806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bc4003a10 0x7f7bc4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 0x7f7bc4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80b43b0 con 0x7f7bc40047c0 2023-11-24T08:04:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc400ba80 con 0x7f7bc4003a10 2023-11-24T08:04:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.371+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc400bc40 con 0x7f7bd8144770 2023-11-24T08:04:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bdd6de640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bddedf640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bc4003a10 0x7f7bc4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bde6e0640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 0x7f7bc4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bdd6de640 1 -- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 msgr2=0x7f7bc4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:31.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bdd6de640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 0x7f7bc4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bdd6de640 1 -- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bc4003a10 msgr2=0x7f7bc4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:31.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bdd6de640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bc4003a10 0x7f7bc4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bdd6de640 1 -- 172.21.15.64:0/2213162382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc4003520 con 0x7f7bc40047c0 2023-11-24T08:04:31.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.372+0000 7f7bde6e0640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8144770 0x7f7bc4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.373+0000 7f7bdd6de640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bc4004c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c240 tx=0x7f7bc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68171efeeb8d25ba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.373+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8005ad0 con 0x7f7bc40047c0 2023-11-24T08:04:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.373+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8005c30 con 0x7f7bc40047c0 2023-11-24T08:04:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.374+0000 7f7bdcedd640 1 -- 172.21.15.64:0/2213162382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc802c8b0 con 0x7f7bc40047c0 2023-11-24T08:04:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.374+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2213162382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 msgr2=0x7f7bc4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.374+0000 7f7bdeee1640 1 --2- 172.21.15.64:0/2213162382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bc4004c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c240 tx=0x7f7bc40057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.374+0000 7f7bde6e0640 1 -- 172.21.15.64:0/2213162382 reap_dead start 2023-11-24T08:04:31.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.374+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2213162382 shutdown_connections 2023-11-24T08:04:31.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.374+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2213162382 >> 172.21.15.64:0/2213162382 conn(0x7f7bd80cec10 msgr2=0x7f7bd80cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:31.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.374+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2213162382 shutdown_connections 2023-11-24T08:04:31.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.374+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2213162382 wait complete. 2023-11-24T08:04:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.376+0000 7f7bdeee1640 1 Processor -- start 2023-11-24T08:04:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.376+0000 7f7bdeee1640 1 -- start start 2023-11-24T08:04:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.377+0000 7f7bdeee1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bd81f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.377+0000 7f7bdeee1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bd80a8460 0x7f7bd80a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.377+0000 7f7bdeee1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8070130 0x7f7bd80705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.377+0000 7f7bdeee1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80a8e20 con 0x7f7bc40047c0 2023-11-24T08:04:31.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.377+0000 7f7bdeee1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8070af0 con 0x7f7bd80a8460 2023-11-24T08:04:31.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.377+0000 7f7bdeee1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd81f33a0 con 0x7f7bd8070130 2023-11-24T08:04:31.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bd81f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bd81f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51330/0 (socket says 172.21.15.64:51330) 2023-11-24T08:04:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 -- 172.21.15.64:0/2160524298 learned_addr learned my addr 172.21.15.64:0/2160524298 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bdd6de640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bd80a8460 0x7f7bd80a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 -- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8070130 msgr2=0x7f7bd80705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8070130 0x7f7bd80705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 -- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bd80a8460 msgr2=0x7f7bd80a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bd80a8460 0x7f7bd80a88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 -- 172.21.15.64:0/2160524298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd8070d40 con 0x7f7bc40047c0 2023-11-24T08:04:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.378+0000 7f7bddedf640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bd81f5da0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4011600 tx=0x7f7bd4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.379+0000 7f7bce7fc640 1 -- 172.21.15.64:0/2160524298 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400f920 con 0x7f7bc40047c0 2023-11-24T08:04:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.379+0000 7f7bce7fc640 1 -- 172.21.15.64:0/2160524298 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd400fa80 con 0x7f7bc40047c0 2023-11-24T08:04:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.379+0000 7f7bce7fc640 1 -- 172.21.15.64:0/2160524298 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400fdf0 con 0x7f7bc40047c0 2023-11-24T08:04:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.379+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd8071020 con 0x7f7bc40047c0 2023-11-24T08:04:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.379+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd80715e0 con 0x7f7bc40047c0 2023-11-24T08:04:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.380+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd804e840 con 0x7f7bc40047c0 2023-11-24T08:04:31.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.381+0000 7f7bce7fc640 1 -- 172.21.15.64:0/2160524298 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bd4015460 con 0x7f7bc40047c0 2023-11-24T08:04:31.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.382+0000 7f7bce7fc640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7bbc06ae40 0x7f7bbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.382+0000 7f7bce7fc640 1 -- 172.21.15.64:0/2160524298 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bd4089b00 con 0x7f7bc40047c0 2023-11-24T08:04:31.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.383+0000 7f7bdd6de640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7bbc06ae40 0x7f7bbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.384+0000 7f7bdd6de640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7bbc06ae40 0x7f7bbc06d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8034000 tx=0x7f7bc8009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.387+0000 7f7bce7fc640 1 -- 172.21.15.64:0/2160524298 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bd408be60 con 0x7f7bc40047c0 2023-11-24T08:04:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.579+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bd8143980 con 0x7f7bc40047c0 2023-11-24T08:04:31.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.580+0000 7f7bce7fc640 1 -- 172.21.15.64:0/2160524298 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bd40220a0 con 0x7f7bc40047c0 2023-11-24T08:04:31.582 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:31.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7bbc06ae40 msgr2=0x7f7bbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7bbc06ae40 0x7f7bbc06d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8034000 tx=0x7f7bc8009030 comp rx=0 tx=0).stop 2023-11-24T08:04:31.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 msgr2=0x7f7bd81f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bd81f5da0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4011600 tx=0x7f7bd4011ac0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 shutdown_connections 2023-11-24T08:04:31.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7bbc06ae40 0x7f7bbc06d300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7bd8070130 0x7f7bd80705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7bd80a8460 0x7f7bd80a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 --2- 172.21.15.64:0/2160524298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc40047c0 0x7f7bd81f5da0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 >> 172.21.15.64:0/2160524298 conn(0x7f7bd80cec10 msgr2=0x7f7bd80cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 shutdown_connections 2023-11-24T08:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.583+0000 7f7bdeee1640 1 -- 172.21.15.64:0/2160524298 wait complete. 2023-11-24T08:04:31.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:31.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.823+0000 7f0c9e57c640 1 Processor -- start 2023-11-24T08:04:31.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.824+0000 7f0c9e57c640 1 -- start start 2023-11-24T08:04:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.825+0000 7f0c9e57c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0c9815b350 0x7f0c980abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.826+0000 7f0c9e57c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c980adb60 0x7f0c980adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.826+0000 7f0c9e57c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9815b730 con 0x7f0c9809cb30 2023-11-24T08:04:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.826+0000 7f0c9e57c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c980ac740 con 0x7f0c980adb60 2023-11-24T08:04:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.826+0000 7f0c9e57c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c980ac460 con 0x7f0c9815b350 2023-11-24T08:04:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.826+0000 7f0c9cb19640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c980adb60 0x7f0c980adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.826+0000 7f0c9cb19640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c980adb60 0x7f0c980adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51332/0 (socket says 172.21.15.64:51332) 2023-11-24T08:04:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.826+0000 7f0c9cb19640 1 -- 172.21.15.64:0/1035814824 learned_addr learned my addr 172.21.15.64:0/1035814824 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.826+0000 7f0c97fff640 1 --2- 172.21.15.64:0/1035814824 >> v2:172.21.15.91:3300/0 conn(0x7f0c9815b350 0x7f0c980abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.827+0000 7f0c9cb19640 1 -- 172.21.15.64:0/1035814824 >> v2:172.21.15.91:3300/0 conn(0x7f0c9815b350 msgr2=0x7f0c980abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.827+0000 7f0c9cb19640 1 --2- 172.21.15.64:0/1035814824 >> v2:172.21.15.91:3300/0 conn(0x7f0c9815b350 0x7f0c980abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.827+0000 7f0c9cb19640 1 -- 172.21.15.64:0/1035814824 >> v1:172.21.15.91:6789/0 conn(0x7f0c9809cb30 legacy=0x7f0c980b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.827+0000 7f0c9cb19640 1 -- 172.21.15.64:0/1035814824 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980b2a50 con 0x7f0c980adb60 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.828+0000 7f0c9cb19640 1 --2- 172.21.15.64:0/1035814824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c980adb60 0x7f0c980adf70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c980b5100 tx=0x7f0c9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c304060d70eeda server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.828+0000 7f0c96ffd640 1 -- 172.21.15.64:0/1035814824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9002ca80 con 0x7f0c980adb60 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.828+0000 7f0c96ffd640 1 -- 172.21.15.64:0/1035814824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9002cbe0 con 0x7f0c980adb60 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.828+0000 7f0c96ffd640 1 -- 172.21.15.64:0/1035814824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900343f0 con 0x7f0c980adb60 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.829+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1035814824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c980adb60 msgr2=0x7f0c980adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.829+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1035814824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c980adb60 0x7f0c980adf70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c980b5100 tx=0x7f0c9002bee0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.829+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1035814824 shutdown_connections 2023-11-24T08:04:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.829+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1035814824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c980adb60 0x7f0c980adf70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.829+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1035814824 >> v2:172.21.15.91:3300/0 conn(0x7f0c9815b350 0x7f0c980abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.829+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1035814824 >> 172.21.15.64:0/1035814824 conn(0x7f0c980b0ba0 msgr2=0x7f0c980b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:31.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.829+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1035814824 shutdown_connections 2023-11-24T08:04:31.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.829+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1035814824 wait complete. 2023-11-24T08:04:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.831+0000 7f0c9e57c640 1 Processor -- start 2023-11-24T08:04:31.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.831+0000 7f0c9e57c640 1 -- start start 2023-11-24T08:04:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c9e57c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9809cb30 0x7f0c9813da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c9e57c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c980adb60 0x7f0c9814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c9e57c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0c9815b350 0x7f0c9809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c9e57c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9813df70 con 0x7f0c9809cb30 2023-11-24T08:04:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c9e57c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981568b0 con 0x7f0c980adb60 2023-11-24T08:04:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c9e57c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98156b90 con 0x7f0c9815b350 2023-11-24T08:04:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c97fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9809cb30 0x7f0c9813da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c97fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9809cb30 0x7f0c9813da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51342/0 (socket says 172.21.15.64:51342) 2023-11-24T08:04:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c97fff640 1 -- 172.21.15.64:0/1146057019 learned_addr learned my addr 172.21.15.64:0/1146057019 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c97fff640 1 -- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0c9815b350 msgr2=0x7f0c9809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c97fff640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0c9815b350 0x7f0c9809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c97fff640 1 -- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c980adb60 msgr2=0x7f0c9814dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c97fff640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c980adb60 0x7f0c9814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.832+0000 7f0c97fff640 1 -- 172.21.15.64:0/1146057019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90009b30 con 0x7f0c9809cb30 2023-11-24T08:04:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.833+0000 7f0c97fff640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9809cb30 0x7f0c9813da30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800d800 tx=0x7f0c8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.833+0000 7f0c94ff9640 1 -- 172.21.15.64:0/1146057019 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800b6c0 con 0x7f0c9809cb30 2023-11-24T08:04:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.833+0000 7f0c94ff9640 1 -- 172.21.15.64:0/1146057019 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8800b820 con 0x7f0c9809cb30 2023-11-24T08:04:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.833+0000 7f0c94ff9640 1 -- 172.21.15.64:0/1146057019 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800bbb0 con 0x7f0c9809cb30 2023-11-24T08:04:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.833+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98156e70 con 0x7f0c9809cb30 2023-11-24T08:04:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.833+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9814e530 con 0x7f0c9809cb30 2023-11-24T08:04:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.835+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9814e780 con 0x7f0c9809cb30 2023-11-24T08:04:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.835+0000 7f0c94ff9640 1 -- 172.21.15.64:0/1146057019 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c8800bd10 con 0x7f0c9809cb30 2023-11-24T08:04:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.836+0000 7f0c94ff9640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8406ae40 0x7f0c8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.836+0000 7f0c94ff9640 1 -- 172.21.15.64:0/1146057019 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c88089ca0 con 0x7f0c9809cb30 2023-11-24T08:04:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.837+0000 7f0c977fe640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8406ae40 0x7f0c8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.838+0000 7f0c977fe640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8406ae40 0x7f0c8406d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c006fd0 tx=0x7f0c8c006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:31.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:31.841+0000 7f0c94ff9640 1 -- 172.21.15.64:0/1146057019 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c88054cb0 con 0x7f0c9809cb30 2023-11-24T08:04:32.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.036+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c98157390 con 0x7f0c9809cb30 2023-11-24T08:04:32.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.037+0000 7f0c94ff9640 1 -- 172.21.15.64:0/1146057019 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c98157390 con 0x7f0c9809cb30 2023-11-24T08:04:32.039 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8406ae40 msgr2=0x7f0c8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8406ae40 0x7f0c8406d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c006fd0 tx=0x7f0c8c006bc0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9809cb30 msgr2=0x7f0c9813da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9809cb30 0x7f0c9813da30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800d800 tx=0x7f0c8800dcc0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 shutdown_connections 2023-11-24T08:04:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8406ae40 0x7f0c8406d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0c9815b350 0x7f0c9809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0c980adb60 0x7f0c9814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 --2- 172.21.15.64:0/1146057019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9809cb30 0x7f0c9813da30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 >> 172.21.15.64:0/1146057019 conn(0x7f0c980b0ba0 msgr2=0x7f0c98070490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:32.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 shutdown_connections 2023-11-24T08:04:32.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.040+0000 7f0c9e57c640 1 -- 172.21.15.64:0/1146057019 wait complete. 2023-11-24T08:04:32.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:32.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.275+0000 7fc122973640 1 Processor -- start 2023-11-24T08:04:32.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.275+0000 7fc122973640 1 -- start start 2023-11-24T08:04:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.276+0000 7fc122973640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc11c15d3b0 0x7fc11c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.276+0000 7fc122973640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc11c0d1d90 0x7fc11c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.276+0000 7fc122973640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0c6bf0 con 0x7fc11c0d0fa0 2023-11-24T08:04:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.276+0000 7fc122973640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c0d1430 con 0x7fc11c0d1d90 2023-11-24T08:04:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.276+0000 7fc122973640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c0cedc0 con 0x7fc11c15d3b0 2023-11-24T08:04:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc121170640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc11c0d0fa0 0x7fc11c0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37760/0 (socket says 172.21.15.64:37760) 2023-11-24T08:04:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc121170640 1 -- 172.21.15.64:0/3270069334 learned_addr learned my addr 172.21.15.64:0/3270069334 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc122172640 1 --2- 172.21.15.64:0/3270069334 >> v2:172.21.15.64:3300/0 conn(0x7fc11c0d1d90 0x7fc11c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc121971640 1 --2- 172.21.15.64:0/3270069334 >> v2:172.21.15.91:3300/0 conn(0x7fc11c15d3b0 0x7fc11c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc12096f640 1 -- 172.21.15.64:0/3270069334 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466993116 0 0) 0x7fc11c0c6bf0 con 0x7fc11c0d0fa0 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc12096f640 1 -- 172.21.15.64:0/3270069334 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc108003520 con 0x7fc11c0d0fa0 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc122172640 1 -- 172.21.15.64:0/3270069334 >> v2:172.21.15.91:3300/0 conn(0x7fc11c15d3b0 msgr2=0x7fc11c0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc122172640 1 --2- 172.21.15.64:0/3270069334 >> v2:172.21.15.91:3300/0 conn(0x7fc11c15d3b0 0x7fc11c0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc122172640 1 -- 172.21.15.64:0/3270069334 >> v1:172.21.15.64:6789/0 conn(0x7fc11c0d0fa0 legacy=0x7fc11c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc122172640 1 -- 172.21.15.64:0/3270069334 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0c6e40 con 0x7fc11c0d1d90 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.277+0000 7fc121971640 1 --2- 172.21.15.64:0/3270069334 >> v2:172.21.15.91:3300/0 conn(0x7fc11c15d3b0 0x7fc11c0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.279+0000 7fc122172640 1 --2- 172.21.15.64:0/3270069334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c0b3ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc11c08b2a0 tx=0x7fc11402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92ffb59758135b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.279+0000 7fc12096f640 1 -- 172.21.15.64:0/3270069334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11402cbf0 con 0x7fc11c0d1d90 2023-11-24T08:04:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.279+0000 7fc12096f640 1 -- 172.21.15.64:0/3270069334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11402cd50 con 0x7fc11c0d1d90 2023-11-24T08:04:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.279+0000 7fc12096f640 1 -- 172.21.15.64:0/3270069334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114034660 con 0x7fc11c0d1d90 2023-11-24T08:04:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.279+0000 7fc122973640 1 -- 172.21.15.64:0/3270069334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 msgr2=0x7fc11c0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.279+0000 7fc122973640 1 --2- 172.21.15.64:0/3270069334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c0b3ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc11c08b2a0 tx=0x7fc11402be40 comp rx=0 tx=0).stop 2023-11-24T08:04:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.280+0000 7fc122973640 1 -- 172.21.15.64:0/3270069334 shutdown_connections 2023-11-24T08:04:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.280+0000 7fc122973640 1 --2- 172.21.15.64:0/3270069334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c0b3ca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.280+0000 7fc122973640 1 --2- 172.21.15.64:0/3270069334 >> v2:172.21.15.91:3300/0 conn(0x7fc11c15d3b0 0x7fc11c0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.280+0000 7fc122973640 1 -- 172.21.15.64:0/3270069334 >> 172.21.15.64:0/3270069334 conn(0x7fc11c0d30a0 msgr2=0x7fc11c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.280+0000 7fc122973640 1 -- 172.21.15.64:0/3270069334 shutdown_connections 2023-11-24T08:04:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.280+0000 7fc122973640 1 -- 172.21.15.64:0/3270069334 wait complete. 2023-11-24T08:04:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.281+0000 7fc122973640 1 Processor -- start 2023-11-24T08:04:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.282+0000 7fc122973640 1 -- start start 2023-11-24T08:04:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.282+0000 7fc122973640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc11c0d0fa0 0x7fc11c0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.282+0000 7fc122973640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc122973640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc11c15d3b0 0x7fc11c0cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc122973640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0b8fc0 con 0x7fc11c0d1d90 2023-11-24T08:04:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc122973640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c14c3c0 con 0x7fc11c0d0fa0 2023-11-24T08:04:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc122973640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0b7fa0 con 0x7fc11c15d3b0 2023-11-24T08:04:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121170640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121170640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c148bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51362/0 (socket says 172.21.15.64:51362) 2023-11-24T08:04:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121170640 1 -- 172.21.15.64:0/2445325059 learned_addr learned my addr 172.21.15.64:0/2445325059 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121971640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc11c0d0fa0 0x7fc11c0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121170640 1 -- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc11c15d3b0 msgr2=0x7fc11c0cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc122172640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc11c15d3b0 0x7fc11c0cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121170640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc11c15d3b0 0x7fc11c0cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121170640 1 -- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc11c0d0fa0 msgr2=0x7fc11c0b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121170640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc11c0d0fa0 0x7fc11c0b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121170640 1 -- 172.21.15.64:0/2445325059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114009a70 con 0x7fc11c0d1d90 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc121971640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc11c0d0fa0 0x7fc11c0b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.283+0000 7fc122172640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc11c15d3b0 0x7fc11c0cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.284+0000 7fc121170640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c148bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c500 tx=0x7fc10c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.284+0000 7fc1127fc640 1 -- 172.21.15.64:0/2445325059 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c005ce0 con 0x7fc11c0d1d90 2023-11-24T08:04:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.284+0000 7fc1127fc640 1 -- 172.21.15.64:0/2445325059 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c005e40 con 0x7fc11c0d1d90 2023-11-24T08:04:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.284+0000 7fc1127fc640 1 -- 172.21.15.64:0/2445325059 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c007330 con 0x7fc11c0d1d90 2023-11-24T08:04:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.284+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c0b8280 con 0x7fc11c0d1d90 2023-11-24T08:04:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.284+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c0b7c50 con 0x7fc11c0d1d90 2023-11-24T08:04:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.285+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0b87a0 con 0x7fc11c0d1d90 2023-11-24T08:04:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.286+0000 7fc1127fc640 1 -- 172.21.15.64:0/2445325059 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc10c007550 con 0x7fc11c0d1d90 2023-11-24T08:04:32.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.288+0000 7fc1127fc640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc10006ae40 0x7fc10006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.288+0000 7fc1127fc640 1 -- 172.21.15.64:0/2445325059 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc10c088cc0 con 0x7fc11c0d1d90 2023-11-24T08:04:32.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.288+0000 7fc121971640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc10006ae40 0x7fc10006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.289+0000 7fc121971640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc10006ae40 0x7fc10006d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc118006fd0 tx=0x7fc118008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:32.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.292+0000 7fc1127fc640 1 -- 172.21.15.64:0/2445325059 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc11c0b87a0 con 0x7fc11c0d1d90 2023-11-24T08:04:32.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.483+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc11c0b87a0 con 0x7fc11c0d1d90 2023-11-24T08:04:32.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.484+0000 7fc1127fc640 1 -- 172.21.15.64:0/2445325059 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc11c0b87a0 con 0x7fc11c0d1d90 2023-11-24T08:04:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:32.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc10006ae40 msgr2=0x7fc10006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc10006ae40 0x7fc10006d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc118006fd0 tx=0x7fc118008040 comp rx=0 tx=0).stop 2023-11-24T08:04:32.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 msgr2=0x7fc11c148bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c148bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c500 tx=0x7fc10c00c9c0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 shutdown_connections 2023-11-24T08:04:32.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc10006ae40 0x7fc10006d300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc11c15d3b0 0x7fc11c0cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c0d1d90 0x7fc11c148bb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 --2- 172.21.15.64:0/2445325059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc11c0d0fa0 0x7fc11c0b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 >> 172.21.15.64:0/2445325059 conn(0x7fc11c0d30a0 msgr2=0x7fc11c0c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:32.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 shutdown_connections 2023-11-24T08:04:32.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.486+0000 7fc122973640 1 -- 172.21.15.64:0/2445325059 wait complete. 2023-11-24T08:04:32.507 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.729+0000 7f40eb01a640 1 Processor -- start 2023-11-24T08:04:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.729+0000 7f40eb01a640 1 -- start start 2023-11-24T08:04:32.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.731+0000 7f40eb01a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40e40cf500 0x7f40e40cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.731+0000 7f40eb01a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e4140900 con 0x7f40e413dfd0 2023-11-24T08:04:32.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.731+0000 7f40eb01a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e415fa70 con 0x7f40e40cf050 2023-11-24T08:04:32.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.731+0000 7f40eb01a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e415c580 con 0x7f40e40cf500 2023-11-24T08:04:32.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.731+0000 7f40ea018640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f40e413dfd0 0x7f40e415be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37772/0 (socket says 172.21.15.64:37772) 2023-11-24T08:04:32.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.731+0000 7f40ea018640 1 -- 172.21.15.64:0/904176612 learned_addr learned my addr 172.21.15.64:0/904176612 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:32.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.732+0000 7f40ea819640 1 --2- 172.21.15.64:0/904176612 >> v2:172.21.15.64:3300/0 conn(0x7f40e40cf500 0x7f40e40cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.732+0000 7f40e9016640 1 -- 172.21.15.64:0/904176612 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001600835 0 0) 0x7f40e4140900 con 0x7f40e413dfd0 2023-11-24T08:04:32.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.732+0000 7f40e9016640 1 -- 172.21.15.64:0/904176612 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40d0003520 con 0x7f40e413dfd0 2023-11-24T08:04:32.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.732+0000 7f40ea819640 1 -- 172.21.15.64:0/904176612 >> v1:172.21.15.91:6789/0 conn(0x7f40e40cf050 legacy=0x7f40e40cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.732+0000 7f40ea819640 1 -- 172.21.15.64:0/904176612 >> v1:172.21.15.64:6789/0 conn(0x7f40e413dfd0 legacy=0x7f40e415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.732+0000 7f40ea819640 1 -- 172.21.15.64:0/904176612 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e415ee60 con 0x7f40e40cf500 2023-11-24T08:04:32.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.733+0000 7f40ea819640 1 --2- 172.21.15.64:0/904176612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40cdec0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009d90 tx=0x7f40dc02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2dcb2248321f54 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:32.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.733+0000 7f40e9016640 1 -- 172.21.15.64:0/904176612 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc02ccf0 con 0x7f40e40cf500 2023-11-24T08:04:32.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40e9016640 1 -- 172.21.15.64:0/904176612 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc02ce50 con 0x7f40e40cf500 2023-11-24T08:04:32.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40e9016640 1 -- 172.21.15.64:0/904176612 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc034660 con 0x7f40e40cf500 2023-11-24T08:04:32.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40eb01a640 1 -- 172.21.15.64:0/904176612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 msgr2=0x7f40e40cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40eb01a640 1 --2- 172.21.15.64:0/904176612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40cdec0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009d90 tx=0x7f40dc02bf40 comp rx=0 tx=0).stop 2023-11-24T08:04:32.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40eb01a640 1 -- 172.21.15.64:0/904176612 shutdown_connections 2023-11-24T08:04:32.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40eb01a640 1 --2- 172.21.15.64:0/904176612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40cdec0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40eb01a640 1 -- 172.21.15.64:0/904176612 >> 172.21.15.64:0/904176612 conn(0x7f40e406a860 msgr2=0x7f40e406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:32.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40eb01a640 1 -- 172.21.15.64:0/904176612 shutdown_connections 2023-11-24T08:04:32.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.734+0000 7f40eb01a640 1 -- 172.21.15.64:0/904176612 wait complete. 2023-11-24T08:04:32.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.736+0000 7f40eb01a640 1 Processor -- start 2023-11-24T08:04:32.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.736+0000 7f40eb01a640 1 -- start start 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40eb01a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40e40cf050 0x7f40e4158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40eb01a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40eb01a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40e413dfd0 0x7f40e40b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40eb01a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40b2570 con 0x7f40e40cf500 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40eb01a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40b18f0 con 0x7f40e40cf050 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40eb01a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40b1bd0 con 0x7f40e413dfd0 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40e9817640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40e9817640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51378/0 (socket says 172.21.15.64:51378) 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.737+0000 7f40e9817640 1 -- 172.21.15.64:0/973769892 learned_addr learned my addr 172.21.15.64:0/973769892 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40ea018640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40e40cf050 0x7f40e4158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40ea819640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40e413dfd0 0x7f40e40b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40e9817640 1 -- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40e413dfd0 msgr2=0x7f40e40b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40e9817640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40e413dfd0 0x7f40e40b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40e9817640 1 -- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40e40cf050 msgr2=0x7f40e4158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40e9817640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40e40cf050 0x7f40e4158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40e9817640 1 -- 172.21.15.64:0/973769892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc009a70 con 0x7f40e40cf500 2023-11-24T08:04:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40ea819640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40e413dfd0 0x7f40e40b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.738+0000 7f40e9817640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40b2990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c450 tx=0x7f40d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.739+0000 7f40daffd640 1 -- 172.21.15.64:0/973769892 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40052d0 con 0x7f40e40cf500 2023-11-24T08:04:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.739+0000 7f40daffd640 1 -- 172.21.15.64:0/973769892 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4005430 con 0x7f40e40cf500 2023-11-24T08:04:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.739+0000 7f40daffd640 1 -- 172.21.15.64:0/973769892 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4012870 con 0x7f40e40cf500 2023-11-24T08:04:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.739+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e40b1eb0 con 0x7f40e40cf500 2023-11-24T08:04:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.739+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e413cda0 con 0x7f40e40cf500 2023-11-24T08:04:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.740+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e40b4090 con 0x7f40e40cf500 2023-11-24T08:04:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.741+0000 7f40daffd640 1 -- 172.21.15.64:0/973769892 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40d40129d0 con 0x7f40e40cf500 2023-11-24T08:04:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.742+0000 7f40daffd640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40c806ae40 0x7f40c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.742+0000 7f40daffd640 1 -- 172.21.15.64:0/973769892 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40d40891f0 con 0x7f40e40cf500 2023-11-24T08:04:32.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.742+0000 7f40ea018640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40c806ae40 0x7f40c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:32.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.743+0000 7f40ea018640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40c806ae40 0x7f40c806d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f40e00011e0 tx=0x7f40e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:32.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.747+0000 7f40daffd640 1 -- 172.21.15.64:0/973769892 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40d4054200 con 0x7f40e40cf500 2023-11-24T08:04:32.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.942+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40e40b26b0 con 0x7f40e40cf500 2023-11-24T08:04:32.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.945+0000 7f40daffd640 1 -- 172.21.15.64:0/973769892 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f40e40b26b0 con 0x7f40e40cf500 2023-11-24T08:04:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:32.947 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40c806ae40 msgr2=0x7f40c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40c806ae40 0x7f40c806d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f40e00011e0 tx=0x7f40e0009040 comp rx=0 tx=0).stop 2023-11-24T08:04:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 msgr2=0x7f40e40b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40b2990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c450 tx=0x7f40d400c910 comp rx=0 tx=0).stop 2023-11-24T08:04:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 shutdown_connections 2023-11-24T08:04:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40c806ae40 0x7f40c806d300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40e413dfd0 0x7f40e40b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40e40cf500 0x7f40e40b2990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 --2- 172.21.15.64:0/973769892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f40e40cf050 0x7f40e4158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 >> 172.21.15.64:0/973769892 conn(0x7f40e406a860 msgr2=0x7f40e40ad360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 shutdown_connections 2023-11-24T08:04:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:32.947+0000 7f40eb01a640 1 -- 172.21.15.64:0/973769892 wait complete. 2023-11-24T08:04:32.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:32.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:33.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.196+0000 7f411bfff640 1 Processor -- start 2023-11-24T08:04:33.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.196+0000 7f411bfff640 1 -- start start 2023-11-24T08:04:33.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.198+0000 7f411bfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f411c0b4140 0x7f411c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.198+0000 7f411bfff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c070d40 con 0x7f411c1439b0 2023-11-24T08:04:33.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.198+0000 7f411bfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c071020 con 0x7f411c1435d0 2023-11-24T08:04:33.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.198+0000 7f411bfff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c071300 con 0x7f411c0b4140 2023-11-24T08:04:33.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.199+0000 7f411b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f411c1439b0 0x7f411c070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37782/0 (socket says 172.21.15.64:37782) 2023-11-24T08:04:33.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.199+0000 7f411b7fe640 1 -- 172.21.15.64:0/1197223821 learned_addr learned my addr 172.21.15.64:0/1197223821 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:33.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.199+0000 7f4113fff640 1 --2- 172.21.15.64:0/1197223821 >> v2:172.21.15.91:3300/0 conn(0x7f411c0b4140 0x7f411c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.199+0000 7f411a7fc640 1 -- 172.21.15.64:0/1197223821 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265878082 0 0) 0x7f411c070d40 con 0x7f411c1439b0 2023-11-24T08:04:33.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.199+0000 7f411a7fc640 1 -- 172.21.15.64:0/1197223821 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4108003520 con 0x7f411c1439b0 2023-11-24T08:04:33.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.199+0000 7f4113fff640 1 -- 172.21.15.64:0/1197223821 >> v1:172.21.15.91:6789/0 conn(0x7f411c1435d0 legacy=0x7f411c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.199+0000 7f4113fff640 1 -- 172.21.15.64:0/1197223821 >> v1:172.21.15.64:6789/0 conn(0x7f411c1439b0 legacy=0x7f411c070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.199+0000 7f411a7fc640 1 -- 172.21.15.64:0/1197223821 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1548531189 0 0) 0x7f4108003520 con 0x7f411c1439b0 2023-11-24T08:04:33.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.200+0000 7f4113fff640 1 -- 172.21.15.64:0/1197223821 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0715e0 con 0x7f411c0b4140 2023-11-24T08:04:33.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.201+0000 7f4113fff640 1 --2- 172.21.15.64:0/1197223821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c0b4140 0x7f411c071de0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41040097a0 tx=0x7f410402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7a8f671ed33d7c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:33.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.201+0000 7f411a7fc640 1 -- 172.21.15.64:0/1197223821 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140019f0 con 0x7f411c0b4140 2023-11-24T08:04:33.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.201+0000 7f411a7fc640 1 -- 172.21.15.64:0/1197223821 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410402e050 con 0x7f411c0b4140 2023-11-24T08:04:33.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.201+0000 7f411a7fc640 1 -- 172.21.15.64:0/1197223821 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104030730 con 0x7f411c0b4140 2023-11-24T08:04:33.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.202+0000 7f411bfff640 1 -- 172.21.15.64:0/1197223821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c0b4140 msgr2=0x7f411c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.202+0000 7f411bfff640 1 --2- 172.21.15.64:0/1197223821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c0b4140 0x7f411c071de0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41040097a0 tx=0x7f410402b790 comp rx=0 tx=0).stop 2023-11-24T08:04:33.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.202+0000 7f411bfff640 1 -- 172.21.15.64:0/1197223821 shutdown_connections 2023-11-24T08:04:33.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.202+0000 7f411bfff640 1 --2- 172.21.15.64:0/1197223821 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c0b4140 0x7f411c071de0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.202+0000 7f411bfff640 1 -- 172.21.15.64:0/1197223821 >> 172.21.15.64:0/1197223821 conn(0x7f411c0cec10 msgr2=0x7f411c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:33.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.202+0000 7f411bfff640 1 -- 172.21.15.64:0/1197223821 shutdown_connections 2023-11-24T08:04:33.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.202+0000 7f411bfff640 1 -- 172.21.15.64:0/1197223821 wait complete. 2023-11-24T08:04:33.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.204+0000 7f411bfff640 1 Processor -- start 2023-11-24T08:04:33.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.204+0000 7f411bfff640 1 -- start start 2023-11-24T08:04:33.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f411bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f411c0b4140 0x7f411c13cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f411bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c1435d0 0x7f411c0af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f411bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c1439b0 0x7f411c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f411bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1f0b00 con 0x7f411c1435d0 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f411bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c140440 con 0x7f411c1439b0 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f411bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1efe80 con 0x7f411c0b4140 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f4113fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c1435d0 0x7f411c0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f4113fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c1435d0 0x7f411c0af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51382/0 (socket says 172.21.15.64:51382) 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.205+0000 7f4113fff640 1 -- 172.21.15.64:0/918691953 learned_addr learned my addr 172.21.15.64:0/918691953 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f411affd640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f411c0b4140 0x7f411c13cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f411b7fe640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c1439b0 0x7f411c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4113fff640 1 -- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f411c0b4140 msgr2=0x7f411c13cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4113fff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f411c0b4140 0x7f411c13cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4113fff640 1 -- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c1439b0 msgr2=0x7f411c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4113fff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c1439b0 0x7f411c0afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4113fff640 1 -- 172.21.15.64:0/918691953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009400 con 0x7f411c1435d0 2023-11-24T08:04:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f411affd640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f411c0b4140 0x7f411c13cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:33.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4113fff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c1435d0 0x7f411c0af580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f41040098c0 tx=0x7f41040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4118ff9640 1 -- 172.21.15.64:0/918691953 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410403c880 con 0x7f411c1435d0 2023-11-24T08:04:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4118ff9640 1 -- 172.21.15.64:0/918691953 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410403c9e0 con 0x7f411c1435d0 2023-11-24T08:04:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f4118ff9640 1 -- 172.21.15.64:0/918691953 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410403cd90 con 0x7f411c1435d0 2023-11-24T08:04:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.206+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c1f0160 con 0x7f411c1435d0 2023-11-24T08:04:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.207+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c1f0720 con 0x7f411c1435d0 2023-11-24T08:04:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.208+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c04e840 con 0x7f411c1435d0 2023-11-24T08:04:33.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.209+0000 7f4118ff9640 1 -- 172.21.15.64:0/918691953 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4104039040 con 0x7f411c1435d0 2023-11-24T08:04:33.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.210+0000 7f4118ff9640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40fc06ae40 0x7f40fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.210+0000 7f4118ff9640 1 -- 172.21.15.64:0/918691953 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41040ab210 con 0x7f411c1435d0 2023-11-24T08:04:33.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.211+0000 7f411affd640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40fc06ae40 0x7f40fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.212+0000 7f411affd640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40fc06ae40 0x7f40fc06d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f410c000f60 tx=0x7f410c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:33.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.215+0000 7f4118ff9640 1 -- 172.21.15.64:0/918691953 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41040762b0 con 0x7f411c1435d0 2023-11-24T08:04:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.406+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f411c1f0c40 con 0x7f411c1435d0 2023-11-24T08:04:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.407+0000 7f4118ff9640 1 -- 172.21.15.64:0/918691953 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41040798d0 con 0x7f411c1435d0 2023-11-24T08:04:33.409 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:33.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40fc06ae40 msgr2=0x7f40fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40fc06ae40 0x7f40fc06d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f410c000f60 tx=0x7f410c009040 comp rx=0 tx=0).stop 2023-11-24T08:04:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c1435d0 msgr2=0x7f411c0af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c1435d0 0x7f411c0af580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f41040098c0 tx=0x7f41040046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 shutdown_connections 2023-11-24T08:04:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f40fc06ae40 0x7f40fc06d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f411c1439b0 0x7f411c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c1435d0 0x7f411c0af580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 --2- 172.21.15.64:0/918691953 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f411c0b4140 0x7f411c13cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 >> 172.21.15.64:0/918691953 conn(0x7f411c0cec10 msgr2=0x7f411c06d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 shutdown_connections 2023-11-24T08:04:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.410+0000 7f411bfff640 1 -- 172.21.15.64:0/918691953 wait complete. 2023-11-24T08:04:33.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:33.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.657+0000 7efdbf59e640 1 Processor -- start 2023-11-24T08:04:33.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.658+0000 7efdbf59e640 1 -- start start 2023-11-24T08:04:33.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.659+0000 7efdbf59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efdc015d3b0 0x7efdc00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.659+0000 7efdbf59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdc00d0fa0 0x7efdc00d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.659+0000 7efdbf59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d1bc0 0x7efdc00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.659+0000 7efdbf59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc00c6bf0 con 0x7efdc00d0fa0 2023-11-24T08:04:33.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbf59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc00d1ff0 con 0x7efdc015d3b0 2023-11-24T08:04:33.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbf59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00d1430 con 0x7efdc00d1bc0 2023-11-24T08:04:33.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbdd9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdc00d0fa0 0x7efdc00d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbdd9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdc00d0fa0 0x7efdc00d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51388/0 (socket says 172.21.15.64:51388) 2023-11-24T08:04:33.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbdd9b640 1 -- 172.21.15.64:0/1844304116 learned_addr learned my addr 172.21.15.64:0/1844304116 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:33.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbe59c640 1 --2- 172.21.15.64:0/1844304116 >> v2:172.21.15.91:3300/0 conn(0x7efdc015d3b0 0x7efdc00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbed9d640 1 --2- 172.21.15.64:0/1844304116 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d1bc0 0x7efdc00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbdd9b640 1 -- 172.21.15.64:0/1844304116 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d1bc0 msgr2=0x7efdc00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbdd9b640 1 --2- 172.21.15.64:0/1844304116 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d1bc0 0x7efdc00b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbdd9b640 1 -- 172.21.15.64:0/1844304116 >> v2:172.21.15.91:3300/0 conn(0x7efdc015d3b0 msgr2=0x7efdc00ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:33.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbdd9b640 1 --2- 172.21.15.64:0/1844304116 >> v2:172.21.15.91:3300/0 conn(0x7efdc015d3b0 0x7efdc00ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.660+0000 7efdbdd9b640 1 -- 172.21.15.64:0/1844304116 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc00cedc0 con 0x7efdc00d0fa0 2023-11-24T08:04:33.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.662+0000 7efdbdd9b640 1 --2- 172.21.15.64:0/1844304116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d0fa0 0x7efdc00d1680 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efdc008b2a0 tx=0x7efda802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47008ee4cf498afa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:33.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.662+0000 7efdbd59a640 1 -- 172.21.15.64:0/1844304116 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda80317e0 con 0x7efdc00d0fa0 2023-11-24T08:04:33.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.662+0000 7efdbd59a640 1 -- 172.21.15.64:0/1844304116 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8031940 con 0x7efdc00d0fa0 2023-11-24T08:04:33.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.662+0000 7efdbd59a640 1 -- 172.21.15.64:0/1844304116 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8031cd0 con 0x7efdc00d0fa0 2023-11-24T08:04:33.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 -- 172.21.15.64:0/1844304116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d0fa0 msgr2=0x7efdc00d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 --2- 172.21.15.64:0/1844304116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d0fa0 0x7efdc00d1680 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efdc008b2a0 tx=0x7efda802b880 comp rx=0 tx=0).stop 2023-11-24T08:04:33.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 -- 172.21.15.64:0/1844304116 shutdown_connections 2023-11-24T08:04:33.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 --2- 172.21.15.64:0/1844304116 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d1bc0 0x7efdc00b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 --2- 172.21.15.64:0/1844304116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d0fa0 0x7efdc00d1680 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 --2- 172.21.15.64:0/1844304116 >> v2:172.21.15.91:3300/0 conn(0x7efdc015d3b0 0x7efdc00ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 -- 172.21.15.64:0/1844304116 >> 172.21.15.64:0/1844304116 conn(0x7efdc00d30a0 msgr2=0x7efdc00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 -- 172.21.15.64:0/1844304116 shutdown_connections 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.663+0000 7efdbf59e640 1 -- 172.21.15.64:0/1844304116 wait complete. 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.665+0000 7efdbf59e640 1 Processor -- start 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.665+0000 7efdbf59e640 1 -- start start 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbf59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d0fa0 0x7efdc00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d1bc0 0x7efdc00ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbf59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc015d3b0 0x7efdc009e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbf59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00ca170 con 0x7efdc00d1bc0 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbf59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc009f740 con 0x7efdc015d3b0 2023-11-24T08:04:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbf59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc009f460 con 0x7efdc00d0fa0 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d1bc0 0x7efdc00ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d1bc0 0x7efdc00ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51392/0 (socket says 172.21.15.64:51392) 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.666+0000 7efdbdd9b640 1 -- 172.21.15.64:0/2080424770 learned_addr learned my addr 172.21.15.64:0/2080424770 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbed9d640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc015d3b0 0x7efdc009e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbdd9b640 1 -- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d0fa0 msgr2=0x7efdc00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbe59c640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d0fa0 0x7efdc00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbdd9b640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d0fa0 0x7efdc00cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbdd9b640 1 -- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc015d3b0 msgr2=0x7efdc009e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbdd9b640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc015d3b0 0x7efdc009e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbdd9b640 1 -- 172.21.15.64:0/2080424770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80094d0 con 0x7efdc00d1bc0 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbed9d640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc015d3b0 0x7efdc009e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:33.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.667+0000 7efdbdd9b640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d1bc0 0x7efdc00ccaa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efda8000c00 tx=0x7efda80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.668+0000 7efdb6ffd640 1 -- 172.21.15.64:0/2080424770 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8032880 con 0x7efdc00d1bc0 2023-11-24T08:04:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.668+0000 7efdb6ffd640 1 -- 172.21.15.64:0/2080424770 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8032a20 con 0x7efdc00d1bc0 2023-11-24T08:04:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.668+0000 7efdb6ffd640 1 -- 172.21.15.64:0/2080424770 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8032dd0 con 0x7efdc00d1bc0 2023-11-24T08:04:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.668+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc00ca350 con 0x7efdc00d1bc0 2023-11-24T08:04:33.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.668+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc00ca910 con 0x7efdc00d1bc0 2023-11-24T08:04:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.669+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc004e840 con 0x7efdc00d1bc0 2023-11-24T08:04:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.670+0000 7efdb6ffd640 1 -- 172.21.15.64:0/2080424770 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efda804e450 con 0x7efdc00d1bc0 2023-11-24T08:04:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.671+0000 7efdb6ffd640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda006ae40 0x7efda006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.671+0000 7efdb6ffd640 1 -- 172.21.15.64:0/2080424770 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efda80ab7a0 con 0x7efdc00d1bc0 2023-11-24T08:04:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.672+0000 7efdbe59c640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda006ae40 0x7efda006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.673+0000 7efdbe59c640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda006ae40 0x7efda006d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009e90 tx=0x7efdb000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:33.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.675+0000 7efdb6ffd640 1 -- 172.21.15.64:0/2080424770 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efda8076900 con 0x7efdc00d1bc0 2023-11-24T08:04:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.877+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efdc00b5910 con 0x7efdc00d1bc0 2023-11-24T08:04:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.878+0000 7efdb6ffd640 1 -- 172.21.15.64:0/2080424770 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7efda8079f20 con 0x7efdc00d1bc0 2023-11-24T08:04:33.880 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda006ae40 msgr2=0x7efda006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda006ae40 0x7efda006d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009e90 tx=0x7efdb000b040 comp rx=0 tx=0).stop 2023-11-24T08:04:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d1bc0 msgr2=0x7efdc00ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d1bc0 0x7efdc00ccaa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efda8000c00 tx=0x7efda80046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 shutdown_connections 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda006ae40 0x7efda006d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc015d3b0 0x7efdc009e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc00d1bc0 0x7efdc00ccaa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 --2- 172.21.15.64:0/2080424770 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc00d0fa0 0x7efdc00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.880+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 >> 172.21.15.64:0/2080424770 conn(0x7efdc00d30a0 msgr2=0x7efdc006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.881+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 shutdown_connections 2023-11-24T08:04:33.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:33.881+0000 7efdbf59e640 1 -- 172.21.15.64:0/2080424770 wait complete. 2023-11-24T08:04:33.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:34.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.131+0000 7f3cb5541640 1 Processor -- start 2023-11-24T08:04:34.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.131+0000 7f3cb5541640 1 -- start start 2023-11-24T08:04:34.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.132+0000 7f3cb5541640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 0x7f3cb00abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.133+0000 7f3cb5541640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb015b730 con 0x7f3cb00adb60 2023-11-24T08:04:34.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.133+0000 7f3cb5541640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb00ac740 con 0x7f3cb009cb30 2023-11-24T08:04:34.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.133+0000 7f3cb5541640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00ac460 con 0x7f3cb015b350 2023-11-24T08:04:34.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.133+0000 7f3caf7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3cb00adb60 0x7f3cb00adfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37784/0 (socket says 172.21.15.64:37784) 2023-11-24T08:04:34.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.133+0000 7f3caf7fe640 1 -- 172.21.15.64:0/2041022629 learned_addr learned my addr 172.21.15.64:0/2041022629 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:34.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.133+0000 7f3caeffd640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 0x7f3cb00abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124894259 0 0) 0x7f3ca8001490 con 0x7f3cb00adb60 2023-11-24T08:04:34.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c9c003520 con 0x7f3cb00adb60 2023-11-24T08:04:34.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3029873042 0 0) 0x7f3c9c003520 con 0x7f3cb00adb60 2023-11-24T08:04:34.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 msgr2=0x7f3cb00abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:34.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 0x7f3cb00abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 >> v1:172.21.15.91:6789/0 conn(0x7f3cb009cb30 legacy=0x7f3cb00b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb00b2a50 con 0x7f3cb00adb60 2023-11-24T08:04:34.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f3ca8001b80 con 0x7f3cb00adb60 2023-11-24T08:04:34.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3caeffd640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 0x7f3cb00abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:04:34.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 >> v1:172.21.15.64:6789/0 conn(0x7f3cb00adb60 legacy=0x7f3cb00adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c9c0039a0 0x7f3c9c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 0x7f3c9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c9c004d00 0x7f3c9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00ac460 con 0x7f3c9c004d00 2023-11-24T08:04:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c00b980 con 0x7f3c9c0039a0 2023-11-24T08:04:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.134+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c00baf0 con 0x7f3cb015b350 2023-11-24T08:04:34.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.135+0000 7f3caf7fe640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c9c004d00 0x7f3c9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.135+0000 7f3caeffd640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c9c0039a0 0x7f3c9c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.135+0000 7f3ca7fff640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 0x7f3c9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.135+0000 7f3caf7fe640 1 -- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 msgr2=0x7f3c9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:34.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.135+0000 7f3caf7fe640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb015b350 0x7f3c9c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.135+0000 7f3caf7fe640 1 -- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c9c0039a0 msgr2=0x7f3c9c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:34.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.135+0000 7f3caf7fe640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c9c0039a0 0x7f3c9c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.135+0000 7f3caf7fe640 1 -- 172.21.15.64:0/2041022629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca0009400 con 0x7f3c9c004d00 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.136+0000 7f3caf7fe640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c9c004d00 0x7f3c9c005180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800a840 tx=0x7f3c9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2a57c21928527b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.136+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca802cea0 con 0x7f3c9c004d00 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca802f070 con 0x7f3c9c004d00 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3cae7fc640 1 -- 172.21.15.64:0/2041022629 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8034890 con 0x7f3c9c004d00 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3cb5541640 1 -- 172.21.15.64:0/2041022629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c9c004d00 msgr2=0x7f3c9c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3cb5541640 1 --2- 172.21.15.64:0/2041022629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c9c004d00 0x7f3c9c005180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800a840 tx=0x7f3c9c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3caf7fe640 1 -- 172.21.15.64:0/2041022629 reap_dead start 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3cb5541640 1 -- 172.21.15.64:0/2041022629 shutdown_connections 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3cb5541640 1 -- 172.21.15.64:0/2041022629 >> 172.21.15.64:0/2041022629 conn(0x7f3cb00b0ba0 msgr2=0x7f3cb00b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:34.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3cb5541640 1 -- 172.21.15.64:0/2041022629 shutdown_connections 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.137+0000 7f3cb5541640 1 -- 172.21.15.64:0/2041022629 wait complete. 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.139+0000 7f3cb5541640 1 Processor -- start 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.139+0000 7f3cb5541640 1 -- start start 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.140+0000 7f3cb5541640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb0145b90 0x7f3cb014ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.140+0000 7f3cb5541640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cb014bf70 0x7f3cb0154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.140+0000 7f3cb5541640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0145f70 0x7f3cb0154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.140+0000 7f3cb5541640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0155f50 con 0x7f3cb0145f70 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.140+0000 7f3cb5541640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01556b0 con 0x7f3cb014bf70 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.140+0000 7f3cb5541640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0155990 con 0x7f3cb0145b90 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0145f70 0x7f3cb0154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0145f70 0x7f3cb0154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51416/0 (socket says 172.21.15.64:51416) 2023-11-24T08:04:34.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 -- 172.21.15.64:0/4136231331 learned_addr learned my addr 172.21.15.64:0/4136231331 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caeffd640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb0145b90 0x7f3cb014ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3ca7fff640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cb014bf70 0x7f3cb0154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 -- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb0145b90 msgr2=0x7f3cb014ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb0145b90 0x7f3cb014ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 -- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cb014bf70 msgr2=0x7f3cb0154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cb014bf70 0x7f3cb0154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 -- 172.21.15.64:0/4136231331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb0155c70 con 0x7f3cb0145f70 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.141+0000 7f3caf7fe640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0145f70 0x7f3cb0154c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800a0b0 tx=0x7f3ca8001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.142+0000 7f3ca77fe640 1 -- 172.21.15.64:0/4136231331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca803f670 con 0x7f3cb0145f70 2023-11-24T08:04:34.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.142+0000 7f3ca77fe640 1 -- 172.21.15.64:0/4136231331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca803f7d0 con 0x7f3cb0145f70 2023-11-24T08:04:34.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.142+0000 7f3ca77fe640 1 -- 172.21.15.64:0/4136231331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca803fb40 con 0x7f3cb0145f70 2023-11-24T08:04:34.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.142+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb0150840 con 0x7f3cb0145f70 2023-11-24T08:04:34.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.142+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb0155370 con 0x7f3cb0145f70 2023-11-24T08:04:34.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.143+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb009d200 con 0x7f3cb0145f70 2023-11-24T08:04:34.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.144+0000 7f3ca77fe640 1 -- 172.21.15.64:0/4136231331 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3ca8031030 con 0x7f3cb0145f70 2023-11-24T08:04:34.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.145+0000 7f3ca77fe640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c9006ae40 0x7f3c9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.145+0000 7f3ca77fe640 1 -- 172.21.15.64:0/4136231331 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ca80ab580 con 0x7f3cb0145f70 2023-11-24T08:04:34.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.146+0000 7f3caeffd640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c9006ae40 0x7f3c9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.147+0000 7f3caeffd640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c9006ae40 0x7f3c9006d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009ba0 tx=0x7f3ca0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:34.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.149+0000 7f3ca77fe640 1 -- 172.21.15.64:0/4136231331 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ca80766b0 con 0x7f3cb0145f70 2023-11-24T08:04:34.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.341+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3cb009d200 con 0x7f3cb0145f70 2023-11-24T08:04:34.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.341+0000 7f3ca77fe640 1 -- 172.21.15.64:0/4136231331 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ca8079cd0 con 0x7f3cb0145f70 2023-11-24T08:04:34.343 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:34.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.344+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c9006ae40 msgr2=0x7f3c9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.344+0000 7f3cb5541640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c9006ae40 0x7f3c9006d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009ba0 tx=0x7f3ca0008040 comp rx=0 tx=0).stop 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.344+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0145f70 msgr2=0x7f3cb0154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.344+0000 7f3cb5541640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0145f70 0x7f3cb0154c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800a0b0 tx=0x7f3ca8001f10 comp rx=0 tx=0).stop 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.345+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 shutdown_connections 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.345+0000 7f3cb5541640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c9006ae40 0x7f3c9006d300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.345+0000 7f3cb5541640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0145f70 0x7f3cb0154c50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.345+0000 7f3cb5541640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3cb014bf70 0x7f3cb0154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.345+0000 7f3cb5541640 1 --2- 172.21.15.64:0/4136231331 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3cb0145b90 0x7f3cb014ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.345+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 >> 172.21.15.64:0/4136231331 conn(0x7f3cb00b0ba0 msgr2=0x7f3cb00b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.345+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 shutdown_connections 2023-11-24T08:04:34.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.345+0000 7f3cb5541640 1 -- 172.21.15.64:0/4136231331 wait complete. 2023-11-24T08:04:34.365 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:34.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.589+0000 7f19011db640 1 Processor -- start 2023-11-24T08:04:34.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.589+0000 7f19011db640 1 -- start start 2023-11-24T08:04:34.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.590+0000 7f19011db640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 0x7f18fc0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.590+0000 7f19011db640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc140900 con 0x7f18fc13dfd0 2023-11-24T08:04:34.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.590+0000 7f19011db640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc15fa70 con 0x7f18fc0cf050 2023-11-24T08:04:34.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.590+0000 7f19011db640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc15c580 con 0x7f18fc0cf500 2023-11-24T08:04:34.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.590+0000 7f18fbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f18fc13dfd0 0x7f18fc15be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37794/0 (socket says 172.21.15.64:37794) 2023-11-24T08:04:34.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.590+0000 7f18fbfff640 1 -- 172.21.15.64:0/1117450556 learned_addr learned my addr 172.21.15.64:0/1117450556 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:34.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f19009da640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 0x7f18fc0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921267613 0 0) 0x7f18fc140900 con 0x7f18fc13dfd0 2023-11-24T08:04:34.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e8003520 con 0x7f18fc13dfd0 2023-11-24T08:04:34.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2020392712 0 0) 0x7f18e8003520 con 0x7f18fc13dfd0 2023-11-24T08:04:34.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 msgr2=0x7f18fc0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:34.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f18faffd640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 0x7f18fc0cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 >> v1:172.21.15.91:6789/0 conn(0x7f18fc0cf050 legacy=0x7f18fc0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0cf950 con 0x7f18fc13dfd0 2023-11-24T08:04:34.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.591+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f18ec001a10 con 0x7f18fc13dfd0 2023-11-24T08:04:34.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 >> v1:172.21.15.64:6789/0 conn(0x7f18fc13dfd0 legacy=0x7f18fc15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18faffd640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 0x7f18e8003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18faffd640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18e8003fe0 0x7f18e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18faffd640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e8004d00 0x7f18e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc15c580 con 0x7f18e8003fe0 2023-11-24T08:04:34.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e800bb40 con 0x7f18e8004d00 2023-11-24T08:04:34.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e800bd00 con 0x7f18fc0cf500 2023-11-24T08:04:34.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18fb7fe640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18e8003fe0 0x7f18e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f18fbfff640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 0x7f18e8003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.592+0000 7f19009da640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e8004d00 0x7f18e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.593+0000 7f18fb7fe640 1 -- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 msgr2=0x7f18e8003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:34.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.593+0000 7f18fb7fe640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 0x7f18e8003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.593+0000 7f18fb7fe640 1 -- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e8004d00 msgr2=0x7f18e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:34.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.593+0000 7f18fb7fe640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18e8004d00 0x7f18e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.593+0000 7f18fb7fe640 1 -- 172.21.15.64:0/1117450556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8003520 con 0x7f18e8003fe0 2023-11-24T08:04:34.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.593+0000 7f18fbfff640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18fc0cf500 0x7f18e8003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:34.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.594+0000 7f18fb7fe640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18e8003fe0 0x7f18e80047c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18e4001750 tx=0x7f18e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab766ec1aaebc580 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:34.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.594+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e40052d0 con 0x7f18e8003fe0 2023-11-24T08:04:34.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.594+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e402e050 con 0x7f18e8003fe0 2023-11-24T08:04:34.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.594+0000 7f18faffd640 1 -- 172.21.15.64:0/1117450556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4003830 con 0x7f18e8003fe0 2023-11-24T08:04:34.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.595+0000 7f19011db640 1 -- 172.21.15.64:0/1117450556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18e8003fe0 msgr2=0x7f18e80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.595+0000 7f19011db640 1 --2- 172.21.15.64:0/1117450556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18e8003fe0 0x7f18e80047c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18e4001750 tx=0x7f18e80057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.595+0000 7f19009da640 1 -- 172.21.15.64:0/1117450556 reap_dead start 2023-11-24T08:04:34.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.595+0000 7f19011db640 1 -- 172.21.15.64:0/1117450556 shutdown_connections 2023-11-24T08:04:34.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.595+0000 7f19011db640 1 -- 172.21.15.64:0/1117450556 >> 172.21.15.64:0/1117450556 conn(0x7f18fc06a860 msgr2=0x7f18fc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:34.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.595+0000 7f19011db640 1 -- 172.21.15.64:0/1117450556 shutdown_connections 2023-11-24T08:04:34.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.595+0000 7f19011db640 1 -- 172.21.15.64:0/1117450556 wait complete. 2023-11-24T08:04:34.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.597+0000 7f19011db640 1 Processor -- start 2023-11-24T08:04:34.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.597+0000 7f19011db640 1 -- start start 2023-11-24T08:04:34.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f19011db640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18e8003fe0 0x7f18fc15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f19011db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc1f6e30 0x7f18fc154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f19011db640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18fc071d90 0x7f18fc155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f19011db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc155c00 con 0x7f18fc1f6e30 2023-11-24T08:04:34.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f19011db640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1f7330 con 0x7f18fc071d90 2023-11-24T08:04:34.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f19011db640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1f7610 con 0x7f18e8003fe0 2023-11-24T08:04:34.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f18fb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc1f6e30 0x7f18fc154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f18fb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc1f6e30 0x7f18fc154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51436/0 (socket says 172.21.15.64:51436) 2023-11-24T08:04:34.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f18fb7fe640 1 -- 172.21.15.64:0/1573400139 learned_addr learned my addr 172.21.15.64:0/1573400139 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f19009da640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18fc071d90 0x7f18fc155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.598+0000 7f18fbfff640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18e8003fe0 0x7f18fc15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18fb7fe640 1 -- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18e8003fe0 msgr2=0x7f18fc15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18fb7fe640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18e8003fe0 0x7f18fc15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18fb7fe640 1 -- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18fc071d90 msgr2=0x7f18fc155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18fb7fe640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18fc071d90 0x7f18fc155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18fb7fe640 1 -- 172.21.15.64:0/1573400139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc1f78f0 con 0x7f18fc1f6e30 2023-11-24T08:04:34.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18fb7fe640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc1f6e30 0x7f18fc154d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18e4008fd0 tx=0x7f18e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:34.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18f8ff9640 1 -- 172.21.15.64:0/1573400139 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e40076c0 con 0x7f18fc1f6e30 2023-11-24T08:04:34.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18f8ff9640 1 -- 172.21.15.64:0/1573400139 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4007a30 con 0x7f18fc1f6e30 2023-11-24T08:04:34.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc1f7bd0 con 0x7f18fc1f6e30 2023-11-24T08:04:34.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.599+0000 7f18f8ff9640 1 -- 172.21.15.64:0/1573400139 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4006be0 con 0x7f18fc1f6e30 2023-11-24T08:04:34.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.600+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc1588d0 con 0x7f18fc1f6e30 2023-11-24T08:04:34.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.601+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc0aecc0 con 0x7f18fc1f6e30 2023-11-24T08:04:34.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.602+0000 7f18f8ff9640 1 -- 172.21.15.64:0/1573400139 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18e4006d40 con 0x7f18fc1f6e30 2023-11-24T08:04:34.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.603+0000 7f18f8ff9640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18dc06ae40 0x7f18dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:34.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.603+0000 7f18f8ff9640 1 -- 172.21.15.64:0/1573400139 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18e40ab6e0 con 0x7f18fc1f6e30 2023-11-24T08:04:34.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.603+0000 7f18fbfff640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18dc06ae40 0x7f18dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:34.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.604+0000 7f18fbfff640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18dc06ae40 0x7f18dc06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f18ec001350 tx=0x7f18ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:34.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.607+0000 7f18f8ff9640 1 -- 172.21.15.64:0/1573400139 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18e4076780 con 0x7f18fc1f6e30 2023-11-24T08:04:34.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.809+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18fc04e840 con 0x7f18fc1f6e30 2023-11-24T08:04:34.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.812+0000 7f18f8ff9640 1 -- 172.21.15.64:0/1573400139 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f18e4079da0 con 0x7f18fc1f6e30 2023-11-24T08:04:34.814 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:34.814 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18dc06ae40 msgr2=0x7f18dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18dc06ae40 0x7f18dc06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f18ec001350 tx=0x7f18ec009040 comp rx=0 tx=0).stop 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc1f6e30 msgr2=0x7f18fc154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc1f6e30 0x7f18fc154d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18e4008fd0 tx=0x7f18e4001120 comp rx=0 tx=0).stop 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 shutdown_connections 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18dc06ae40 0x7f18dc06d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18fc071d90 0x7f18fc155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc1f6e30 0x7f18fc154d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 --2- 172.21.15.64:0/1573400139 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18e8003fe0 0x7f18fc15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:34.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 >> 172.21.15.64:0/1573400139 conn(0x7f18fc06a860 msgr2=0x7f18fc0aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:34.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 shutdown_connections 2023-11-24T08:04:34.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:34.815+0000 7f19011db640 1 -- 172.21.15.64:0/1573400139 wait complete. 2023-11-24T08:04:34.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:34.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:35.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.069+0000 7f74098bb640 1 Processor -- start 2023-11-24T08:04:35.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.070+0000 7f74098bb640 1 -- start start 2023-11-24T08:04:35.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.071+0000 7f74098bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 0x7f74040ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.071+0000 7f74098bb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74040c6bf0 con 0x7f74040d0fa0 2023-11-24T08:04:35.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.071+0000 7f74098bb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74040d13d0 con 0x7f740415d3b0 2023-11-24T08:04:35.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.071+0000 7f74098bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74040cedc0 con 0x7f74040d1d90 2023-11-24T08:04:35.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.072+0000 7f74090ba640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f74040d0fa0 0x7f74040b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37810/0 (socket says 172.21.15.64:37810) 2023-11-24T08:04:35.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.072+0000 7f74090ba640 1 -- 172.21.15.64:0/1149795843 learned_addr learned my addr 172.21.15.64:0/1149795843 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:35.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.072+0000 7f7403fff640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 0x7f74040ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.072+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362370773 0 0) 0x7f74040c6bf0 con 0x7f74040d0fa0 2023-11-24T08:04:35.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.072+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f0003520 con 0x7f74040d0fa0 2023-11-24T08:04:35.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2628623122 0 0) 0x7f73f0003520 con 0x7f74040d0fa0 2023-11-24T08:04:35.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 msgr2=0x7f74040ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:35.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 0x7f74040ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 >> v1:172.21.15.91:6789/0 conn(0x7f740415d3b0 legacy=0x7f74040d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040c6e40 con 0x7f74040d0fa0 2023-11-24T08:04:35.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f73fc001670 con 0x7f74040d0fa0 2023-11-24T08:04:35.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 >> v1:172.21.15.64:6789/0 conn(0x7f74040d0fa0 legacy=0x7f74040b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 0x7f73f0003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f73f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73f0004d00 0x7f73f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74040cedc0 con 0x7f73f0003fe0 2023-11-24T08:04:35.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f000bbc0 con 0x7f73f0004d00 2023-11-24T08:04:35.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f000bd80 con 0x7f74040d1d90 2023-11-24T08:04:35.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f7403fff640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f73f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.073+0000 7f74090ba640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73f0004d00 0x7f73f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.074+0000 7f74088b9640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 0x7f73f0003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.074+0000 7f7403fff640 1 -- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 msgr2=0x7f73f0003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:35.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.074+0000 7f7403fff640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 0x7f73f0003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.074+0000 7f7403fff640 1 -- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73f0004d00 msgr2=0x7f73f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:35.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.074+0000 7f7403fff640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73f0004d00 0x7f73f0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.074+0000 7f7403fff640 1 -- 172.21.15.64:0/1149795843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0003520 con 0x7f73f0003fe0 2023-11-24T08:04:35.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.074+0000 7f74090ba640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f73f0004d00 0x7f73f0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:35.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.074+0000 7f74088b9640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74040d1d90 0x7f73f0003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.075+0000 7f7403fff640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f73f00047c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009ac0 tx=0x7f73f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93dbc18831ca9f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.075+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec02c6b0 con 0x7f73f0003fe0 2023-11-24T08:04:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.075+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec02c810 con 0x7f73f0003fe0 2023-11-24T08:04:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.075+0000 7f74037fe640 1 -- 172.21.15.64:0/1149795843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec02cba0 con 0x7f73f0003fe0 2023-11-24T08:04:35.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.075+0000 7f74098bb640 1 -- 172.21.15.64:0/1149795843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 msgr2=0x7f73f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.076+0000 7f74098bb640 1 --2- 172.21.15.64:0/1149795843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f73f00047c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009ac0 tx=0x7f73f00057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.076+0000 7f74090ba640 1 -- 172.21.15.64:0/1149795843 reap_dead start 2023-11-24T08:04:35.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.076+0000 7f74098bb640 1 -- 172.21.15.64:0/1149795843 shutdown_connections 2023-11-24T08:04:35.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.076+0000 7f74098bb640 1 -- 172.21.15.64:0/1149795843 >> 172.21.15.64:0/1149795843 conn(0x7f74040d30a0 msgr2=0x7f74040d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:35.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.076+0000 7f74098bb640 1 -- 172.21.15.64:0/1149795843 shutdown_connections 2023-11-24T08:04:35.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.076+0000 7f74098bb640 1 -- 172.21.15.64:0/1149795843 wait complete. 2023-11-24T08:04:35.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.078+0000 7f74098bb640 1 Processor -- start 2023-11-24T08:04:35.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.078+0000 7f74098bb640 1 -- start start 2023-11-24T08:04:35.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74098bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f740409ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74098bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f740414f500 0x7f740409f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74098bb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f740409d660 0x7f740409dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74098bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740409d3b0 con 0x7f73f0003fe0 2023-11-24T08:04:35.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74098bb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740409e020 con 0x7f740409d660 2023-11-24T08:04:35.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74098bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740409fdf0 con 0x7f740414f500 2023-11-24T08:04:35.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74088b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f740409ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74088b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f740409ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51446/0 (socket says 172.21.15.64:51446) 2023-11-24T08:04:35.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74088b9640 1 -- 172.21.15.64:0/256168151 learned_addr learned my addr 172.21.15.64:0/256168151 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:35.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f74090ba640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f740409d660 0x7f740409dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.079+0000 7f7403fff640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f740414f500 0x7f740409f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74088b9640 1 -- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f740414f500 msgr2=0x7f740409f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74088b9640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f740414f500 0x7f740409f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74088b9640 1 -- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f740409d660 msgr2=0x7f740409dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74088b9640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f740409d660 0x7f740409dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74088b9640 1 -- 172.21.15.64:0/256168151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040a00d0 con 0x7f73f0003fe0 2023-11-24T08:04:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74090ba640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f740409d660 0x7f740409dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74088b9640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f740409ce70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73f400db10 tx=0x7f73f40020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74017fa640 1 -- 172.21.15.64:0/256168151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4003da0 con 0x7f73f0003fe0 2023-11-24T08:04:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74017fa640 1 -- 172.21.15.64:0/256168151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f4006b50 con 0x7f73f0003fe0 2023-11-24T08:04:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74040a03b0 con 0x7f73f0003fe0 2023-11-24T08:04:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74017fa640 1 -- 172.21.15.64:0/256168151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400b740 con 0x7f73f0003fe0 2023-11-24T08:04:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.080+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74040a0970 con 0x7f73f0003fe0 2023-11-24T08:04:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.082+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74040a0b30 con 0x7f73f0003fe0 2023-11-24T08:04:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.083+0000 7f74017fa640 1 -- 172.21.15.64:0/256168151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73f400b8a0 con 0x7f73f0003fe0 2023-11-24T08:04:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.084+0000 7f74017fa640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73e406ae40 0x7f73e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.084+0000 7f74017fa640 1 -- 172.21.15.64:0/256168151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73f4088fd0 con 0x7f73f0003fe0 2023-11-24T08:04:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.084+0000 7f7403fff640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73e406ae40 0x7f73e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.086+0000 7f7403fff640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73e406ae40 0x7f73e406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f73ec036000 tx=0x7f73ec030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:35.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.088+0000 7f74017fa640 1 -- 172.21.15.64:0/256168151 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f73f4054110 con 0x7f73f0003fe0 2023-11-24T08:04:35.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.285+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f740404e840 con 0x7f73f0003fe0 2023-11-24T08:04:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.286+0000 7f74017fa640 1 -- 172.21.15.64:0/256168151 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73f4057730 con 0x7f73f0003fe0 2023-11-24T08:04:35.288 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:35.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73e406ae40 msgr2=0x7f73e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73e406ae40 0x7f73e406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f73ec036000 tx=0x7f73ec030000 comp rx=0 tx=0).stop 2023-11-24T08:04:35.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 msgr2=0x7f740409ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f740409ce70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73f400db10 tx=0x7f73f40020c0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 shutdown_connections 2023-11-24T08:04:35.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f73e406ae40 0x7f73e406d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f740409d660 0x7f740409dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f740414f500 0x7f740409f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 --2- 172.21.15.64:0/256168151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0003fe0 0x7f740409ce70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.289+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 >> 172.21.15.64:0/256168151 conn(0x7f74040d30a0 msgr2=0x7f740406af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:35.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.290+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 shutdown_connections 2023-11-24T08:04:35.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.290+0000 7f74098bb640 1 -- 172.21.15.64:0/256168151 wait complete. 2023-11-24T08:04:35.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:35.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.527+0000 7fe0ad101640 1 Processor -- start 2023-11-24T08:04:35.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.528+0000 7fe0ad101640 1 -- start start 2023-11-24T08:04:35.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.529+0000 7fe0ad101640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0a815b350 0x7fe0a80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.529+0000 7fe0ad101640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe0a809cb30 0x7fe0a80b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.529+0000 7fe0ad101640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a80adb60 0x7fe0a80adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.529+0000 7fe0ad101640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a815b730 con 0x7fe0a815b350 2023-11-24T08:04:35.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.529+0000 7fe0ad101640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a80ac740 con 0x7fe0a809cb30 2023-11-24T08:04:35.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.529+0000 7fe0ad101640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80ac460 con 0x7fe0a80adb60 2023-11-24T08:04:35.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0a815b350 0x7fe0a80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0a815b350 0x7fe0a80abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51462/0 (socket says 172.21.15.64:51462) 2023-11-24T08:04:35.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a6d9d640 1 -- 172.21.15.64:0/3443292310 learned_addr learned my addr 172.21.15.64:0/3443292310 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:35.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe09ffff640 1 --2- 172.21.15.64:0/3443292310 >> v2:172.21.15.91:3300/0 conn(0x7fe0a809cb30 0x7fe0a80b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a759e640 1 --2- 172.21.15.64:0/3443292310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a80adb60 0x7fe0a80adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a6d9d640 1 -- 172.21.15.64:0/3443292310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a80adb60 msgr2=0x7fe0a80adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:35.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a6d9d640 1 --2- 172.21.15.64:0/3443292310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a80adb60 0x7fe0a80adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a6d9d640 1 -- 172.21.15.64:0/3443292310 >> v2:172.21.15.91:3300/0 conn(0x7fe0a809cb30 msgr2=0x7fe0a80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:35.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a6d9d640 1 --2- 172.21.15.64:0/3443292310 >> v2:172.21.15.91:3300/0 conn(0x7fe0a809cb30 0x7fe0a80b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a6d9d640 1 -- 172.21.15.64:0/3443292310 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a80b2a50 con 0x7fe0a815b350 2023-11-24T08:04:35.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe09ffff640 1 --2- 172.21.15.64:0/3443292310 >> v2:172.21.15.91:3300/0 conn(0x7fe0a809cb30 0x7fe0a80b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:35.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.530+0000 7fe0a759e640 1 --2- 172.21.15.64:0/3443292310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a80adb60 0x7fe0a80adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.531+0000 7fe0a6d9d640 1 --2- 172.21.15.64:0/3443292310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a815b350 0x7fe0a80abe80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80d0060 tx=0x7fe09802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89968d0dcfe09f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0a659c640 1 -- 172.21.15.64:0/3443292310 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098031880 con 0x7fe0a815b350 2023-11-24T08:04:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0a659c640 1 -- 172.21.15.64:0/3443292310 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0980319e0 con 0x7fe0a815b350 2023-11-24T08:04:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0a659c640 1 -- 172.21.15.64:0/3443292310 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098031d70 con 0x7fe0a815b350 2023-11-24T08:04:35.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0ad101640 1 -- 172.21.15.64:0/3443292310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a815b350 msgr2=0x7fe0a80abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0ad101640 1 --2- 172.21.15.64:0/3443292310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a815b350 0x7fe0a80abe80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80d0060 tx=0x7fe09802b920 comp rx=0 tx=0).stop 2023-11-24T08:04:35.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0ad101640 1 -- 172.21.15.64:0/3443292310 shutdown_connections 2023-11-24T08:04:35.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0ad101640 1 --2- 172.21.15.64:0/3443292310 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a80adb60 0x7fe0a80adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0ad101640 1 --2- 172.21.15.64:0/3443292310 >> v2:172.21.15.91:3300/0 conn(0x7fe0a809cb30 0x7fe0a80b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0ad101640 1 --2- 172.21.15.64:0/3443292310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a815b350 0x7fe0a80abe80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.532+0000 7fe0ad101640 1 -- 172.21.15.64:0/3443292310 >> 172.21.15.64:0/3443292310 conn(0x7fe0a80b0ba0 msgr2=0x7fe0a80b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:35.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.533+0000 7fe0ad101640 1 -- 172.21.15.64:0/3443292310 shutdown_connections 2023-11-24T08:04:35.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.533+0000 7fe0ad101640 1 -- 172.21.15.64:0/3443292310 wait complete. 2023-11-24T08:04:35.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.534+0000 7fe0ad101640 1 Processor -- start 2023-11-24T08:04:35.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.535+0000 7fe0ad101640 1 -- start start 2023-11-24T08:04:35.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.535+0000 7fe0ad101640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a809cb30 0x7fe0a813d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.535+0000 7fe0ad101640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0a80adb60 0x7fe0a81402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0ad101640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a815b350 0x7fe0a809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0ad101640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a813e250 con 0x7fe0a809cb30 2023-11-24T08:04:35.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0ad101640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a813e7d0 con 0x7fe0a80adb60 2023-11-24T08:04:35.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0ad101640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8140d80 con 0x7fe0a815b350 2023-11-24T08:04:35.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0a6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a809cb30 0x7fe0a813d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0a6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a809cb30 0x7fe0a813d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51474/0 (socket says 172.21.15.64:51474) 2023-11-24T08:04:35.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0a6d9d640 1 -- 172.21.15.64:0/42934591 learned_addr learned my addr 172.21.15.64:0/42934591 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:35.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0a6d9d640 1 -- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a815b350 msgr2=0x7fe0a809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0a6d9d640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a815b350 0x7fe0a809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe09ffff640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0a80adb60 0x7fe0a81402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0a6d9d640 1 -- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0a80adb60 msgr2=0x7fe0a81402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0a6d9d640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0a80adb60 0x7fe0a81402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe0a6d9d640 1 -- 172.21.15.64:0/42934591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980094d0 con 0x7fe0a809cb30 2023-11-24T08:04:35.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.536+0000 7fe09ffff640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0a80adb60 0x7fe0a81402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:35.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.537+0000 7fe0a6d9d640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a809cb30 0x7fe0a813d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe09802b960 tx=0x7fe0980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:35.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.537+0000 7fe09f7fe640 1 -- 172.21.15.64:0/42934591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098031870 con 0x7fe0a809cb30 2023-11-24T08:04:35.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.537+0000 7fe09f7fe640 1 -- 172.21.15.64:0/42934591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0980443f0 con 0x7fe0a809cb30 2023-11-24T08:04:35.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.537+0000 7fe09f7fe640 1 -- 172.21.15.64:0/42934591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980447a0 con 0x7fe0a809cb30 2023-11-24T08:04:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.537+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a8141060 con 0x7fe0a809cb30 2023-11-24T08:04:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.537+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a813df00 con 0x7fe0a809cb30 2023-11-24T08:04:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.538+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a809d200 con 0x7fe0a809cb30 2023-11-24T08:04:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.539+0000 7fe09f7fe640 1 -- 172.21.15.64:0/42934591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe09804d450 con 0x7fe0a809cb30 2023-11-24T08:04:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.540+0000 7fe09f7fe640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe08806ae40 0x7fe08806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.541+0000 7fe09f7fe640 1 -- 172.21.15.64:0/42934591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0980abd30 con 0x7fe0a809cb30 2023-11-24T08:04:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.541+0000 7fe09ffff640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe08806ae40 0x7fe08806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.542+0000 7fe09ffff640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe08806ae40 0x7fe08806d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe0a809bd50 tx=0x7fe090008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.545+0000 7fe09f7fe640 1 -- 172.21.15.64:0/42934591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe098076e70 con 0x7fe0a809cb30 2023-11-24T08:04:35.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.746+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe0a804e920 con 0x7fe0a809cb30 2023-11-24T08:04:35.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.747+0000 7fe09f7fe640 1 -- 172.21.15.64:0/42934591 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe098035020 con 0x7fe0a809cb30 2023-11-24T08:04:35.749 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:35.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe08806ae40 msgr2=0x7fe08806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe08806ae40 0x7fe08806d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe0a809bd50 tx=0x7fe090008040 comp rx=0 tx=0).stop 2023-11-24T08:04:35.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a809cb30 msgr2=0x7fe0a813d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:35.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a809cb30 0x7fe0a813d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe09802b960 tx=0x7fe0980046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 shutdown_connections 2023-11-24T08:04:35.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe08806ae40 0x7fe08806d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0a815b350 0x7fe0a809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0a80adb60 0x7fe0a81402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 --2- 172.21.15.64:0/42934591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a809cb30 0x7fe0a813d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:35.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 >> 172.21.15.64:0/42934591 conn(0x7fe0a80b0ba0 msgr2=0x7fe0a8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:35.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.750+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 shutdown_connections 2023-11-24T08:04:35.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.751+0000 7fe0ad101640 1 -- 172.21.15.64:0/42934591 wait complete. 2023-11-24T08:04:35.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:35.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.994+0000 7f163222e640 1 Processor -- start 2023-11-24T08:04:35.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.994+0000 7f163222e640 1 -- start start 2023-11-24T08:04:35.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.996+0000 7f163222e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f162c0d1d90 0x7f162c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.996+0000 7f163222e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f162c0d0fa0 0x7f162c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:35.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.996+0000 7f163222e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162c0c6bf0 con 0x7f162c15d3b0 2023-11-24T08:04:35.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.996+0000 7f163222e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c0d13f0 con 0x7f162c0d1d90 2023-11-24T08:04:35.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.996+0000 7f163222e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c0cedc0 con 0x7f162c0d0fa0 2023-11-24T08:04:35.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.996+0000 7f163122c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f162c15d3b0 0x7f162c0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37812/0 (socket says 172.21.15.64:37812) 2023-11-24T08:04:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.996+0000 7f163122c640 1 -- 172.21.15.64:0/2012461356 learned_addr learned my addr 172.21.15.64:0/2012461356 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.996+0000 7f1630a2b640 1 --2- 172.21.15.64:0/2012461356 >> v2:172.21.15.64:3300/0 conn(0x7f162c0d1d90 0x7f162c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.997+0000 7f1631a2d640 1 --2- 172.21.15.64:0/2012461356 >> v2:172.21.15.91:3300/0 conn(0x7f162c0d0fa0 0x7f162c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.997+0000 7f1623fff640 1 -- 172.21.15.64:0/2012461356 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194437238 0 0) 0x7f162c0c6bf0 con 0x7f162c15d3b0 2023-11-24T08:04:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.997+0000 7f1623fff640 1 -- 172.21.15.64:0/2012461356 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1618003520 con 0x7f162c15d3b0 2023-11-24T08:04:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.997+0000 7f1630a2b640 1 -- 172.21.15.64:0/2012461356 >> v2:172.21.15.91:3300/0 conn(0x7f162c0d0fa0 msgr2=0x7f162c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.997+0000 7f1630a2b640 1 --2- 172.21.15.64:0/2012461356 >> v2:172.21.15.91:3300/0 conn(0x7f162c0d0fa0 0x7f162c0b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.997+0000 7f1630a2b640 1 -- 172.21.15.64:0/2012461356 >> v1:172.21.15.64:6789/0 conn(0x7f162c15d3b0 legacy=0x7f162c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.997+0000 7f1630a2b640 1 -- 172.21.15.64:0/2012461356 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c0c6e40 con 0x7f162c0d1d90 2023-11-24T08:04:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.998+0000 7f1630a2b640 1 --2- 172.21.15.64:0/2012461356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d1d90 0x7f162c0ce6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f162c072b10 tx=0x7f161c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28041dd9acb1cff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f1623fff640 1 -- 172.21.15.64:0/2012461356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c02e070 con 0x7f162c0d1d90 2023-11-24T08:04:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f1623fff640 1 -- 172.21.15.64:0/2012461356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c0315a0 con 0x7f162c0d1d90 2023-11-24T08:04:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f1623fff640 1 -- 172.21.15.64:0/2012461356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c031910 con 0x7f162c0d1d90 2023-11-24T08:04:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f163222e640 1 -- 172.21.15.64:0/2012461356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d1d90 msgr2=0x7f162c0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f163222e640 1 --2- 172.21.15.64:0/2012461356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d1d90 0x7f162c0ce6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f162c072b10 tx=0x7f161c02b840 comp rx=0 tx=0).stop 2023-11-24T08:04:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f163222e640 1 -- 172.21.15.64:0/2012461356 shutdown_connections 2023-11-24T08:04:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f163222e640 1 --2- 172.21.15.64:0/2012461356 >> v2:172.21.15.91:3300/0 conn(0x7f162c0d0fa0 0x7f162c0b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f163222e640 1 --2- 172.21.15.64:0/2012461356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d1d90 0x7f162c0ce6c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f163222e640 1 -- 172.21.15.64:0/2012461356 >> 172.21.15.64:0/2012461356 conn(0x7f162c0d30a0 msgr2=0x7f162c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f163222e640 1 -- 172.21.15.64:0/2012461356 shutdown_connections 2023-11-24T08:04:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:35.999+0000 7f163222e640 1 -- 172.21.15.64:0/2012461356 wait complete. 2023-11-24T08:04:36.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.001+0000 7f163222e640 1 Processor -- start 2023-11-24T08:04:36.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.001+0000 7f163222e640 1 -- start start 2023-11-24T08:04:36.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.002+0000 7f163222e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d0fa0 0x7f162c0b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.002+0000 7f163222e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f162c0d1d90 0x7f162c15b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.002+0000 7f163222e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f162c15d3b0 0x7f162c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.002+0000 7f163222e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0b75b0 con 0x7f162c0d0fa0 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.002+0000 7f163222e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0a0c60 con 0x7f162c15d3b0 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.002+0000 7f163222e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0b8d00 con 0x7f162c0d1d90 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d0fa0 0x7f162c0b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d0fa0 0x7f162c0b87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51488/0 (socket says 172.21.15.64:51488) 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 -- 172.21.15.64:0/1657431179 learned_addr learned my addr 172.21.15.64:0/1657431179 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f1631a2d640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f162c15d3b0 0x7f162c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 -- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f162c0d1d90 msgr2=0x7f162c15b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f162c0d1d90 0x7f162c15b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 -- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f162c15d3b0 msgr2=0x7f162c0cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f162c15d3b0 0x7f162c0cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 -- 172.21.15.64:0/1657431179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c009400 con 0x7f162c0d0fa0 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f1631a2d640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f162c15d3b0 0x7f162c0cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.003+0000 7f163122c640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d0fa0 0x7f162c0b87c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f162800c390 tx=0x7f162800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.004+0000 7f1621ffb640 1 -- 172.21.15.64:0/1657431179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628005ce0 con 0x7f162c0d0fa0 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.004+0000 7f1621ffb640 1 -- 172.21.15.64:0/1657431179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1628005e40 con 0x7f162c0d0fa0 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.004+0000 7f1621ffb640 1 -- 172.21.15.64:0/1657431179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16280072f0 con 0x7f162c0d0fa0 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.004+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c0b6a30 con 0x7f162c0d0fa0 2023-11-24T08:04:36.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.004+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c0b6ff0 con 0x7f162c0d0fa0 2023-11-24T08:04:36.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.005+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c04e840 con 0x7f162c0d0fa0 2023-11-24T08:04:36.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.006+0000 7f1621ffb640 1 -- 172.21.15.64:0/1657431179 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1628007450 con 0x7f162c0d0fa0 2023-11-24T08:04:36.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.007+0000 7f1621ffb640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161006ae40 0x7f161006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.007+0000 7f1621ffb640 1 -- 172.21.15.64:0/1657431179 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1628088ba0 con 0x7f162c0d0fa0 2023-11-24T08:04:36.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.008+0000 7f1630a2b640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161006ae40 0x7f161006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.009+0000 7f1630a2b640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161006ae40 0x7f161006d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f161c036000 tx=0x7f161c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.012+0000 7f1621ffb640 1 -- 172.21.15.64:0/1657431179 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1628053ce0 con 0x7f162c0d0fa0 2023-11-24T08:04:36.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.205+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f162c0ca6d0 con 0x7f162c0d0fa0 2023-11-24T08:04:36.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.206+0000 7f1621ffb640 1 -- 172.21.15.64:0/1657431179 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1628017020 con 0x7f162c0d0fa0 2023-11-24T08:04:36.208 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.208+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161006ae40 msgr2=0x7f161006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.208+0000 7f163222e640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161006ae40 0x7f161006d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f161c036000 tx=0x7f161c030000 comp rx=0 tx=0).stop 2023-11-24T08:04:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.208+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d0fa0 msgr2=0x7f162c0b87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.208+0000 7f163222e640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d0fa0 0x7f162c0b87c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f162800c390 tx=0x7f162800c850 comp rx=0 tx=0).stop 2023-11-24T08:04:36.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.209+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 shutdown_connections 2023-11-24T08:04:36.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.209+0000 7f163222e640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161006ae40 0x7f161006d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.209+0000 7f163222e640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f162c15d3b0 0x7f162c0cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.209+0000 7f163222e640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f162c0d1d90 0x7f162c15b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.209+0000 7f163222e640 1 --2- 172.21.15.64:0/1657431179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f162c0d0fa0 0x7f162c0b87c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.209+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 >> 172.21.15.64:0/1657431179 conn(0x7f162c0d30a0 msgr2=0x7f162c0c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:36.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.209+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 shutdown_connections 2023-11-24T08:04:36.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.209+0000 7f163222e640 1 -- 172.21.15.64:0/1657431179 wait complete. 2023-11-24T08:04:36.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:36.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.453+0000 7f69daad1640 1 Processor -- start 2023-11-24T08:04:36.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.453+0000 7f69daad1640 1 -- start start 2023-11-24T08:04:36.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.455+0000 7f69daad1640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f69d40cf050 0x7f69d40cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.455+0000 7f69daad1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d4140900 con 0x7f69d413dfd0 2023-11-24T08:04:36.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.455+0000 7f69daad1640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d415fa70 con 0x7f69d40cd7b0 2023-11-24T08:04:36.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.455+0000 7f69daad1640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d40cfa40 con 0x7f69d40cf050 2023-11-24T08:04:36.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.455+0000 7f69d9acf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f69d413dfd0 0x7f69d415be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37818/0 (socket says 172.21.15.64:37818) 2023-11-24T08:04:36.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.456+0000 7f69d9acf640 1 -- 172.21.15.64:0/3163160679 learned_addr learned my addr 172.21.15.64:0/3163160679 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.456+0000 7f69d92ce640 1 --2- 172.21.15.64:0/3163160679 >> v2:172.21.15.91:3300/0 conn(0x7f69d40cf050 0x7f69d40cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.456+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925093305 0 0) 0x7f69d4140900 con 0x7f69d413dfd0 2023-11-24T08:04:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.456+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c0003520 con 0x7f69d413dfd0 2023-11-24T08:04:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1307856784 0 0) 0x7f69c0003520 con 0x7f69d413dfd0 2023-11-24T08:04:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 >> v2:172.21.15.91:3300/0 conn(0x7f69d40cf050 msgr2=0x7f69d40cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 --2- 172.21.15.64:0/3163160679 >> v2:172.21.15.91:3300/0 conn(0x7f69d40cf050 0x7f69d40cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 >> v1:172.21.15.91:6789/0 conn(0x7f69d40cd7b0 legacy=0x7f69d40cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d415ee60 con 0x7f69d413dfd0 2023-11-24T08:04:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f69d0001670 con 0x7f69d413dfd0 2023-11-24T08:04:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 >> v1:172.21.15.64:6789/0 conn(0x7f69d413dfd0 legacy=0x7f69d415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69c00039e0 0x7f69c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c00047c0 0x7f69c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69c0005180 0x7f69c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40cfa40 con 0x7f69c0005180 2023-11-24T08:04:36.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c000c0b0 con 0x7f69c00047c0 2023-11-24T08:04:36.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c000c270 con 0x7f69c00039e0 2023-11-24T08:04:36.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69da2d0640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69c0005180 0x7f69c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d92ce640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c00047c0 0x7f69c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.457+0000 7f69d9acf640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69c00039e0 0x7f69c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.458+0000 7f69da2d0640 1 -- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69c00039e0 msgr2=0x7f69c0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:36.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.458+0000 7f69da2d0640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69c00039e0 0x7f69c0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.458+0000 7f69da2d0640 1 -- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c00047c0 msgr2=0x7f69c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:36.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.458+0000 7f69da2d0640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c00047c0 0x7f69c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.458+0000 7f69da2d0640 1 -- 172.21.15.64:0/3163160679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c0003520 con 0x7f69c0005180 2023-11-24T08:04:36.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.458+0000 7f69d92ce640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c00047c0 0x7f69c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:36.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.459+0000 7f69da2d0640 1 -- 172.21.15.64:0/3163160679 reap_dead start 2023-11-24T08:04:36.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.459+0000 7f69da2d0640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69c0005180 0x7f69c000b980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f69cc001e50 tx=0x7f69c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8417cf485df615d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc003b70 con 0x7f69c0005180 2023-11-24T08:04:36.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc003d10 con 0x7f69c0005180 2023-11-24T08:04:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69d8acd640 1 -- 172.21.15.64:0/3163160679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc005310 con 0x7f69c0005180 2023-11-24T08:04:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69daad1640 1 -- 172.21.15.64:0/3163160679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69c0005180 msgr2=0x7f69c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69daad1640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69c0005180 0x7f69c000b980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f69cc001e50 tx=0x7f69c00057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69daad1640 1 -- 172.21.15.64:0/3163160679 shutdown_connections 2023-11-24T08:04:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69daad1640 1 --2- 172.21.15.64:0/3163160679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69c0005180 0x7f69c000b980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69daad1640 1 -- 172.21.15.64:0/3163160679 >> 172.21.15.64:0/3163160679 conn(0x7f69d406a860 msgr2=0x7f69d406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:36.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69daad1640 1 -- 172.21.15.64:0/3163160679 shutdown_connections 2023-11-24T08:04:36.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.460+0000 7f69daad1640 1 -- 172.21.15.64:0/3163160679 wait complete. 2023-11-24T08:04:36.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.462+0000 7f69daad1640 1 Processor -- start 2023-11-24T08:04:36.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.462+0000 7f69daad1640 1 -- start start 2023-11-24T08:04:36.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.463+0000 7f69daad1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c0005180 0x7f69d4159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.463+0000 7f69daad1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d413b890 0x7f69d40b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.463+0000 7f69daad1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d40b3540 0x7f69d41f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.463+0000 7f69daad1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40b2b00 con 0x7f69d413b890 2023-11-24T08:04:36.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.463+0000 7f69daad1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d415abd0 con 0x7f69c0005180 2023-11-24T08:04:36.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.463+0000 7f69daad1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d406c7b0 con 0x7f69d40b3540 2023-11-24T08:04:36.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d92ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d413b890 0x7f69d40b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d92ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d413b890 0x7f69d40b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51494/0 (socket says 172.21.15.64:51494) 2023-11-24T08:04:36.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d92ce640 1 -- 172.21.15.64:0/2073637054 learned_addr learned my addr 172.21.15.64:0/2073637054 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:36.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69da2d0640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d40b3540 0x7f69d41f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d9acf640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c0005180 0x7f69d4159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d92ce640 1 -- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d40b3540 msgr2=0x7f69d41f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d92ce640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d40b3540 0x7f69d41f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d92ce640 1 -- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c0005180 msgr2=0x7f69d4159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d92ce640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c0005180 0x7f69d4159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.464+0000 7f69d92ce640 1 -- 172.21.15.64:0/2073637054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d415ae20 con 0x7f69d413b890 2023-11-24T08:04:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.465+0000 7f69d92ce640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d413b890 0x7f69d40b3000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f69c4011540 tx=0x7f69c4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.465+0000 7f69ca7fc640 1 -- 172.21.15.64:0/2073637054 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4015870 con 0x7f69d413b890 2023-11-24T08:04:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.465+0000 7f69ca7fc640 1 -- 172.21.15.64:0/2073637054 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c40159d0 con 0x7f69d413b890 2023-11-24T08:04:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.465+0000 7f69ca7fc640 1 -- 172.21.15.64:0/2073637054 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4015d40 con 0x7f69d413b890 2023-11-24T08:04:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.465+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d415b100 con 0x7f69d413b890 2023-11-24T08:04:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.465+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d406cce0 con 0x7f69d413b890 2023-11-24T08:04:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.466+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d404e840 con 0x7f69d413b890 2023-11-24T08:04:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.467+0000 7f69ca7fc640 1 -- 172.21.15.64:0/2073637054 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69c400f460 con 0x7f69d413b890 2023-11-24T08:04:36.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.468+0000 7f69ca7fc640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806ae70 0x7f69b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.468+0000 7f69ca7fc640 1 -- 172.21.15.64:0/2073637054 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69c4089e10 con 0x7f69d413b890 2023-11-24T08:04:36.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.469+0000 7f69d9acf640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806ae70 0x7f69b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.470+0000 7f69d9acf640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806ae70 0x7f69b806d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69d413f540 tx=0x7f69d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.473+0000 7f69ca7fc640 1 -- 172.21.15.64:0/2073637054 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69c408d030 con 0x7f69d413b890 2023-11-24T08:04:36.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.669+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69d40ce9a0 con 0x7f69d413b890 2023-11-24T08:04:36.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.672+0000 7f69ca7fc640 1 -- 172.21.15.64:0/2073637054 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f69c4022090 con 0x7f69d413b890 2023-11-24T08:04:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:36.674 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:03:56.934213+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806ae70 msgr2=0x7f69b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806ae70 0x7f69b806d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69d413f540 tx=0x7f69d0009040 comp rx=0 tx=0).stop 2023-11-24T08:04:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d413b890 msgr2=0x7f69d40b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d413b890 0x7f69d40b3000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f69c4011540 tx=0x7f69c4011a00 comp rx=0 tx=0).stop 2023-11-24T08:04:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 shutdown_connections 2023-11-24T08:04:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806ae70 0x7f69b806d330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d40b3540 0x7f69d41f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d413b890 0x7f69d40b3000 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 --2- 172.21.15.64:0/2073637054 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69c0005180 0x7f69d4159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 >> 172.21.15.64:0/2073637054 conn(0x7f69d406a860 msgr2=0x7f69d40cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:36.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 shutdown_connections 2023-11-24T08:04:36.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.675+0000 7f69daad1640 1 -- 172.21.15.64:0/2073637054 wait complete. 2023-11-24T08:04:36.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:36.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.927+0000 7ff65bfff640 1 Processor -- start 2023-11-24T08:04:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.927+0000 7ff65bfff640 1 -- start start 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.928+0000 7ff65bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff65c1435d0 0x7ff65c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.929+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.929+0000 7ff65bfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c070d40 con 0x7ff65c1439b0 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.929+0000 7ff65bfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c071020 con 0x7ff65c1435d0 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.929+0000 7ff65bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c071300 con 0x7ff65c0b4140 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.929+0000 7ff65affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff65c1435d0 0x7ff65c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.929+0000 7ff65affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff65c1435d0 0x7ff65c071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51500/0 (socket says 172.21.15.64:51500) 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.929+0000 7ff65affd640 1 -- 172.21.15.64:0/621361979 learned_addr learned my addr 172.21.15.64:0/621361979 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.929+0000 7ff65b7fe640 1 --2- 172.21.15.64:0/621361979 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.930+0000 7ff65affd640 1 -- 172.21.15.64:0/621361979 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 msgr2=0x7ff65c070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.930+0000 7ff65affd640 1 --2- 172.21.15.64:0/621361979 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.930+0000 7ff65affd640 1 -- 172.21.15.64:0/621361979 >> v1:172.21.15.91:6789/0 conn(0x7ff65c1439b0 legacy=0x7ff65c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.930+0000 7ff65affd640 1 -- 172.21.15.64:0/621361979 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0715e0 con 0x7ff65c1435d0 2023-11-24T08:04:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.931+0000 7ff65affd640 1 --2- 172.21.15.64:0/621361979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c071de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009e50 tx=0x7ff64c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9903a36f1a2703 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.931+0000 7ff659ffb640 1 -- 172.21.15.64:0/621361979 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c02cb00 con 0x7ff65c1435d0 2023-11-24T08:04:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.931+0000 7ff659ffb640 1 -- 172.21.15.64:0/621361979 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c02cc60 con 0x7ff65c1435d0 2023-11-24T08:04:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff659ffb640 1 -- 172.21.15.64:0/621361979 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c0343f0 con 0x7ff65c1435d0 2023-11-24T08:04:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff65bfff640 1 -- 172.21.15.64:0/621361979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 msgr2=0x7ff65c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff65bfff640 1 --2- 172.21.15.64:0/621361979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c071de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009e50 tx=0x7ff64c02bf60 comp rx=0 tx=0).stop 2023-11-24T08:04:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff65bfff640 1 -- 172.21.15.64:0/621361979 shutdown_connections 2023-11-24T08:04:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff65bfff640 1 --2- 172.21.15.64:0/621361979 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff65bfff640 1 --2- 172.21.15.64:0/621361979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c071de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff65bfff640 1 -- 172.21.15.64:0/621361979 >> 172.21.15.64:0/621361979 conn(0x7ff65c0cec10 msgr2=0x7ff65c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff65bfff640 1 -- 172.21.15.64:0/621361979 shutdown_connections 2023-11-24T08:04:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.932+0000 7ff65bfff640 1 -- 172.21.15.64:0/621361979 wait complete. 2023-11-24T08:04:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.934+0000 7ff65bfff640 1 Processor -- start 2023-11-24T08:04:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.934+0000 7ff65bfff640 1 -- start start 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.935+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c1f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.935+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c1f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.935+0000 7ff65bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff65c1439b0 0x7ff65c0a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.935+0000 7ff65bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c15c120 con 0x7ff65c1435d0 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.935+0000 7ff65bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0a9850 con 0x7ff65c1439b0 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.935+0000 7ff65bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c1f3ba0 con 0x7ff65c0b4140 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.935+0000 7ff65a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c1f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c1f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51502/0 (socket says 172.21.15.64:51502) 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65a7fc640 1 -- 172.21.15.64:0/1129677305 learned_addr learned my addr 172.21.15.64:0/1129677305 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65affd640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c1f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65a7fc640 1 -- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 msgr2=0x7ff65c1f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65b7fe640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff65c1439b0 0x7ff65c0a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65a7fc640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c1f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65a7fc640 1 -- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff65c1439b0 msgr2=0x7ff65c0a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65a7fc640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff65c1439b0 0x7ff65c0a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65a7fc640 1 -- 172.21.15.64:0/1129677305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c009b30 con 0x7ff65c1435d0 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65affd640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c1f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff65a7fc640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c1f3660 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff64400c3f0 tx=0x7ff64400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.936+0000 7ff6537fe640 1 -- 172.21.15.64:0/1129677305 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440052d0 con 0x7ff65c1435d0 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.937+0000 7ff6537fe640 1 -- 172.21.15.64:0/1129677305 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644005450 con 0x7ff65c1435d0 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.937+0000 7ff6537fe640 1 -- 172.21.15.64:0/1129677305 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644003830 con 0x7ff65c1435d0 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.937+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c0a89a0 con 0x7ff65c1435d0 2023-11-24T08:04:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.937+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c0a8f20 con 0x7ff65c1435d0 2023-11-24T08:04:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.938+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c04e840 con 0x7ff65c1435d0 2023-11-24T08:04:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.939+0000 7ff6537fe640 1 -- 172.21.15.64:0/1129677305 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff644003990 con 0x7ff65c1435d0 2023-11-24T08:04:36.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.940+0000 7ff6537fe640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff63c06ae70 0x7ff63c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:36.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.940+0000 7ff6537fe640 1 -- 172.21.15.64:0/1129677305 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff644088c10 con 0x7ff65c1435d0 2023-11-24T08:04:36.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.941+0000 7ff65affd640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff63c06ae70 0x7ff63c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:36.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.942+0000 7ff65affd640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff63c06ae70 0x7ff63c06d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff64c035000 tx=0x7ff64c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:36.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:36.945+0000 7ff6537fe640 1 -- 172.21.15.64:0/1129677305 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff644053c20 con 0x7ff65c1435d0 2023-11-24T08:04:37.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.146+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff65c06c720 con 0x7ff65c1435d0 2023-11-24T08:04:37.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.147+0000 7ff6537fe640 1 -- 172.21.15.64:0/1129677305 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6440170a0 con 0x7ff65c1435d0 2023-11-24T08:04:37.149 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.149+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff63c06ae70 msgr2=0x7ff63c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.149+0000 7ff65bfff640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff63c06ae70 0x7ff63c06d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff64c035000 tx=0x7ff64c030000 comp rx=0 tx=0).stop 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.149+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 msgr2=0x7ff65c1f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.149+0000 7ff65bfff640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c1f3660 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff64400c3f0 tx=0x7ff64400c8b0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.150+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 shutdown_connections 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.150+0000 7ff65bfff640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff63c06ae70 0x7ff63c06d330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.150+0000 7ff65bfff640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff65c1439b0 0x7ff65c0a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.150+0000 7ff65bfff640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff65c1435d0 0x7ff65c1f3660 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.150+0000 7ff65bfff640 1 --2- 172.21.15.64:0/1129677305 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff65c0b4140 0x7ff65c1f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.150+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 >> 172.21.15.64:0/1129677305 conn(0x7ff65c0cec10 msgr2=0x7ff65c0b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:37.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.150+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 shutdown_connections 2023-11-24T08:04:37.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.150+0000 7ff65bfff640 1 -- 172.21.15.64:0/1129677305 wait complete. 2023-11-24T08:04:37.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:37.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.395+0000 7fe57d3a0640 1 Processor -- start 2023-11-24T08:04:37.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.395+0000 7fe57d3a0640 1 -- start start 2023-11-24T08:04:37.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.397+0000 7fe57d3a0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe57815f270 0x7fe5780cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.397+0000 7fe57d3a0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe57806d9e0 0x7fe5780cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.397+0000 7fe57d3a0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57815f930 con 0x7fe578144770 2023-11-24T08:04:37.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.397+0000 7fe57d3a0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5780b43b0 con 0x7fe57806d9e0 2023-11-24T08:04:37.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.397+0000 7fe57d3a0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780b3b10 con 0x7fe57815f270 2023-11-24T08:04:37.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe57cb9f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe57806d9e0 0x7fe5780cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe57cb9f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe57806d9e0 0x7fe5780cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46588/0 (socket says 172.21.15.64:46588) 2023-11-24T08:04:37.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe57cb9f640 1 -- 172.21.15.64:0/587232434 learned_addr learned my addr 172.21.15.64:0/587232434 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe56f7fe640 1 --2- 172.21.15.64:0/587232434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe57815f270 0x7fe5780cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe57cb9f640 1 -- 172.21.15.64:0/587232434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe57815f270 msgr2=0x7fe5780cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe57cb9f640 1 --2- 172.21.15.64:0/587232434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe57815f270 0x7fe5780cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe57cb9f640 1 -- 172.21.15.64:0/587232434 >> v1:172.21.15.64:6789/0 conn(0x7fe578144770 legacy=0x7fe57815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe57cb9f640 1 -- 172.21.15.64:0/587232434 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5780b3df0 con 0x7fe57806d9e0 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.398+0000 7fe56f7fe640 1 --2- 172.21.15.64:0/587232434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe57815f270 0x7fe5780cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.399+0000 7fe57cb9f640 1 --2- 172.21.15.64:0/587232434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57806d9e0 0x7fe5780cd9e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe570009870 tx=0x7fe57002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70aa990e5da46581 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe5777fe640 1 -- 172.21.15.64:0/587232434 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570030840 con 0x7fe57806d9e0 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe5777fe640 1 -- 172.21.15.64:0/587232434 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5700309a0 con 0x7fe57806d9e0 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe5777fe640 1 -- 172.21.15.64:0/587232434 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570030d30 con 0x7fe57806d9e0 2023-11-24T08:04:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe57d3a0640 1 -- 172.21.15.64:0/587232434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57806d9e0 msgr2=0x7fe5780cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/587232434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57806d9e0 0x7fe5780cd9e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe570009870 tx=0x7fe57002b980 comp rx=0 tx=0).stop 2023-11-24T08:04:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe57d3a0640 1 -- 172.21.15.64:0/587232434 shutdown_connections 2023-11-24T08:04:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/587232434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57806d9e0 0x7fe5780cd9e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/587232434 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe57815f270 0x7fe5780cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe57d3a0640 1 -- 172.21.15.64:0/587232434 >> 172.21.15.64:0/587232434 conn(0x7fe5780cec10 msgr2=0x7fe5780cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe57d3a0640 1 -- 172.21.15.64:0/587232434 shutdown_connections 2023-11-24T08:04:37.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.400+0000 7fe57d3a0640 1 -- 172.21.15.64:0/587232434 wait complete. 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.402+0000 7fe57d3a0640 1 Processor -- start 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.402+0000 7fe57d3a0640 1 -- start start 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.403+0000 7fe57d3a0640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe57806d9e0 0x7fe578159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.403+0000 7fe57d3a0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe578144770 0x7fe578070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.403+0000 7fe57d3a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57815f270 0x7fe57815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.403+0000 7fe57d3a0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57815d710 con 0x7fe57815f270 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.403+0000 7fe57d3a0640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57815cb00 con 0x7fe57806d9e0 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.403+0000 7fe57d3a0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57815cde0 con 0x7fe578144770 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57815f270 0x7fe57815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57815f270 0x7fe57815dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46602/0 (socket says 172.21.15.64:46602) 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 -- 172.21.15.64:0/134919981 learned_addr learned my addr 172.21.15.64:0/134919981 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe577fff640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe57806d9e0 0x7fe578159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 -- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe578144770 msgr2=0x7fe578070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe578144770 0x7fe578070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 -- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe57806d9e0 msgr2=0x7fe578159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe57806d9e0 0x7fe578159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 -- 172.21.15.64:0/134919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700094d0 con 0x7fe57815f270 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe577fff640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe57806d9e0 0x7fe578159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.404+0000 7fe57cb9f640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57815f270 0x7fe57815dba0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe570009870 tx=0x7fe570004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.405+0000 7fe5757fa640 1 -- 172.21.15.64:0/134919981 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700307d0 con 0x7fe57815f270 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.405+0000 7fe5757fa640 1 -- 172.21.15.64:0/134919981 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570043400 con 0x7fe57815f270 2023-11-24T08:04:37.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.405+0000 7fe5757fa640 1 -- 172.21.15.64:0/134919981 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700437b0 con 0x7fe57815f270 2023-11-24T08:04:37.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.405+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57815d0c0 con 0x7fe57815f270 2023-11-24T08:04:37.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.405+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe578159cd0 con 0x7fe57815f270 2023-11-24T08:04:37.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.406+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57815d880 con 0x7fe57815f270 2023-11-24T08:04:37.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.407+0000 7fe5757fa640 1 -- 172.21.15.64:0/134919981 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe570043910 con 0x7fe57815f270 2023-11-24T08:04:37.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.408+0000 7fe5757fa640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe55806ae40 0x7fe55806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.408+0000 7fe5757fa640 1 -- 172.21.15.64:0/134919981 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe57003e070 con 0x7fe57815f270 2023-11-24T08:04:37.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.409+0000 7fe577fff640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe55806ae40 0x7fe55806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.410+0000 7fe577fff640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe55806ae40 0x7fe55806d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe5680012e0 tx=0x7fe568009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:37.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.412+0000 7fe5757fa640 1 -- 172.21.15.64:0/134919981 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe570077a50 con 0x7fe57815f270 2023-11-24T08:04:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.603+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe57804e840 con 0x7fe57815f270 2023-11-24T08:04:37.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.604+0000 7fe5757fa640 1 -- 172.21.15.64:0/134919981 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe57007b070 con 0x7fe57815f270 2023-11-24T08:04:37.606 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:37.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe55806ae40 msgr2=0x7fe55806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe55806ae40 0x7fe55806d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe5680012e0 tx=0x7fe568009040 comp rx=0 tx=0).stop 2023-11-24T08:04:37.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57815f270 msgr2=0x7fe57815dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57815f270 0x7fe57815dba0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe570009870 tx=0x7fe570004730 comp rx=0 tx=0).stop 2023-11-24T08:04:37.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 shutdown_connections 2023-11-24T08:04:37.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe55806ae40 0x7fe55806d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe57815f270 0x7fe57815dba0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe578144770 0x7fe578070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.606+0000 7fe57d3a0640 1 --2- 172.21.15.64:0/134919981 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe57806d9e0 0x7fe578159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.607+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 >> 172.21.15.64:0/134919981 conn(0x7fe5780cec10 msgr2=0x7fe5780af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:37.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.607+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 shutdown_connections 2023-11-24T08:04:37.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.607+0000 7fe57d3a0640 1 -- 172.21.15.64:0/134919981 wait complete. 2023-11-24T08:04:37.625 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:37.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.848+0000 7ff071827640 1 Processor -- start 2023-11-24T08:04:37.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.848+0000 7ff071827640 1 -- start start 2023-11-24T08:04:37.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff071827640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff06c144770 0x7ff06c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff071827640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff06c0b35a0 0x7ff06c15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff071827640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c06c9d0 0x7ff06c06ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff071827640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0b43b0 con 0x7ff06c0b35a0 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff071827640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c15f930 con 0x7ff06c144770 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff071827640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0b3b10 con 0x7ff06c06c9d0 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff06bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff06c0b35a0 0x7ff06c15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff06bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff06c0b35a0 0x7ff06c15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46606/0 (socket says 172.21.15.64:46606) 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.850+0000 7ff06bfff640 1 -- 172.21.15.64:0/3618013520 learned_addr learned my addr 172.21.15.64:0/3618013520 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.851+0000 7ff070825640 1 --2- 172.21.15.64:0/3618013520 >> v2:172.21.15.91:3300/0 conn(0x7ff06c144770 0x7ff06c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.851+0000 7ff071026640 1 --2- 172.21.15.64:0/3618013520 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c06c9d0 0x7ff06c06ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.851+0000 7ff06bfff640 1 -- 172.21.15.64:0/3618013520 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c06c9d0 msgr2=0x7ff06c06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.851+0000 7ff06bfff640 1 --2- 172.21.15.64:0/3618013520 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c06c9d0 0x7ff06c06ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.851+0000 7ff06bfff640 1 -- 172.21.15.64:0/3618013520 >> v2:172.21.15.91:3300/0 conn(0x7ff06c144770 msgr2=0x7ff06c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.851+0000 7ff06bfff640 1 --2- 172.21.15.64:0/3618013520 >> v2:172.21.15.91:3300/0 conn(0x7ff06c144770 0x7ff06c15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.851+0000 7ff06bfff640 1 -- 172.21.15.64:0/3618013520 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c0b3df0 con 0x7ff06c0b35a0 2023-11-24T08:04:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.851+0000 7ff071026640 1 --2- 172.21.15.64:0/3618013520 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c06c9d0 0x7ff06c06ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.852+0000 7ff06bfff640 1 --2- 172.21.15.64:0/3618013520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0b35a0 0x7ff06c15f0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff06c06df90 tx=0x7ff05802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a3e2a3765c53b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.852+0000 7ff06b7fe640 1 -- 172.21.15.64:0/3618013520 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058031860 con 0x7ff06c0b35a0 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.852+0000 7ff06b7fe640 1 -- 172.21.15.64:0/3618013520 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0580319c0 con 0x7ff06c0b35a0 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.852+0000 7ff06b7fe640 1 -- 172.21.15.64:0/3618013520 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058031d50 con 0x7ff06c0b35a0 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 -- 172.21.15.64:0/3618013520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0b35a0 msgr2=0x7ff06c15f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 --2- 172.21.15.64:0/3618013520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0b35a0 0x7ff06c15f0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff06c06df90 tx=0x7ff05802b900 comp rx=0 tx=0).stop 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 -- 172.21.15.64:0/3618013520 shutdown_connections 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 --2- 172.21.15.64:0/3618013520 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c06c9d0 0x7ff06c06ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 --2- 172.21.15.64:0/3618013520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0b35a0 0x7ff06c15f0a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 --2- 172.21.15.64:0/3618013520 >> v2:172.21.15.91:3300/0 conn(0x7ff06c144770 0x7ff06c15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 -- 172.21.15.64:0/3618013520 >> 172.21.15.64:0/3618013520 conn(0x7ff06c0cec10 msgr2=0x7ff06c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 -- 172.21.15.64:0/3618013520 shutdown_connections 2023-11-24T08:04:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.853+0000 7ff071827640 1 -- 172.21.15.64:0/3618013520 wait complete. 2023-11-24T08:04:37.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.855+0000 7ff071827640 1 Processor -- start 2023-11-24T08:04:37.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.855+0000 7ff071827640 1 -- start start 2023-11-24T08:04:37.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff071827640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c06c9d0 0x7ff06c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff071827640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c0b35a0 0x7ff06c0ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff071827640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff06c144770 0x7ff06c0acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff071827640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c134c90 con 0x7ff06c06c9d0 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff071827640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c134010 con 0x7ff06c144770 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff071827640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c1342f0 con 0x7ff06c0b35a0 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff070825640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c06c9d0 0x7ff06c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff070825640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c06c9d0 0x7ff06c15a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46610/0 (socket says 172.21.15.64:46610) 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff070825640 1 -- 172.21.15.64:0/4095491376 learned_addr learned my addr 172.21.15.64:0/4095491376 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff06bfff640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c0b35a0 0x7ff06c0ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff070825640 1 -- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c0b35a0 msgr2=0x7ff06c0ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff070825640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c0b35a0 0x7ff06c0ac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff070825640 1 -- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff06c144770 msgr2=0x7ff06c0acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff070825640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff06c144770 0x7ff06c0acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.856+0000 7ff070825640 1 -- 172.21.15.64:0/4095491376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580094d0 con 0x7ff06c06c9d0 2023-11-24T08:04:37.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.857+0000 7ff070825640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c06c9d0 0x7ff06c15a900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d5f0 tx=0x7ff06000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:37.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.857+0000 7ff0697fa640 1 -- 172.21.15.64:0/4095491376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600125f0 con 0x7ff06c06c9d0 2023-11-24T08:04:37.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.857+0000 7ff0697fa640 1 -- 172.21.15.64:0/4095491376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff060012750 con 0x7ff06c06c9d0 2023-11-24T08:04:37.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.857+0000 7ff0697fa640 1 -- 172.21.15.64:0/4095491376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060012b00 con 0x7ff06c06c9d0 2023-11-24T08:04:37.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.857+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c1345d0 con 0x7ff06c06c9d0 2023-11-24T08:04:37.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.857+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c15ae40 con 0x7ff06c06c9d0 2023-11-24T08:04:37.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.858+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c134f00 con 0x7ff06c06c9d0 2023-11-24T08:04:37.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.859+0000 7ff0697fa640 1 -- 172.21.15.64:0/4095491376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff060012c60 con 0x7ff06c06c9d0 2023-11-24T08:04:37.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.861+0000 7ff0697fa640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff05006ae40 0x7ff05006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:37.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.861+0000 7ff0697fa640 1 -- 172.21.15.64:0/4095491376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff060089b10 con 0x7ff06c06c9d0 2023-11-24T08:04:37.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.861+0000 7ff06bfff640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff05006ae40 0x7ff05006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:37.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.862+0000 7ff06bfff640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff05006ae40 0x7ff05006d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff058036000 tx=0x7ff05803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:37.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:37.865+0000 7ff0697fa640 1 -- 172.21.15.64:0/4095491376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff060054b20 con 0x7ff06c06c9d0 2023-11-24T08:04:38.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.067+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff06c04e840 con 0x7ff06c06c9d0 2023-11-24T08:04:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.068+0000 7ff0697fa640 1 -- 172.21.15.64:0/4095491376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff060058140 con 0x7ff06c06c9d0 2023-11-24T08:04:38.070 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff05006ae40 msgr2=0x7ff05006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff05006ae40 0x7ff05006d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff058036000 tx=0x7ff05803c040 comp rx=0 tx=0).stop 2023-11-24T08:04:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c06c9d0 msgr2=0x7ff06c15a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c06c9d0 0x7ff06c15a900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d5f0 tx=0x7ff06000dab0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 shutdown_connections 2023-11-24T08:04:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff05006ae40 0x7ff05006d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff06c144770 0x7ff06c0acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff06c0b35a0 0x7ff06c0ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 --2- 172.21.15.64:0/4095491376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c06c9d0 0x7ff06c15a900 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 >> 172.21.15.64:0/4095491376 conn(0x7ff06c0cec10 msgr2=0x7ff06c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:38.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 shutdown_connections 2023-11-24T08:04:38.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.071+0000 7ff071827640 1 -- 172.21.15.64:0/4095491376 wait complete. 2023-11-24T08:04:38.160 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:04:38.160 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:04:38.161 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:04:38.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:38.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.396+0000 7f6a1a4c2640 1 Processor -- start 2023-11-24T08:04:38.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.396+0000 7f6a1a4c2640 1 -- start start 2023-11-24T08:04:38.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.397+0000 7f6a1a4c2640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a140d1900 0x7f6a140b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.398+0000 7f6a1a4c2640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140cf0c0 con 0x7f6a140d14a0 2023-11-24T08:04:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.398+0000 7f6a1a4c2640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140b01b0 con 0x7f6a1415d5b0 2023-11-24T08:04:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.398+0000 7f6a1a4c2640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1415d990 con 0x7f6a140d1900 2023-11-24T08:04:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.398+0000 7f6a18cbf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6a140d14a0 0x7f6a140b0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41962/0 (socket says 172.21.15.64:41962) 2023-11-24T08:04:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.398+0000 7f6a18cbf640 1 -- 172.21.15.64:0/2659395058 learned_addr learned my addr 172.21.15.64:0/2659395058 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.398+0000 7f6a19cc1640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a140d1900 0x7f6a140b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.398+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379194975 0 0) 0x7f6a140cf0c0 con 0x7f6a140d14a0 2023-11-24T08:04:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a00003520 con 0x7f6a140d14a0 2023-11-24T08:04:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1420615706 0 0) 0x7f6a00003520 con 0x7f6a140d14a0 2023-11-24T08:04:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a140d1900 msgr2=0x7f6a140b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a140d1900 0x7f6a140b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 >> v1:172.21.15.91:6789/0 conn(0x7f6a1415d5b0 legacy=0x7f6a140d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140ceb00 con 0x7f6a140d14a0 2023-11-24T08:04:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f6a04001670 con 0x7f6a140d14a0 2023-11-24T08:04:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 >> v1:172.21.15.64:6789/0 conn(0x7f6a140d14a0 legacy=0x7f6a140b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a00003a10 0x7f6a00003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a140d1900 0x7f6a000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a00004d00 0x7f6a00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1415d990 con 0x7f6a00004d00 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0000bac0 con 0x7f6a00003a10 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0000bc80 con 0x7f6a140d1900 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.399+0000 7f6a19cc1640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a00004d00 0x7f6a00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.400+0000 7f6a194c0640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a00003a10 0x7f6a00003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.400+0000 7f6a18cbf640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a140d1900 0x7f6a000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.400+0000 7f6a19cc1640 1 -- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a140d1900 msgr2=0x7f6a000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.400+0000 7f6a19cc1640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a140d1900 0x7f6a000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.400+0000 7f6a19cc1640 1 -- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a00003a10 msgr2=0x7f6a00003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.400+0000 7f6a19cc1640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a00003a10 0x7f6a00003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.400+0000 7f6a19cc1640 1 -- 172.21.15.64:0/2659395058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00003520 con 0x7f6a00004d00 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.401+0000 7f6a19cc1640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a00004d00 0x7f6a00005180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009a90 tx=0x7f6a000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf10d7aad97ab1f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.401+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c02c680 con 0x7f6a00004d00 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.401+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c02c7e0 con 0x7f6a00004d00 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.401+0000 7f6a0bfff640 1 -- 172.21.15.64:0/2659395058 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c02cb70 con 0x7f6a00004d00 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.402+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2659395058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a00004d00 msgr2=0x7f6a00005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.402+0000 7f6a1a4c2640 1 --2- 172.21.15.64:0/2659395058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a00004d00 0x7f6a00005180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009a90 tx=0x7f6a000057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.402+0000 7f6a19cc1640 1 -- 172.21.15.64:0/2659395058 reap_dead start 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.402+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2659395058 shutdown_connections 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.402+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2659395058 >> 172.21.15.64:0/2659395058 conn(0x7f6a1409c3b0 msgr2=0x7f6a140d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.402+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2659395058 shutdown_connections 2023-11-24T08:04:38.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.402+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2659395058 wait complete. 2023-11-24T08:04:38.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.404+0000 7f6a1a4c2640 1 Processor -- start 2023-11-24T08:04:38.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.404+0000 7f6a1a4c2640 1 -- start start 2023-11-24T08:04:38.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a1a4c2640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a00004d00 0x7f6a141f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a1a4c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1414f460 0x7f6a140b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a1a4c2640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a140b81e0 0x7f6a140c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a1a4c2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140b8660 con 0x7f6a1414f460 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a1a4c2640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140b2d90 con 0x7f6a140b81e0 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a1a4c2640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140c8a40 con 0x7f6a00004d00 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a18cbf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1414f460 0x7f6a140b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a18cbf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1414f460 0x7f6a140b7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46626/0 (socket says 172.21.15.64:46626) 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.405+0000 7f6a18cbf640 1 -- 172.21.15.64:0/2288132158 learned_addr learned my addr 172.21.15.64:0/2288132158 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a19cc1640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a140b81e0 0x7f6a140c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a194c0640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a00004d00 0x7f6a141f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a18cbf640 1 -- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a00004d00 msgr2=0x7f6a141f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a18cbf640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a00004d00 0x7f6a141f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a18cbf640 1 -- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a140b81e0 msgr2=0x7f6a140c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a18cbf640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a140b81e0 0x7f6a140c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a18cbf640 1 -- 172.21.15.64:0/2288132158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140b2740 con 0x7f6a1414f460 2023-11-24T08:04:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a18cbf640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1414f460 0x7f6a140b7ca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400e810 tx=0x7f6a0400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a09ffb640 1 -- 172.21.15.64:0/2288132158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400c870 con 0x7f6a1414f460 2023-11-24T08:04:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a09ffb640 1 -- 172.21.15.64:0/2288132158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0400c9d0 con 0x7f6a1414f460 2023-11-24T08:04:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a09ffb640 1 -- 172.21.15.64:0/2288132158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400cd80 con 0x7f6a1414f460 2023-11-24T08:04:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.406+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a140b2a20 con 0x7f6a1414f460 2023-11-24T08:04:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.407+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a140c8210 con 0x7f6a1414f460 2023-11-24T08:04:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.408+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a140b25a0 con 0x7f6a1414f460 2023-11-24T08:04:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.409+0000 7f6a09ffb640 1 -- 172.21.15.64:0/2288132158 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a0401d460 con 0x7f6a1414f460 2023-11-24T08:04:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.410+0000 7f6a09ffb640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69f806ae70 0x7f69f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.410+0000 7f6a09ffb640 1 -- 172.21.15.64:0/2288132158 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6a040896a0 con 0x7f6a1414f460 2023-11-24T08:04:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.410+0000 7f6a194c0640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69f806ae70 0x7f69f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.412+0000 7f6a194c0640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69f806ae70 0x7f69f806d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a10007fd0 tx=0x7f6a10009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.415+0000 7f6a09ffb640 1 -- 172.21.15.64:0/2288132158 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a140b25a0 con 0x7f6a1414f460 2023-11-24T08:04:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.610+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a140b25a0 con 0x7f6a1414f460 2023-11-24T08:04:38.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.612+0000 7f6a09ffb640 1 -- 172.21.15.64:0/2288132158 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7f6a140b25a0 con 0x7f6a1414f460 2023-11-24T08:04:38.614 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:38.614 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.614+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69f806ae70 msgr2=0x7f69f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69f806ae70 0x7f69f806d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a10007fd0 tx=0x7f6a10009040 comp rx=0 tx=0).stop 2023-11-24T08:04:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1414f460 msgr2=0x7f6a140b7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1414f460 0x7f6a140b7ca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400e810 tx=0x7f6a0400ecd0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 shutdown_connections 2023-11-24T08:04:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69f806ae70 0x7f69f806d330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a140b81e0 0x7f6a140c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1414f460 0x7f6a140b7ca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 --2- 172.21.15.64:0/2288132158 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a00004d00 0x7f6a141f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 >> 172.21.15.64:0/2288132158 conn(0x7f6a1409c3b0 msgr2=0x7f6a1406c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:38.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 shutdown_connections 2023-11-24T08:04:38.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.615+0000 7f6a1a4c2640 1 -- 172.21.15.64:0/2288132158 wait complete. 2023-11-24T08:04:38.635 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:38.635 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:38.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.866+0000 7fda751f4640 1 Processor -- start 2023-11-24T08:04:38.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.867+0000 7fda751f4640 1 -- start start 2023-11-24T08:04:38.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.868+0000 7fda751f4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fda70153ea0 0x7fda700ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.868+0000 7fda751f4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda700ce650 con 0x7fda70070850 2023-11-24T08:04:38.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.868+0000 7fda751f4640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda70154710 con 0x7fda70154280 2023-11-24T08:04:38.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.868+0000 7fda751f4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7015e0a0 con 0x7fda70153ea0 2023-11-24T08:04:38.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.869+0000 7fda6ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fda70153ea0 0x7fda700ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.869+0000 7fda6ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fda70153ea0 0x7fda700ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46636/0 (socket says 172.21.15.64:46636) 2023-11-24T08:04:38.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.869+0000 7fda6ffff640 1 -- 172.21.15.64:0/2188967161 learned_addr learned my addr 172.21.15.64:0/2188967161 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:38.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.869+0000 7fda6ffff640 1 -- 172.21.15.64:0/2188967161 >> v1:172.21.15.91:6789/0 conn(0x7fda70154280 legacy=0x7fda7015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.869+0000 7fda6ffff640 1 -- 172.21.15.64:0/2188967161 >> v1:172.21.15.64:6789/0 conn(0x7fda70070850 legacy=0x7fda70071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.869+0000 7fda6ffff640 1 -- 172.21.15.64:0/2188967161 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7015e380 con 0x7fda70153ea0 2023-11-24T08:04:38.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.870+0000 7fda6ffff640 1 --2- 172.21.15.64:0/2188967161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70153ea0 0x7fda700ad450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda60009870 tx=0x7fda6002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1fe87c3331ff9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:38.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.871+0000 7fda6f7fe640 1 -- 172.21.15.64:0/2188967161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6002f840 con 0x7fda70153ea0 2023-11-24T08:04:38.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.871+0000 7fda6f7fe640 1 -- 172.21.15.64:0/2188967161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6002f9a0 con 0x7fda70153ea0 2023-11-24T08:04:38.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.871+0000 7fda6f7fe640 1 -- 172.21.15.64:0/2188967161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6002fd30 con 0x7fda70153ea0 2023-11-24T08:04:38.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.871+0000 7fda751f4640 1 -- 172.21.15.64:0/2188967161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70153ea0 msgr2=0x7fda700ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.871+0000 7fda751f4640 1 --2- 172.21.15.64:0/2188967161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70153ea0 0x7fda700ad450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda60009870 tx=0x7fda6002ba20 comp rx=0 tx=0).stop 2023-11-24T08:04:38.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.872+0000 7fda751f4640 1 -- 172.21.15.64:0/2188967161 shutdown_connections 2023-11-24T08:04:38.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.872+0000 7fda751f4640 1 --2- 172.21.15.64:0/2188967161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70153ea0 0x7fda700ad450 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.872+0000 7fda751f4640 1 -- 172.21.15.64:0/2188967161 >> 172.21.15.64:0/2188967161 conn(0x7fda700cec10 msgr2=0x7fda700cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:38.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.872+0000 7fda751f4640 1 -- 172.21.15.64:0/2188967161 shutdown_connections 2023-11-24T08:04:38.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.872+0000 7fda751f4640 1 -- 172.21.15.64:0/2188967161 wait complete. 2023-11-24T08:04:38.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.873+0000 7fda751f4640 1 Processor -- start 2023-11-24T08:04:38.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.874+0000 7fda751f4640 1 -- start start 2023-11-24T08:04:38.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda751f4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda70070850 0x7fda70146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda751f4640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda70153ea0 0x7fda701588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda751f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70154280 0x7fda700b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda751f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda70156850 con 0x7fda70154280 2023-11-24T08:04:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda751f4640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda70156b30 con 0x7fda70153ea0 2023-11-24T08:04:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda751f4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda70156e10 con 0x7fda70070850 2023-11-24T08:04:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda749f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70154280 0x7fda700b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda749f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70154280 0x7fda700b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46638/0 (socket says 172.21.15.64:46638) 2023-11-24T08:04:38.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda749f3640 1 -- 172.21.15.64:0/1488609308 learned_addr learned my addr 172.21.15.64:0/1488609308 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:38.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda67fff640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda70153ea0 0x7fda701588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.875+0000 7fda6ffff640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda70070850 0x7fda70146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda749f3640 1 -- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda70070850 msgr2=0x7fda70146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda749f3640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda70070850 0x7fda70146720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda749f3640 1 -- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda70153ea0 msgr2=0x7fda701588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda749f3640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda70153ea0 0x7fda701588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda749f3640 1 -- 172.21.15.64:0/1488609308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda600094d0 con 0x7fda70154280 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda749f3640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70154280 0x7fda700b4310 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fda6800c7a0 tx=0x7fda6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda6d7fa640 1 -- 172.21.15.64:0/1488609308 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68003b70 con 0x7fda70154280 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda6d7fa640 1 -- 172.21.15.64:0/1488609308 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda68003cf0 con 0x7fda70154280 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda6d7fa640 1 -- 172.21.15.64:0/1488609308 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68003ea0 con 0x7fda70154280 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.876+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda701570f0 con 0x7fda70154280 2023-11-24T08:04:38.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.877+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda70146c60 con 0x7fda70154280 2023-11-24T08:04:38.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.878+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda70157610 con 0x7fda70154280 2023-11-24T08:04:38.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.879+0000 7fda6d7fa640 1 -- 172.21.15.64:0/1488609308 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda6800aa90 con 0x7fda70154280 2023-11-24T08:04:38.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.879+0000 7fda6d7fa640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fda5006ae40 0x7fda5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:38.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.880+0000 7fda6d7fa640 1 -- 172.21.15.64:0/1488609308 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fda68089510 con 0x7fda70154280 2023-11-24T08:04:38.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.880+0000 7fda6ffff640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fda5006ae40 0x7fda5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:38.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.881+0000 7fda6ffff640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fda5006ae40 0x7fda5006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda60036000 tx=0x7fda60008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:38.884+0000 7fda6d7fa640 1 -- 172.21.15.64:0/1488609308 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda70157610 con 0x7fda70154280 2023-11-24T08:04:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.079+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda70157610 con 0x7fda70154280 2023-11-24T08:04:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.080+0000 7fda6d7fa640 1 -- 172.21.15.64:0/1488609308 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda70157610 con 0x7fda70154280 2023-11-24T08:04:39.082 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:39.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fda5006ae40 msgr2=0x7fda5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fda5006ae40 0x7fda5006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda60036000 tx=0x7fda60008030 comp rx=0 tx=0).stop 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70154280 msgr2=0x7fda700b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70154280 0x7fda700b4310 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fda6800c7a0 tx=0x7fda6800cc60 comp rx=0 tx=0).stop 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 shutdown_connections 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fda5006ae40 0x7fda5006d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda70154280 0x7fda700b4310 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fda70153ea0 0x7fda701588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 --2- 172.21.15.64:0/1488609308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fda70070850 0x7fda70146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.082+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 >> 172.21.15.64:0/1488609308 conn(0x7fda700cec10 msgr2=0x7fda700b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.083+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 shutdown_connections 2023-11-24T08:04:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.083+0000 7fda751f4640 1 -- 172.21.15.64:0/1488609308 wait complete. 2023-11-24T08:04:39.105 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:39.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.337+0000 7fb8790ff640 1 Processor -- start 2023-11-24T08:04:39.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.337+0000 7fb8790ff640 1 -- start start 2023-11-24T08:04:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.338+0000 7fb8790ff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8740d1d90 0x7fb8740ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.338+0000 7fb8790ff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740c6bf0 con 0x7fb87415d3b0 2023-11-24T08:04:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.338+0000 7fb8790ff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740d13d0 con 0x7fb8740d0fa0 2023-11-24T08:04:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.338+0000 7fb8790ff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740cedc0 con 0x7fb8740d1d90 2023-11-24T08:04:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.339+0000 7fb873fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb87415d3b0 0x7fb8740d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41980/0 (socket says 172.21.15.64:41980) 2023-11-24T08:04:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.339+0000 7fb873fff640 1 -- 172.21.15.64:0/559550715 learned_addr learned my addr 172.21.15.64:0/559550715 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.339+0000 7fb8737fe640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8740d1d90 0x7fb8740ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.339+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735848668 0 0) 0x7fb8740c6bf0 con 0x7fb87415d3b0 2023-11-24T08:04:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.339+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb85c003520 con 0x7fb87415d3b0 2023-11-24T08:04:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2127983594 0 0) 0x7fb85c003520 con 0x7fb87415d3b0 2023-11-24T08:04:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8740d1d90 msgr2=0x7fb8740ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8740d1d90 0x7fb8740ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 >> v1:172.21.15.91:6789/0 conn(0x7fb8740d0fa0 legacy=0x7fb8740b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740c6e40 con 0x7fb87415d3b0 2023-11-24T08:04:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fb8640019f0 con 0x7fb87415d3b0 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 >> v1:172.21.15.64:6789/0 conn(0x7fb87415d3b0 legacy=0x7fb8740d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb85c003a10 0x7fb85c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c0047c0 0x7fb85c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8740d1d90 0x7fb85c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740cedc0 con 0x7fb85c0047c0 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c00bb70 con 0x7fb85c003a10 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c00bd30 con 0x7fb8740d1d90 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.340+0000 7fb8737fe640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c0047c0 0x7fb85c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.341+0000 7fb873fff640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb85c003a10 0x7fb85c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.341+0000 7fb8737fe640 1 -- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8740d1d90 msgr2=0x7fb85c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.341+0000 7fb8737fe640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8740d1d90 0x7fb85c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.341+0000 7fb8737fe640 1 -- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb85c003a10 msgr2=0x7fb85c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.341+0000 7fb8737fe640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb85c003a10 0x7fb85c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.341+0000 7fb8737fe640 1 -- 172.21.15.64:0/559550715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c003520 con 0x7fb85c0047c0 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.342+0000 7fb8737fe640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c0047c0 0x7fb85c004c40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb868009a90 tx=0x7fb85c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4843d4519cbbfc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.342+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86802c6b0 con 0x7fb85c0047c0 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.342+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86802c810 con 0x7fb85c0047c0 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.342+0000 7fb872ffd640 1 -- 172.21.15.64:0/559550715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86802cba0 con 0x7fb85c0047c0 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.343+0000 7fb8790ff640 1 -- 172.21.15.64:0/559550715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c0047c0 msgr2=0x7fb85c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.343+0000 7fb8790ff640 1 --2- 172.21.15.64:0/559550715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c0047c0 0x7fb85c004c40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb868009a90 tx=0x7fb85c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.343+0000 7fb8788fe640 1 -- 172.21.15.64:0/559550715 reap_dead start 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.343+0000 7fb8790ff640 1 -- 172.21.15.64:0/559550715 shutdown_connections 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.343+0000 7fb8790ff640 1 -- 172.21.15.64:0/559550715 >> 172.21.15.64:0/559550715 conn(0x7fb8740d30a0 msgr2=0x7fb8740d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.343+0000 7fb8790ff640 1 -- 172.21.15.64:0/559550715 shutdown_connections 2023-11-24T08:04:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.343+0000 7fb8790ff640 1 -- 172.21.15.64:0/559550715 wait complete. 2023-11-24T08:04:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.345+0000 7fb8790ff640 1 Processor -- start 2023-11-24T08:04:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.345+0000 7fb8790ff640 1 -- start start 2023-11-24T08:04:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8790ff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb85c0047c0 0x7fb874146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8790ff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8741580f0 0x7fb87415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8790ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb874144060 0x7fb87415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8790ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb874157a60 con 0x7fb874144060 2023-11-24T08:04:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8790ff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8741588d0 con 0x7fb8741580f0 2023-11-24T08:04:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8790ff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb874147040 con 0x7fb85c0047c0 2023-11-24T08:04:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8788fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb874144060 0x7fb87415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8788fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb874144060 0x7fb87415ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46660/0 (socket says 172.21.15.64:46660) 2023-11-24T08:04:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.346+0000 7fb8788fe640 1 -- 172.21.15.64:0/1007528513 learned_addr learned my addr 172.21.15.64:0/1007528513 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb873fff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb85c0047c0 0x7fb874146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb8737fe640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8741580f0 0x7fb87415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb8788fe640 1 -- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb85c0047c0 msgr2=0x7fb874146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb8788fe640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb85c0047c0 0x7fb874146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb8788fe640 1 -- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8741580f0 msgr2=0x7fb87415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb8788fe640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8741580f0 0x7fb87415fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb8788fe640 1 -- 172.21.15.64:0/1007528513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8741585f0 con 0x7fb874144060 2023-11-24T08:04:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb8788fe640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb874144060 0x7fb87415ab00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb874072b10 tx=0x7fb86c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb870ff9640 1 -- 172.21.15.64:0/1007528513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c007340 con 0x7fb874144060 2023-11-24T08:04:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb870ff9640 1 -- 172.21.15.64:0/1007528513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c0074a0 con 0x7fb874144060 2023-11-24T08:04:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb870ff9640 1 -- 172.21.15.64:0/1007528513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c004260 con 0x7fb874144060 2023-11-24T08:04:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.347+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87415a020 con 0x7fb874144060 2023-11-24T08:04:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.348+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87415a5e0 con 0x7fb874144060 2023-11-24T08:04:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.349+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87415a7d0 con 0x7fb874144060 2023-11-24T08:04:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.350+0000 7fb870ff9640 1 -- 172.21.15.64:0/1007528513 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb86c0057c0 con 0x7fb874144060 2023-11-24T08:04:39.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.351+0000 7fb870ff9640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.351+0000 7fb870ff9640 1 -- 172.21.15.64:0/1007528513 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb86c088530 con 0x7fb874144060 2023-11-24T08:04:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.351+0000 7fb873fff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.353+0000 7fb873fff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb864005d70 tx=0x7fb864009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:39.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.355+0000 7fb870ff9640 1 -- 172.21.15.64:0/1007528513 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb86c053550 con 0x7fb874144060 2023-11-24T08:04:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.554+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb87404e840 con 0x7fb874144060 2023-11-24T08:04:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.555+0000 7fb870ff9640 1 -- 172.21.15.64:0/1007528513 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb86c056b70 con 0x7fb874144060 2023-11-24T08:04:39.557 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.557+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 msgr2=0x7fb86006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.557+0000 7fb8790ff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb864005d70 tx=0x7fb864009040 comp rx=0 tx=0).stop 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.557+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb874144060 msgr2=0x7fb87415ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.557+0000 7fb8790ff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb874144060 0x7fb87415ab00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb874072b10 tx=0x7fb86c00c820 comp rx=0 tx=0).stop 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.558+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 shutdown_connections 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.558+0000 7fb8790ff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.558+0000 7fb8790ff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb874144060 0x7fb87415ab00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.558+0000 7fb8790ff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8741580f0 0x7fb87415fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.558+0000 7fb8790ff640 1 --2- 172.21.15.64:0/1007528513 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb85c0047c0 0x7fb874146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.558+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 >> 172.21.15.64:0/1007528513 conn(0x7fb8740d30a0 msgr2=0x7fb8740d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.558+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 shutdown_connections 2023-11-24T08:04:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.558+0000 7fb8790ff640 1 -- 172.21.15.64:0/1007528513 wait complete. 2023-11-24T08:04:39.579 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:39.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.809+0000 7f6795e64640 1 Processor -- start 2023-11-24T08:04:39.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.809+0000 7f6795e64640 1 -- start start 2023-11-24T08:04:39.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.811+0000 7f6795e64640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6790144770 0x7f679015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.811+0000 7f6795e64640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679015f930 con 0x7f67900b35a0 2023-11-24T08:04:39.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.811+0000 7f6795e64640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67900b3b10 con 0x7f679015f0a0 2023-11-24T08:04:39.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.811+0000 7f6795e64640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900b43b0 con 0x7f6790144770 2023-11-24T08:04:39.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.811+0000 7f678ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f67900b35a0 0x7f679006c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41982/0 (socket says 172.21.15.64:41982) 2023-11-24T08:04:39.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.811+0000 7f678ffff640 1 -- 172.21.15.64:0/457620697 learned_addr learned my addr 172.21.15.64:0/457620697 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:39.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f6794e62640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6790144770 0x7f679015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853060119 0 0) 0x7f679015f930 con 0x7f67900b35a0 2023-11-24T08:04:39.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6780003520 con 0x7f67900b35a0 2023-11-24T08:04:39.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 105082390 0 0) 0x7f6780003520 con 0x7f67900b35a0 2023-11-24T08:04:39.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6790144770 msgr2=0x7f679015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:39.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6790144770 0x7f679015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 >> v1:172.21.15.91:6789/0 conn(0x7f679015f0a0 legacy=0x7f67900ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900b3df0 con 0x7f67900b35a0 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f677c001670 con 0x7f67900b35a0 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 >> v1:172.21.15.64:6789/0 conn(0x7f67900b35a0 legacy=0x7f679006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6790144770 0x7f6780003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6780003f90 0x7f67800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6780005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900b43b0 con 0x7f6780004d00 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678000bae0 con 0x7f6780003f90 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.812+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678000bca0 con 0x7f6790144770 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.813+0000 7f6795663640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6780005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.813+0000 7f678ffff640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6780003f90 0x7f67800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.813+0000 7f6795663640 1 -- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6790144770 msgr2=0x7f6780003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.813+0000 7f6795663640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6790144770 0x7f6780003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.813+0000 7f6795663640 1 -- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6780003f90 msgr2=0x7f67800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.813+0000 7f6795663640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6780003f90 0x7f67800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.813+0000 7f6795663640 1 -- 172.21.15.64:0/457620697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6780003520 con 0x7f6780004d00 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.814+0000 7f6795663640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6780005180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6788001e50 tx=0x7f67800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff68968159439e0e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788003b70 con 0x7f6780004d00 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6788003cd0 con 0x7f6780004d00 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f678f7fe640 1 -- 172.21.15.64:0/457620697 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788005310 con 0x7f6780004d00 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f6795e64640 1 -- 172.21.15.64:0/457620697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 msgr2=0x7f6780005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f6795e64640 1 --2- 172.21.15.64:0/457620697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6780005180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6788001e50 tx=0x7f67800057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f6795663640 1 -- 172.21.15.64:0/457620697 reap_dead start 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f6795e64640 1 -- 172.21.15.64:0/457620697 shutdown_connections 2023-11-24T08:04:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f6795e64640 1 -- 172.21.15.64:0/457620697 >> 172.21.15.64:0/457620697 conn(0x7f67900cec10 msgr2=0x7f67900cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f6795e64640 1 -- 172.21.15.64:0/457620697 shutdown_connections 2023-11-24T08:04:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.815+0000 7f6795e64640 1 -- 172.21.15.64:0/457620697 wait complete. 2023-11-24T08:04:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.817+0000 7f6795e64640 1 Processor -- start 2023-11-24T08:04:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.817+0000 7f6795e64640 1 -- start start 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.818+0000 7f6795e64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6790159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.818+0000 7f6795e64640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67900cb870 0x7f679013b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.818+0000 7f6795e64640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67900c9730 0x7f67900c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.818+0000 7f6795e64640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900cbcf0 con 0x7f6780004d00 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.818+0000 7f6795e64640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900cb090 con 0x7f67900c9730 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.818+0000 7f6795e64640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900ca0f0 con 0x7f67900cb870 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6794e62640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6790159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6794e62640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6790159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46674/0 (socket says 172.21.15.64:46674) 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6794e62640 1 -- 172.21.15.64:0/2849105071 learned_addr learned my addr 172.21.15.64:0/2849105071 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f678ffff640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67900cb870 0x7f679013b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6794e62640 1 -- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67900cb870 msgr2=0x7f679013b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6795663640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67900c9730 0x7f67900c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6794e62640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67900cb870 0x7f679013b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6794e62640 1 -- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67900c9730 msgr2=0x7f67900c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6794e62640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67900c9730 0x7f67900c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.819+0000 7f6794e62640 1 -- 172.21.15.64:0/2849105071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679015aa10 con 0x7f6780004d00 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.820+0000 7f6794e62640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6790159820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f678400e7f0 tx=0x7f678400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.820+0000 7f678d7fa640 1 -- 172.21.15.64:0/2849105071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400eea0 con 0x7f6780004d00 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.820+0000 7f678d7fa640 1 -- 172.21.15.64:0/2849105071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678400c940 con 0x7f6780004d00 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.820+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6790159e00 con 0x7f6780004d00 2023-11-24T08:04:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.820+0000 7f678d7fa640 1 -- 172.21.15.64:0/2849105071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400ccd0 con 0x7f6780004d00 2023-11-24T08:04:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.820+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679015a3c0 con 0x7f6780004d00 2023-11-24T08:04:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.821+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679004e840 con 0x7f6780004d00 2023-11-24T08:04:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.822+0000 7f678d7fa640 1 -- 172.21.15.64:0/2849105071 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6784013470 con 0x7f6780004d00 2023-11-24T08:04:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.823+0000 7f678d7fa640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677406ae70 0x7f677406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.823+0000 7f678d7fa640 1 -- 172.21.15.64:0/2849105071 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f67840896d0 con 0x7f6780004d00 2023-11-24T08:04:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.824+0000 7f678ffff640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677406ae70 0x7f677406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.825+0000 7f678ffff640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677406ae70 0x7f677406d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f677c001eb0 tx=0x7f677c001d60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:39.827+0000 7f678d7fa640 1 -- 172.21.15.64:0/2849105071 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6784054780 con 0x7f6780004d00 2023-11-24T08:04:40.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.022+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67900b2460 con 0x7f6780004d00 2023-11-24T08:04:40.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.025+0000 7f678d7fa640 1 -- 172.21.15.64:0/2849105071 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6784057da0 con 0x7f6780004d00 2023-11-24T08:04:40.027 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:40.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677406ae70 msgr2=0x7f677406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:40.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677406ae70 0x7f677406d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f677c001eb0 tx=0x7f677c001d60 comp rx=0 tx=0).stop 2023-11-24T08:04:40.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 msgr2=0x7f6790159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:40.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6790159820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f678400e7f0 tx=0x7f678400ecb0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 shutdown_connections 2023-11-24T08:04:40.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677406ae70 0x7f677406d330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67900c9730 0x7f67900c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67900cb870 0x7f679013b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 --2- 172.21.15.64:0/2849105071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6780004d00 0x7f6790159820 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.027+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 >> 172.21.15.64:0/2849105071 conn(0x7f67900cec10 msgr2=0x7f679015b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.028+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 shutdown_connections 2023-11-24T08:04:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.028+0000 7f6795e64640 1 -- 172.21.15.64:0/2849105071 wait complete. 2023-11-24T08:04:40.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T08:04:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.274+0000 7ffb79bff640 1 Processor -- start 2023-11-24T08:04:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.274+0000 7ffb79bff640 1 -- start start 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.275+0000 7ffb79bff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb74143570 0x7ffb740cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb79bff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ffb74143950 0x7ffb740cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb79bff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 0x7ffb7415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb79bff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb74071700 con 0x7ffb74143570 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb79bff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7415b410 con 0x7ffb74143950 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb79bff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7415b130 con 0x7ffb7415e6b0 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb78bfd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb74143570 0x7ffb740cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb78bfd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb74143570 0x7ffb740cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46688/0 (socket says 172.21.15.64:46688) 2023-11-24T08:04:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb78bfd640 1 -- 172.21.15.64:0/2278757447 learned_addr learned my addr 172.21.15.64:0/2278757447 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:40.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb73fff640 1 --2- 172.21.15.64:0/2278757447 >> v2:172.21.15.91:3300/0 conn(0x7ffb74143950 0x7ffb740cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:40.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.276+0000 7ffb793fe640 1 --2- 172.21.15.64:0/2278757447 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 0x7ffb7415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:40.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.277+0000 7ffb78bfd640 1 -- 172.21.15.64:0/2278757447 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 msgr2=0x7ffb7415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:40.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.277+0000 7ffb78bfd640 1 --2- 172.21.15.64:0/2278757447 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 0x7ffb7415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.277+0000 7ffb78bfd640 1 -- 172.21.15.64:0/2278757447 >> v2:172.21.15.91:3300/0 conn(0x7ffb74143950 msgr2=0x7ffb740cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:40.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.277+0000 7ffb78bfd640 1 --2- 172.21.15.64:0/2278757447 >> v2:172.21.15.91:3300/0 conn(0x7ffb74143950 0x7ffb740cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.277+0000 7ffb78bfd640 1 -- 172.21.15.64:0/2278757447 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7415dbd0 con 0x7ffb74143570 2023-11-24T08:04:40.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.277+0000 7ffb73fff640 1 --2- 172.21.15.64:0/2278757447 >> v2:172.21.15.91:3300/0 conn(0x7ffb74143950 0x7ffb740cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:40.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.278+0000 7ffb78bfd640 1 --2- 172.21.15.64:0/2278757447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb740cf050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffb74070f10 tx=0x7ffb6802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6eaa6f79360a33 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.278+0000 7ffb737fe640 1 -- 172.21.15.64:0/2278757447 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6802ee00 con 0x7ffb74143570 2023-11-24T08:04:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.278+0000 7ffb737fe640 1 -- 172.21.15.64:0/2278757447 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb68036440 con 0x7ffb74143570 2023-11-24T08:04:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.278+0000 7ffb737fe640 1 -- 172.21.15.64:0/2278757447 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb680367d0 con 0x7ffb74143570 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 -- 172.21.15.64:0/2278757447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 msgr2=0x7ffb740cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 --2- 172.21.15.64:0/2278757447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb740cf050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffb74070f10 tx=0x7ffb6802e050 comp rx=0 tx=0).stop 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 -- 172.21.15.64:0/2278757447 shutdown_connections 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 --2- 172.21.15.64:0/2278757447 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 0x7ffb7415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 --2- 172.21.15.64:0/2278757447 >> v2:172.21.15.91:3300/0 conn(0x7ffb74143950 0x7ffb740cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 --2- 172.21.15.64:0/2278757447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb740cf050 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 -- 172.21.15.64:0/2278757447 >> 172.21.15.64:0/2278757447 conn(0x7ffb7406a860 msgr2=0x7ffb7406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 -- 172.21.15.64:0/2278757447 shutdown_connections 2023-11-24T08:04:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.279+0000 7ffb79bff640 1 -- 172.21.15.64:0/2278757447 wait complete. 2023-11-24T08:04:40.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.281+0000 7ffb79bff640 1 Processor -- start 2023-11-24T08:04:40.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.281+0000 7ffb79bff640 1 -- start start 2023-11-24T08:04:40.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb79bff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb74159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:40.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb79bff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffb74143950 0x7ffb74158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:40.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb79bff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 0x7ffb7413d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:40.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb79bff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74159150 con 0x7ffb74143570 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb79bff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7413d110 con 0x7ffb74143950 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb79bff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7413a610 con 0x7ffb7415e6b0 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb78bfd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb74159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb78bfd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb74159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46704/0 (socket says 172.21.15.64:46704) 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb78bfd640 1 -- 172.21.15.64:0/88534224 learned_addr learned my addr 172.21.15.64:0/88534224 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.282+0000 7ffb73fff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffb74143950 0x7ffb74158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb793fe640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 0x7ffb7413d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb78bfd640 1 -- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 msgr2=0x7ffb7413d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb78bfd640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 0x7ffb7413d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb78bfd640 1 -- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffb74143950 msgr2=0x7ffb74158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb78bfd640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffb74143950 0x7ffb74158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:40.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb78bfd640 1 -- 172.21.15.64:0/88534224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6800b770 con 0x7ffb74143570 2023-11-24T08:04:40.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb73fff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffb74143950 0x7ffb74158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:40.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb78bfd640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb74159310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffb6802e090 tx=0x7ffb680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:40.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb717fa640 1 -- 172.21.15.64:0/88534224 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68034850 con 0x7ffb74143570 2023-11-24T08:04:40.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb717fa640 1 -- 172.21.15.64:0/88534224 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb680349b0 con 0x7ffb74143570 2023-11-24T08:04:40.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7413a330 con 0x7ffb74143570 2023-11-24T08:04:40.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb717fa640 1 -- 172.21.15.64:0/88534224 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68034d60 con 0x7ffb74143570 2023-11-24T08:04:40.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.283+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7413cda0 con 0x7ffb74143570 2023-11-24T08:04:40.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.285+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7404e840 con 0x7ffb74143570 2023-11-24T08:04:40.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.286+0000 7ffb717fa640 1 -- 172.21.15.64:0/88534224 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb6803d040 con 0x7ffb74143570 2023-11-24T08:04:40.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.286+0000 7ffb717fa640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffb6006ae70 0x7ffb6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:40.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.287+0000 7ffb717fa640 1 -- 172.21.15.64:0/88534224 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ffb680ae4f0 con 0x7ffb74143570 2023-11-24T08:04:40.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.287+0000 7ffb73fff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffb6006ae70 0x7ffb6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:40.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.288+0000 7ffb73fff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffb6006ae70 0x7ffb6006d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb740cb2d0 tx=0x7ffb64008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:40.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.291+0000 7ffb717fa640 1 -- 172.21.15.64:0/88534224 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb68079470 con 0x7ffb74143570 2023-11-24T08:04:40.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:40.555+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7ffb740b3910 con 0x7ffb74143570 2023-11-24T08:04:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.032+0000 7ffb717fa640 1 -- 172.21.15.64:0/88534224 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v62) v1 ==== 184+0+0 (secure 0 0 0) 0x7ffb6807ca90 con 0x7ffb74143570 2023-11-24T08:04:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffb6006ae70 msgr2=0x7ffb6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffb6006ae70 0x7ffb6006d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb740cb2d0 tx=0x7ffb64008040 comp rx=0 tx=0).stop 2023-11-24T08:04:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 msgr2=0x7ffb74159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb74159310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffb6802e090 tx=0x7ffb680046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 shutdown_connections 2023-11-24T08:04:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffb6006ae70 0x7ffb6006d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffb7415e6b0 0x7ffb7413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffb74143950 0x7ffb74158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 --2- 172.21.15.64:0/88534224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb74143570 0x7ffb74159310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 >> 172.21.15.64:0/88534224 conn(0x7ffb7406a860 msgr2=0x7ffb740ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.035+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 shutdown_connections 2023-11-24T08:04:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.036+0000 7ffb79bff640 1 -- 172.21.15.64:0/88534224 wait complete. 2023-11-24T08:04:41.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:04:41.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.286+0000 7fd707fff640 1 Processor -- start 2023-11-24T08:04:41.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.287+0000 7fd707fff640 1 -- start start 2023-11-24T08:04:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd707fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd70806dae0 0x7fd70815b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd707fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd70815bbd0 0x7fd708070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd707fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd708144ca0 con 0x7fd7081448c0 2023-11-24T08:04:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd707fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70815c450 con 0x7fd70815bbd0 2023-11-24T08:04:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd707fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7080715b0 con 0x7fd70806dae0 2023-11-24T08:04:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd7077fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd70815bbd0 0x7fd708070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd7077fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd70815bbd0 0x7fd708070070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46714/0 (socket says 172.21.15.64:46714) 2023-11-24T08:04:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd7077fe640 1 -- 172.21.15.64:0/933251761 learned_addr learned my addr 172.21.15.64:0/933251761 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:41.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.289+0000 7fd6fffff640 1 --2- 172.21.15.64:0/933251761 >> v2:172.21.15.91:3300/0 conn(0x7fd70806dae0 0x7fd70815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:41.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.290+0000 7fd7077fe640 1 -- 172.21.15.64:0/933251761 >> v2:172.21.15.91:3300/0 conn(0x7fd70806dae0 msgr2=0x7fd70815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:41.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.290+0000 7fd7077fe640 1 --2- 172.21.15.64:0/933251761 >> v2:172.21.15.91:3300/0 conn(0x7fd70806dae0 0x7fd70815b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.290+0000 7fd7077fe640 1 -- 172.21.15.64:0/933251761 >> v1:172.21.15.91:6789/0 conn(0x7fd7081448c0 legacy=0x7fd70806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:41.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.290+0000 7fd7077fe640 1 -- 172.21.15.64:0/933251761 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd708071890 con 0x7fd70815bbd0 2023-11-24T08:04:41.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.291+0000 7fd7077fe640 1 --2- 172.21.15.64:0/933251761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd70815bbd0 0x7fd708070070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd7080b1a90 tx=0x7fd70002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33c4e1f0b6c2d2f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:41.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.291+0000 7fd7067fc640 1 -- 172.21.15.64:0/933251761 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70002ca80 con 0x7fd70815bbd0 2023-11-24T08:04:41.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.291+0000 7fd7067fc640 1 -- 172.21.15.64:0/933251761 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70002cbe0 con 0x7fd70815bbd0 2023-11-24T08:04:41.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.291+0000 7fd7067fc640 1 -- 172.21.15.64:0/933251761 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7000343f0 con 0x7fd70815bbd0 2023-11-24T08:04:41.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.292+0000 7fd707fff640 1 -- 172.21.15.64:0/933251761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd70815bbd0 msgr2=0x7fd708070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:41.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.292+0000 7fd707fff640 1 --2- 172.21.15.64:0/933251761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd70815bbd0 0x7fd708070070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd7080b1a90 tx=0x7fd70002bee0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.292+0000 7fd707fff640 1 -- 172.21.15.64:0/933251761 shutdown_connections 2023-11-24T08:04:41.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.292+0000 7fd707fff640 1 --2- 172.21.15.64:0/933251761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd70815bbd0 0x7fd708070070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.292+0000 7fd707fff640 1 --2- 172.21.15.64:0/933251761 >> v2:172.21.15.91:3300/0 conn(0x7fd70806dae0 0x7fd70815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.292+0000 7fd707fff640 1 -- 172.21.15.64:0/933251761 >> 172.21.15.64:0/933251761 conn(0x7fd7080b0aa0 msgr2=0x7fd7080b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:41.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.292+0000 7fd707fff640 1 -- 172.21.15.64:0/933251761 shutdown_connections 2023-11-24T08:04:41.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.292+0000 7fd707fff640 1 -- 172.21.15.64:0/933251761 wait complete. 2023-11-24T08:04:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.294+0000 7fd707fff640 1 Processor -- start 2023-11-24T08:04:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.294+0000 7fd707fff640 1 -- start start 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd707fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd70806dae0 0x7fd70813a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd707fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7081448c0 0x7fd708137640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd707fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd70815bbd0 0x7fd708143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd707fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708143400 con 0x7fd7081448c0 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd707fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081427f0 con 0x7fd70806dae0 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd707fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd708142ad0 con 0x7fd70815bbd0 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd6fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7081448c0 0x7fd708137640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd6fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7081448c0 0x7fd708137640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46716/0 (socket says 172.21.15.64:46716) 2023-11-24T08:04:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.295+0000 7fd6fffff640 1 -- 172.21.15.64:0/2251156649 learned_addr learned my addr 172.21.15.64:0/2251156649 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd706ffd640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd70806dae0 0x7fd70813a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd7077fe640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd70815bbd0 0x7fd708143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd6fffff640 1 -- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd70815bbd0 msgr2=0x7fd708143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd6fffff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd70815bbd0 0x7fd708143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd6fffff640 1 -- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd70806dae0 msgr2=0x7fd70813a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd6fffff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd70806dae0 0x7fd70813a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd6fffff640 1 -- 172.21.15.64:0/2251156649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700009b30 con 0x7fd7081448c0 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd7077fe640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd70815bbd0 0x7fd708143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.296+0000 7fd6fffff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7081448c0 0x7fd708137640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c7f0 tx=0x7fd6f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.297+0000 7fd704ff9640 1 -- 172.21.15.64:0/2251156649 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0007820 con 0x7fd7081448c0 2023-11-24T08:04:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.297+0000 7fd704ff9640 1 -- 172.21.15.64:0/2251156649 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f0007980 con 0x7fd7081448c0 2023-11-24T08:04:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.297+0000 7fd704ff9640 1 -- 172.21.15.64:0/2251156649 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0007d30 con 0x7fd7081448c0 2023-11-24T08:04:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.297+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7080724f0 con 0x7fd7081448c0 2023-11-24T08:04:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.297+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708143090 con 0x7fd7081448c0 2023-11-24T08:04:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.298+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70804e840 con 0x7fd7081448c0 2023-11-24T08:04:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.299+0000 7fd704ff9640 1 -- 172.21.15.64:0/2251156649 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6f00113f0 con 0x7fd7081448c0 2023-11-24T08:04:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.300+0000 7fd704ff9640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6e806ae40 0x7fd6e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.301+0000 7fd704ff9640 1 -- 172.21.15.64:0/2251156649 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd6f0089a20 con 0x7fd7081448c0 2023-11-24T08:04:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.301+0000 7fd706ffd640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6e806ae40 0x7fd6e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:41.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.302+0000 7fd706ffd640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6e806ae40 0x7fd6e806d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8007fd0 tx=0x7fd6f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:41.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.305+0000 7fd704ff9640 1 -- 172.21.15.64:0/2251156649 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6f0054a40 con 0x7fd7081448c0 2023-11-24T08:04:41.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.502+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd708141720 con 0x7fd7081448c0 2023-11-24T08:04:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.509+0000 7fd704ff9640 1 -- 172.21.15.64:0/2251156649 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd6f001f0c0 con 0x7fd7081448c0 2023-11-24T08:04:41.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6e806ae40 msgr2=0x7fd6e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:41.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6e806ae40 0x7fd6e806d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8007fd0 tx=0x7fd6f8009040 comp rx=0 tx=0).stop 2023-11-24T08:04:41.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7081448c0 msgr2=0x7fd708137640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:41.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7081448c0 0x7fd708137640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c7f0 tx=0x7fd6f000ccb0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 shutdown_connections 2023-11-24T08:04:41.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6e806ae40 0x7fd6e806d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd70815bbd0 0x7fd708143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7081448c0 0x7fd708137640 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 --2- 172.21.15.64:0/2251156649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd70806dae0 0x7fd70813a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:41.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 >> 172.21.15.64:0/2251156649 conn(0x7fd7080b0aa0 msgr2=0x7fd7080b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:41.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 shutdown_connections 2023-11-24T08:04:41.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:41.512+0000 7fd707fff640 1 -- 172.21.15.64:0/2251156649 wait complete. 2023-11-24T08:04:41.535 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:04:41.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:04:41.656 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:04:41.779 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:04:41.779 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:04:41.779 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:41.842 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:04:41.842 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:41.901 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:42.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.126+0000 7fa922b4e640 1 Processor -- start 2023-11-24T08:04:42.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.127+0000 7fa922b4e640 1 -- start start 2023-11-24T08:04:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.128+0000 7fa922b4e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c0cd320 0x7fa91c15d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.129+0000 7fa922b4e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa91c1431c0 0x7fa91c06dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.129+0000 7fa922b4e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c15e910 con 0x7fa91c15de50 2023-11-24T08:04:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.129+0000 7fa922b4e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c06bdf0 con 0x7fa91c1431c0 2023-11-24T08:04:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.129+0000 7fa922b4e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c15e370 con 0x7fa91c0cd320 2023-11-24T08:04:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.129+0000 7fa92234d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa91c1431c0 0x7fa91c06dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.129+0000 7fa92234d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa91c1431c0 0x7fa91c06dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46718/0 (socket says 172.21.15.64:46718) 2023-11-24T08:04:42.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.129+0000 7fa92234d640 1 -- 172.21.15.64:0/2306902351 learned_addr learned my addr 172.21.15.64:0/2306902351 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:42.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.129+0000 7fa921b4c640 1 --2- 172.21.15.64:0/2306902351 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c0cd320 0x7fa91c15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.130+0000 7fa92234d640 1 -- 172.21.15.64:0/2306902351 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c0cd320 msgr2=0x7fa91c15d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:42.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.130+0000 7fa92234d640 1 --2- 172.21.15.64:0/2306902351 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c0cd320 0x7fa91c15d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.130+0000 7fa92234d640 1 -- 172.21.15.64:0/2306902351 >> v1:172.21.15.91:6789/0 conn(0x7fa91c15de50 legacy=0x7fa91c142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.130+0000 7fa92234d640 1 -- 172.21.15.64:0/2306902351 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c15e630 con 0x7fa91c1431c0 2023-11-24T08:04:42.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.131+0000 7fa92234d640 1 --2- 172.21.15.64:0/2306902351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c06dda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa914009e50 tx=0x7fa91402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11749045b1bcfc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:42.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.131+0000 7fa920b4a640 1 -- 172.21.15.64:0/2306902351 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91402ca70 con 0x7fa91c1431c0 2023-11-24T08:04:42.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.131+0000 7fa920b4a640 1 -- 172.21.15.64:0/2306902351 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91402cbd0 con 0x7fa91c1431c0 2023-11-24T08:04:42.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.131+0000 7fa920b4a640 1 -- 172.21.15.64:0/2306902351 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9140343f0 con 0x7fa91c1431c0 2023-11-24T08:04:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.132+0000 7fa922b4e640 1 -- 172.21.15.64:0/2306902351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 msgr2=0x7fa91c06dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.132+0000 7fa922b4e640 1 --2- 172.21.15.64:0/2306902351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c06dda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa914009e50 tx=0x7fa91402bed0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.132+0000 7fa922b4e640 1 -- 172.21.15.64:0/2306902351 shutdown_connections 2023-11-24T08:04:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.132+0000 7fa922b4e640 1 --2- 172.21.15.64:0/2306902351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c06dda0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.132+0000 7fa922b4e640 1 --2- 172.21.15.64:0/2306902351 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c0cd320 0x7fa91c15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.132+0000 7fa922b4e640 1 -- 172.21.15.64:0/2306902351 >> 172.21.15.64:0/2306902351 conn(0x7fa91c0cbf60 msgr2=0x7fa91c0cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:42.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.132+0000 7fa922b4e640 1 -- 172.21.15.64:0/2306902351 shutdown_connections 2023-11-24T08:04:42.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.132+0000 7fa922b4e640 1 -- 172.21.15.64:0/2306902351 wait complete. 2023-11-24T08:04:42.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.135+0000 7fa922b4e640 1 Processor -- start 2023-11-24T08:04:42.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.135+0000 7fa922b4e640 1 -- start start 2023-11-24T08:04:42.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.136+0000 7fa922b4e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa91c0cd320 0x7fa91c140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.136+0000 7fa922b4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.136+0000 7fa922b4e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c15de50 0x7fa91c0adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.136+0000 7fa922b4e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c140550 con 0x7fa91c1431c0 2023-11-24T08:04:42.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.136+0000 7fa922b4e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c155f40 con 0x7fa91c0cd320 2023-11-24T08:04:42.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.136+0000 7fa922b4e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c156220 con 0x7fa91c15de50 2023-11-24T08:04:42.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92134b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92134b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c1556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46724/0 (socket says 172.21.15.64:46724) 2023-11-24T08:04:42.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92134b640 1 -- 172.21.15.64:0/1430659403 learned_addr learned my addr 172.21.15.64:0/1430659403 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:42.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa921b4c640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa91c0cd320 0x7fa91c140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92234d640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c15de50 0x7fa91c0adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92134b640 1 -- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c15de50 msgr2=0x7fa91c0adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92134b640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c15de50 0x7fa91c0adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92134b640 1 -- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa91c0cd320 msgr2=0x7fa91c140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92134b640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa91c0cd320 0x7fa91c140010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.137+0000 7fa92134b640 1 -- 172.21.15.64:0/1430659403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914009b30 con 0x7fa91c1431c0 2023-11-24T08:04:42.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.138+0000 7fa92134b640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c1556c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00c400 tx=0x7fa90c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:42.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.138+0000 7fa9127fc640 1 -- 172.21.15.64:0/1430659403 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0052d0 con 0x7fa91c1431c0 2023-11-24T08:04:42.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.138+0000 7fa9127fc640 1 -- 172.21.15.64:0/1430659403 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c005430 con 0x7fa91c1431c0 2023-11-24T08:04:42.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.138+0000 7fa9127fc640 1 -- 172.21.15.64:0/1430659403 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c003830 con 0x7fa91c1431c0 2023-11-24T08:04:42.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.138+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c156500 con 0x7fa91c1431c0 2023-11-24T08:04:42.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.138+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c155c00 con 0x7fa91c1431c0 2023-11-24T08:04:42.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.140+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c071ca0 con 0x7fa91c1431c0 2023-11-24T08:04:42.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.140+0000 7fa9127fc640 1 -- 172.21.15.64:0/1430659403 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa90c003990 con 0x7fa91c1431c0 2023-11-24T08:04:42.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.141+0000 7fa9127fc640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa90006ae40 0x7fa90006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.141+0000 7fa9127fc640 1 -- 172.21.15.64:0/1430659403 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa90c088940 con 0x7fa91c1431c0 2023-11-24T08:04:42.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.142+0000 7fa921b4c640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa90006ae40 0x7fa90006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.143+0000 7fa921b4c640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa90006ae40 0x7fa90006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa918009b40 tx=0x7fa918009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:42.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.146+0000 7fa9127fc640 1 -- 172.21.15.64:0/1430659403 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa90c053a80 con 0x7fa91c1431c0 2023-11-24T08:04:42.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.344+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa91c15e910 con 0x7fa91c1431c0 2023-11-24T08:04:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.346+0000 7fa9127fc640 1 -- 172.21.15.64:0/1430659403 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7fa91c15e910 con 0x7fa91c1431c0 2023-11-24T08:04:42.348 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:42.349 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.349+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa90006ae40 msgr2=0x7fa90006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.349+0000 7fa922b4e640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa90006ae40 0x7fa90006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa918009b40 tx=0x7fa918009190 comp rx=0 tx=0).stop 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.349+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 msgr2=0x7fa91c1556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.349+0000 7fa922b4e640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c1556c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00c400 tx=0x7fa90c00c8c0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.349+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 shutdown_connections 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.349+0000 7fa922b4e640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa90006ae40 0x7fa90006d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.349+0000 7fa922b4e640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa91c15de50 0x7fa91c0adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.349+0000 7fa922b4e640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa91c1431c0 0x7fa91c1556c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.350+0000 7fa922b4e640 1 --2- 172.21.15.64:0/1430659403 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa91c0cd320 0x7fa91c140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.350+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 >> 172.21.15.64:0/1430659403 conn(0x7fa91c0cbf60 msgr2=0x7fa91c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.350+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 shutdown_connections 2023-11-24T08:04:42.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.350+0000 7fa922b4e640 1 -- 172.21.15.64:0/1430659403 wait complete. 2023-11-24T08:04:42.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:42.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:42.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.600+0000 7f04f5485640 1 Processor -- start 2023-11-24T08:04:42.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.601+0000 7f04f5485640 1 -- start start 2023-11-24T08:04:42.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.602+0000 7f04f5485640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f04f015d3b0 0x7f04f00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.602+0000 7f04f5485640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f00c6bf0 con 0x7f04f00d0fa0 2023-11-24T08:04:42.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.602+0000 7f04f5485640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f00d1430 con 0x7f04f00d1d90 2023-11-24T08:04:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.602+0000 7f04f5485640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f00cedc0 con 0x7f04f015d3b0 2023-11-24T08:04:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.603+0000 7f04e7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f04f00d0fa0 0x7f04f00d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41994/0 (socket says 172.21.15.64:41994) 2023-11-24T08:04:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.603+0000 7f04e7fff640 1 -- 172.21.15.64:0/3519479664 learned_addr learned my addr 172.21.15.64:0/3519479664 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.603+0000 7f04effff640 1 --2- 172.21.15.64:0/3519479664 >> v2:172.21.15.91:3300/0 conn(0x7f04f015d3b0 0x7f04f00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.603+0000 7f04ef7fe640 1 -- 172.21.15.64:0/3519479664 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445258425 0 0) 0x7f04f00c6bf0 con 0x7f04f00d0fa0 2023-11-24T08:04:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.603+0000 7f04ef7fe640 1 -- 172.21.15.64:0/3519479664 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04dc003520 con 0x7f04f00d0fa0 2023-11-24T08:04:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.604+0000 7f04ef7fe640 1 -- 172.21.15.64:0/3519479664 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3625521663 0 0) 0x7f04dc003520 con 0x7f04f00d0fa0 2023-11-24T08:04:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.604+0000 7f04effff640 1 -- 172.21.15.64:0/3519479664 >> v1:172.21.15.91:6789/0 conn(0x7f04f00d1d90 legacy=0x7f04f00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.604+0000 7f04effff640 1 -- 172.21.15.64:0/3519479664 >> v1:172.21.15.64:6789/0 conn(0x7f04f00d0fa0 legacy=0x7f04f00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.604+0000 7f04effff640 1 -- 172.21.15.64:0/3519479664 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f00c6e40 con 0x7f04f015d3b0 2023-11-24T08:04:42.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.605+0000 7f04effff640 1 --2- 172.21.15.64:0/3519479664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f015d3b0 0x7f04f00ce6c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f04e00097a0 tx=0x7f04e002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e376cb7b8a7abfa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:42.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.606+0000 7f04ef7fe640 1 -- 172.21.15.64:0/3519479664 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8001670 con 0x7f04f015d3b0 2023-11-24T08:04:42.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.606+0000 7f04ef7fe640 1 -- 172.21.15.64:0/3519479664 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e002e050 con 0x7f04f015d3b0 2023-11-24T08:04:42.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.606+0000 7f04ef7fe640 1 -- 172.21.15.64:0/3519479664 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e0030710 con 0x7f04f015d3b0 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.606+0000 7f04f5485640 1 -- 172.21.15.64:0/3519479664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f015d3b0 msgr2=0x7f04f00ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.606+0000 7f04f5485640 1 --2- 172.21.15.64:0/3519479664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f015d3b0 0x7f04f00ce6c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f04e00097a0 tx=0x7f04e002b790 comp rx=0 tx=0).stop 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.606+0000 7f04f5485640 1 -- 172.21.15.64:0/3519479664 shutdown_connections 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.606+0000 7f04f5485640 1 --2- 172.21.15.64:0/3519479664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f015d3b0 0x7f04f00ce6c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.606+0000 7f04f5485640 1 -- 172.21.15.64:0/3519479664 >> 172.21.15.64:0/3519479664 conn(0x7f04f00d30a0 msgr2=0x7f04f00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.607+0000 7f04f5485640 1 -- 172.21.15.64:0/3519479664 shutdown_connections 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.607+0000 7f04f5485640 1 -- 172.21.15.64:0/3519479664 wait complete. 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.608+0000 7f04f5485640 1 Processor -- start 2023-11-24T08:04:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.609+0000 7f04f5485640 1 -- start start 2023-11-24T08:04:42.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04f5485640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f00d0fa0 0x7f04f01f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04f5485640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f00d1d90 0x7f04f00b8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04f5485640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04f015d3b0 0x7f04f0157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04f5485640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00b7cc0 con 0x7f04f00d0fa0 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04f5485640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00b8690 con 0x7f04f00d1d90 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04f5485640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f0158270 con 0x7f04f015d3b0 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f00d0fa0 0x7f04f01f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f00d0fa0 0x7f04f01f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46732/0 (socket says 172.21.15.64:46732) 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04effff640 1 -- 172.21.15.64:0/303884949 learned_addr learned my addr 172.21.15.64:0/303884949 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04e7fff640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f00d1d90 0x7f04f00b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04f4c84640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04f015d3b0 0x7f04f0157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04effff640 1 -- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04f015d3b0 msgr2=0x7f04f0157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04effff640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04f015d3b0 0x7f04f0157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04effff640 1 -- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f00d1d90 msgr2=0x7f04f00b8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04effff640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f00d1d90 0x7f04f00b8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04effff640 1 -- 172.21.15.64:0/303884949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e0009400 con 0x7f04f00d0fa0 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.610+0000 7f04f4c84640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04f015d3b0 0x7f04f0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.611+0000 7f04effff640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f00d0fa0 0x7f04f01f5d20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04e0000c00 tx=0x7f04e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.611+0000 7f04ed7fa640 1 -- 172.21.15.64:0/303884949 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e003d890 con 0x7f04f00d0fa0 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.611+0000 7f04ed7fa640 1 -- 172.21.15.64:0/303884949 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e003da10 con 0x7f04f00d0fa0 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.611+0000 7f04ed7fa640 1 -- 172.21.15.64:0/303884949 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e003ddc0 con 0x7f04f00d0fa0 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.611+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f00b70b0 con 0x7f04f00d0fa0 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.612+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f00b7670 con 0x7f04f00d0fa0 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.613+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f004e840 con 0x7f04f00d0fa0 2023-11-24T08:04:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.614+0000 7f04ed7fa640 1 -- 172.21.15.64:0/303884949 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04e0039040 con 0x7f04f00d0fa0 2023-11-24T08:04:42.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.614+0000 7f04ed7fa640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f04d006ae40 0x7f04d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:42.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.615+0000 7f04ed7fa640 1 -- 172.21.15.64:0/303884949 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f04e00ab6e0 con 0x7f04f00d0fa0 2023-11-24T08:04:42.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.615+0000 7f04e7fff640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f04d006ae40 0x7f04d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:42.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.616+0000 7f04e7fff640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f04d006ae40 0x7f04d006d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04f00d3f80 tx=0x7f04d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:42.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.619+0000 7f04ed7fa640 1 -- 172.21.15.64:0/303884949 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04e0076780 con 0x7f04f00d0fa0 2023-11-24T08:04:42.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.811+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04f006b750 con 0x7f04f00d0fa0 2023-11-24T08:04:42.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.812+0000 7f04ed7fa640 1 -- 172.21.15.64:0/303884949 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f04e004d0a0 con 0x7f04f00d0fa0 2023-11-24T08:04:42.814 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.814+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f04d006ae40 msgr2=0x7f04d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.814+0000 7f04f5485640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f04d006ae40 0x7f04d006d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04f00d3f80 tx=0x7f04d8009040 comp rx=0 tx=0).stop 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.814+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f00d0fa0 msgr2=0x7f04f01f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.814+0000 7f04f5485640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f00d0fa0 0x7f04f01f5d20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04e0000c00 tx=0x7f04e00046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.815+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 shutdown_connections 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.815+0000 7f04f5485640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f04d006ae40 0x7f04d006d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.815+0000 7f04f5485640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f04f015d3b0 0x7f04f0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.815+0000 7f04f5485640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f04f00d1d90 0x7f04f00b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.815+0000 7f04f5485640 1 --2- 172.21.15.64:0/303884949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f00d0fa0 0x7f04f01f5d20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.815+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 >> 172.21.15.64:0/303884949 conn(0x7f04f00d30a0 msgr2=0x7f04f015ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.815+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 shutdown_connections 2023-11-24T08:04:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:42.815+0000 7f04f5485640 1 -- 172.21.15.64:0/303884949 wait complete. 2023-11-24T08:04:42.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:43.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.044+0000 7faea359e640 1 Processor -- start 2023-11-24T08:04:43.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.045+0000 7faea359e640 1 -- start start 2023-11-24T08:04:43.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.046+0000 7faea359e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faea41439e0 0x7faea40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7faea359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7faea406a440 0x7faea406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7faea359e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea40b4dc0 con 0x7faea406d570 2023-11-24T08:04:43.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7faea359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea406ae30 con 0x7faea406a440 2023-11-24T08:04:43.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7faea359e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea4143dc0 con 0x7faea41439e0 2023-11-24T08:04:43.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7fae9bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7faea406a440 0x7faea406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7fae9bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7faea406a440 0x7faea406a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46744/0 (socket says 172.21.15.64:46744) 2023-11-24T08:04:43.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7fae9bfff640 1 -- 172.21.15.64:0/582167480 learned_addr learned my addr 172.21.15.64:0/582167480 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:43.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7faea259c640 1 --2- 172.21.15.64:0/582167480 >> v2:172.21.15.91:3300/0 conn(0x7faea41439e0 0x7faea40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7fae9bfff640 1 -- 172.21.15.64:0/582167480 >> v2:172.21.15.91:3300/0 conn(0x7faea41439e0 msgr2=0x7faea40b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:43.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7fae9bfff640 1 --2- 172.21.15.64:0/582167480 >> v2:172.21.15.91:3300/0 conn(0x7faea41439e0 0x7faea40b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7fae9bfff640 1 -- 172.21.15.64:0/582167480 >> v1:172.21.15.91:6789/0 conn(0x7faea406d570 legacy=0x7faea406d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:43.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.047+0000 7fae9bfff640 1 -- 172.21.15.64:0/582167480 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4161350 con 0x7faea406a440 2023-11-24T08:04:43.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.049+0000 7fae9bfff640 1 --2- 172.21.15.64:0/582167480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea406a440 0x7faea406a8f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faea406c4c0 tx=0x7fae8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bc471e20d34dce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.049+0000 7faea1d9b640 1 -- 172.21.15.64:0/582167480 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c031870 con 0x7faea406a440 2023-11-24T08:04:43.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.049+0000 7faea1d9b640 1 -- 172.21.15.64:0/582167480 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c0319d0 con 0x7faea406a440 2023-11-24T08:04:43.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.049+0000 7faea1d9b640 1 -- 172.21.15.64:0/582167480 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c031d60 con 0x7faea406a440 2023-11-24T08:04:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.049+0000 7faea359e640 1 -- 172.21.15.64:0/582167480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea406a440 msgr2=0x7faea406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.049+0000 7faea359e640 1 --2- 172.21.15.64:0/582167480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea406a440 0x7faea406a8f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faea406c4c0 tx=0x7fae8c02b910 comp rx=0 tx=0).stop 2023-11-24T08:04:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.050+0000 7faea359e640 1 -- 172.21.15.64:0/582167480 shutdown_connections 2023-11-24T08:04:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.050+0000 7faea359e640 1 --2- 172.21.15.64:0/582167480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea406a440 0x7faea406a8f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.050+0000 7faea359e640 1 --2- 172.21.15.64:0/582167480 >> v2:172.21.15.91:3300/0 conn(0x7faea41439e0 0x7faea40b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.050+0000 7faea359e640 1 -- 172.21.15.64:0/582167480 >> 172.21.15.64:0/582167480 conn(0x7faea406bbb0 msgr2=0x7faea40b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.050+0000 7faea359e640 1 -- 172.21.15.64:0/582167480 shutdown_connections 2023-11-24T08:04:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.050+0000 7faea359e640 1 -- 172.21.15.64:0/582167480 wait complete. 2023-11-24T08:04:43.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.052+0000 7faea359e640 1 Processor -- start 2023-11-24T08:04:43.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.052+0000 7faea359e640 1 -- start start 2023-11-24T08:04:43.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea359e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faea406a440 0x7faea40b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faea406d570 0x7faea41448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea41439e0 0x7faea4144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea4134220 con 0x7faea41439e0 2023-11-24T08:04:43.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea359e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea4131390 con 0x7faea406a440 2023-11-24T08:04:43.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea4145490 con 0x7faea406d570 2023-11-24T08:04:43.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea41439e0 0x7faea4144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea41439e0 0x7faea4144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46758/0 (socket says 172.21.15.64:46758) 2023-11-24T08:04:43.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea2d9d640 1 -- 172.21.15.64:0/2212025407 learned_addr learned my addr 172.21.15.64:0/2212025407 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:43.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7faea259c640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faea406a440 0x7faea40b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.053+0000 7fae9bfff640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faea406d570 0x7faea41448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7faea2d9d640 1 -- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faea406d570 msgr2=0x7faea41448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7faea2d9d640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faea406d570 0x7faea41448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7faea2d9d640 1 -- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faea406a440 msgr2=0x7faea40b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7faea2d9d640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faea406a440 0x7faea40b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7faea2d9d640 1 -- 172.21.15.64:0/2212025407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c0094d0 con 0x7faea41439e0 2023-11-24T08:04:43.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7fae9bfff640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faea406d570 0x7faea41448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:43.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7faea2d9d640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea41439e0 0x7faea4144e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00c780 tx=0x7fae9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7fae9affd640 1 -- 172.21.15.64:0/2212025407 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c004b60 con 0x7faea41439e0 2023-11-24T08:04:43.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7fae9affd640 1 -- 172.21.15.64:0/2212025407 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c00ce90 con 0x7faea41439e0 2023-11-24T08:04:43.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7fae9affd640 1 -- 172.21.15.64:0/2212025407 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00a850 con 0x7faea41439e0 2023-11-24T08:04:43.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea4145770 con 0x7faea41439e0 2023-11-24T08:04:43.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.054+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea4131930 con 0x7faea41439e0 2023-11-24T08:04:43.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.056+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea4131b20 con 0x7faea41439e0 2023-11-24T08:04:43.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.056+0000 7fae9affd640 1 -- 172.21.15.64:0/2212025407 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae9c00a9b0 con 0x7faea41439e0 2023-11-24T08:04:43.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.057+0000 7fae9affd640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae8406ae40 0x7fae8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.058+0000 7fae9affd640 1 -- 172.21.15.64:0/2212025407 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fae9c089900 con 0x7faea41439e0 2023-11-24T08:04:43.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.058+0000 7faea259c640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae8406ae40 0x7fae8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.059+0000 7faea259c640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae8406ae40 0x7fae8406d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae94006fd0 tx=0x7fae94008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.062+0000 7fae9affd640 1 -- 172.21.15.64:0/2212025407 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae9c054880 con 0x7faea41439e0 2023-11-24T08:04:43.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.257+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faea406fb90 con 0x7faea41439e0 2023-11-24T08:04:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.258+0000 7fae9affd640 1 -- 172.21.15.64:0/2212025407 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7faea406fb90 con 0x7faea41439e0 2023-11-24T08:04:43.260 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:43.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae8406ae40 msgr2=0x7fae8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae8406ae40 0x7fae8406d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae94006fd0 tx=0x7fae94008040 comp rx=0 tx=0).stop 2023-11-24T08:04:43.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea41439e0 msgr2=0x7faea4144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea41439e0 0x7faea4144e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00c780 tx=0x7fae9c00cc40 comp rx=0 tx=0).stop 2023-11-24T08:04:43.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 shutdown_connections 2023-11-24T08:04:43.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae8406ae40 0x7fae8406d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea41439e0 0x7faea4144e10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faea406d570 0x7faea41448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 --2- 172.21.15.64:0/2212025407 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faea406a440 0x7faea40b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 >> 172.21.15.64:0/2212025407 conn(0x7faea406bbb0 msgr2=0x7faea406f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:43.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 shutdown_connections 2023-11-24T08:04:43.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.261+0000 7faea359e640 1 -- 172.21.15.64:0/2212025407 wait complete. 2023-11-24T08:04:43.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.509+0000 7f8a732e0640 1 Processor -- start 2023-11-24T08:04:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.509+0000 7f8a732e0640 1 -- start start 2023-11-24T08:04:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a732e0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 0x7f8a6c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a732e0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8a6c09cb30 0x7f8a6c0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a732e0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c15b730 con 0x7f8a6c0adb60 2023-11-24T08:04:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a732e0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6c0ac740 con 0x7f8a6c09cb30 2023-11-24T08:04:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a732e0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0ac460 con 0x7f8a6c15b350 2023-11-24T08:04:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a7087b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8a6c09cb30 0x7f8a6c0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a7187d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8a6c0adb60 0x7f8a6c0adfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42002/0 (socket says 172.21.15.64:42002) 2023-11-24T08:04:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a7187d640 1 -- 172.21.15.64:0/1592264559 learned_addr learned my addr 172.21.15.64:0/1592264559 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.511+0000 7f8a7107c640 1 --2- 172.21.15.64:0/1592264559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 0x7f8a6c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.512+0000 7f8a7087b640 1 -- 172.21.15.64:0/1592264559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 msgr2=0x7f8a6c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.512+0000 7f8a7087b640 1 --2- 172.21.15.64:0/1592264559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 0x7f8a6c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.512+0000 7f8a7087b640 1 -- 172.21.15.64:0/1592264559 >> v1:172.21.15.64:6789/0 conn(0x7f8a6c0adb60 legacy=0x7f8a6c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.512+0000 7f8a7087b640 1 -- 172.21.15.64:0/1592264559 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c0b2a50 con 0x7f8a6c09cb30 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.512+0000 7f8a5ffff640 1 -- 172.21.15.64:0/1592264559 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164920458 0 0) 0x7f8a64001490 con 0x7f8a6c0adb60 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.513+0000 7f8a7087b640 1 --2- 172.21.15.64:0/1592264559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c09cb30 0x7f8a6c0b3530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8a680097a0 tx=0x7f8a6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd89a7d77ae54d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.513+0000 7f8a5ffff640 1 -- 172.21.15.64:0/1592264559 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68031830 con 0x7f8a6c09cb30 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.513+0000 7f8a5ffff640 1 -- 172.21.15.64:0/1592264559 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68031990 con 0x7f8a6c09cb30 2023-11-24T08:04:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.513+0000 7f8a5ffff640 1 -- 172.21.15.64:0/1592264559 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68031d20 con 0x7f8a6c09cb30 2023-11-24T08:04:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.514+0000 7f8a732e0640 1 -- 172.21.15.64:0/1592264559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c09cb30 msgr2=0x7f8a6c0b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.514+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1592264559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c09cb30 0x7f8a6c0b3530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8a680097a0 tx=0x7f8a6802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.514+0000 7f8a732e0640 1 -- 172.21.15.64:0/1592264559 shutdown_connections 2023-11-24T08:04:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.514+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1592264559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c09cb30 0x7f8a6c0b3530 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.514+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1592264559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 0x7f8a6c0abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.514+0000 7f8a732e0640 1 -- 172.21.15.64:0/1592264559 >> 172.21.15.64:0/1592264559 conn(0x7f8a6c0b0ba0 msgr2=0x7f8a6c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.514+0000 7f8a732e0640 1 -- 172.21.15.64:0/1592264559 shutdown_connections 2023-11-24T08:04:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.514+0000 7f8a732e0640 1 -- 172.21.15.64:0/1592264559 wait complete. 2023-11-24T08:04:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.516+0000 7f8a732e0640 1 Processor -- start 2023-11-24T08:04:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.516+0000 7f8a732e0640 1 -- start start 2023-11-24T08:04:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a732e0640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a6c09cb30 0x7f8a6c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a732e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0adb60 0x7f8a6c1418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a732e0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 0x7f8a6c141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a732e0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c1f78a0 con 0x7f8a6c0adb60 2023-11-24T08:04:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a732e0640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c1f75c0 con 0x7f8a6c09cb30 2023-11-24T08:04:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a732e0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c156570 con 0x7f8a6c15b350 2023-11-24T08:04:43.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a7087b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0adb60 0x7f8a6c1418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a7087b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0adb60 0x7f8a6c1418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46780/0 (socket says 172.21.15.64:46780) 2023-11-24T08:04:43.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.517+0000 7f8a7087b640 1 -- 172.21.15.64:0/1169203887 learned_addr learned my addr 172.21.15.64:0/1169203887 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a7087b640 1 -- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 msgr2=0x7f8a6c141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a7087b640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 0x7f8a6c141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a7107c640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a6c09cb30 0x7f8a6c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a7087b640 1 -- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a6c09cb30 msgr2=0x7f8a6c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a7087b640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a6c09cb30 0x7f8a6c06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a7087b640 1 -- 172.21.15.64:0/1169203887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68009400 con 0x7f8a6c0adb60 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a7087b640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0adb60 0x7f8a6c1418e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8a6802bee0 tx=0x7f8a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a5dffb640 1 -- 172.21.15.64:0/1169203887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6803a6f0 con 0x7f8a6c0adb60 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a5dffb640 1 -- 172.21.15.64:0/1169203887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6803a870 con 0x7f8a6c0adb60 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a5dffb640 1 -- 172.21.15.64:0/1169203887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6803ac20 con 0x7f8a6c0adb60 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c156850 con 0x7f8a6c0adb60 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.518+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c06e2c0 con 0x7f8a6c0adb60 2023-11-24T08:04:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.520+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c09b720 con 0x7f8a6c0adb60 2023-11-24T08:04:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.521+0000 7f8a5dffb640 1 -- 172.21.15.64:0/1169203887 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a68030380 con 0x7f8a6c0adb60 2023-11-24T08:04:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.521+0000 7f8a5dffb640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a5006ae40 0x7f8a5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.522+0000 7f8a5dffb640 1 -- 172.21.15.64:0/1169203887 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8a680ab110 con 0x7f8a6c0adb60 2023-11-24T08:04:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.522+0000 7f8a7107c640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a5006ae40 0x7f8a5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.523+0000 7f8a7107c640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a5006ae40 0x7f8a5006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.526+0000 7f8a5dffb640 1 -- 172.21.15.64:0/1169203887 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a68076120 con 0x7f8a6c0adb60 2023-11-24T08:04:43.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.719+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a6c1563d0 con 0x7f8a6c0adb60 2023-11-24T08:04:43.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.720+0000 7f8a5dffb640 1 -- 172.21.15.64:0/1169203887 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a6c1563d0 con 0x7f8a6c0adb60 2023-11-24T08:04:43.722 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:43.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a5006ae40 msgr2=0x7f8a5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a5006ae40 0x7f8a5006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60008040 comp rx=0 tx=0).stop 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0adb60 msgr2=0x7f8a6c1418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0adb60 0x7f8a6c1418e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8a6802bee0 tx=0x7f8a680046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 shutdown_connections 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a5006ae40 0x7f8a5006d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8a6c15b350 0x7f8a6c141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0adb60 0x7f8a6c1418e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 --2- 172.21.15.64:0/1169203887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8a6c09cb30 0x7f8a6c06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.723+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 >> 172.21.15.64:0/1169203887 conn(0x7f8a6c0b0ba0 msgr2=0x7f8a6c0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.724+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 shutdown_connections 2023-11-24T08:04:43.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.724+0000 7f8a732e0640 1 -- 172.21.15.64:0/1169203887 wait complete. 2023-11-24T08:04:43.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:04:43.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.972+0000 7f5708978640 1 Processor -- start 2023-11-24T08:04:43.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.972+0000 7f5708978640 1 -- start start 2023-11-24T08:04:43.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.974+0000 7f5708978640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f570415d770 0x7f57040d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.974+0000 7f5708978640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f57040d3980 0x7f57040b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.974+0000 7f5708978640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57040d0b40 con 0x7f57040d2740 2023-11-24T08:04:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.974+0000 7f5708978640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570415db50 con 0x7f570415d770 2023-11-24T08:04:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.974+0000 7f5708978640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57040d0860 con 0x7f57040d3980 2023-11-24T08:04:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.974+0000 7f570259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f570415d770 0x7f57040d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.975+0000 7f570259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f570415d770 0x7f57040d2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46786/0 (socket says 172.21.15.64:46786) 2023-11-24T08:04:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.975+0000 7f570259c640 1 -- 172.21.15.64:0/3134185261 learned_addr learned my addr 172.21.15.64:0/3134185261 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.975+0000 7f5702d9d640 1 --2- 172.21.15.64:0/3134185261 >> v2:172.21.15.91:3300/0 conn(0x7f57040d3980 0x7f57040b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.975+0000 7f570259c640 1 -- 172.21.15.64:0/3134185261 >> v2:172.21.15.91:3300/0 conn(0x7f57040d3980 msgr2=0x7f57040b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.975+0000 7f570259c640 1 --2- 172.21.15.64:0/3134185261 >> v2:172.21.15.91:3300/0 conn(0x7f57040d3980 0x7f57040b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.975+0000 7f570259c640 1 -- 172.21.15.64:0/3134185261 >> v1:172.21.15.91:6789/0 conn(0x7f57040d2740 legacy=0x7f57040d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.975+0000 7f570259c640 1 -- 172.21.15.64:0/3134185261 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040d1720 con 0x7f570415d770 2023-11-24T08:04:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.976+0000 7f570259c640 1 --2- 172.21.15.64:0/3134185261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570415d770 0x7f57040d2200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57040ce2f0 tx=0x7f56f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a7b5a59c6f847d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.977+0000 7f570159a640 1 -- 172.21.15.64:0/3134185261 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4031870 con 0x7f570415d770 2023-11-24T08:04:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.977+0000 7f570159a640 1 -- 172.21.15.64:0/3134185261 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f40319d0 con 0x7f570415d770 2023-11-24T08:04:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.977+0000 7f570159a640 1 -- 172.21.15.64:0/3134185261 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4031d60 con 0x7f570415d770 2023-11-24T08:04:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.977+0000 7f5708978640 1 -- 172.21.15.64:0/3134185261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570415d770 msgr2=0x7f57040d2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.977+0000 7f5708978640 1 --2- 172.21.15.64:0/3134185261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570415d770 0x7f57040d2200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57040ce2f0 tx=0x7f56f402b910 comp rx=0 tx=0).stop 2023-11-24T08:04:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.977+0000 7f5708978640 1 -- 172.21.15.64:0/3134185261 shutdown_connections 2023-11-24T08:04:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.977+0000 7f5708978640 1 --2- 172.21.15.64:0/3134185261 >> v2:172.21.15.91:3300/0 conn(0x7f57040d3980 0x7f57040b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.977+0000 7f5708978640 1 --2- 172.21.15.64:0/3134185261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570415d770 0x7f57040d2200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.978+0000 7f5708978640 1 -- 172.21.15.64:0/3134185261 >> 172.21.15.64:0/3134185261 conn(0x7f57040721a0 msgr2=0x7f570406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:43.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.978+0000 7f5708978640 1 -- 172.21.15.64:0/3134185261 shutdown_connections 2023-11-24T08:04:43.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.978+0000 7f5708978640 1 -- 172.21.15.64:0/3134185261 wait complete. 2023-11-24T08:04:43.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.979+0000 7f5708978640 1 Processor -- start 2023-11-24T08:04:43.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.980+0000 7f5708978640 1 -- start start 2023-11-24T08:04:43.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f5708978640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57040d3980 0x7f570414e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f5708978640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f570415d770 0x7f57040c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f5708978640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f570415e3e0 0x7f57041f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f5708978640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570415f220 con 0x7f57040d3980 2023-11-24T08:04:43.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f5708978640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040c8470 con 0x7f570415d770 2023-11-24T08:04:43.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f5708978640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570415e980 con 0x7f570415e3e0 2023-11-24T08:04:43.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f570259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57040d3980 0x7f570414e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f570259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57040d3980 0x7f570414e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46792/0 (socket says 172.21.15.64:46792) 2023-11-24T08:04:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.981+0000 7f570259c640 1 -- 172.21.15.64:0/3629276025 learned_addr learned my addr 172.21.15.64:0/3629276025 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.982+0000 7f5702d9d640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f570415e3e0 0x7f57041f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.982+0000 7f570259c640 1 -- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f570415e3e0 msgr2=0x7f57041f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.982+0000 7f570259c640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f570415e3e0 0x7f57041f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.982+0000 7f570259c640 1 -- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f570415d770 msgr2=0x7f57040c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.982+0000 7f570259c640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f570415d770 0x7f57040c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.982+0000 7f570259c640 1 -- 172.21.15.64:0/3629276025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f40094d0 con 0x7f57040d3980 2023-11-24T08:04:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.982+0000 7f570259c640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57040d3980 0x7f570414e0c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f56f4000c00 tx=0x7f56f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.982+0000 7f56f2ffd640 1 -- 172.21.15.64:0/3629276025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f403abe0 con 0x7f57040d3980 2023-11-24T08:04:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.983+0000 7f56f2ffd640 1 -- 172.21.15.64:0/3629276025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f403ad40 con 0x7f57040d3980 2023-11-24T08:04:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.983+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570415ec60 con 0x7f57040d3980 2023-11-24T08:04:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.983+0000 7f56f2ffd640 1 -- 172.21.15.64:0/3629276025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4044630 con 0x7f57040d3980 2023-11-24T08:04:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.983+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57040c8680 con 0x7f57040d3980 2023-11-24T08:04:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.984+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5704067c50 con 0x7f57040d3980 2023-11-24T08:04:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.985+0000 7f56f2ffd640 1 -- 172.21.15.64:0/3629276025 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56f404d450 con 0x7f57040d3980 2023-11-24T08:04:43.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.986+0000 7f56f2ffd640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56e406ae70 0x7f56e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:43.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.987+0000 7f56f2ffd640 1 -- 172.21.15.64:0/3629276025 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f56f402d040 con 0x7f57040d3980 2023-11-24T08:04:43.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.987+0000 7f5701d9b640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56e406ae70 0x7f56e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:43.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.988+0000 7f5701d9b640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56e406ae70 0x7f56e406d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f570415e040 tx=0x7f56f80079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:43.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:43.991+0000 7f56f2ffd640 1 -- 172.21.15.64:0/3629276025 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56f4076c90 con 0x7f57040d3980 2023-11-24T08:04:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.201+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5704074770 con 0x7f57040d3980 2023-11-24T08:04:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.202+0000 7f56f2ffd640 1 -- 172.21.15.64:0/3629276025 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f56f4035020 con 0x7f57040d3980 2023-11-24T08:04:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56e406ae70 msgr2=0x7f56e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56e406ae70 0x7f56e406d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f570415e040 tx=0x7f56f80079b0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57040d3980 msgr2=0x7f570414e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57040d3980 0x7f570414e0c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f56f4000c00 tx=0x7f56f40046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 shutdown_connections 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56e406ae70 0x7f56e406d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f570415e3e0 0x7f57041f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f570415d770 0x7f57040c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 --2- 172.21.15.64:0/3629276025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57040d3980 0x7f570414e0c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 >> 172.21.15.64:0/3629276025 conn(0x7f57040721a0 msgr2=0x7f57040b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 shutdown_connections 2023-11-24T08:04:44.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.205+0000 7f5708978640 1 -- 172.21.15.64:0/3629276025 wait complete. 2023-11-24T08:04:44.227 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:04:44.227 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:44.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.525+0000 7f3f3bb61640 1 Processor -- start 2023-11-24T08:04:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.525+0000 7f3f3bb61640 1 -- start start 2023-11-24T08:04:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.527+0000 7f3f3bb61640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f34153f10 0x7f3f3415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.527+0000 7f3f3bb61640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3f3415b050 0x7f3f340b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.527+0000 7f3f3bb61640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f340ad6b0 con 0x7f3f3406ddb0 2023-11-24T08:04:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.527+0000 7f3f3bb61640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f340b25c0 con 0x7f3f34153f10 2023-11-24T08:04:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.527+0000 7f3f3bb61640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f340b2860 con 0x7f3f3415b050 2023-11-24T08:04:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.527+0000 7f3f398fd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f34153f10 0x7f3f3415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.528+0000 7f3f398fd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f34153f10 0x7f3f3415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46806/0 (socket says 172.21.15.64:46806) 2023-11-24T08:04:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.528+0000 7f3f398fd640 1 -- 172.21.15.64:0/348727648 learned_addr learned my addr 172.21.15.64:0/348727648 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.528+0000 7f3f390fc640 1 --2- 172.21.15.64:0/348727648 >> v2:172.21.15.91:3300/0 conn(0x7f3f3415b050 0x7f3f340b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.528+0000 7f3f398fd640 1 -- 172.21.15.64:0/348727648 >> v2:172.21.15.91:3300/0 conn(0x7f3f3415b050 msgr2=0x7f3f340b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.528+0000 7f3f398fd640 1 --2- 172.21.15.64:0/348727648 >> v2:172.21.15.91:3300/0 conn(0x7f3f3415b050 0x7f3f340b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.528+0000 7f3f398fd640 1 -- 172.21.15.64:0/348727648 >> v1:172.21.15.91:6789/0 conn(0x7f3f3406ddb0 legacy=0x7f3f3415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:44.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.528+0000 7f3f398fd640 1 -- 172.21.15.64:0/348727648 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340b2b40 con 0x7f3f34153f10 2023-11-24T08:04:44.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.529+0000 7f3f398fd640 1 --2- 172.21.15.64:0/348727648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34153f10 0x7f3f3415aaa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3f3406e9a0 tx=0x7f3f3002deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a7e8d00eae10c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:44.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.530+0000 7f3f388fb640 1 -- 172.21.15.64:0/348727648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3002ec60 con 0x7f3f34153f10 2023-11-24T08:04:44.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.530+0000 7f3f388fb640 1 -- 172.21.15.64:0/348727648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30032070 con 0x7f3f34153f10 2023-11-24T08:04:44.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.530+0000 7f3f388fb640 1 -- 172.21.15.64:0/348727648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30038400 con 0x7f3f34153f10 2023-11-24T08:04:44.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.531+0000 7f3f3bb61640 1 -- 172.21.15.64:0/348727648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34153f10 msgr2=0x7f3f3415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.531+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/348727648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34153f10 0x7f3f3415aaa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3f3406e9a0 tx=0x7f3f3002deb0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.531+0000 7f3f3bb61640 1 -- 172.21.15.64:0/348727648 shutdown_connections 2023-11-24T08:04:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.531+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/348727648 >> v2:172.21.15.91:3300/0 conn(0x7f3f3415b050 0x7f3f340b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.531+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/348727648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34153f10 0x7f3f3415aaa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.531+0000 7f3f3bb61640 1 -- 172.21.15.64:0/348727648 >> 172.21.15.64:0/348727648 conn(0x7f3f340b0d80 msgr2=0x7f3f340b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.531+0000 7f3f3bb61640 1 -- 172.21.15.64:0/348727648 shutdown_connections 2023-11-24T08:04:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.531+0000 7f3f3bb61640 1 -- 172.21.15.64:0/348727648 wait complete. 2023-11-24T08:04:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.533+0000 7f3f3bb61640 1 Processor -- start 2023-11-24T08:04:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.533+0000 7f3f3bb61640 1 -- start start 2023-11-24T08:04:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f3bb61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3406ddb0 0x7f3f341499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f3bb61640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f34153f10 0x7f3f340cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f3bb61640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3415b050 0x7f3f3406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f3bb61640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f341e44a0 con 0x7f3f3406ddb0 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f3bb61640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3414a4a0 con 0x7f3f34153f10 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f3bb61640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3414a780 con 0x7f3f3415b050 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f398fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3406ddb0 0x7f3f341499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f398fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3406ddb0 0x7f3f341499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46812/0 (socket says 172.21.15.64:46812) 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f398fd640 1 -- 172.21.15.64:0/2249123028 learned_addr learned my addr 172.21.15.64:0/2249123028 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f390fc640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f34153f10 0x7f3f340cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f3a0fe640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3415b050 0x7f3f3406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.535+0000 7f3f398fd640 1 -- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3415b050 msgr2=0x7f3f3406ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f398fd640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3415b050 0x7f3f3406ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f398fd640 1 -- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f34153f10 msgr2=0x7f3f340cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f398fd640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f34153f10 0x7f3f340cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f398fd640 1 -- 172.21.15.64:0/2249123028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3000b760 con 0x7f3f3406ddb0 2023-11-24T08:04:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f3a0fe640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3415b050 0x7f3f3406ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f398fd640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3406ddb0 0x7f3f341499f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3f30000c00 tx=0x7f3f300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f2a7fc640 1 -- 172.21.15.64:0/2249123028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30038d40 con 0x7f3f3406ddb0 2023-11-24T08:04:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f2a7fc640 1 -- 172.21.15.64:0/2249123028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f300423f0 con 0x7f3f3406ddb0 2023-11-24T08:04:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3414aa60 con 0x7f3f3406ddb0 2023-11-24T08:04:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.536+0000 7f3f2a7fc640 1 -- 172.21.15.64:0/2249123028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30038ea0 con 0x7f3f3406ddb0 2023-11-24T08:04:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.537+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3414fb10 con 0x7f3f3406ddb0 2023-11-24T08:04:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.538+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f34074770 con 0x7f3f3406ddb0 2023-11-24T08:04:44.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.540+0000 7f3f2a7fc640 1 -- 172.21.15.64:0/2249123028 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f300427a0 con 0x7f3f3406ddb0 2023-11-24T08:04:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.540+0000 7f3f2a7fc640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1806ae70 0x7f3f1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.541+0000 7f3f2a7fc640 1 -- 172.21.15.64:0/2249123028 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3f300abcb0 con 0x7f3f3406ddb0 2023-11-24T08:04:44.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.541+0000 7f3f390fc640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1806ae70 0x7f3f1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.542+0000 7f3f390fc640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1806ae70 0x7f3f1806d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3f24006fd0 tx=0x7f3f24008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:44.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.547+0000 7f3f2a7fc640 1 -- 172.21.15.64:0/2249123028 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f30076c30 con 0x7f3f3406ddb0 2023-11-24T08:04:44.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.744+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f34074770 con 0x7f3f3406ddb0 2023-11-24T08:04:44.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.747+0000 7f3f2a7fc640 1 -- 172.21.15.64:0/2249123028 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7f3f3007a250 con 0x7f3f3406ddb0 2023-11-24T08:04:44.749 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:44.749 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:44.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.749+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1806ae70 msgr2=0x7f3f1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.749+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1806ae70 0x7f3f1806d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3f24006fd0 tx=0x7f3f24008040 comp rx=0 tx=0).stop 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.749+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3406ddb0 msgr2=0x7f3f341499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.749+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3406ddb0 0x7f3f341499f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3f30000c00 tx=0x7f3f300046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.750+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 shutdown_connections 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.750+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1806ae70 0x7f3f1806d330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.750+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3415b050 0x7f3f3406ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.750+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f34153f10 0x7f3f340cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.750+0000 7f3f3bb61640 1 --2- 172.21.15.64:0/2249123028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3406ddb0 0x7f3f341499f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.750+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 >> 172.21.15.64:0/2249123028 conn(0x7f3f340b0d80 msgr2=0x7f3f3414c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:44.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.750+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 shutdown_connections 2023-11-24T08:04:44.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.750+0000 7f3f3bb61640 1 -- 172.21.15.64:0/2249123028 wait complete. 2023-11-24T08:04:44.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:44.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:44.772 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:04:44.787 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:04:44.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.974+0000 7f1c0ffff640 1 Processor -- start 2023-11-24T08:04:44.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.974+0000 7f1c0ffff640 1 -- start start 2023-11-24T08:04:44.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.976+0000 7f1c0ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1c101618d0 0x7f1c10070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.976+0000 7f1c0ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1c10071310 0x7f1c1006a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.976+0000 7f1c0ffff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c100b3290 con 0x7f1c100b2eb0 2023-11-24T08:04:44.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.976+0000 7f1c0ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c10161d10 con 0x7f1c101618d0 2023-11-24T08:04:44.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.976+0000 7f1c0ffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c10071780 con 0x7f1c10071310 2023-11-24T08:04:44.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.977+0000 7f1c07fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1c101618d0 0x7f1c10070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.977+0000 7f1c07fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1c101618d0 0x7f1c10070dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46814/0 (socket says 172.21.15.64:46814) 2023-11-24T08:04:44.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.977+0000 7f1c07fff640 1 -- 172.21.15.64:0/668277145 learned_addr learned my addr 172.21.15.64:0/668277145 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:44.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.977+0000 7f1c0f7fe640 1 --2- 172.21.15.64:0/668277145 >> v2:172.21.15.91:3300/0 conn(0x7f1c10071310 0x7f1c1006a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.977+0000 7f1c07fff640 1 -- 172.21.15.64:0/668277145 >> v2:172.21.15.91:3300/0 conn(0x7f1c10071310 msgr2=0x7f1c1006a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:44.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.977+0000 7f1c07fff640 1 --2- 172.21.15.64:0/668277145 >> v2:172.21.15.91:3300/0 conn(0x7f1c10071310 0x7f1c1006a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.977+0000 7f1c07fff640 1 -- 172.21.15.64:0/668277145 >> v1:172.21.15.91:6789/0 conn(0x7f1c100b2eb0 legacy=0x7f1c1015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:44.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.977+0000 7f1c07fff640 1 -- 172.21.15.64:0/668277145 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1015e350 con 0x7f1c101618d0 2023-11-24T08:04:44.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.978+0000 7f1c07fff640 1 --2- 172.21.15.64:0/668277145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c101618d0 0x7f1c10070dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1c100663a0 tx=0x7f1bf802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234789f2ce459f15 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:44.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.979+0000 7f1c0e7fc640 1 -- 172.21.15.64:0/668277145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8031870 con 0x7f1c101618d0 2023-11-24T08:04:44.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.979+0000 7f1c0e7fc640 1 -- 172.21.15.64:0/668277145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf80319d0 con 0x7f1c101618d0 2023-11-24T08:04:44.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.979+0000 7f1c0e7fc640 1 -- 172.21.15.64:0/668277145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8031d60 con 0x7f1c101618d0 2023-11-24T08:04:44.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.979+0000 7f1c0ffff640 1 -- 172.21.15.64:0/668277145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c101618d0 msgr2=0x7f1c10070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.979+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/668277145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c101618d0 0x7f1c10070dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1c100663a0 tx=0x7f1bf802b910 comp rx=0 tx=0).stop 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.980+0000 7f1c0ffff640 1 -- 172.21.15.64:0/668277145 shutdown_connections 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.980+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/668277145 >> v2:172.21.15.91:3300/0 conn(0x7f1c10071310 0x7f1c1006a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.980+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/668277145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c101618d0 0x7f1c10070dd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.980+0000 7f1c0ffff640 1 -- 172.21.15.64:0/668277145 >> 172.21.15.64:0/668277145 conn(0x7f1c100b15c0 msgr2=0x7f1c100b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.980+0000 7f1c0ffff640 1 -- 172.21.15.64:0/668277145 shutdown_connections 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.980+0000 7f1c0ffff640 1 -- 172.21.15.64:0/668277145 wait complete. 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.981+0000 7f1c0ffff640 1 Processor -- start 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.982+0000 7f1c0ffff640 1 -- start start 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.982+0000 7f1c0ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c10071310 0x7f1c100ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100b2eb0 0x7f1c1015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c0ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c101618d0 0x7f1c10143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c0ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101431d0 con 0x7f1c100b2eb0 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c0ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101425c0 con 0x7f1c10071310 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c0ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101428a0 con 0x7f1c101618d0 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100b2eb0 0x7f1c1015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100b2eb0 0x7f1c1015d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46830/0 (socket says 172.21.15.64:46830) 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c07fff640 1 -- 172.21.15.64:0/2569434065 learned_addr learned my addr 172.21.15.64:0/2569434065 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c0f7fe640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c101618d0 0x7f1c10143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c0effd640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c10071310 0x7f1c100ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c07fff640 1 -- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c101618d0 msgr2=0x7f1c10143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c07fff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c101618d0 0x7f1c10143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c07fff640 1 -- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c10071310 msgr2=0x7f1c100ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c07fff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c10071310 0x7f1c100ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.983+0000 7f1c07fff640 1 -- 172.21.15.64:0/2569434065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf80094d0 con 0x7f1c100b2eb0 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.984+0000 7f1c07fff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100b2eb0 0x7f1c1015d830 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80090b0 tx=0x7f1bf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:44.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.984+0000 7f1c077fe640 1 -- 172.21.15.64:0/2569434065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf803ad20 con 0x7f1c100b2eb0 2023-11-24T08:04:44.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.984+0000 7f1c077fe640 1 -- 172.21.15.64:0/2569434065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf80443f0 con 0x7f1c100b2eb0 2023-11-24T08:04:44.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.984+0000 7f1c077fe640 1 -- 172.21.15.64:0/2569434065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf803aea0 con 0x7f1c100b2eb0 2023-11-24T08:04:44.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.984+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c10142b80 con 0x7f1c100b2eb0 2023-11-24T08:04:44.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.984+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c100ce580 con 0x7f1c100b2eb0 2023-11-24T08:04:44.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.986+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c10143340 con 0x7f1c100b2eb0 2023-11-24T08:04:44.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.986+0000 7f1c077fe640 1 -- 172.21.15.64:0/2569434065 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1bf804d450 con 0x7f1c100b2eb0 2023-11-24T08:04:44.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.987+0000 7f1c077fe640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1bf006ae40 0x7f1bf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:44.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.988+0000 7f1c077fe640 1 -- 172.21.15.64:0/2569434065 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1bf802d040 con 0x7f1c100b2eb0 2023-11-24T08:04:44.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.988+0000 7f1c0effd640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1bf006ae40 0x7f1bf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:44.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.989+0000 7f1c0effd640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1bf006ae40 0x7f1bf006d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c00006fd0 tx=0x7f1c00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:44.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:44.992+0000 7f1c077fe640 1 -- 172.21.15.64:0/2569434065 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1bf8076ce0 con 0x7f1c100b2eb0 2023-11-24T08:04:45.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.188+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c1004e840 con 0x7f1c100b2eb0 2023-11-24T08:04:45.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.188+0000 7f1c077fe640 1 -- 172.21.15.64:0/2569434065 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bf8035070 con 0x7f1c100b2eb0 2023-11-24T08:04:45.190 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1bf006ae40 msgr2=0x7f1bf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1bf006ae40 0x7f1bf006d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c00006fd0 tx=0x7f1c00008040 comp rx=0 tx=0).stop 2023-11-24T08:04:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100b2eb0 msgr2=0x7f1c1015d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100b2eb0 0x7f1c1015d830 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80090b0 tx=0x7f1bf80046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 shutdown_connections 2023-11-24T08:04:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1bf006ae40 0x7f1bf006d300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1c101618d0 0x7f1c10143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100b2eb0 0x7f1c1015d830 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 --2- 172.21.15.64:0/2569434065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1c10071310 0x7f1c100ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 >> 172.21.15.64:0/2569434065 conn(0x7f1c100b15c0 msgr2=0x7f1c100af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 shutdown_connections 2023-11-24T08:04:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.191+0000 7f1c0ffff640 1 -- 172.21.15.64:0/2569434065 wait complete. 2023-11-24T08:04:45.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:45.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.408+0000 7fb2e13b3640 1 Processor -- start 2023-11-24T08:04:45.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.408+0000 7fb2e13b3640 1 -- start start 2023-11-24T08:04:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.410+0000 7fb2e13b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc1439b0 0x7fb2dc070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.410+0000 7fb2e13b3640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc070d40 con 0x7fb2dc1435d0 2023-11-24T08:04:45.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.410+0000 7fb2e13b3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc071020 con 0x7fb2dc0b4140 2023-11-24T08:04:45.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.410+0000 7fb2e13b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc071300 con 0x7fb2dc1439b0 2023-11-24T08:04:45.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.411+0000 7fb2dbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb2dc1435d0 0x7fb2dc0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42008/0 (socket says 172.21.15.64:42008) 2023-11-24T08:04:45.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.411+0000 7fb2dbfff640 1 -- 172.21.15.64:0/1191012648 learned_addr learned my addr 172.21.15.64:0/1191012648 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.411+0000 7fb2e0bb2640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc1439b0 0x7fb2dc070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.411+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724595177 0 0) 0x7fb2dc070d40 con 0x7fb2dc1435d0 2023-11-24T08:04:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.411+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c4005730 con 0x7fb2dc1435d0 2023-11-24T08:04:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.411+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856508527 0 0) 0x7fb2c4005730 con 0x7fb2dc1435d0 2023-11-24T08:04:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.411+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc1439b0 msgr2=0x7fb2dc070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc1439b0 0x7fb2dc070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 >> v1:172.21.15.91:6789/0 conn(0x7fb2dc0b4140 legacy=0x7fb2dc070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0715e0 con 0x7fb2dc1435d0 2023-11-24T08:04:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fb2cc001670 con 0x7fb2dc1435d0 2023-11-24T08:04:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 >> v1:172.21.15.64:6789/0 conn(0x7fb2dc1435d0 legacy=0x7fb2dc0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc1439b0 0x7fb2c4005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c40061f0 0x7fb2c40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4006f10 0x7fb2c4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc071300 con 0x7fb2c4006f10 2023-11-24T08:04:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c400dd50 con 0x7fb2c40061f0 2023-11-24T08:04:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c400df10 con 0x7fb2dc1439b0 2023-11-24T08:04:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2e0bb2640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4006f10 0x7fb2c4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2dbfff640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc1439b0 0x7fb2c4005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.412+0000 7fb2db7fe640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c40061f0 0x7fb2c40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.413+0000 7fb2e0bb2640 1 -- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc1439b0 msgr2=0x7fb2c4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:45.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.413+0000 7fb2e0bb2640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc1439b0 0x7fb2c4005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.413+0000 7fb2e0bb2640 1 -- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c40061f0 msgr2=0x7fb2c40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:45.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.413+0000 7fb2e0bb2640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c40061f0 0x7fb2c40069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.413+0000 7fb2e0bb2640 1 -- 172.21.15.64:0/1191012648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4005730 con 0x7fb2c4006f10 2023-11-24T08:04:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.414+0000 7fb2e0bb2640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4006f10 0x7fb2c4007390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009f90 tx=0x7fb2c4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de072105d4392f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.414+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d402cbd0 con 0x7fb2c4006f10 2023-11-24T08:04:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.414+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d400f040 con 0x7fb2c4006f10 2023-11-24T08:04:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.414+0000 7fb2daffd640 1 -- 172.21.15.64:0/1191012648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4034440 con 0x7fb2c4006f10 2023-11-24T08:04:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.415+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1191012648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4006f10 msgr2=0x7fb2c4007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.415+0000 7fb2e13b3640 1 --2- 172.21.15.64:0/1191012648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4006f10 0x7fb2c4007390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009f90 tx=0x7fb2c4007a00 comp rx=0 tx=0).stop 2023-11-24T08:04:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.415+0000 7fb2e0bb2640 1 -- 172.21.15.64:0/1191012648 reap_dead start 2023-11-24T08:04:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.415+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1191012648 shutdown_connections 2023-11-24T08:04:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.415+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1191012648 >> 172.21.15.64:0/1191012648 conn(0x7fb2dc0cec10 msgr2=0x7fb2dc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.415+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1191012648 shutdown_connections 2023-11-24T08:04:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.415+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1191012648 wait complete. 2023-11-24T08:04:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.417+0000 7fb2e13b3640 1 Processor -- start 2023-11-24T08:04:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.417+0000 7fb2e13b3640 1 -- start start 2023-11-24T08:04:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.418+0000 7fb2e13b3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c4006f10 0x7fb2dc0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.418+0000 7fb2e13b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2dc0ac870 0x7fb2dc139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.418+0000 7fb2e13b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc156600 0x7fb2dc157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.418+0000 7fb2e13b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc139bb0 con 0x7fb2dc0ac870 2023-11-24T08:04:45.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.418+0000 7fb2e13b3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0acd30 con 0x7fb2c4006f10 2023-11-24T08:04:45.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.418+0000 7fb2e13b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0ad010 con 0x7fb2dc156600 2023-11-24T08:04:45.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2dc0ac870 0x7fb2dc139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2dc0ac870 0x7fb2dc139670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46858/0 (socket says 172.21.15.64:46858) 2023-11-24T08:04:45.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 -- 172.21.15.64:0/1878606628 learned_addr learned my addr 172.21.15.64:0/1878606628 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:45.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2dbfff640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c4006f10 0x7fb2dc0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2e0bb2640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc156600 0x7fb2dc157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 -- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc156600 msgr2=0x7fb2dc157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc156600 0x7fb2dc157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 -- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c4006f10 msgr2=0x7fb2dc0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c4006f10 0x7fb2dc0addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 -- 172.21.15.64:0/1878606628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0ad2f0 con 0x7fb2dc0ac870 2023-11-24T08:04:45.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.419+0000 7fb2db7fe640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2dc0ac870 0x7fb2dc139670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e550 tx=0x7fb2d000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:45.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.420+0000 7fb2d8ff9640 1 -- 172.21.15.64:0/1878606628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00052e0 con 0x7fb2dc0ac870 2023-11-24T08:04:45.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.420+0000 7fb2d8ff9640 1 -- 172.21.15.64:0/1878606628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0005440 con 0x7fb2dc0ac870 2023-11-24T08:04:45.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.420+0000 7fb2d8ff9640 1 -- 172.21.15.64:0/1878606628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0003810 con 0x7fb2dc0ac870 2023-11-24T08:04:45.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.420+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc0ad5d0 con 0x7fb2dc0ac870 2023-11-24T08:04:45.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.420+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc156ab0 con 0x7fb2dc0ac870 2023-11-24T08:04:45.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.421+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc0ae310 con 0x7fb2dc0ac870 2023-11-24T08:04:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.422+0000 7fb2d8ff9640 1 -- 172.21.15.64:0/1878606628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb2d0003970 con 0x7fb2dc0ac870 2023-11-24T08:04:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.423+0000 7fb2d8ff9640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb2c806ae70 0x7fb2c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.423+0000 7fb2d8ff9640 1 -- 172.21.15.64:0/1878606628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb2d00894c0 con 0x7fb2dc0ac870 2023-11-24T08:04:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.424+0000 7fb2dbfff640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb2c806ae70 0x7fb2c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.425+0000 7fb2dbfff640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb2c806ae70 0x7fb2c806d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0059f0 tx=0x7fb2cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.428+0000 7fb2d8ff9640 1 -- 172.21.15.64:0/1878606628 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2d0054440 con 0x7fb2dc0ac870 2023-11-24T08:04:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.616+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2dc04e840 con 0x7fb2dc0ac870 2023-11-24T08:04:45.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.617+0000 7fb2d8ff9640 1 -- 172.21.15.64:0/1878606628 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2d0057a60 con 0x7fb2dc0ac870 2023-11-24T08:04:45.619 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.619+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb2c806ae70 msgr2=0x7fb2c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb2c806ae70 0x7fb2c806d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0059f0 tx=0x7fb2cc009040 comp rx=0 tx=0).stop 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2dc0ac870 msgr2=0x7fb2dc139670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2dc0ac870 0x7fb2dc139670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e550 tx=0x7fb2d000ea10 comp rx=0 tx=0).stop 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 shutdown_connections 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb2c806ae70 0x7fb2c806d330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2dc156600 0x7fb2dc157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2dc0ac870 0x7fb2dc139670 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 --2- 172.21.15.64:0/1878606628 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb2c4006f10 0x7fb2dc0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 >> 172.21.15.64:0/1878606628 conn(0x7fb2dc0cec10 msgr2=0x7fb2dc0b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:45.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 shutdown_connections 2023-11-24T08:04:45.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.620+0000 7fb2e13b3640 1 -- 172.21.15.64:0/1878606628 wait complete. 2023-11-24T08:04:45.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:45.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.842+0000 7fcafffff640 1 Processor -- start 2023-11-24T08:04:45.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.842+0000 7fcafffff640 1 -- start start 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcafffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb001435d0 0x7fcb00071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcafffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb001439b0 0x7fcb000b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcafffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb00070d40 con 0x7fcb000b3f70 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcafffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb000b4400 con 0x7fcb001439b0 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcafffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00071020 con 0x7fcb001435d0 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcaff7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcb000b3f70 0x7fcb00070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42018/0 (socket says 172.21.15.64:42018) 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcaff7fe640 1 -- 172.21.15.64:0/2153596448 learned_addr learned my addr 172.21.15.64:0/2153596448 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcafeffd640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb001435d0 0x7fcb00071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.845+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/2153596448 >> v2:172.21.15.91:3300/0 conn(0x7fcb001439b0 0x7fcb000b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427007019 0 0) 0x7fcb00070d40 con 0x7fcb000b3f70 2023-11-24T08:04:45.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaec005730 con 0x7fcb000b3f70 2023-11-24T08:04:45.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1525632484 0 0) 0x7fcaec005730 con 0x7fcb000b3f70 2023-11-24T08:04:45.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb001435d0 msgr2=0x7fcb00071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:45.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb001435d0 0x7fcb00071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 >> v2:172.21.15.91:3300/0 conn(0x7fcb001439b0 msgr2=0x7fcb000b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 --2- 172.21.15.64:0/2153596448 >> v2:172.21.15.91:3300/0 conn(0x7fcb001439b0 0x7fcb000b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00071300 con 0x7fcb000b3f70 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fcaf8001670 con 0x7fcb000b3f70 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 >> v1:172.21.15.64:6789/0 conn(0x7fcb000b3f70 legacy=0x7fcb00070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec005bf0 0x7fcaec006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcaec006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb001435d0 0x7fcaec007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b4400 con 0x7fcaec0069d0 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00071020 con 0x7fcaec005bf0 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.846+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec00dbc0 con 0x7fcb001435d0 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.847+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcaec006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.847+0000 7fcafeffd640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec005bf0 0x7fcaec006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.847+0000 7fcaff7fe640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb001435d0 0x7fcaec007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.847+0000 7fcaf67fc640 1 -- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb001435d0 msgr2=0x7fcaec007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.847+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb001435d0 0x7fcaec007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.847+0000 7fcaf67fc640 1 -- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec005bf0 msgr2=0x7fcaec006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.847+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec005bf0 0x7fcaec006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.847+0000 7fcaf67fc640 1 -- 172.21.15.64:0/2153596448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec005730 con 0x7fcaec0069d0 2023-11-24T08:04:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.848+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcaec006e50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009ac0 tx=0x7fcaec007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59dbba99ea022149 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.848+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae802c710 con 0x7fcaec0069d0 2023-11-24T08:04:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.848+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae802c870 con 0x7fcaec0069d0 2023-11-24T08:04:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.848+0000 7fcafe7fc640 1 -- 172.21.15.64:0/2153596448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae802cc00 con 0x7fcaec0069d0 2023-11-24T08:04:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.849+0000 7fcafffff640 1 -- 172.21.15.64:0/2153596448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 msgr2=0x7fcaec006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.849+0000 7fcafffff640 1 --2- 172.21.15.64:0/2153596448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcaec006e50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009ac0 tx=0x7fcaec007a00 comp rx=0 tx=0).stop 2023-11-24T08:04:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.849+0000 7fcaff7fe640 1 -- 172.21.15.64:0/2153596448 reap_dead start 2023-11-24T08:04:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.849+0000 7fcafffff640 1 -- 172.21.15.64:0/2153596448 shutdown_connections 2023-11-24T08:04:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.849+0000 7fcafffff640 1 -- 172.21.15.64:0/2153596448 >> 172.21.15.64:0/2153596448 conn(0x7fcb000cec10 msgr2=0x7fcb000cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.849+0000 7fcafffff640 1 -- 172.21.15.64:0/2153596448 shutdown_connections 2023-11-24T08:04:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.849+0000 7fcafffff640 1 -- 172.21.15.64:0/2153596448 wait complete. 2023-11-24T08:04:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.851+0000 7fcafffff640 1 Processor -- start 2023-11-24T08:04:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.851+0000 7fcafffff640 1 -- start start 2023-11-24T08:04:45.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcb000c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb000ae060 0x7fcb000ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb00153980 0x7fcb0015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000acdb0 con 0x7fcaec0069d0 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000acfc0 con 0x7fcb000ae060 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000ad2a0 con 0x7fcb00153980 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcb000c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcb000c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46870/0 (socket says 172.21.15.64:46870) 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.852+0000 7fcafeffd640 1 -- 172.21.15.64:0/3130834623 learned_addr learned my addr 172.21.15.64:0/3130834623 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb000ae060 0x7fcb000ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcafeffd640 1 -- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb00153980 msgr2=0x7fcb0015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcafeffd640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb00153980 0x7fcb0015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcafeffd640 1 -- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb000ae060 msgr2=0x7fcb000ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcafeffd640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb000ae060 0x7fcb000ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:45.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcafeffd640 1 -- 172.21.15.64:0/3130834623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb000ad580 con 0x7fcaec0069d0 2023-11-24T08:04:45.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb000ae060 0x7fcb000ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:45.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcafeffd640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcb000c3ca0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0007f30 tx=0x7fcaf0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:45.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcaf7fff640 1 -- 172.21.15.64:0/3130834623 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000e030 con 0x7fcaec0069d0 2023-11-24T08:04:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcaf7fff640 1 -- 172.21.15.64:0/3130834623 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf0016470 con 0x7fcaec0069d0 2023-11-24T08:04:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcaf7fff640 1 -- 172.21.15.64:0/3130834623 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0016820 con 0x7fcaec0069d0 2023-11-24T08:04:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.853+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb000ad860 con 0x7fcaec0069d0 2023-11-24T08:04:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.854+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb00153e00 con 0x7fcaec0069d0 2023-11-24T08:04:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.855+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00154080 con 0x7fcaec0069d0 2023-11-24T08:04:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.856+0000 7fcaf7fff640 1 -- 172.21.15.64:0/3130834623 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcaf0016980 con 0x7fcaec0069d0 2023-11-24T08:04:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.857+0000 7fcaf7fff640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcae006ae70 0x7fcae006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.857+0000 7fcaf7fff640 1 -- 172.21.15.64:0/3130834623 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fcaf000f070 con 0x7fcaec0069d0 2023-11-24T08:04:45.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.857+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcae006ae70 0x7fcae006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.858+0000 7fcaf67fc640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcae006ae70 0x7fcae006d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcae8036000 tx=0x7fcae8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:45.865+0000 7fcaf7fff640 1 -- 172.21.15.64:0/3130834623 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb00154080 con 0x7fcaec0069d0 2023-11-24T08:04:46.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.050+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb00154080 con 0x7fcaec0069d0 2023-11-24T08:04:46.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.051+0000 7fcaf7fff640 1 -- 172.21.15.64:0/3130834623 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb00154080 con 0x7fcaec0069d0 2023-11-24T08:04:46.053 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:46.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcae006ae70 msgr2=0x7fcae006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcae006ae70 0x7fcae006d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcae8036000 tx=0x7fcae8030000 comp rx=0 tx=0).stop 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 msgr2=0x7fcb000c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcb000c3ca0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0007f30 tx=0x7fcaf0011660 comp rx=0 tx=0).stop 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 shutdown_connections 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcae006ae70 0x7fcae006d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb00153980 0x7fcb0015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb000ae060 0x7fcb000ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.054+0000 7fcafffff640 1 --2- 172.21.15.64:0/3130834623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0069d0 0x7fcb000c3ca0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.055+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 >> 172.21.15.64:0/3130834623 conn(0x7fcb000cec10 msgr2=0x7fcb000b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.055+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 shutdown_connections 2023-11-24T08:04:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.055+0000 7fcafffff640 1 -- 172.21.15.64:0/3130834623 wait complete. 2023-11-24T08:04:46.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:04:46.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.290+0000 7fb451de6640 1 Processor -- start 2023-11-24T08:04:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.291+0000 7fb451de6640 1 -- start start 2023-11-24T08:04:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb451de6640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c1416f0 0x7fb44c15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb451de6640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c070bf0 con 0x7fb44c06d230 2023-11-24T08:04:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb451de6640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c06d850 con 0x7fb44c141e80 2023-11-24T08:04:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb451de6640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c06db30 con 0x7fb44c1416f0 2023-11-24T08:04:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb4515e5640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb44c06d230 0x7fb44c0cd620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42030/0 (socket says 172.21.15.64:42030) 2023-11-24T08:04:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb4515e5640 1 -- 172.21.15.64:0/2486051558 learned_addr learned my addr 172.21.15.64:0/2486051558 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947502170 0 0) 0x7fb44c070bf0 con 0x7fb44c06d230 2023-11-24T08:04:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb43c003520 con 0x7fb44c06d230 2023-11-24T08:04:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb450de4640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c1416f0 0x7fb44c15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 343634105 0 0) 0x7fb43c003520 con 0x7fb44c06d230 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c1416f0 msgr2=0x7fb44c15ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c1416f0 0x7fb44c15ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 >> v1:172.21.15.91:6789/0 conn(0x7fb44c141e80 legacy=0x7fb44c06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0705a0 con 0x7fb44c06d230 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fb4440019f0 con 0x7fb44c06d230 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 >> v1:172.21.15.64:6789/0 conn(0x7fb44c06d230 legacy=0x7fb44c0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.293+0000 7fb44b7fe640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c1416f0 0x7fb43c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44b7fe640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb43c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44b7fe640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb43c004d70 0x7fb43c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c06db30 con 0x7fb43c004090 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c00bbe0 con 0x7fb43c004d70 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c00bda0 con 0x7fb44c1416f0 2023-11-24T08:04:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44bfff640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb43c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb450de4640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c1416f0 0x7fb43c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb4515e5640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb43c004d70 0x7fb43c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44bfff640 1 -- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c1416f0 msgr2=0x7fb43c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44bfff640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c1416f0 0x7fb43c003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44bfff640 1 -- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb43c004d70 msgr2=0x7fb43c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44bfff640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb43c004d70 0x7fb43c0051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb44bfff640 1 -- 172.21.15.64:0/2486051558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c003520 con 0x7fb43c004090 2023-11-24T08:04:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.294+0000 7fb4515e5640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb43c004d70 0x7fb43c0051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.295+0000 7fb44bfff640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb43c0047c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb438001750 tx=0x7fb43c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9428e1f83f75deda server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4380052d0 con 0x7fb43c004090 2023-11-24T08:04:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43802e050 con 0x7fb43c004090 2023-11-24T08:04:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb44b7fe640 1 -- 172.21.15.64:0/2486051558 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb438003830 con 0x7fb43c004090 2023-11-24T08:04:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb451de6640 1 -- 172.21.15.64:0/2486051558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 msgr2=0x7fb43c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb451de6640 1 --2- 172.21.15.64:0/2486051558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb43c0047c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb438001750 tx=0x7fb43c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb4515e5640 1 -- 172.21.15.64:0/2486051558 reap_dead start 2023-11-24T08:04:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb451de6640 1 -- 172.21.15.64:0/2486051558 shutdown_connections 2023-11-24T08:04:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb451de6640 1 -- 172.21.15.64:0/2486051558 >> 172.21.15.64:0/2486051558 conn(0x7fb44c0b0840 msgr2=0x7fb44c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.296+0000 7fb451de6640 1 -- 172.21.15.64:0/2486051558 shutdown_connections 2023-11-24T08:04:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.297+0000 7fb451de6640 1 -- 172.21.15.64:0/2486051558 wait complete. 2023-11-24T08:04:46.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.298+0000 7fb451de6640 1 Processor -- start 2023-11-24T08:04:46.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.298+0000 7fb451de6640 1 -- start start 2023-11-24T08:04:46.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.299+0000 7fb451de6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb44c0ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:46.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.299+0000 7fb451de6640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c15e220 0x7fb44c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:46.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.299+0000 7fb451de6640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb44c0b2c50 0x7fb44c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:46.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.299+0000 7fb451de6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c0b1c40 con 0x7fb43c004090 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.299+0000 7fb451de6640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c0b1e90 con 0x7fb44c0b2c50 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.299+0000 7fb451de6640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c0b2170 con 0x7fb44c15e220 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.299+0000 7fb450de4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb44c0ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.299+0000 7fb450de4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb44c0ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46898/0 (socket says 172.21.15.64:46898) 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb450de4640 1 -- 172.21.15.64:0/2071986057 learned_addr learned my addr 172.21.15.64:0/2071986057 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb450de4640 1 -- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c15e220 msgr2=0x7fb44c15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb450de4640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c15e220 0x7fb44c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb450de4640 1 -- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb44c0b2c50 msgr2=0x7fb44c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb450de4640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb44c0b2c50 0x7fb44c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb450de4640 1 -- 172.21.15.64:0/2071986057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0b2450 con 0x7fb43c004090 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb450de4640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb44c0ad280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb44000f580 tx=0x7fb44000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb4497fa640 1 -- 172.21.15.64:0/2071986057 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440014870 con 0x7fb43c004090 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb4497fa640 1 -- 172.21.15.64:0/2071986057 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4400149d0 con 0x7fb43c004090 2023-11-24T08:04:46.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb4497fa640 1 -- 172.21.15.64:0/2071986057 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440014d80 con 0x7fb43c004090 2023-11-24T08:04:46.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.300+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c0b2730 con 0x7fb43c004090 2023-11-24T08:04:46.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.301+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c15a970 con 0x7fb43c004090 2023-11-24T08:04:46.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.302+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c04e840 con 0x7fb43c004090 2023-11-24T08:04:46.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.302+0000 7fb4497fa640 1 -- 172.21.15.64:0/2071986057 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb440015410 con 0x7fb43c004090 2023-11-24T08:04:46.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.303+0000 7fb4497fa640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb43006ae70 0x7fb43006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:46.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.303+0000 7fb4497fa640 1 -- 172.21.15.64:0/2071986057 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb44008a140 con 0x7fb43c004090 2023-11-24T08:04:46.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.304+0000 7fb44bfff640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb43006ae70 0x7fb43006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:46.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.305+0000 7fb44bfff640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb43006ae70 0x7fb43006d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb438033000 tx=0x7fb43802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:46.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.308+0000 7fb4497fa640 1 -- 172.21.15.64:0/2071986057 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4400550d0 con 0x7fb43c004090 2023-11-24T08:04:46.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.511+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb44c140590 con 0x7fb43c004090 2023-11-24T08:04:46.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.511+0000 7fb4497fa640 1 -- 172.21.15.64:0/2071986057 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb4400586f0 con 0x7fb43c004090 2023-11-24T08:04:46.513 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb43006ae70 msgr2=0x7fb43006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb43006ae70 0x7fb43006d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb438033000 tx=0x7fb43802f000 comp rx=0 tx=0).stop 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 msgr2=0x7fb44c0ad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb44c0ad280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb44000f580 tx=0x7fb44000fa40 comp rx=0 tx=0).stop 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 shutdown_connections 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb43006ae70 0x7fb43006d330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb44c0b2c50 0x7fb44c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb44c15e220 0x7fb44c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 --2- 172.21.15.64:0/2071986057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb43c004090 0x7fb44c0ad280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 >> 172.21.15.64:0/2071986057 conn(0x7fb44c0b0840 msgr2=0x7fb44c1445e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:46.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 shutdown_connections 2023-11-24T08:04:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:46.514+0000 7fb451de6640 1 -- 172.21.15.64:0/2071986057 wait complete. 2023-11-24T08:04:46.534 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T08:04:46.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:46.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T08:04:46.625 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:04:46.625 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:04:46.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:46.625 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:04:46.688 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:04:46.688 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:46.750 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:04:46.751 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:46.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.038+0000 7f8cc4c15640 1 Processor -- start 2023-11-24T08:04:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.038+0000 7f8cc4c15640 1 -- start start 2023-11-24T08:04:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cc4c15640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8cc0161930 0x7f8cc00ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cc4c15640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc00719c0 0x7f8cc00ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cc4c15640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc00ad510 con 0x7f8cc006d520 2023-11-24T08:04:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cc4c15640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc00acc50 con 0x7f8cc0161930 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cc4c15640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0071e60 con 0x7f8cc00719c0 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cbe59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8cc0161930 0x7f8cc00ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cbe59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8cc0161930 0x7f8cc00ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56622/0 (socket says 172.21.15.64:56622) 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cbe59c640 1 -- 172.21.15.64:0/1399664281 learned_addr learned my addr 172.21.15.64:0/1399664281 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cbdd9b640 1 --2- 172.21.15.64:0/1399664281 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc00719c0 0x7f8cc00ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cbe59c640 1 -- 172.21.15.64:0/1399664281 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc00719c0 msgr2=0x7f8cc00ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cbe59c640 1 --2- 172.21.15.64:0/1399664281 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc00719c0 0x7f8cc00ad9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cbe59c640 1 -- 172.21.15.64:0/1399664281 >> v1:172.21.15.91:6789/0 conn(0x7f8cc006d520 legacy=0x7f8cc00b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.040+0000 7f8cbe59c640 1 -- 172.21.15.64:0/1399664281 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00acec0 con 0x7f8cc0161930 2023-11-24T08:04:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.042+0000 7f8cbe59c640 1 --2- 172.21.15.64:0/1399664281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00ae6a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009e50 tx=0x7f8cb002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693237c31b0fe3d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.042+0000 7f8cbd59a640 1 -- 172.21.15.64:0/1399664281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb002cb00 con 0x7f8cc0161930 2023-11-24T08:04:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.042+0000 7f8cbd59a640 1 -- 172.21.15.64:0/1399664281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb002cc60 con 0x7f8cc0161930 2023-11-24T08:04:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.042+0000 7f8cbd59a640 1 -- 172.21.15.64:0/1399664281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00343f0 con 0x7f8cc0161930 2023-11-24T08:04:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.043+0000 7f8cc4c15640 1 -- 172.21.15.64:0/1399664281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 msgr2=0x7f8cc00ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.043+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/1399664281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00ae6a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009e50 tx=0x7f8cb002bf60 comp rx=0 tx=0).stop 2023-11-24T08:04:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.043+0000 7f8cc4c15640 1 -- 172.21.15.64:0/1399664281 shutdown_connections 2023-11-24T08:04:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.043+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/1399664281 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc00719c0 0x7f8cc00ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.043+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/1399664281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00ae6a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.043+0000 7f8cc4c15640 1 -- 172.21.15.64:0/1399664281 >> 172.21.15.64:0/1399664281 conn(0x7f8cc00b0d80 msgr2=0x7f8cc00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.043+0000 7f8cc4c15640 1 -- 172.21.15.64:0/1399664281 shutdown_connections 2023-11-24T08:04:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.043+0000 7f8cc4c15640 1 -- 172.21.15.64:0/1399664281 wait complete. 2023-11-24T08:04:47.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.045+0000 7f8cc4c15640 1 Processor -- start 2023-11-24T08:04:47.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.045+0000 7f8cc4c15640 1 -- start start 2023-11-24T08:04:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cc4c15640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc006d520 0x7f8cc014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cc4c15640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8cc00719c0 0x7f8cc0099070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cc4c15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cc4c15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc009c140 con 0x7f8cc0161930 2023-11-24T08:04:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cc4c15640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc009c420 con 0x7f8cc00719c0 2023-11-24T08:04:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cc4c15640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc009c700 con 0x7f8cc006d520 2023-11-24T08:04:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cbed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cbed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56626/0 (socket says 172.21.15.64:56626) 2023-11-24T08:04:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.046+0000 7f8cbed9d640 1 -- 172.21.15.64:0/2597245387 learned_addr learned my addr 172.21.15.64:0/2597245387 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cbe59c640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc006d520 0x7f8cc014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cbdd9b640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8cc00719c0 0x7f8cc0099070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cbed9d640 1 -- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc006d520 msgr2=0x7f8cc014f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cbed9d640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc006d520 0x7f8cc014f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cbed9d640 1 -- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8cc00719c0 msgr2=0x7f8cc0099070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cbed9d640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8cc00719c0 0x7f8cc0099070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cbed9d640 1 -- 172.21.15.64:0/2597245387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb0009b30 con 0x7f8cc0161930 2023-11-24T08:04:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cbed9d640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00995b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c390 tx=0x7f8cb800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.047+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/2597245387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80035f0 con 0x7f8cc0161930 2023-11-24T08:04:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.048+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/2597245387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8003750 con 0x7f8cc0161930 2023-11-24T08:04:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.048+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/2597245387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8003b00 con 0x7f8cc0161930 2023-11-24T08:04:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.048+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc009c9e0 con 0x7f8cc0161930 2023-11-24T08:04:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.048+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc014f740 con 0x7f8cc0161930 2023-11-24T08:04:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.049+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc0074770 con 0x7f8cc0161930 2023-11-24T08:04:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.050+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/2597245387 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8cb8003c60 con 0x7f8cc0161930 2023-11-24T08:04:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.051+0000 7f8cb6ffd640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ca006ae70 0x7f8ca006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.051+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/2597245387 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8cb8088860 con 0x7f8cc0161930 2023-11-24T08:04:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.052+0000 7f8cbe59c640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ca006ae70 0x7f8ca006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.053+0000 7f8cbe59c640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ca006ae70 0x7f8ca006d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0035000 tx=0x7f8cb0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.055+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/2597245387 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8cb80537e0 con 0x7f8cc0161930 2023-11-24T08:04:47.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.254+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cc00cb4b0 con 0x7f8cc0161930 2023-11-24T08:04:47.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.256+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/2597245387 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7f8cb8056e00 con 0x7f8cc0161930 2023-11-24T08:04:47.258 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:47.258 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:47.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.258+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ca006ae70 msgr2=0x7f8ca006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.258+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ca006ae70 0x7f8ca006d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0035000 tx=0x7f8cb0030000 comp rx=0 tx=0).stop 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.258+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 msgr2=0x7f8cc00995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.258+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00995b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c390 tx=0x7f8cb800c850 comp rx=0 tx=0).stop 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.259+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 shutdown_connections 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.259+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ca006ae70 0x7f8ca006d330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.259+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0161930 0x7f8cc00995b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.259+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8cc00719c0 0x7f8cc0099070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.259+0000 7f8cc4c15640 1 --2- 172.21.15.64:0/2597245387 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8cc006d520 0x7f8cc014f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.259+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 >> 172.21.15.64:0/2597245387 conn(0x7f8cc00b0d80 msgr2=0x7f8cc0164090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.259+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 shutdown_connections 2023-11-24T08:04:47.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.259+0000 7f8cc4c15640 1 -- 172.21.15.64:0/2597245387 wait complete. 2023-11-24T08:04:47.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:47.283 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.510+0000 7f791aa39640 1 Processor -- start 2023-11-24T08:04:47.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.510+0000 7f791aa39640 1 -- start start 2023-11-24T08:04:47.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791aa39640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7914144770 0x7f791415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791aa39640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f79140b35a0 0x7f791415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791aa39640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f791406c9d0 0x7f791406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791aa39640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79140b43b0 con 0x7f791406c9d0 2023-11-24T08:04:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791aa39640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791415f930 con 0x7f79140b35a0 2023-11-24T08:04:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791aa39640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79140b3b10 con 0x7f7914144770 2023-11-24T08:04:47.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791a238640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f791406c9d0 0x7f791406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791a238640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f791406c9d0 0x7f791406ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56632/0 (socket says 172.21.15.64:56632) 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791a238640 1 -- 172.21.15.64:0/1328573932 learned_addr learned my addr 172.21.15.64:0/1328573932 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f7919236640 1 --2- 172.21.15.64:0/1328573932 >> v2:172.21.15.91:3300/0 conn(0x7f79140b35a0 0x7f791415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f7919a37640 1 --2- 172.21.15.64:0/1328573932 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7914144770 0x7f791415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791a238640 1 -- 172.21.15.64:0/1328573932 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7914144770 msgr2=0x7f791415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791a238640 1 --2- 172.21.15.64:0/1328573932 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7914144770 0x7f791415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791a238640 1 -- 172.21.15.64:0/1328573932 >> v2:172.21.15.91:3300/0 conn(0x7f79140b35a0 msgr2=0x7f791415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.512+0000 7f791a238640 1 --2- 172.21.15.64:0/1328573932 >> v2:172.21.15.91:3300/0 conn(0x7f79140b35a0 0x7f791415f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.513+0000 7f791a238640 1 -- 172.21.15.64:0/1328573932 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79140b3df0 con 0x7f791406c9d0 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.514+0000 7f791a238640 1 --2- 172.21.15.64:0/1328573932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791406c9d0 0x7f791406ce60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f790c009e50 tx=0x7f790c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d838fe8baadef68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.514+0000 7f7918a35640 1 -- 172.21.15.64:0/1328573932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c02ca70 con 0x7f791406c9d0 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.514+0000 7f7918a35640 1 -- 172.21.15.64:0/1328573932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c02cbd0 con 0x7f791406c9d0 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.514+0000 7f7918a35640 1 -- 172.21.15.64:0/1328573932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c0343f0 con 0x7f791406c9d0 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 -- 172.21.15.64:0/1328573932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791406c9d0 msgr2=0x7f791406ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 --2- 172.21.15.64:0/1328573932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791406c9d0 0x7f791406ce60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f790c009e50 tx=0x7f790c02bed0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 -- 172.21.15.64:0/1328573932 shutdown_connections 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 --2- 172.21.15.64:0/1328573932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791406c9d0 0x7f791406ce60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 --2- 172.21.15.64:0/1328573932 >> v2:172.21.15.91:3300/0 conn(0x7f79140b35a0 0x7f791415f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 --2- 172.21.15.64:0/1328573932 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7914144770 0x7f791415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 -- 172.21.15.64:0/1328573932 >> 172.21.15.64:0/1328573932 conn(0x7f79140cec10 msgr2=0x7f79140cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:47.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 -- 172.21.15.64:0/1328573932 shutdown_connections 2023-11-24T08:04:47.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.515+0000 7f791aa39640 1 -- 172.21.15.64:0/1328573932 wait complete. 2023-11-24T08:04:47.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.517+0000 7f791aa39640 1 Processor -- start 2023-11-24T08:04:47.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.517+0000 7f791aa39640 1 -- start start 2023-11-24T08:04:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791aa39640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f791406c9d0 0x7f79140cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791aa39640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79140b35a0 0x7f791413fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791aa39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7914144770 0x7f7914140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791aa39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79141505e0 con 0x7f7914144770 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791aa39640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79140c5ac0 con 0x7f79140b35a0 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791aa39640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79140c5220 con 0x7f791406c9d0 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791a238640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7914144770 0x7f7914140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791a238640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7914144770 0x7f7914140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56638/0 (socket says 172.21.15.64:56638) 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791a238640 1 -- 172.21.15.64:0/939677688 learned_addr learned my addr 172.21.15.64:0/939677688 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f7919a37640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f791406c9d0 0x7f79140cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f7919236640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79140b35a0 0x7f791413fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791a238640 1 -- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f791406c9d0 msgr2=0x7f79140cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.518+0000 7f791a238640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f791406c9d0 0x7f79140cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f791a238640 1 -- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79140b35a0 msgr2=0x7f791413fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f791a238640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79140b35a0 0x7f791413fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f791a238640 1 -- 172.21.15.64:0/939677688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c009b30 con 0x7f7914144770 2023-11-24T08:04:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f7919236640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79140b35a0 0x7f791413fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f791a238640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7914144770 0x7f7914140290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f790c009e50 tx=0x7f790c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f790a7fc640 1 -- 172.21.15.64:0/939677688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c009510 con 0x7f7914144770 2023-11-24T08:04:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f790a7fc640 1 -- 172.21.15.64:0/939677688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c03e450 con 0x7f7914144770 2023-11-24T08:04:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f790a7fc640 1 -- 172.21.15.64:0/939677688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c03e800 con 0x7f7914144770 2023-11-24T08:04:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79140c5500 con 0x7f7914144770 2023-11-24T08:04:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.519+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7914141db0 con 0x7f7914144770 2023-11-24T08:04:47.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.520+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7914141c10 con 0x7f7914144770 2023-11-24T08:04:47.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.521+0000 7f790a7fc640 1 -- 172.21.15.64:0/939677688 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f790c03e960 con 0x7f7914144770 2023-11-24T08:04:47.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.522+0000 7f790a7fc640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78f806ae70 0x7f78f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.522+0000 7f790a7fc640 1 -- 172.21.15.64:0/939677688 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f790c0ac3e0 con 0x7f7914144770 2023-11-24T08:04:47.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.523+0000 7f7919a37640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78f806ae70 0x7f78f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.524+0000 7f7919a37640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78f806ae70 0x7f78f806d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7910006fd0 tx=0x7f7910008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.528+0000 7f790a7fc640 1 -- 172.21.15.64:0/939677688 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7914141c10 con 0x7f7914144770 2023-11-24T08:04:47.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.720+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7914141c10 con 0x7f7914144770 2023-11-24T08:04:47.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.720+0000 7f790a7fc640 1 -- 172.21.15.64:0/939677688 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7914141c10 con 0x7f7914144770 2023-11-24T08:04:47.721 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78f806ae70 msgr2=0x7f78f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78f806ae70 0x7f78f806d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7910006fd0 tx=0x7f7910008040 comp rx=0 tx=0).stop 2023-11-24T08:04:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7914144770 msgr2=0x7f7914140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7914144770 0x7f7914140290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f790c009e50 tx=0x7f790c004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 shutdown_connections 2023-11-24T08:04:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78f806ae70 0x7f78f806d330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7914144770 0x7f7914140290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f79140b35a0 0x7f791413fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 --2- 172.21.15.64:0/939677688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f791406c9d0 0x7f79140cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.722+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 >> 172.21.15.64:0/939677688 conn(0x7f79140cec10 msgr2=0x7f79140b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.724+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 shutdown_connections 2023-11-24T08:04:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.724+0000 7f791aa39640 1 -- 172.21.15.64:0/939677688 wait complete. 2023-11-24T08:04:47.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.962+0000 7fde6cc8c640 1 Processor -- start 2023-11-24T08:04:47.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.963+0000 7fde6cc8c640 1 -- start start 2023-11-24T08:04:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.964+0000 7fde6cc8c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde680b4140 0x7fde68071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.964+0000 7fde6cc8c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 0x7fde68070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.964+0000 7fde6cc8c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde68070d40 con 0x7fde681435d0 2023-11-24T08:04:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.964+0000 7fde6cc8c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde68071020 con 0x7fde680b4140 2023-11-24T08:04:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.964+0000 7fde6cc8c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde68071300 con 0x7fde681439b0 2023-11-24T08:04:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.965+0000 7fde5ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde680b4140 0x7fde68071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.965+0000 7fde5ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde680b4140 0x7fde68071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56650/0 (socket says 172.21.15.64:56650) 2023-11-24T08:04:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.965+0000 7fde5ffff640 1 -- 172.21.15.64:0/3630358336 learned_addr learned my addr 172.21.15.64:0/3630358336 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.965+0000 7fde5ffff640 1 -- 172.21.15.64:0/3630358336 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 msgr2=0x7fde68070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.965+0000 7fde5ffff640 1 --2- 172.21.15.64:0/3630358336 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 0x7fde68070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.965+0000 7fde5ffff640 1 -- 172.21.15.64:0/3630358336 >> v1:172.21.15.91:6789/0 conn(0x7fde681435d0 legacy=0x7fde680b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.965+0000 7fde5ffff640 1 -- 172.21.15.64:0/3630358336 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde680715e0 con 0x7fde680b4140 2023-11-24T08:04:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.966+0000 7fde5ffff640 1 --2- 172.21.15.64:0/3630358336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680b4140 0x7fde68071de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fde50009870 tx=0x7fde5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d533e2c72e31adf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.967+0000 7fde66ffd640 1 -- 172.21.15.64:0/3630358336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50031640 con 0x7fde680b4140 2023-11-24T08:04:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.967+0000 7fde66ffd640 1 -- 172.21.15.64:0/3630358336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde500317a0 con 0x7fde680b4140 2023-11-24T08:04:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.967+0000 7fde66ffd640 1 -- 172.21.15.64:0/3630358336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50031b30 con 0x7fde680b4140 2023-11-24T08:04:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.968+0000 7fde6cc8c640 1 -- 172.21.15.64:0/3630358336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680b4140 msgr2=0x7fde68071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.968+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/3630358336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680b4140 0x7fde68071de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fde50009870 tx=0x7fde5002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.968+0000 7fde6cc8c640 1 -- 172.21.15.64:0/3630358336 shutdown_connections 2023-11-24T08:04:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.968+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/3630358336 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 0x7fde68070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.968+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/3630358336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde680b4140 0x7fde68071de0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.968+0000 7fde6cc8c640 1 -- 172.21.15.64:0/3630358336 >> 172.21.15.64:0/3630358336 conn(0x7fde680cec10 msgr2=0x7fde680cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.968+0000 7fde6cc8c640 1 -- 172.21.15.64:0/3630358336 shutdown_connections 2023-11-24T08:04:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.968+0000 7fde6cc8c640 1 -- 172.21.15.64:0/3630358336 wait complete. 2023-11-24T08:04:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.970+0000 7fde6cc8c640 1 Processor -- start 2023-11-24T08:04:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.970+0000 7fde6cc8c640 1 -- start start 2023-11-24T08:04:47.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde6cc8c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde680b4140 0x7fde681f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde6cc8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde681435d0 0x7fde681f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde6cc8c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 0x7fde681f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde6cc8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde681f0e30 con 0x7fde681435d0 2023-11-24T08:04:47.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde6cc8c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680a9130 con 0x7fde680b4140 2023-11-24T08:04:47.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde6cc8c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde681f4da0 con 0x7fde681439b0 2023-11-24T08:04:47.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde681435d0 0x7fde681f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde681435d0 0x7fde681f4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56656/0 (socket says 172.21.15.64:56656) 2023-11-24T08:04:47.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde5ffff640 1 -- 172.21.15.64:0/4058505145 learned_addr learned my addr 172.21.15.64:0/4058505145 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:47.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.972+0000 7fde677fe640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde680b4140 0x7fde681f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde67fff640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 0x7fde681f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde5ffff640 1 -- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 msgr2=0x7fde681f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde5ffff640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 0x7fde681f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde5ffff640 1 -- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde680b4140 msgr2=0x7fde681f08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde5ffff640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde680b4140 0x7fde681f08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde5ffff640 1 -- 172.21.15.64:0/4058505145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500094d0 con 0x7fde681435d0 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde67fff640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 0x7fde681f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde5ffff640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde681435d0 0x7fde681f4860 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fde50009870 tx=0x7fde500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde64ff9640 1 -- 172.21.15.64:0/4058505145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5003ac60 con 0x7fde681435d0 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde64ff9640 1 -- 172.21.15.64:0/4058505145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5003ade0 con 0x7fde681435d0 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde64ff9640 1 -- 172.21.15.64:0/4058505145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50044630 con 0x7fde681435d0 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.973+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde681f3c10 con 0x7fde681435d0 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.974+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6813b910 con 0x7fde681435d0 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.975+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6804e840 con 0x7fde681435d0 2023-11-24T08:04:47.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.976+0000 7fde64ff9640 1 -- 172.21.15.64:0/4058505145 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde5002f050 con 0x7fde681435d0 2023-11-24T08:04:47.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.977+0000 7fde64ff9640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fde4806ae40 0x7fde4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:47.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.977+0000 7fde64ff9640 1 -- 172.21.15.64:0/4058505145 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fde500ab870 con 0x7fde681435d0 2023-11-24T08:04:47.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.977+0000 7fde677fe640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fde4806ae40 0x7fde4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:47.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.978+0000 7fde677fe640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fde4806ae40 0x7fde4806d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde58004ea0 tx=0x7fde58009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:47.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:47.981+0000 7fde64ff9640 1 -- 172.21.15.64:0/4058505145 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde500769a0 con 0x7fde681435d0 2023-11-24T08:04:48.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.179+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde68140d70 con 0x7fde681435d0 2023-11-24T08:04:48.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.180+0000 7fde64ff9640 1 -- 172.21.15.64:0/4058505145 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde50079fc0 con 0x7fde681435d0 2023-11-24T08:04:48.181 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:48.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fde4806ae40 msgr2=0x7fde4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fde4806ae40 0x7fde4806d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde58004ea0 tx=0x7fde58009040 comp rx=0 tx=0).stop 2023-11-24T08:04:48.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde681435d0 msgr2=0x7fde681f4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde681435d0 0x7fde681f4860 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fde50009870 tx=0x7fde500046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 shutdown_connections 2023-11-24T08:04:48.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fde4806ae40 0x7fde4806d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fde681439b0 0x7fde681f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde681435d0 0x7fde681f4860 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 --2- 172.21.15.64:0/4058505145 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fde680b4140 0x7fde681f08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 >> 172.21.15.64:0/4058505145 conn(0x7fde680cec10 msgr2=0x7fde680ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:48.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 shutdown_connections 2023-11-24T08:04:48.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.183+0000 7fde6cc8c640 1 -- 172.21.15.64:0/4058505145 wait complete. 2023-11-24T08:04:48.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.436+0000 7f5597e61640 1 Processor -- start 2023-11-24T08:04:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.436+0000 7f5597e61640 1 -- start start 2023-11-24T08:04:48.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.438+0000 7f5597e61640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f559015b350 0x7f55900abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.438+0000 7f5597e61640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559015b730 con 0x7f559009cb30 2023-11-24T08:04:48.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.438+0000 7f5597e61640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55900ac740 con 0x7f55900adb60 2023-11-24T08:04:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.438+0000 7f5597e61640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55900ac460 con 0x7f559015b350 2023-11-24T08:04:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.438+0000 7f55953fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f559009cb30 0x7f55900b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42656/0 (socket says 172.21.15.64:42656) 2023-11-24T08:04:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.438+0000 7f55953fc640 1 -- 172.21.15.64:0/3889095693 learned_addr learned my addr 172.21.15.64:0/3889095693 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.438+0000 7f5595bfd640 1 --2- 172.21.15.64:0/3889095693 >> v2:172.21.15.64:3300/0 conn(0x7f559015b350 0x7f55900abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.439+0000 7f5594bfb640 1 -- 172.21.15.64:0/3889095693 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073708560 0 0) 0x7f5580001490 con 0x7f559009cb30 2023-11-24T08:04:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.439+0000 7f5594bfb640 1 -- 172.21.15.64:0/3889095693 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f557c003520 con 0x7f559009cb30 2023-11-24T08:04:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.439+0000 7f5595bfd640 1 -- 172.21.15.64:0/3889095693 >> v1:172.21.15.91:6789/0 conn(0x7f55900adb60 legacy=0x7f55900adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.439+0000 7f5595bfd640 1 -- 172.21.15.64:0/3889095693 >> v1:172.21.15.64:6789/0 conn(0x7f559009cb30 legacy=0x7f55900b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.439+0000 7f5595bfd640 1 -- 172.21.15.64:0/3889095693 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55900b2a50 con 0x7f559015b350 2023-11-24T08:04:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.440+0000 7f5595bfd640 1 --2- 172.21.15.64:0/3889095693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559015b350 0x7f55900abe80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f558c0097a0 tx=0x7f558c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2864ef3d53c72d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.440+0000 7f5594bfb640 1 -- 172.21.15.64:0/3889095693 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c030610 con 0x7f559015b350 2023-11-24T08:04:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.440+0000 7f5594bfb640 1 -- 172.21.15.64:0/3889095693 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c030770 con 0x7f559015b350 2023-11-24T08:04:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.440+0000 7f5594bfb640 1 -- 172.21.15.64:0/3889095693 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c030ae0 con 0x7f559015b350 2023-11-24T08:04:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.441+0000 7f5597e61640 1 -- 172.21.15.64:0/3889095693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559015b350 msgr2=0x7f55900abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.441+0000 7f5597e61640 1 --2- 172.21.15.64:0/3889095693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559015b350 0x7f55900abe80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f558c0097a0 tx=0x7f558c02b960 comp rx=0 tx=0).stop 2023-11-24T08:04:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.441+0000 7f5597e61640 1 -- 172.21.15.64:0/3889095693 shutdown_connections 2023-11-24T08:04:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.441+0000 7f5597e61640 1 --2- 172.21.15.64:0/3889095693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559015b350 0x7f55900abe80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.441+0000 7f5597e61640 1 -- 172.21.15.64:0/3889095693 >> 172.21.15.64:0/3889095693 conn(0x7f55900b0ba0 msgr2=0x7f55900b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.441+0000 7f5597e61640 1 -- 172.21.15.64:0/3889095693 shutdown_connections 2023-11-24T08:04:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.441+0000 7f5597e61640 1 -- 172.21.15.64:0/3889095693 wait complete. 2023-11-24T08:04:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.443+0000 7f5597e61640 1 Processor -- start 2023-11-24T08:04:48.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.443+0000 7f5597e61640 1 -- start start 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5597e61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559009cb30 0x7f559014e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5597e61640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55900adb60 0x7f559014acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5597e61640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f559015b350 0x7f559013c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5597e61640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559014e540 con 0x7f559009cb30 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5597e61640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590147340 con 0x7f559015b350 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5597e61640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559014f380 con 0x7f55900adb60 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5595bfd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559009cb30 0x7f559014e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5595bfd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559009cb30 0x7f559014e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56682/0 (socket says 172.21.15.64:56682) 2023-11-24T08:04:48.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.444+0000 7f5595bfd640 1 -- 172.21.15.64:0/2143121973 learned_addr learned my addr 172.21.15.64:0/2143121973 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:48.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f55963fe640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f559015b350 0x7f559013c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f5595bfd640 1 -- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55900adb60 msgr2=0x7f559014acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f5595bfd640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55900adb60 0x7f559014acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f5595bfd640 1 -- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f559015b350 msgr2=0x7f559013c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f5595bfd640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f559015b350 0x7f559013c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f5595bfd640 1 -- 172.21.15.64:0/2143121973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c009400 con 0x7f559009cb30 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f5595bfd640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559009cb30 0x7f559014e000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f558c0097a0 tx=0x7f558c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f55867fc640 1 -- 172.21.15.64:0/2143121973 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c03f070 con 0x7f559009cb30 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f55867fc640 1 -- 172.21.15.64:0/2143121973 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c030e30 con 0x7f559009cb30 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f55867fc640 1 -- 172.21.15.64:0/2143121973 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c004230 con 0x7f559009cb30 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.445+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559014e770 con 0x7f559009cb30 2023-11-24T08:04:48.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.446+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559014ed30 con 0x7f559009cb30 2023-11-24T08:04:48.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.447+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559009d200 con 0x7f559009cb30 2023-11-24T08:04:48.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.448+0000 7f55867fc640 1 -- 172.21.15.64:0/2143121973 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f558c043070 con 0x7f559009cb30 2023-11-24T08:04:48.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.448+0000 7f55867fc640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f557406ae40 0x7f557406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.449+0000 7f55867fc640 1 -- 172.21.15.64:0/2143121973 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f558c0ab360 con 0x7f559009cb30 2023-11-24T08:04:48.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.449+0000 7f55953fc640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f557406ae40 0x7f557406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.451+0000 7f55953fc640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f557406ae40 0x7f557406d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5580005c00 tx=0x7f55800014d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:48.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.453+0000 7f55867fc640 1 -- 172.21.15.64:0/2143121973 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f558c076400 con 0x7f559009cb30 2023-11-24T08:04:48.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.647+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f559004e920 con 0x7f559009cb30 2023-11-24T08:04:48.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.648+0000 7f55867fc640 1 -- 172.21.15.64:0/2143121973 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f558c079a20 con 0x7f559009cb30 2023-11-24T08:04:48.649 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:48.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.650+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f557406ae40 msgr2=0x7f557406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.650+0000 7f5597e61640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f557406ae40 0x7f557406d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5580005c00 tx=0x7f55800014d0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.650+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559009cb30 msgr2=0x7f559014e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.650+0000 7f5597e61640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559009cb30 0x7f559014e000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f558c0097a0 tx=0x7f558c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.651+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 shutdown_connections 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.651+0000 7f5597e61640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f557406ae40 0x7f557406d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.651+0000 7f5597e61640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f559015b350 0x7f559013c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.651+0000 7f5597e61640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55900adb60 0x7f559014acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.651+0000 7f5597e61640 1 --2- 172.21.15.64:0/2143121973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f559009cb30 0x7f559014e000 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.651+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 >> 172.21.15.64:0/2143121973 conn(0x7f55900b0ba0 msgr2=0x7f5590151ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.651+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 shutdown_connections 2023-11-24T08:04:48.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.651+0000 7f5597e61640 1 -- 172.21.15.64:0/2143121973 wait complete. 2023-11-24T08:04:48.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:04:48.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.895+0000 7faa55fc9640 1 Processor -- start 2023-11-24T08:04:48.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.895+0000 7faa55fc9640 1 -- start start 2023-11-24T08:04:48.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.896+0000 7faa55fc9640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa500b3b60 0x7faa5006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.896+0000 7faa55fc9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.896+0000 7faa55fc9640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa500ad400 con 0x7faa5006da60 2023-11-24T08:04:48.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.896+0000 7faa55fc9640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5006dee0 con 0x7faa500b3b60 2023-11-24T08:04:48.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.897+0000 7faa55fc9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa500acb50 con 0x7faa500717a0 2023-11-24T08:04:48.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.897+0000 7faa4f7fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faa500b3b60 0x7faa5006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.897+0000 7faa4ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.897+0000 7faa4effd640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7faa5006da60 0x7faa50071090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:48856/0 (socket says 172.21.15.64:48856) 2023-11-24T08:04:48.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.897+0000 7faa4effd640 1 -- 172.21.15.64:0/1397895190 learned_addr learned my addr 172.21.15.64:0/1397895190 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:48.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.898+0000 7faa4f7fe640 1 -- 172.21.15.64:0/1397895190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 msgr2=0x7faa500ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:48.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.898+0000 7faa4f7fe640 1 --2- 172.21.15.64:0/1397895190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.898+0000 7faa4f7fe640 1 -- 172.21.15.64:0/1397895190 >> v1:172.21.15.91:6789/0 conn(0x7faa5006da60 legacy=0x7faa50071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.899+0000 7faa4f7fe640 1 -- 172.21.15.64:0/1397895190 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa500acdf0 con 0x7faa500b3b60 2023-11-24T08:04:48.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.899+0000 7faa4ffff640 1 --2- 172.21.15.64:0/1397895190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:04:48.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.898+0000 7faa4e7fc640 1 -- 172.21.15.64:0/1397895190 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047667040 0 0) 0x7faa500ad400 con 0x7faa5006da60 2023-11-24T08:04:48.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.900+0000 7faa4f7fe640 1 --2- 172.21.15.64:0/1397895190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 0x7faa5006d520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faa400097a0 tx=0x7faa4002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9185c83506215f22 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:48.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.901+0000 7faa4e7fc640 1 -- 172.21.15.64:0/1397895190 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4002d040 con 0x7faa500b3b60 2023-11-24T08:04:48.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.901+0000 7faa4e7fc640 1 -- 172.21.15.64:0/1397895190 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa40031810 con 0x7faa500b3b60 2023-11-24T08:04:48.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.901+0000 7faa4e7fc640 1 -- 172.21.15.64:0/1397895190 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40031ba0 con 0x7faa500b3b60 2023-11-24T08:04:48.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.902+0000 7faa55fc9640 1 -- 172.21.15.64:0/1397895190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 msgr2=0x7faa5006d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.902+0000 7faa55fc9640 1 --2- 172.21.15.64:0/1397895190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 0x7faa5006d520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faa400097a0 tx=0x7faa4002b8b0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.902+0000 7faa55fc9640 1 -- 172.21.15.64:0/1397895190 shutdown_connections 2023-11-24T08:04:48.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.902+0000 7faa55fc9640 1 --2- 172.21.15.64:0/1397895190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.902+0000 7faa55fc9640 1 --2- 172.21.15.64:0/1397895190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 0x7faa5006d520 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.902+0000 7faa55fc9640 1 -- 172.21.15.64:0/1397895190 >> 172.21.15.64:0/1397895190 conn(0x7faa500b0d80 msgr2=0x7faa500b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:48.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.902+0000 7faa55fc9640 1 -- 172.21.15.64:0/1397895190 shutdown_connections 2023-11-24T08:04:48.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.902+0000 7faa55fc9640 1 -- 172.21.15.64:0/1397895190 wait complete. 2023-11-24T08:04:48.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.904+0000 7faa55fc9640 1 Processor -- start 2023-11-24T08:04:48.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.904+0000 7faa55fc9640 1 -- start start 2023-11-24T08:04:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.905+0000 7faa55fc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa5006da60 0x7faa5015a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa55fc9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa55fc9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 0x7faa5014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa55fc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500c8510 con 0x7faa5006da60 2023-11-24T08:04:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa55fc9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50159a50 con 0x7faa500b3b60 2023-11-24T08:04:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa55fc9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa50159d30 con 0x7faa500717a0 2023-11-24T08:04:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa5006da60 0x7faa5015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa5006da60 0x7faa5015a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56686/0 (socket says 172.21.15.64:56686) 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4f7fe640 1 -- 172.21.15.64:0/4130956553 learned_addr learned my addr 172.21.15.64:0/4130956553 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4effd640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4ffff640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 0x7faa5014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4f7fe640 1 -- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 msgr2=0x7faa500afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4f7fe640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4f7fe640 1 -- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 msgr2=0x7faa5014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4f7fe640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 0x7faa5014c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:48.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.906+0000 7faa4f7fe640 1 -- 172.21.15.64:0/4130956553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48009a70 con 0x7faa5006da60 2023-11-24T08:04:48.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.907+0000 7faa4f7fe640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa5006da60 0x7faa5015a810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faa400098c0 tx=0x7faa400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:48.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.907+0000 7faa37fff640 1 -- 172.21.15.64:0/4130956553 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40006e10 con 0x7faa5006da60 2023-11-24T08:04:48.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.907+0000 7faa37fff640 1 -- 172.21.15.64:0/4130956553 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4003ad60 con 0x7faa5006da60 2023-11-24T08:04:48.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.907+0000 7faa37fff640 1 -- 172.21.15.64:0/4130956553 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40044620 con 0x7faa5006da60 2023-11-24T08:04:48.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.907+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa40009400 con 0x7faa5006da60 2023-11-24T08:04:48.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.907+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5015a2f0 con 0x7faa5006da60 2023-11-24T08:04:48.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.909+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa50074770 con 0x7faa5006da60 2023-11-24T08:04:48.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.909+0000 7faa37fff640 1 -- 172.21.15.64:0/4130956553 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa4004d440 con 0x7faa5006da60 2023-11-24T08:04:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.910+0000 7faa37fff640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faa3c06ae40 0x7faa3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.910+0000 7faa37fff640 1 -- 172.21.15.64:0/4130956553 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7faa400abb40 con 0x7faa5006da60 2023-11-24T08:04:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.911+0000 7faa4effd640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faa3c06ae40 0x7faa3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:48.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.912+0000 7faa4effd640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faa3c06ae40 0x7faa3c06d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa501f7720 tx=0x7faa44007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:48.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:48.915+0000 7faa37fff640 1 -- 172.21.15.64:0/4130956553 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa40076a30 con 0x7faa5006da60 2023-11-24T08:04:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.117+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa5006e320 con 0x7faa5006da60 2023-11-24T08:04:49.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.118+0000 7faa37fff640 1 -- 172.21.15.64:0/4130956553 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7faa40035020 con 0x7faa5006da60 2023-11-24T08:04:49.119 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:04:49.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faa3c06ae40 msgr2=0x7faa3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faa3c06ae40 0x7faa3c06d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa501f7720 tx=0x7faa44007c00 comp rx=0 tx=0).stop 2023-11-24T08:04:49.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa5006da60 msgr2=0x7faa5015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa5006da60 0x7faa5015a810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faa400098c0 tx=0x7faa400046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 shutdown_connections 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faa3c06ae40 0x7faa3c06d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faa500b3b60 0x7faa5014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faa500717a0 0x7faa500afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 --2- 172.21.15.64:0/4130956553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa5006da60 0x7faa5015a810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.120+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 >> 172.21.15.64:0/4130956553 conn(0x7faa500b0d80 msgr2=0x7faa50155e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.121+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 shutdown_connections 2023-11-24T08:04:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.121+0000 7faa55fc9640 1 -- 172.21.15.64:0/4130956553 wait complete. 2023-11-24T08:04:49.140 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:04:49.140 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:49.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:49.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.425+0000 7f242a0d1640 1 Processor -- start 2023-11-24T08:04:49.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.425+0000 7f242a0d1640 1 -- start start 2023-11-24T08:04:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.427+0000 7f242a0d1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242406d250 0x7f242415d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.427+0000 7f242a0d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f242415ddb0 0x7f2424070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.427+0000 7f242a0d1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2424161670 con 0x7f2424157770 2023-11-24T08:04:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.427+0000 7f242a0d1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242415e610 con 0x7f242415ddb0 2023-11-24T08:04:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.427+0000 7f242a0d1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2424160d40 con 0x7f242406d250 2023-11-24T08:04:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.427+0000 7f24290cf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2424157770 0x7f242406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42672/0 (socket says 172.21.15.64:42672) 2023-11-24T08:04:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.427+0000 7f24290cf640 1 -- 172.21.15.64:0/1001211535 learned_addr learned my addr 172.21.15.64:0/1001211535 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.427+0000 7f24298d0640 1 --2- 172.21.15.64:0/1001211535 >> v2:172.21.15.64:3300/0 conn(0x7f242415ddb0 0x7f2424070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.428+0000 7f24288ce640 1 --2- 172.21.15.64:0/1001211535 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242406d250 0x7f242415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.428+0000 7f241bfff640 1 -- 172.21.15.64:0/1001211535 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773331829 0 0) 0x7f2424161670 con 0x7f2424157770 2023-11-24T08:04:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.428+0000 7f241bfff640 1 -- 172.21.15.64:0/1001211535 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2410003520 con 0x7f2424157770 2023-11-24T08:04:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.428+0000 7f24298d0640 1 -- 172.21.15.64:0/1001211535 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242406d250 msgr2=0x7f242415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.428+0000 7f24298d0640 1 --2- 172.21.15.64:0/1001211535 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242406d250 0x7f242415d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.428+0000 7f24298d0640 1 -- 172.21.15.64:0/1001211535 >> v1:172.21.15.64:6789/0 conn(0x7f2424157770 legacy=0x7f242406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.428+0000 7f24298d0640 1 -- 172.21.15.64:0/1001211535 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2424161020 con 0x7f242415ddb0 2023-11-24T08:04:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.428+0000 7f24288ce640 1 --2- 172.21.15.64:0/1001211535 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242406d250 0x7f242415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:49.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.429+0000 7f24298d0640 1 --2- 172.21.15.64:0/1001211535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242415ddb0 0x7f2424070720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f241c009d90 tx=0x7f241c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57b9cef370cb314 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:49.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.429+0000 7f241bfff640 1 -- 172.21.15.64:0/1001211535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c02cc50 con 0x7f242415ddb0 2023-11-24T08:04:49.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.429+0000 7f241bfff640 1 -- 172.21.15.64:0/1001211535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c02cdb0 con 0x7f242415ddb0 2023-11-24T08:04:49.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.429+0000 7f241bfff640 1 -- 172.21.15.64:0/1001211535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c034640 con 0x7f242415ddb0 2023-11-24T08:04:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.430+0000 7f242a0d1640 1 -- 172.21.15.64:0/1001211535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242415ddb0 msgr2=0x7f2424070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.430+0000 7f242a0d1640 1 --2- 172.21.15.64:0/1001211535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242415ddb0 0x7f2424070720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f241c009d90 tx=0x7f241c02bea0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.430+0000 7f242a0d1640 1 -- 172.21.15.64:0/1001211535 shutdown_connections 2023-11-24T08:04:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.430+0000 7f242a0d1640 1 --2- 172.21.15.64:0/1001211535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f242415ddb0 0x7f2424070720 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.430+0000 7f242a0d1640 1 --2- 172.21.15.64:0/1001211535 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242406d250 0x7f242415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.430+0000 7f242a0d1640 1 -- 172.21.15.64:0/1001211535 >> 172.21.15.64:0/1001211535 conn(0x7f242406a860 msgr2=0x7f242406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.430+0000 7f242a0d1640 1 -- 172.21.15.64:0/1001211535 shutdown_connections 2023-11-24T08:04:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.430+0000 7f242a0d1640 1 -- 172.21.15.64:0/1001211535 wait complete. 2023-11-24T08:04:49.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.432+0000 7f242a0d1640 1 Processor -- start 2023-11-24T08:04:49.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.432+0000 7f242a0d1640 1 -- start start 2023-11-24T08:04:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f242a0d1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f242406d250 0x7f2424141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f242a0d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424157770 0x7f2424154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f242a0d1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242415ddb0 0x7f24240b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f242a0d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424141a90 con 0x7f2424157770 2023-11-24T08:04:49.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f242a0d1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24240cb020 con 0x7f242406d250 2023-11-24T08:04:49.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f242a0d1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24240cb300 con 0x7f242415ddb0 2023-11-24T08:04:49.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f24288ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424157770 0x7f2424154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f24288ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424157770 0x7f2424154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56708/0 (socket says 172.21.15.64:56708) 2023-11-24T08:04:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.433+0000 7f24288ce640 1 -- 172.21.15.64:0/3524392806 learned_addr learned my addr 172.21.15.64:0/3524392806 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f24288ce640 1 -- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242415ddb0 msgr2=0x7f24240b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f24288ce640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242415ddb0 0x7f24240b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f24288ce640 1 -- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f242406d250 msgr2=0x7f2424141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f24288ce640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f242406d250 0x7f2424141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f24288ce640 1 -- 172.21.15.64:0/3524392806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c009a70 con 0x7f2424157770 2023-11-24T08:04:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f24288ce640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424157770 0x7f2424154690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f241400e730 tx=0x7f241400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:49.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f2419ffb640 1 -- 172.21.15.64:0/3524392806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414004230 con 0x7f2424157770 2023-11-24T08:04:49.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f2419ffb640 1 -- 172.21.15.64:0/3524392806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2414004390 con 0x7f2424157770 2023-11-24T08:04:49.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f2419ffb640 1 -- 172.21.15.64:0/3524392806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400c8f0 con 0x7f2424157770 2023-11-24T08:04:49.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2424161c80 con 0x7f2424157770 2023-11-24T08:04:49.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.434+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24240cb8c0 con 0x7f2424157770 2023-11-24T08:04:49.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.436+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242404e840 con 0x7f2424157770 2023-11-24T08:04:49.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.436+0000 7f2419ffb640 1 -- 172.21.15.64:0/3524392806 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f241400ca50 con 0x7f2424157770 2023-11-24T08:04:49.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.437+0000 7f2419ffb640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f240806ae40 0x7f240806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.437+0000 7f2419ffb640 1 -- 172.21.15.64:0/3524392806 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2414089360 con 0x7f2424157770 2023-11-24T08:04:49.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.438+0000 7f24290cf640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f240806ae40 0x7f240806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:49.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.439+0000 7f24290cf640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f240806ae40 0x7f240806d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2420000f60 tx=0x7f24200078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:49.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.441+0000 7f2419ffb640 1 -- 172.21.15.64:0/3524392806 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24140542e0 con 0x7f2424157770 2023-11-24T08:04:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.639+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2424156f50 con 0x7f2424157770 2023-11-24T08:04:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.641+0000 7f2419ffb640 1 -- 172.21.15.64:0/3524392806 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7f2414057900 con 0x7f2424157770 2023-11-24T08:04:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:49.643 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f240806ae40 msgr2=0x7f240806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f240806ae40 0x7f240806d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2420000f60 tx=0x7f24200078f0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424157770 msgr2=0x7f2424154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424157770 0x7f2424154690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f241400e730 tx=0x7f241400ebf0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 shutdown_connections 2023-11-24T08:04:49.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f240806ae40 0x7f240806d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f242415ddb0 0x7f24240b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2424157770 0x7f2424154690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 --2- 172.21.15.64:0/3524392806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f242406d250 0x7f2424141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 >> 172.21.15.64:0/3524392806 conn(0x7f242406a860 msgr2=0x7f2424155710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 shutdown_connections 2023-11-24T08:04:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.644+0000 7f242a0d1640 1 -- 172.21.15.64:0/3524392806 wait complete. 2023-11-24T08:04:49.663 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:49.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:49.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.891+0000 7f45faa61640 1 Processor -- start 2023-11-24T08:04:49.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.891+0000 7f45faa61640 1 -- start start 2023-11-24T08:04:49.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.893+0000 7f45faa61640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f45f4162600 0x7f45f4162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.893+0000 7f45faa61640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f4162fa0 con 0x7f45f4162220 2023-11-24T08:04:49.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.893+0000 7f45faa61640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40706a0 con 0x7f45f406c3c0 2023-11-24T08:04:49.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.893+0000 7f45faa61640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f4070c60 con 0x7f45f4162600 2023-11-24T08:04:49.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.893+0000 7f45f3fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f45f4162220 0x7f45f40b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42674/0 (socket says 172.21.15.64:42674) 2023-11-24T08:04:49.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.893+0000 7f45f3fff640 1 -- 172.21.15.64:0/2149850180 learned_addr learned my addr 172.21.15.64:0/2149850180 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:49.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.893+0000 7f45f37fe640 1 --2- 172.21.15.64:0/2149850180 >> v2:172.21.15.64:3300/0 conn(0x7f45f4162600 0x7f45f4162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:49.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.894+0000 7f45f2ffd640 1 -- 172.21.15.64:0/2149850180 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231204392 0 0) 0x7f45f4162fa0 con 0x7f45f4162220 2023-11-24T08:04:49.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.894+0000 7f45f2ffd640 1 -- 172.21.15.64:0/2149850180 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e0003520 con 0x7f45f4162220 2023-11-24T08:04:49.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.894+0000 7f45f37fe640 1 -- 172.21.15.64:0/2149850180 >> v1:172.21.15.91:6789/0 conn(0x7f45f406c3c0 legacy=0x7f45f40710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.894+0000 7f45f37fe640 1 -- 172.21.15.64:0/2149850180 >> v1:172.21.15.64:6789/0 conn(0x7f45f4162220 legacy=0x7f45f40b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.894+0000 7f45f37fe640 1 -- 172.21.15.64:0/2149850180 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40722b0 con 0x7f45f4162600 2023-11-24T08:04:49.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.895+0000 7f45f37fe640 1 --2- 172.21.15.64:0/2149850180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f4162600 0x7f45f4162a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0097a0 tx=0x7f45dc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e05f1b158676ae9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:49.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.895+0000 7f45f2ffd640 1 -- 172.21.15.64:0/2149850180 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc030610 con 0x7f45f4162600 2023-11-24T08:04:49.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.895+0000 7f45f2ffd640 1 -- 172.21.15.64:0/2149850180 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc030770 con 0x7f45f4162600 2023-11-24T08:04:49.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.895+0000 7f45f2ffd640 1 -- 172.21.15.64:0/2149850180 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc030b00 con 0x7f45f4162600 2023-11-24T08:04:49.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.896+0000 7f45faa61640 1 -- 172.21.15.64:0/2149850180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f4162600 msgr2=0x7f45f4162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.896+0000 7f45faa61640 1 --2- 172.21.15.64:0/2149850180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f4162600 0x7f45f4162a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0097a0 tx=0x7f45dc02b960 comp rx=0 tx=0).stop 2023-11-24T08:04:49.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.896+0000 7f45faa61640 1 -- 172.21.15.64:0/2149850180 shutdown_connections 2023-11-24T08:04:49.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.896+0000 7f45faa61640 1 --2- 172.21.15.64:0/2149850180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f4162600 0x7f45f4162a60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.896+0000 7f45faa61640 1 -- 172.21.15.64:0/2149850180 >> 172.21.15.64:0/2149850180 conn(0x7f45f40b0ba0 msgr2=0x7f45f40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:49.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.896+0000 7f45faa61640 1 -- 172.21.15.64:0/2149850180 shutdown_connections 2023-11-24T08:04:49.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.896+0000 7f45faa61640 1 -- 172.21.15.64:0/2149850180 wait complete. 2023-11-24T08:04:49.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.898+0000 7f45faa61640 1 Processor -- start 2023-11-24T08:04:49.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.898+0000 7f45faa61640 1 -- start start 2023-11-24T08:04:49.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45faa61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f406c3c0 0x7f45f415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45faa61640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45f4162220 0x7f45f4156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45faa61640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45f4162600 0x7f45f40aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45faa61640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40cbf90 con 0x7f45f406c3c0 2023-11-24T08:04:49.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45faa61640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f415c490 con 0x7f45f4162600 2023-11-24T08:04:49.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45faa61640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4156a90 con 0x7f45f4162220 2023-11-24T08:04:49.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f406c3c0 0x7f45f415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:49.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f406c3c0 0x7f45f415bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56734/0 (socket says 172.21.15.64:56734) 2023-11-24T08:04:49.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.899+0000 7f45f3fff640 1 -- 172.21.15.64:0/3475254824 learned_addr learned my addr 172.21.15.64:0/3475254824 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:49.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f3fff640 1 -- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45f4162220 msgr2=0x7f45f4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f3fff640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45f4162220 0x7f45f4156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f3fff640 1 -- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45f4162600 msgr2=0x7f45f40aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:49.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f3fff640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45f4162600 0x7f45f40aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:49.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f3fff640 1 -- 172.21.15.64:0/3475254824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc009400 con 0x7f45f406c3c0 2023-11-24T08:04:49.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f3fff640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f406c3c0 0x7f45f415bf50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45e400cbf0 tx=0x7f45e400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:49.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f0ff9640 1 -- 172.21.15.64:0/3475254824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e400ace0 con 0x7f45f406c3c0 2023-11-24T08:04:49.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f0ff9640 1 -- 172.21.15.64:0/3475254824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e400ae40 con 0x7f45f406c3c0 2023-11-24T08:04:49.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45f0ff9640 1 -- 172.21.15.64:0/3475254824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e4012620 con 0x7f45f406c3c0 2023-11-24T08:04:49.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.900+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f415a6c0 con 0x7f45f406c3c0 2023-11-24T08:04:49.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.901+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f415ac80 con 0x7f45f406c3c0 2023-11-24T08:04:49.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.902+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f4067960 con 0x7f45f406c3c0 2023-11-24T08:04:49.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.902+0000 7f45f0ff9640 1 -- 172.21.15.64:0/3475254824 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45e4007070 con 0x7f45f406c3c0 2023-11-24T08:04:49.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.903+0000 7f45f0ff9640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f45d406ae40 0x7f45d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:49.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.903+0000 7f45f0ff9640 1 -- 172.21.15.64:0/3475254824 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f45e40895e0 con 0x7f45f406c3c0 2023-11-24T08:04:49.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.904+0000 7f45f37fe640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f45d406ae40 0x7f45d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:49.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.905+0000 7f45f37fe640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f45d406ae40 0x7f45d406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45dc036000 tx=0x7f45dc03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:49.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:49.908+0000 7f45f0ff9640 1 -- 172.21.15.64:0/3475254824 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45e4054690 con 0x7f45f406c3c0 2023-11-24T08:04:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.106+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45f40cc0d0 con 0x7f45f406c3c0 2023-11-24T08:04:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.106+0000 7f45f0ff9640 1 -- 172.21.15.64:0/3475254824 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45e4057cb0 con 0x7f45f406c3c0 2023-11-24T08:04:50.108 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:50.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f45d406ae40 msgr2=0x7f45d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f45d406ae40 0x7f45d406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45dc036000 tx=0x7f45dc03b040 comp rx=0 tx=0).stop 2023-11-24T08:04:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f406c3c0 msgr2=0x7f45f415bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f406c3c0 0x7f45f415bf50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45e400cbf0 tx=0x7f45e400a430 comp rx=0 tx=0).stop 2023-11-24T08:04:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 shutdown_connections 2023-11-24T08:04:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f45d406ae40 0x7f45d406d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f45f4162600 0x7f45f40aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f45f4162220 0x7f45f4156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 --2- 172.21.15.64:0/3475254824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45f406c3c0 0x7f45f415bf50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 >> 172.21.15.64:0/3475254824 conn(0x7f45f40b0ba0 msgr2=0x7f45f415e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 shutdown_connections 2023-11-24T08:04:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.109+0000 7f45faa61640 1 -- 172.21.15.64:0/3475254824 wait complete. 2023-11-24T08:04:50.131 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:50.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.364+0000 7f7d7fd0f640 1 Processor -- start 2023-11-24T08:04:50.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.364+0000 7f7d7fd0f640 1 -- start start 2023-11-24T08:04:50.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.366+0000 7f7d7fd0f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d780b48d0 0x7f7d780b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.366+0000 7f7d7fd0f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d780b23e0 con 0x7f7d780b3210 2023-11-24T08:04:50.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.366+0000 7f7d7fd0f640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d780d41b0 con 0x7f7d78160bd0 2023-11-24T08:04:50.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.366+0000 7f7d7fd0f640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d780cdc70 con 0x7f7d780b48d0 2023-11-24T08:04:50.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.366+0000 7f7d7e2ac640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7d780b3210 0x7f7d78167290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42688/0 (socket says 172.21.15.64:42688) 2023-11-24T08:04:50.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.366+0000 7f7d7e2ac640 1 -- 172.21.15.64:0/1528855759 learned_addr learned my addr 172.21.15.64:0/1528855759 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:50.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.366+0000 7f7d7d2aa640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d780b48d0 0x7f7d780b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537150128 0 0) 0x7f7d780b23e0 con 0x7f7d780b3210 2023-11-24T08:04:50.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d64003520 con 0x7f7d780b3210 2023-11-24T08:04:50.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4114102488 0 0) 0x7f7d64003520 con 0x7f7d780b3210 2023-11-24T08:04:50.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d780b48d0 msgr2=0x7f7d780b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:50.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d780b48d0 0x7f7d780b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 >> v1:172.21.15.91:6789/0 conn(0x7f7d78160bd0 legacy=0x7f7d780b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78160fb0 con 0x7f7d780b3210 2023-11-24T08:04:50.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f7d700019f0 con 0x7f7d780b3210 2023-11-24T08:04:50.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 >> v1:172.21.15.64:6789/0 conn(0x7f7d780b3210 legacy=0x7f7d78167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d780b48d0 0x7f7d640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d64003f40 0x7f7d640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d64004d00 0x7f7d64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d780cdc70 con 0x7f7d64004d00 2023-11-24T08:04:50.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6400bb40 con 0x7f7d64003f40 2023-11-24T08:04:50.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6400bd00 con 0x7f7d780b48d0 2023-11-24T08:04:50.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.367+0000 7f7d7e2ac640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d64004d00 0x7f7d64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.368+0000 7f7d7daab640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d780b48d0 0x7f7d640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.368+0000 7f7d7d2aa640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d64003f40 0x7f7d640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.368+0000 7f7d7e2ac640 1 -- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d780b48d0 msgr2=0x7f7d640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:50.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.368+0000 7f7d7e2ac640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d780b48d0 0x7f7d640039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.368+0000 7f7d7e2ac640 1 -- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d64003f40 msgr2=0x7f7d640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:50.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.368+0000 7f7d7e2ac640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d64003f40 0x7f7d640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.368+0000 7f7d7e2ac640 1 -- 172.21.15.64:0/1528855759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d64003520 con 0x7f7d64004d00 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.368+0000 7f7d7d2aa640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d64003f40 0x7f7d640047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.369+0000 7f7d7e2ac640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d64004d00 0x7f7d64005180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000a830 tx=0x7f7d640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5098bf4a760bc44e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.369+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70034630 con 0x7f7d64004d00 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.369+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7002f070 con 0x7f7d64004d00 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.369+0000 7f7d7caa9640 1 -- 172.21.15.64:0/1528855759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d700349d0 con 0x7f7d64004d00 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.370+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1528855759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d64004d00 msgr2=0x7f7d64005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.370+0000 7f7d7fd0f640 1 --2- 172.21.15.64:0/1528855759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d64004d00 0x7f7d64005180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000a830 tx=0x7f7d640057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.370+0000 7f7d7e2ac640 1 -- 172.21.15.64:0/1528855759 reap_dead start 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.370+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1528855759 shutdown_connections 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.370+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1528855759 >> 172.21.15.64:0/1528855759 conn(0x7f7d7809c5a0 msgr2=0x7f7d780d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.370+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1528855759 shutdown_connections 2023-11-24T08:04:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.370+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1528855759 wait complete. 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.372+0000 7f7d7fd0f640 1 Processor -- start 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.372+0000 7f7d7fd0f640 1 -- start start 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7fd0f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d64004d00 0x7f7d7815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7fd0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d780cad00 0x7f7d78154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7fd0f640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d780cc7a0 0x7f7d780ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7fd0f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d781545b0 con 0x7f7d780cad00 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7fd0f640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d780ca970 con 0x7f7d780cc7a0 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7fd0f640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d780ca3b0 con 0x7f7d64004d00 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7d2aa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d780cad00 0x7f7d78154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7d2aa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d780cad00 0x7f7d78154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56748/0 (socket says 172.21.15.64:56748) 2023-11-24T08:04:50.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.373+0000 7f7d7d2aa640 1 -- 172.21.15.64:0/1697483948 learned_addr learned my addr 172.21.15.64:0/1697483948 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7daab640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d64004d00 0x7f7d7815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7e2ac640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d780cc7a0 0x7f7d780ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7d2aa640 1 -- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d64004d00 msgr2=0x7f7d7815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7d2aa640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d64004d00 0x7f7d7815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7d2aa640 1 -- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d780cc7a0 msgr2=0x7f7d780ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7d2aa640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d780cc7a0 0x7f7d780ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7d2aa640 1 -- 172.21.15.64:0/1697483948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780ca690 con 0x7f7d780cad00 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7e2ac640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d780cc7a0 0x7f7d780ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:50.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7d2aa640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d780cad00 0x7f7d78154070 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d68010890 tx=0x7f7d68010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d6e7fc640 1 -- 172.21.15.64:0/1697483948 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68010ee0 con 0x7f7d780cad00 2023-11-24T08:04:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d6e7fc640 1 -- 172.21.15.64:0/1697483948 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6800eb60 con 0x7f7d780cad00 2023-11-24T08:04:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d6e7fc640 1 -- 172.21.15.64:0/1697483948 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6801c400 con 0x7f7d780cad00 2023-11-24T08:04:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.374+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d780c7760 con 0x7f7d780cad00 2023-11-24T08:04:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.375+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d78154a00 con 0x7f7d780cad00 2023-11-24T08:04:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.376+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d780cd8b0 con 0x7f7d780cad00 2023-11-24T08:04:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.377+0000 7f7d6e7fc640 1 -- 172.21.15.64:0/1697483948 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d6801c560 con 0x7f7d780cad00 2023-11-24T08:04:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.378+0000 7f7d6e7fc640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d5c06ae70 0x7f7d5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.378+0000 7f7d6e7fc640 1 -- 172.21.15.64:0/1697483948 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7d68089a30 con 0x7f7d780cad00 2023-11-24T08:04:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.378+0000 7f7d7daab640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d5c06ae70 0x7f7d5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.379+0000 7f7d7daab640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d5c06ae70 0x7f7d5c06d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7d74007fd0 tx=0x7f7d74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:50.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.382+0000 7f7d6e7fc640 1 -- 172.21.15.64:0/1697483948 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d68054b90 con 0x7f7d780cad00 2023-11-24T08:04:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.576+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d780a1210 con 0x7f7d780cad00 2023-11-24T08:04:50.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.577+0000 7f7d6e7fc640 1 -- 172.21.15.64:0/1697483948 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d680581b0 con 0x7f7d780cad00 2023-11-24T08:04:50.578 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:50.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.579+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d5c06ae70 msgr2=0x7f7d5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.579+0000 7f7d7fd0f640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d5c06ae70 0x7f7d5c06d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7d74007fd0 tx=0x7f7d74009040 comp rx=0 tx=0).stop 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.579+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d780cad00 msgr2=0x7f7d78154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.579+0000 7f7d7fd0f640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d780cad00 0x7f7d78154070 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d68010890 tx=0x7f7d68010c90 comp rx=0 tx=0).stop 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.580+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 shutdown_connections 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.580+0000 7f7d7fd0f640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d5c06ae70 0x7f7d5c06d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.580+0000 7f7d7fd0f640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d780cc7a0 0x7f7d780ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.580+0000 7f7d7fd0f640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d780cad00 0x7f7d78154070 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.580+0000 7f7d7fd0f640 1 --2- 172.21.15.64:0/1697483948 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d64004d00 0x7f7d7815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.580+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 >> 172.21.15.64:0/1697483948 conn(0x7f7d7809c5a0 msgr2=0x7f7d780cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.580+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 shutdown_connections 2023-11-24T08:04:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.580+0000 7f7d7fd0f640 1 -- 172.21.15.64:0/1697483948 wait complete. 2023-11-24T08:04:50.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.832+0000 7f4f58973640 1 Processor -- start 2023-11-24T08:04:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.832+0000 7f4f58973640 1 -- start start 2023-11-24T08:04:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.833+0000 7f4f58973640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 0x7f4f540b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.833+0000 7f4f58973640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4f540b2450 0x7f4f5415c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f58973640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f540b13b0 con 0x7f4f540d0ca0 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f58973640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f540b11c0 con 0x7f4f540b2450 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f58973640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5415c640 con 0x7f4f5415af50 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f53fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4f540d0ca0 0x7f4f540d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42694/0 (socket says 172.21.15.64:42694) 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f53fff640 1 -- 172.21.15.64:0/1372824669 learned_addr learned my addr 172.21.15.64:0/1372824669 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f537fe640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 0x7f4f540b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f52ffd640 1 --2- 172.21.15.64:0/1372824669 >> v2:172.21.15.91:3300/0 conn(0x7f4f540b2450 0x7f4f5415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462103226 0 0) 0x7f4f540b13b0 con 0x7f4f540d0ca0 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.834+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f40003520 con 0x7f4f540d0ca0 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1042092015 0 0) 0x7f4f40003520 con 0x7f4f540d0ca0 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 msgr2=0x7f4f540b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 0x7f4f540b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 >> v2:172.21.15.91:3300/0 conn(0x7f4f540b2450 msgr2=0x7f4f5415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 --2- 172.21.15.64:0/1372824669 >> v2:172.21.15.91:3300/0 conn(0x7f4f540b2450 0x7f4f5415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540cc830 con 0x7f4f540d0ca0 2023-11-24T08:04:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f4f4c001670 con 0x7f4f540d0ca0 2023-11-24T08:04:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 >> v1:172.21.15.64:6789/0 conn(0x7f4f540d0ca0 legacy=0x7f4f540d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 0x7f4f400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f40003fb0 0x7f4f400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40004d00 0x7f4f40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4000b980 con 0x7f4f40004d00 2023-11-24T08:04:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4000bac0 con 0x7f4f40003fb0 2023-11-24T08:04:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4000bc80 con 0x7f4f5415af50 2023-11-24T08:04:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.835+0000 7f4f53fff640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40004d00 0x7f4f40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f537fe640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 0x7f4f400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f52ffd640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f40003fb0 0x7f4f400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f53fff640 1 -- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 msgr2=0x7f4f400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f53fff640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 0x7f4f400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f53fff640 1 -- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f40003fb0 msgr2=0x7f4f400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f53fff640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f40003fb0 0x7f4f400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f53fff640 1 -- 172.21.15.64:0/1372824669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f40003520 con 0x7f4f40004d00 2023-11-24T08:04:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f537fe640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f5415af50 0x7f4f400039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.836+0000 7f4f52ffd640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f40003fb0 0x7f4f400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.837+0000 7f4f53fff640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40004d00 0x7f4f40005180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c0e0 tx=0x7f4f400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2d46dd7a5c30e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.837+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c005ad0 con 0x7f4f40004d00 2023-11-24T08:04:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.837+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c005c50 con 0x7f4f40004d00 2023-11-24T08:04:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.837+0000 7f4f527fc640 1 -- 172.21.15.64:0/1372824669 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c02c810 con 0x7f4f40004d00 2023-11-24T08:04:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.838+0000 7f4f58973640 1 -- 172.21.15.64:0/1372824669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40004d00 msgr2=0x7f4f40005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.838+0000 7f4f58973640 1 --2- 172.21.15.64:0/1372824669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40004d00 0x7f4f40005180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c0e0 tx=0x7f4f400057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.838+0000 7f4f53fff640 1 -- 172.21.15.64:0/1372824669 reap_dead start 2023-11-24T08:04:50.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.838+0000 7f4f58973640 1 -- 172.21.15.64:0/1372824669 shutdown_connections 2023-11-24T08:04:50.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.838+0000 7f4f58973640 1 -- 172.21.15.64:0/1372824669 >> 172.21.15.64:0/1372824669 conn(0x7f4f54099a00 msgr2=0x7f4f54099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:50.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.838+0000 7f4f58973640 1 -- 172.21.15.64:0/1372824669 shutdown_connections 2023-11-24T08:04:50.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.838+0000 7f4f58973640 1 -- 172.21.15.64:0/1372824669 wait complete. 2023-11-24T08:04:50.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.840+0000 7f4f58973640 1 Processor -- start 2023-11-24T08:04:50.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.840+0000 7f4f58973640 1 -- start start 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f58973640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f40004d00 0x7f4f54157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f58973640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f540c4dd0 0x7f4f540c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f58973640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f540b6680 0x7f4f54158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f58973640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f54157e60 con 0x7f4f540c4dd0 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f58973640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f54158140 con 0x7f4f540b6680 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f58973640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f54158420 con 0x7f4f40004d00 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f540c4dd0 0x7f4f540c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f540c4dd0 0x7f4f540c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56758/0 (socket says 172.21.15.64:56758) 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f52ffd640 1 -- 172.21.15.64:0/338877650 learned_addr learned my addr 172.21.15.64:0/338877650 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.841+0000 7f4f537fe640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f40004d00 0x7f4f54157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f52ffd640 1 -- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f40004d00 msgr2=0x7f4f54157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f53fff640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f540b6680 0x7f4f54158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f52ffd640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f40004d00 0x7f4f54157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f52ffd640 1 -- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f540b6680 msgr2=0x7f4f54158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f52ffd640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f540b6680 0x7f4f54158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f52ffd640 1 -- 172.21.15.64:0/338877650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f54158700 con 0x7f4f540c4dd0 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f52ffd640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f540c4dd0 0x7f4f540c9e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0108f0 tx=0x7f4f3c010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f4bfff640 1 -- 172.21.15.64:0/338877650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c00eb60 con 0x7f4f540c4dd0 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f4bfff640 1 -- 172.21.15.64:0/338877650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c00ecc0 con 0x7f4f540c4dd0 2023-11-24T08:04:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f4bfff640 1 -- 172.21.15.64:0/338877650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c01c400 con 0x7f4f540c4dd0 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f54157b70 con 0x7f4f540c4dd0 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.842+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f54159470 con 0x7f4f540c4dd0 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.844+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5404e840 con 0x7f4f540c4dd0 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.844+0000 7f4f4bfff640 1 -- 172.21.15.64:0/338877650 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f3c01c560 con 0x7f4f540c4dd0 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.845+0000 7f4f4bfff640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4f3406ae70 0x7f4f3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.846+0000 7f4f4bfff640 1 -- 172.21.15.64:0/338877650 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4f3c089c40 con 0x7f4f540c4dd0 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.846+0000 7f4f537fe640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4f3406ae70 0x7f4f3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.847+0000 7f4f537fe640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4f3406ae70 0x7f4f3406d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f44007440 tx=0x7f4f4400d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:50.850+0000 7f4f4bfff640 1 -- 172.21.15.64:0/338877650 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f3c054d80 con 0x7f4f540c4dd0 2023-11-24T08:04:51.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.049+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f5415a930 con 0x7f4f540c4dd0 2023-11-24T08:04:51.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.050+0000 7f4f4bfff640 1 -- 172.21.15.64:0/338877650 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f3c0583a0 con 0x7f4f540c4dd0 2023-11-24T08:04:51.055 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:51.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4f3406ae70 msgr2=0x7f4f3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4f3406ae70 0x7f4f3406d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f44007440 tx=0x7f4f4400d040 comp rx=0 tx=0).stop 2023-11-24T08:04:51.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f540c4dd0 msgr2=0x7f4f540c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f540c4dd0 0x7f4f540c9e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0108f0 tx=0x7f4f3c010db0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 shutdown_connections 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4f3406ae70 0x7f4f3406d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4f540b6680 0x7f4f54158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f540c4dd0 0x7f4f540c9e80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 --2- 172.21.15.64:0/338877650 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4f40004d00 0x7f4f54157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 >> 172.21.15.64:0/338877650 conn(0x7f4f54099a00 msgr2=0x7f4f540ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.052+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 shutdown_connections 2023-11-24T08:04:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.053+0000 7f4f58973640 1 -- 172.21.15.64:0/338877650 wait complete. 2023-11-24T08:04:51.072 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:04:51.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.295+0000 7fb027fff640 1 Processor -- start 2023-11-24T08:04:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.296+0000 7fb027fff640 1 -- start start 2023-11-24T08:04:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.297+0000 7fb027fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb028070550 0x7fb028160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.298+0000 7fb027fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0280b3040 con 0x7fb0280b2550 2023-11-24T08:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.298+0000 7fb027fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb028070cf0 con 0x7fb028161530 2023-11-24T08:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.298+0000 7fb027fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0281619a0 con 0x7fb028070550 2023-11-24T08:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.298+0000 7fb026ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb0280b2550 0x7fb0280b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42708/0 (socket says 172.21.15.64:42708) 2023-11-24T08:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.298+0000 7fb026ffd640 1 -- 172.21.15.64:0/986609140 learned_addr learned my addr 172.21.15.64:0/986609140 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.298+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223336119 0 0) 0x7fb0280b3040 con 0x7fb0280b2550 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.298+0000 7fb0267fc640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb028070550 0x7fb028160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.298+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb010003520 con 0x7fb0280b2550 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3511576677 0 0) 0x7fb010003520 con 0x7fb0280b2550 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb028070550 msgr2=0x7fb028160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb028070550 0x7fb028160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 >> v1:172.21.15.91:6789/0 conn(0x7fb028161530 legacy=0x7fb0280ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb028070fd0 con 0x7fb0280b2550 2023-11-24T08:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fb018001670 con 0x7fb0280b2550 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 >> v1:172.21.15.64:6789/0 conn(0x7fb0280b2550 legacy=0x7fb0280b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010003a10 0x7fb010003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0100047c0 0x7fb010004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb028070550 0x7fb01000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0281619a0 con 0x7fb010003a10 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100054e0 con 0x7fb0100047c0 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01000bec0 con 0x7fb028070550 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb026ffd640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010003a10 0x7fb010003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb0267fc640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0100047c0 0x7fb010004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb0277fe640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb028070550 0x7fb01000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb026ffd640 1 -- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb028070550 msgr2=0x7fb01000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.299+0000 7fb026ffd640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb028070550 0x7fb01000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.300+0000 7fb026ffd640 1 -- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0100047c0 msgr2=0x7fb010004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.300+0000 7fb026ffd640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0100047c0 0x7fb010004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.300+0000 7fb026ffd640 1 -- 172.21.15.64:0/986609140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010003520 con 0x7fb010003a10 2023-11-24T08:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.300+0000 7fb0267fc640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0100047c0 0x7fb010004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.301+0000 7fb026ffd640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010003a10 0x7fb010003e30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb01800c0e0 tx=0x7fb0100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939e241c1833270e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.301+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018005ad0 con 0x7fb010003a10 2023-11-24T08:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.301+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb018005c30 con 0x7fb010003a10 2023-11-24T08:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.301+0000 7fb025ffb640 1 -- 172.21.15.64:0/986609140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01802ca30 con 0x7fb010003a10 2023-11-24T08:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.301+0000 7fb027fff640 1 -- 172.21.15.64:0/986609140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010003a10 msgr2=0x7fb010003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.301+0000 7fb027fff640 1 --2- 172.21.15.64:0/986609140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010003a10 0x7fb010003e30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb01800c0e0 tx=0x7fb0100057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.301+0000 7fb0277fe640 1 -- 172.21.15.64:0/986609140 reap_dead start 2023-11-24T08:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.302+0000 7fb027fff640 1 -- 172.21.15.64:0/986609140 shutdown_connections 2023-11-24T08:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.302+0000 7fb027fff640 1 -- 172.21.15.64:0/986609140 >> 172.21.15.64:0/986609140 conn(0x7fb0280b0840 msgr2=0x7fb0280b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.302+0000 7fb027fff640 1 -- 172.21.15.64:0/986609140 shutdown_connections 2023-11-24T08:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.302+0000 7fb027fff640 1 -- 172.21.15.64:0/986609140 wait complete. 2023-11-24T08:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.303+0000 7fb027fff640 1 Processor -- start 2023-11-24T08:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.304+0000 7fb027fff640 1 -- start start 2023-11-24T08:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb027fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb010003a10 0x7fb0280c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb027fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0280ac870 0x7fb02815dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb027fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0281553e0 0x7fb028155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb027fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb028155da0 con 0x7fb0281553e0 2023-11-24T08:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb027fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280ad900 con 0x7fb0280ac870 2023-11-24T08:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb027fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280adf20 con 0x7fb010003a10 2023-11-24T08:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0281553e0 0x7fb028155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0281553e0 0x7fb028155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56780/0 (socket says 172.21.15.64:56780) 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0277fe640 1 -- 172.21.15.64:0/2764660692 learned_addr learned my addr 172.21.15.64:0/2764660692 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb026ffd640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb010003a10 0x7fb0280c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0267fc640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0280ac870 0x7fb02815dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0277fe640 1 -- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb010003a10 msgr2=0x7fb0280c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0277fe640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb010003a10 0x7fb0280c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0277fe640 1 -- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0280ac870 msgr2=0x7fb02815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0277fe640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0280ac870 0x7fb02815dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0277fe640 1 -- 172.21.15.64:0/2764660692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280ad060 con 0x7fb0281553e0 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.305+0000 7fb0267fc640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0280ac870 0x7fb02815dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.306+0000 7fb0277fe640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0281553e0 0x7fb028155860 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb028161910 tx=0x7fb02000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.306+0000 7fb00f7fe640 1 -- 172.21.15.64:0/2764660692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020003b70 con 0x7fb0281553e0 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.306+0000 7fb00f7fe640 1 -- 172.21.15.64:0/2764660692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb020003cd0 con 0x7fb0281553e0 2023-11-24T08:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.306+0000 7fb00f7fe640 1 -- 172.21.15.64:0/2764660692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200052b0 con 0x7fb0281553e0 2023-11-24T08:04:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.306+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0280ad340 con 0x7fb0281553e0 2023-11-24T08:04:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.306+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0280accf0 con 0x7fb0281553e0 2023-11-24T08:04:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.307+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0280adcd0 con 0x7fb0281553e0 2023-11-24T08:04:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.308+0000 7fb00f7fe640 1 -- 172.21.15.64:0/2764660692 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb020005410 con 0x7fb0281553e0 2023-11-24T08:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.309+0000 7fb00f7fe640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb01406ae70 0x7fb01406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.309+0000 7fb00f7fe640 1 -- 172.21.15.64:0/2764660692 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb020088e20 con 0x7fb0281553e0 2023-11-24T08:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.310+0000 7fb026ffd640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb01406ae70 0x7fb01406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.311+0000 7fb026ffd640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb01406ae70 0x7fb01406d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb018034000 tx=0x7fb018009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.314+0000 7fb00f7fe640 1 -- 172.21.15.64:0/2764660692 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb020053da0 con 0x7fb0281553e0 2023-11-24T08:04:51.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.521+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb02804e840 con 0x7fb0281553e0 2023-11-24T08:04:51.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.522+0000 7fb00f7fe640 1 -- 172.21.15.64:0/2764660692 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb020018080 con 0x7fb0281553e0 2023-11-24T08:04:51.523 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:04:51.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.524+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb01406ae70 msgr2=0x7fb01406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.524+0000 7fb027fff640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb01406ae70 0x7fb01406d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb018034000 tx=0x7fb018009030 comp rx=0 tx=0).stop 2023-11-24T08:04:51.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.524+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0281553e0 msgr2=0x7fb028155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:51.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.524+0000 7fb027fff640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0281553e0 0x7fb028155860 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb028161910 tx=0x7fb02000dc00 comp rx=0 tx=0).stop 2023-11-24T08:04:51.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.525+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 shutdown_connections 2023-11-24T08:04:51.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.525+0000 7fb027fff640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb01406ae70 0x7fb01406d330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.525+0000 7fb027fff640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0281553e0 0x7fb028155860 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.525+0000 7fb027fff640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb0280ac870 0x7fb02815dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.525+0000 7fb027fff640 1 --2- 172.21.15.64:0/2764660692 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb010003a10 0x7fb0280c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:51.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.525+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 >> 172.21.15.64:0/2764660692 conn(0x7fb0280b0840 msgr2=0x7fb02815cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:51.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.525+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 shutdown_connections 2023-11-24T08:04:51.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:51.525+0000 7fb027fff640 1 -- 172.21.15.64:0/2764660692 wait complete. 2023-11-24T08:04:51.547 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi091.front.sepia.ceph.com... 2023-11-24T08:04:51.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:51.547 DEBUG:teuthology.orchestra.run.smithi091:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T08:04:51.653 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:04:51.653 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:04:51.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:51.654 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:04:51.720 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:04:51.720 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:51.784 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:04:51.785 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:51.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:52.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.062+0000 7fb65bfff640 1 Processor -- start 2023-11-24T08:04:52.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.062+0000 7fb65bfff640 1 -- start start 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.063+0000 7fb65bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c143570 0x7fb65c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.063+0000 7fb65bfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb65c143950 0x7fb65c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.063+0000 7fb65bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb65c15e6b0 0x7fb65c15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.063+0000 7fb65bfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c071700 con 0x7fb65c15e6b0 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.063+0000 7fb65bfff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c15b410 con 0x7fb65c143950 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.063+0000 7fb65bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c15b130 con 0x7fb65c143570 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.064+0000 7fb65b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb65c15e6b0 0x7fb65c15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.064+0000 7fb65b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb65c15e6b0 0x7fb65c15a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56796/0 (socket says 172.21.15.64:56796) 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.064+0000 7fb65b7fe640 1 -- 172.21.15.64:0/534056740 learned_addr learned my addr 172.21.15.64:0/534056740 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.064+0000 7fb653fff640 1 --2- 172.21.15.64:0/534056740 >> v2:172.21.15.91:3300/0 conn(0x7fb65c143950 0x7fb65c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.064+0000 7fb65affd640 1 --2- 172.21.15.64:0/534056740 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c143570 0x7fb65c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.065+0000 7fb65b7fe640 1 -- 172.21.15.64:0/534056740 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c143570 msgr2=0x7fb65c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.065+0000 7fb65b7fe640 1 --2- 172.21.15.64:0/534056740 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c143570 0x7fb65c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.065+0000 7fb65b7fe640 1 -- 172.21.15.64:0/534056740 >> v2:172.21.15.91:3300/0 conn(0x7fb65c143950 msgr2=0x7fb65c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.065+0000 7fb65b7fe640 1 --2- 172.21.15.64:0/534056740 >> v2:172.21.15.91:3300/0 conn(0x7fb65c143950 0x7fb65c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.065+0000 7fb65b7fe640 1 -- 172.21.15.64:0/534056740 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c15dbd0 con 0x7fb65c15e6b0 2023-11-24T08:04:52.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.066+0000 7fb65b7fe640 1 --2- 172.21.15.64:0/534056740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c15e6b0 0x7fb65c15a4b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0cfd00 tx=0x7fb65402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20397601b7ff8549 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.066+0000 7fb65a7fc640 1 -- 172.21.15.64:0/534056740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65402c9f0 con 0x7fb65c15e6b0 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.066+0000 7fb65a7fc640 1 -- 172.21.15.64:0/534056740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65402cb50 con 0x7fb65c15e6b0 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.066+0000 7fb65a7fc640 1 -- 172.21.15.64:0/534056740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6540343f0 con 0x7fb65c15e6b0 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 -- 172.21.15.64:0/534056740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c15e6b0 msgr2=0x7fb65c15a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 --2- 172.21.15.64:0/534056740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c15e6b0 0x7fb65c15a4b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0cfd00 tx=0x7fb65402be50 comp rx=0 tx=0).stop 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 -- 172.21.15.64:0/534056740 shutdown_connections 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 --2- 172.21.15.64:0/534056740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c15e6b0 0x7fb65c15a4b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 --2- 172.21.15.64:0/534056740 >> v2:172.21.15.91:3300/0 conn(0x7fb65c143950 0x7fb65c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 --2- 172.21.15.64:0/534056740 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c143570 0x7fb65c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 -- 172.21.15.64:0/534056740 >> 172.21.15.64:0/534056740 conn(0x7fb65c06a860 msgr2=0x7fb65c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 -- 172.21.15.64:0/534056740 shutdown_connections 2023-11-24T08:04:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.067+0000 7fb65bfff640 1 -- 172.21.15.64:0/534056740 wait complete. 2023-11-24T08:04:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.069+0000 7fb65bfff640 1 Processor -- start 2023-11-24T08:04:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.069+0000 7fb65bfff640 1 -- start start 2023-11-24T08:04:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb65bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb65c143570 0x7fb65c1f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb65bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c143950 0x7fb65c0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb65bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c15e6b0 0x7fb65c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb65bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0c8b00 con 0x7fb65c143950 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb65bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1f2f00 con 0x7fb65c143570 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb65bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1f6c50 con 0x7fb65c15e6b0 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb653fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c143950 0x7fb65c0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb653fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c143950 0x7fb65c0c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56804/0 (socket says 172.21.15.64:56804) 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb653fff640 1 -- 172.21.15.64:0/1115949725 learned_addr learned my addr 172.21.15.64:0/1115949725 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb65affd640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb65c143570 0x7fb65c1f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.070+0000 7fb65b7fe640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c15e6b0 0x7fb65c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb653fff640 1 -- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c15e6b0 msgr2=0x7fb65c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb653fff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c15e6b0 0x7fb65c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb653fff640 1 -- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb65c143570 msgr2=0x7fb65c1f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb653fff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb65c143570 0x7fb65c1f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb653fff640 1 -- 172.21.15.64:0/1115949725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb654009b30 con 0x7fb65c143950 2023-11-24T08:04:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb653fff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c143950 0x7fb65c0c85c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb64400d7d0 tx=0x7fb64400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb6537fe640 1 -- 172.21.15.64:0/1115949725 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400b6c0 con 0x7fb65c143950 2023-11-24T08:04:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb6537fe640 1 -- 172.21.15.64:0/1115949725 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64400b840 con 0x7fb65c143950 2023-11-24T08:04:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c1f6f30 con 0x7fb65c143950 2023-11-24T08:04:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb6537fe640 1 -- 172.21.15.64:0/1115949725 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400bbf0 con 0x7fb65c143950 2023-11-24T08:04:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.071+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c1f3170 con 0x7fb65c143950 2023-11-24T08:04:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.072+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c1f3300 con 0x7fb65c143950 2023-11-24T08:04:52.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.073+0000 7fb6537fe640 1 -- 172.21.15.64:0/1115949725 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb64400bd50 con 0x7fb65c143950 2023-11-24T08:04:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.074+0000 7fb6537fe640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb63c06ae40 0x7fb63c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.074+0000 7fb6537fe640 1 -- 172.21.15.64:0/1115949725 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb644089bb0 con 0x7fb65c143950 2023-11-24T08:04:52.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.075+0000 7fb65affd640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb63c06ae40 0x7fb63c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.076+0000 7fb65affd640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb63c06ae40 0x7fb63c06d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb64c006fd0 tx=0x7fb64c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.079+0000 7fb6537fe640 1 -- 172.21.15.64:0/1115949725 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb644054bc0 con 0x7fb65c143950 2023-11-24T08:04:52.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.279+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb65c04e840 con 0x7fb65c143950 2023-11-24T08:04:52.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.282+0000 7fb6537fe640 1 -- 172.21.15.64:0/1115949725 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7fb6440581e0 con 0x7fb65c143950 2023-11-24T08:04:52.284 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:52.284 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb63c06ae40 msgr2=0x7fb63c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb63c06ae40 0x7fb63c06d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb64c006fd0 tx=0x7fb64c008040 comp rx=0 tx=0).stop 2023-11-24T08:04:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c143950 msgr2=0x7fb65c0c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c143950 0x7fb65c0c85c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb64400d7d0 tx=0x7fb64400dc90 comp rx=0 tx=0).stop 2023-11-24T08:04:52.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 shutdown_connections 2023-11-24T08:04:52.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb63c06ae40 0x7fb63c06d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb65c15e6b0 0x7fb65c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb65c143950 0x7fb65c0c85c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 --2- 172.21.15.64:0/1115949725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb65c143570 0x7fb65c1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.285+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 >> 172.21.15.64:0/1115949725 conn(0x7fb65c06a860 msgr2=0x7fb65c0cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:52.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.286+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 shutdown_connections 2023-11-24T08:04:52.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.286+0000 7fb65bfff640 1 -- 172.21.15.64:0/1115949725 wait complete. 2023-11-24T08:04:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:52.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.538+0000 7f6c2ffff640 1 Processor -- start 2023-11-24T08:04:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.538+0000 7f6c2ffff640 1 -- start start 2023-11-24T08:04:52.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.540+0000 7f6c2ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c3006a440 0x7f6c3006a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6c300adde0 0x7f6c3015f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2ffff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c300cf440 con 0x7f6c300cefb0 2023-11-24T08:04:52.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3006fa40 con 0x7f6c3006a440 2023-11-24T08:04:52.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2ffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3006ee00 con 0x7f6c300adde0 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c3006a440 0x7f6c3006a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c3006a440 0x7f6c3006a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56814/0 (socket says 172.21.15.64:56814) 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2effd640 1 -- 172.21.15.64:0/2995395564 learned_addr learned my addr 172.21.15.64:0/2995395564 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2f7fe640 1 --2- 172.21.15.64:0/2995395564 >> v2:172.21.15.91:3300/0 conn(0x7f6c300adde0 0x7f6c3015f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2effd640 1 -- 172.21.15.64:0/2995395564 >> v2:172.21.15.91:3300/0 conn(0x7f6c300adde0 msgr2=0x7f6c3015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2effd640 1 --2- 172.21.15.64:0/2995395564 >> v2:172.21.15.91:3300/0 conn(0x7f6c300adde0 0x7f6c3015f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2effd640 1 -- 172.21.15.64:0/2995395564 >> v1:172.21.15.91:6789/0 conn(0x7f6c300cefb0 legacy=0x7f6c3015ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.541+0000 7f6c2effd640 1 -- 172.21.15.64:0/2995395564 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3006ad60 con 0x7f6c3006a440 2023-11-24T08:04:52.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.542+0000 7f6c2f7fe640 1 --2- 172.21.15.64:0/2995395564 >> v2:172.21.15.91:3300/0 conn(0x7f6c300adde0 0x7f6c3015f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:52.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.543+0000 7f6c2effd640 1 --2- 172.21.15.64:0/2995395564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c3006a440 0x7f6c3006a820 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6c300af1d0 tx=0x7f6c2002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a304fea7adc40753 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:52.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.543+0000 7f6c2e7fc640 1 -- 172.21.15.64:0/2995395564 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200317e0 con 0x7f6c3006a440 2023-11-24T08:04:52.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.543+0000 7f6c2e7fc640 1 -- 172.21.15.64:0/2995395564 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20031940 con 0x7f6c3006a440 2023-11-24T08:04:52.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.543+0000 7f6c2e7fc640 1 -- 172.21.15.64:0/2995395564 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20031cd0 con 0x7f6c3006a440 2023-11-24T08:04:52.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.544+0000 7f6c2ffff640 1 -- 172.21.15.64:0/2995395564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c3006a440 msgr2=0x7f6c3006a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.544+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/2995395564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c3006a440 0x7f6c3006a820 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6c300af1d0 tx=0x7f6c2002b880 comp rx=0 tx=0).stop 2023-11-24T08:04:52.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.544+0000 7f6c2ffff640 1 -- 172.21.15.64:0/2995395564 shutdown_connections 2023-11-24T08:04:52.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.544+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/2995395564 >> v2:172.21.15.91:3300/0 conn(0x7f6c300adde0 0x7f6c3015f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.544+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/2995395564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c3006a440 0x7f6c3006a820 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.544+0000 7f6c2ffff640 1 -- 172.21.15.64:0/2995395564 >> 172.21.15.64:0/2995395564 conn(0x7f6c300cfd80 msgr2=0x7f6c300d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:52.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.544+0000 7f6c2ffff640 1 -- 172.21.15.64:0/2995395564 shutdown_connections 2023-11-24T08:04:52.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.544+0000 7f6c2ffff640 1 -- 172.21.15.64:0/2995395564 wait complete. 2023-11-24T08:04:52.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.546+0000 7f6c2ffff640 1 Processor -- start 2023-11-24T08:04:52.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.547+0000 7f6c2ffff640 1 -- start start 2023-11-24T08:04:52.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3006a440 0x7f6c3013a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c300adde0 0x7f6c300b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c30152260 0x7f6c301526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30152c20 con 0x7f6c30152260 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300b4720 con 0x7f6c300adde0 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300b3140 con 0x7f6c3006a440 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c30152260 0x7f6c301526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c30152260 0x7f6c301526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56824/0 (socket says 172.21.15.64:56824) 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.548+0000 7f6c2f7fe640 1 -- 172.21.15.64:0/1988859193 learned_addr learned my addr 172.21.15.64:0/1988859193 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c267fc640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c300adde0 0x7f6c300b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c2effd640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3006a440 0x7f6c3013a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c2f7fe640 1 -- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3006a440 msgr2=0x7f6c3013a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c2f7fe640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3006a440 0x7f6c3013a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c2f7fe640 1 -- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c300adde0 msgr2=0x7f6c300b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c2f7fe640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c300adde0 0x7f6c300b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c2f7fe640 1 -- 172.21.15.64:0/1988859193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200094d0 con 0x7f6c30152260 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c267fc640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c300adde0 0x7f6c300b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c2f7fe640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c30152260 0x7f6c301526e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c28007ef0 tx=0x7f6c2800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c27fff640 1 -- 172.21.15.64:0/1988859193 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28003b90 con 0x7f6c30152260 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c27fff640 1 -- 172.21.15.64:0/1988859193 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2800f040 con 0x7f6c30152260 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c27fff640 1 -- 172.21.15.64:0/1988859193 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800de80 con 0x7f6c30152260 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.549+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c300b3420 con 0x7f6c30152260 2023-11-24T08:04:52.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.550+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c300b39e0 con 0x7f6c30152260 2023-11-24T08:04:52.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.551+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c30066f80 con 0x7f6c30152260 2023-11-24T08:04:52.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.552+0000 7f6c27fff640 1 -- 172.21.15.64:0/1988859193 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c28004030 con 0x7f6c30152260 2023-11-24T08:04:52.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.553+0000 7f6c27fff640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1006ae40 0x7f6c1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:52.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.553+0000 7f6c27fff640 1 -- 172.21.15.64:0/1988859193 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6c280898e0 con 0x7f6c30152260 2023-11-24T08:04:52.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.553+0000 7f6c2effd640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1006ae40 0x7f6c1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:52.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.555+0000 7f6c2effd640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1006ae40 0x7f6c1006d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c20036000 tx=0x7f6c2003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:52.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.558+0000 7f6c27fff640 1 -- 172.21.15.64:0/1988859193 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c280549a0 con 0x7f6c30152260 2023-11-24T08:04:52.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.753+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c3004e840 con 0x7f6c30152260 2023-11-24T08:04:52.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.754+0000 7f6c27fff640 1 -- 172.21.15.64:0/1988859193 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c2801a0a0 con 0x7f6c30152260 2023-11-24T08:04:52.756 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:52.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1006ae40 msgr2=0x7f6c1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1006ae40 0x7f6c1006d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c20036000 tx=0x7f6c2003d040 comp rx=0 tx=0).stop 2023-11-24T08:04:52.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c30152260 msgr2=0x7f6c301526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:52.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c30152260 0x7f6c301526e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c28007ef0 tx=0x7f6c2800d4a0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 shutdown_connections 2023-11-24T08:04:52.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1006ae40 0x7f6c1006d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c30152260 0x7f6c301526e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c300adde0 0x7f6c300b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 --2- 172.21.15.64:0/1988859193 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3006a440 0x7f6c3013a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:52.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 >> 172.21.15.64:0/1988859193 conn(0x7f6c300cfd80 msgr2=0x7f6c30071280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:52.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 shutdown_connections 2023-11-24T08:04:52.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:52.757+0000 7f6c2ffff640 1 -- 172.21.15.64:0/1988859193 wait complete. 2023-11-24T08:04:52.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:53.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.014+0000 7feeb3fff640 1 Processor -- start 2023-11-24T08:04:53.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.014+0000 7feeb3fff640 1 -- start start 2023-11-24T08:04:53.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.016+0000 7feeb3fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feeb415f270 0x7feeb40cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.016+0000 7feeb3fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb415f930 con 0x7feeb4144770 2023-11-24T08:04:53.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.016+0000 7feeb3fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb40b3b10 con 0x7feeb406d9e0 2023-11-24T08:04:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.016+0000 7feeb3fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb40b43b0 con 0x7feeb415f270 2023-11-24T08:04:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.017+0000 7feeb2ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7feeb4144770 0x7feeb415eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42714/0 (socket says 172.21.15.64:42714) 2023-11-24T08:04:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.017+0000 7feeb2ffd640 1 -- 172.21.15.64:0/1546230578 learned_addr learned my addr 172.21.15.64:0/1546230578 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.017+0000 7feeaa7fc640 1 --2- 172.21.15.64:0/1546230578 >> v2:172.21.15.91:3300/0 conn(0x7feeb415f270 0x7feeb40cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.017+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053027106 0 0) 0x7feeb415f930 con 0x7feeb4144770 2023-11-24T08:04:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.017+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea0003520 con 0x7feeb4144770 2023-11-24T08:04:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3164397484 0 0) 0x7feea0003520 con 0x7feeb4144770 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 >> v2:172.21.15.91:3300/0 conn(0x7feeb415f270 msgr2=0x7feeb40cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 --2- 172.21.15.64:0/1546230578 >> v2:172.21.15.91:3300/0 conn(0x7feeb415f270 0x7feeb40cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 >> v1:172.21.15.91:6789/0 conn(0x7feeb406d9e0 legacy=0x7feeb40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb40b3df0 con 0x7feeb4144770 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7feea40019f0 con 0x7feeb4144770 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 >> v1:172.21.15.64:6789/0 conn(0x7feeb4144770 legacy=0x7feeb415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea0003a30 0x7feea0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea00047c0 0x7feea0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea00051d0 0x7feea000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40b43b0 con 0x7feea00047c0 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea000c100 con 0x7feea00051d0 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea000c2c0 con 0x7feea0003a30 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeaa7fc640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea00047c0 0x7feea0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.018+0000 7feeb37fe640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea00051d0 0x7feea000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.019+0000 7feeb2ffd640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea0003a30 0x7feea0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.019+0000 7feeaa7fc640 1 -- 172.21.15.64:0/1546230578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea0003a30 msgr2=0x7feea0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.019+0000 7feeaa7fc640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea0003a30 0x7feea0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.019+0000 7feeaa7fc640 1 -- 172.21.15.64:0/1546230578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea00051d0 msgr2=0x7feea000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.019+0000 7feeaa7fc640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feea00051d0 0x7feea000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.019+0000 7feeaa7fc640 1 -- 172.21.15.64:0/1546230578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea0003520 con 0x7feea00047c0 2023-11-24T08:04:53.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.019+0000 7feeb37fe640 1 -- 172.21.15.64:0/1546230578 reap_dead start 2023-11-24T08:04:53.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.020+0000 7feeaa7fc640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea00047c0 0x7feea0004c40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009ac0 tx=0x7feea00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f56df40635ce8b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.020+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c02c6b0 con 0x7feea00047c0 2023-11-24T08:04:53.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.020+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c02c810 con 0x7feea00047c0 2023-11-24T08:04:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.020+0000 7feeb27fc640 1 -- 172.21.15.64:0/1546230578 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c02cba0 con 0x7feea00047c0 2023-11-24T08:04:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.021+0000 7feeb3fff640 1 -- 172.21.15.64:0/1546230578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea00047c0 msgr2=0x7feea0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.021+0000 7feeb3fff640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea00047c0 0x7feea0004c40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009ac0 tx=0x7feea00057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.021+0000 7feeb3fff640 1 -- 172.21.15.64:0/1546230578 shutdown_connections 2023-11-24T08:04:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.021+0000 7feeb3fff640 1 --2- 172.21.15.64:0/1546230578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea00047c0 0x7feea0004c40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.021+0000 7feeb3fff640 1 -- 172.21.15.64:0/1546230578 >> 172.21.15.64:0/1546230578 conn(0x7feeb40cec10 msgr2=0x7feeb40cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.021+0000 7feeb3fff640 1 -- 172.21.15.64:0/1546230578 shutdown_connections 2023-11-24T08:04:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.021+0000 7feeb3fff640 1 -- 172.21.15.64:0/1546230578 wait complete. 2023-11-24T08:04:53.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.023+0000 7feeb3fff640 1 Processor -- start 2023-11-24T08:04:53.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.023+0000 7feeb3fff640 1 -- start start 2023-11-24T08:04:53.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.024+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea00047c0 0x7feeb4152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.024+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb4071490 0x7feeb415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.024+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feeb415bb60 0x7feeb415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.024+0000 7feeb3fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb4070110 con 0x7feeb4071490 2023-11-24T08:04:53.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.024+0000 7feeb3fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40703f0 con 0x7feeb415bb60 2023-11-24T08:04:53.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.024+0000 7feeb3fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40706d0 con 0x7feea00047c0 2023-11-24T08:04:53.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.024+0000 7feeaa7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb4071490 0x7feeb415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeaa7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb4071490 0x7feeb415b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56842/0 (socket says 172.21.15.64:56842) 2023-11-24T08:04:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeaa7fc640 1 -- 172.21.15.64:0/2742453777 learned_addr learned my addr 172.21.15.64:0/2742453777 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeb2ffd640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea00047c0 0x7feeb4152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeb37fe640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feeb415bb60 0x7feeb415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeaa7fc640 1 -- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea00047c0 msgr2=0x7feeb4152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeaa7fc640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea00047c0 0x7feeb4152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeaa7fc640 1 -- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feeb415bb60 msgr2=0x7feeb415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeaa7fc640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feeb415bb60 0x7feeb415c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeaa7fc640 1 -- 172.21.15.64:0/2742453777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb40709b0 con 0x7feeb4071490 2023-11-24T08:04:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.025+0000 7feeaa7fc640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb4071490 0x7feeb415b620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee9c02c250 tx=0x7fee9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.026+0000 7feeabfff640 1 -- 172.21.15.64:0/2742453777 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c003810 con 0x7feeb4071490 2023-11-24T08:04:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.026+0000 7feeabfff640 1 -- 172.21.15.64:0/2742453777 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c003970 con 0x7feeb4071490 2023-11-24T08:04:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.026+0000 7feeabfff640 1 -- 172.21.15.64:0/2742453777 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c003d00 con 0x7feeb4071490 2023-11-24T08:04:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.026+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb4070c90 con 0x7feeb4071490 2023-11-24T08:04:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.026+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb4152e40 con 0x7feeb4071490 2023-11-24T08:04:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.027+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb404e840 con 0x7feeb4071490 2023-11-24T08:04:53.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.028+0000 7feeabfff640 1 -- 172.21.15.64:0/2742453777 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee9c047450 con 0x7feeb4071490 2023-11-24T08:04:53.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.029+0000 7feeabfff640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fee9406ae70 0x7fee9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.029+0000 7feeabfff640 1 -- 172.21.15.64:0/2742453777 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fee9c0abaf0 con 0x7feeb4071490 2023-11-24T08:04:53.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.030+0000 7feeb2ffd640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fee9406ae70 0x7fee9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.031+0000 7feeb2ffd640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fee9406ae70 0x7fee9406d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feea40013d0 tx=0x7feea4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.035+0000 7feeabfff640 1 -- 172.21.15.64:0/2742453777 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee9c076bb0 con 0x7feeb4071490 2023-11-24T08:04:53.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.231+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feeb41409a0 con 0x7feeb4071490 2023-11-24T08:04:53.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.232+0000 7feeabfff640 1 -- 172.21.15.64:0/2742453777 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee9c04c020 con 0x7feeb4071490 2023-11-24T08:04:53.234 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:53.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fee9406ae70 msgr2=0x7fee9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fee9406ae70 0x7fee9406d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feea40013d0 tx=0x7feea4009040 comp rx=0 tx=0).stop 2023-11-24T08:04:53.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb4071490 msgr2=0x7feeb415b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb4071490 0x7feeb415b620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee9c02c250 tx=0x7fee9c004730 comp rx=0 tx=0).stop 2023-11-24T08:04:53.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 shutdown_connections 2023-11-24T08:04:53.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fee9406ae70 0x7fee9406d330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feeb415bb60 0x7feeb415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb4071490 0x7feeb415b620 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 --2- 172.21.15.64:0/2742453777 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feea00047c0 0x7feeb4152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.235+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 >> 172.21.15.64:0/2742453777 conn(0x7feeb40cec10 msgr2=0x7feeb4140bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:53.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.236+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 shutdown_connections 2023-11-24T08:04:53.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.236+0000 7feeb3fff640 1 -- 172.21.15.64:0/2742453777 wait complete. 2023-11-24T08:04:53.255 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:53.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.485+0000 7fd717478640 1 Processor -- start 2023-11-24T08:04:53.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.486+0000 7fd717478640 1 -- start start 2023-11-24T08:04:53.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.487+0000 7fd717478640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd7100d1d90 0x7fd7100ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.488+0000 7fd717478640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7100c6bf0 con 0x7fd7100d0fa0 2023-11-24T08:04:53.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.488+0000 7fd717478640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7100d13f0 con 0x7fd71015d3b0 2023-11-24T08:04:53.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.488+0000 7fd717478640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7100cedc0 con 0x7fd7100d1d90 2023-11-24T08:04:53.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.488+0000 7fd716c77640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd7100d0fa0 0x7fd7100b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42726/0 (socket says 172.21.15.64:42726) 2023-11-24T08:04:53.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.488+0000 7fd716c77640 1 -- 172.21.15.64:0/1592189816 learned_addr learned my addr 172.21.15.64:0/1592189816 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:53.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.488+0000 7fd715c75640 1 --2- 172.21.15.64:0/1592189816 >> v2:172.21.15.91:3300/0 conn(0x7fd7100d1d90 0x7fd7100ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.488+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237187582 0 0) 0x7fd7100c6bf0 con 0x7fd7100d0fa0 2023-11-24T08:04:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.488+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6fc003520 con 0x7fd7100d0fa0 2023-11-24T08:04:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3223694248 0 0) 0x7fd6fc003520 con 0x7fd7100d0fa0 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 >> v2:172.21.15.91:3300/0 conn(0x7fd7100d1d90 msgr2=0x7fd7100ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 --2- 172.21.15.64:0/1592189816 >> v2:172.21.15.91:3300/0 conn(0x7fd7100d1d90 0x7fd7100ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 >> v1:172.21.15.91:6789/0 conn(0x7fd71015d3b0 legacy=0x7fd7100d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100c6e40 con 0x7fd7100d0fa0 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fd7080019f0 con 0x7fd7100d0fa0 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 >> v1:172.21.15.64:6789/0 conn(0x7fd7100d0fa0 legacy=0x7fd7100b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc003a30 0x7fd6fc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc0047c0 0x7fd6fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6fc0052a0 0x7fd6fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100cedc0 con 0x7fd6fc003a30 2023-11-24T08:04:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc00c170 con 0x7fd6fc0047c0 2023-11-24T08:04:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc00c330 con 0x7fd6fc0052a0 2023-11-24T08:04:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.489+0000 7fd716476640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc003a30 0x7fd6fc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.490+0000 7fd715c75640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc0047c0 0x7fd6fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.490+0000 7fd716476640 1 -- 172.21.15.64:0/1592189816 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6fc0052a0 msgr2=0x7fd6fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.490+0000 7fd716476640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6fc0052a0 0x7fd6fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.490+0000 7fd716476640 1 -- 172.21.15.64:0/1592189816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc0047c0 msgr2=0x7fd6fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:53.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.490+0000 7fd716476640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc0047c0 0x7fd6fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.490+0000 7fd716476640 1 -- 172.21.15.64:0/1592189816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc003520 con 0x7fd6fc003a30 2023-11-24T08:04:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.490+0000 7fd715c75640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc0047c0 0x7fd6fc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:53.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.490+0000 7fd716c77640 1 -- 172.21.15.64:0/1592189816 reap_dead start 2023-11-24T08:04:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.491+0000 7fd716476640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc003a30 0x7fd6fc003e50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001750 tx=0x7fd6fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb443ec2addb0cb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.491+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0052d0 con 0x7fd6fc003a30 2023-11-24T08:04:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.491+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c02e050 con 0x7fd6fc003a30 2023-11-24T08:04:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.491+0000 7fd715474640 1 -- 172.21.15.64:0/1592189816 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c003830 con 0x7fd6fc003a30 2023-11-24T08:04:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.492+0000 7fd717478640 1 -- 172.21.15.64:0/1592189816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc003a30 msgr2=0x7fd6fc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.492+0000 7fd717478640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc003a30 0x7fd6fc003e50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001750 tx=0x7fd6fc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.492+0000 7fd717478640 1 -- 172.21.15.64:0/1592189816 shutdown_connections 2023-11-24T08:04:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.492+0000 7fd717478640 1 --2- 172.21.15.64:0/1592189816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc003a30 0x7fd6fc003e50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.492+0000 7fd717478640 1 -- 172.21.15.64:0/1592189816 >> 172.21.15.64:0/1592189816 conn(0x7fd7100d30a0 msgr2=0x7fd7100d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.492+0000 7fd717478640 1 -- 172.21.15.64:0/1592189816 shutdown_connections 2023-11-24T08:04:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.492+0000 7fd717478640 1 -- 172.21.15.64:0/1592189816 wait complete. 2023-11-24T08:04:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.494+0000 7fd717478640 1 Processor -- start 2023-11-24T08:04:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.494+0000 7fd717478640 1 -- start start 2023-11-24T08:04:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.495+0000 7fd717478640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc003a30 0x7fd7101598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.495+0000 7fd717478640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7100b7e90 0x7fd7100b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.495+0000 7fd717478640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7100a0080 0x7fd7100a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.495+0000 7fd717478640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100b8850 con 0x7fd7100b7e90 2023-11-24T08:04:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.495+0000 7fd717478640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100cb560 con 0x7fd6fc003a30 2023-11-24T08:04:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.495+0000 7fd717478640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100a0d60 con 0x7fd7100a0080 2023-11-24T08:04:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.495+0000 7fd715c75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7100b7e90 0x7fd7100b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.495+0000 7fd715c75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7100b7e90 0x7fd7100b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56862/0 (socket says 172.21.15.64:56862) 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd715c75640 1 -- 172.21.15.64:0/769394968 learned_addr learned my addr 172.21.15.64:0/769394968 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd716476640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc003a30 0x7fd7101598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd716c77640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7100a0080 0x7fd7100a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd715c75640 1 -- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7100a0080 msgr2=0x7fd7100a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd715c75640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7100a0080 0x7fd7100a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd715c75640 1 -- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc003a30 msgr2=0x7fd7101598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd715c75640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc003a30 0x7fd7101598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd715c75640 1 -- 172.21.15.64:0/769394968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7101f6bd0 con 0x7fd7100b7e90 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.496+0000 7fd715c75640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7100b7e90 0x7fd7100b8310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd700010a00 tx=0x7fd700010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.497+0000 7fd706ffd640 1 -- 172.21.15.64:0/769394968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000ec20 con 0x7fd7100b7e90 2023-11-24T08:04:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.497+0000 7fd706ffd640 1 -- 172.21.15.64:0/769394968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70000ed80 con 0x7fd7100b7e90 2023-11-24T08:04:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.497+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd710159e00 con 0x7fd7100b7e90 2023-11-24T08:04:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.497+0000 7fd706ffd640 1 -- 172.21.15.64:0/769394968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70001c620 con 0x7fd7100b7e90 2023-11-24T08:04:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.497+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7101f62a0 con 0x7fd7100b7e90 2023-11-24T08:04:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.498+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71004e840 con 0x7fd7100b7e90 2023-11-24T08:04:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.499+0000 7fd706ffd640 1 -- 172.21.15.64:0/769394968 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd70001c780 con 0x7fd7100b7e90 2023-11-24T08:04:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.500+0000 7fd706ffd640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6f406ae70 0x7fd6f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.500+0000 7fd706ffd640 1 -- 172.21.15.64:0/769394968 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd700089c40 con 0x7fd7100b7e90 2023-11-24T08:04:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.501+0000 7fd716476640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6f406ae70 0x7fd6f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.502+0000 7fd716476640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6f406ae70 0x7fd6f406d330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd70c033000 tx=0x7fd70c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.507+0000 7fd706ffd640 1 -- 172.21.15.64:0/769394968 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd700054d00 con 0x7fd7100b7e90 2023-11-24T08:04:53.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.701+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7100ce620 con 0x7fd7100b7e90 2023-11-24T08:04:53.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.702+0000 7fd706ffd640 1 -- 172.21.15.64:0/769394968 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd700058320 con 0x7fd7100b7e90 2023-11-24T08:04:53.704 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:53.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6f406ae70 msgr2=0x7fd6f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6f406ae70 0x7fd6f406d330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd70c033000 tx=0x7fd70c009030 comp rx=0 tx=0).stop 2023-11-24T08:04:53.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7100b7e90 msgr2=0x7fd7100b8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7100b7e90 0x7fd7100b8310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd700010a00 tx=0x7fd700010ec0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 shutdown_connections 2023-11-24T08:04:53.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd6f406ae70 0x7fd6f406d330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7100a0080 0x7fd7100a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7100b7e90 0x7fd7100b8310 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 --2- 172.21.15.64:0/769394968 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6fc003a30 0x7fd7101598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 >> 172.21.15.64:0/769394968 conn(0x7fd7100d30a0 msgr2=0x7fd710160370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:53.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 shutdown_connections 2023-11-24T08:04:53.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.705+0000 7fd717478640 1 -- 172.21.15.64:0/769394968 wait complete. 2023-11-24T08:04:53.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:04:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.948+0000 7fea49fac640 1 Processor -- start 2023-11-24T08:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.948+0000 7fea49fac640 1 -- start start 2023-11-24T08:04:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.949+0000 7fea49fac640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fea4415d5d0 0x7fea440ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.949+0000 7fea49fac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440cf710 0x7fea440cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.949+0000 7fea49fac640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea440d04b0 con 0x7fea440d1670 2023-11-24T08:04:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.949+0000 7fea49fac640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea440ce4e0 con 0x7fea4415d5d0 2023-11-24T08:04:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.949+0000 7fea49fac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea440cdb80 con 0x7fea440cf710 2023-11-24T08:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.950+0000 7fea43fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440cf710 0x7fea440cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.950+0000 7fea48faa640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fea4415d5d0 0x7fea440ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.950+0000 7fea43fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440cf710 0x7fea440cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:51684/0 (socket says 172.21.15.64:51684) 2023-11-24T08:04:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.950+0000 7fea43fff640 1 -- 172.21.15.64:0/1941096763 learned_addr learned my addr 172.21.15.64:0/1941096763 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.950+0000 7fea48faa640 1 -- 172.21.15.64:0/1941096763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440cf710 msgr2=0x7fea440cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:53.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.950+0000 7fea48faa640 1 --2- 172.21.15.64:0/1941096763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440cf710 0x7fea440cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.951+0000 7fea48faa640 1 -- 172.21.15.64:0/1941096763 >> v1:172.21.15.91:6789/0 conn(0x7fea440d1670 legacy=0x7fea440d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.951+0000 7fea48faa640 1 -- 172.21.15.64:0/1941096763 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea440d01d0 con 0x7fea4415d5d0 2023-11-24T08:04:53.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.951+0000 7fea43fff640 1 --2- 172.21.15.64:0/1941096763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440cf710 0x7fea440cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:04:53.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.952+0000 7fea48faa640 1 --2- 172.21.15.64:0/1941096763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea4415d5d0 0x7fea440ce900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea380097a0 tx=0x7fea3802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a12f8331e7e0f355 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.952+0000 7fea437fe640 1 -- 172.21.15.64:0/1941096763 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38031780 con 0x7fea4415d5d0 2023-11-24T08:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.952+0000 7fea437fe640 1 -- 172.21.15.64:0/1941096763 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea380318e0 con 0x7fea4415d5d0 2023-11-24T08:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.952+0000 7fea437fe640 1 -- 172.21.15.64:0/1941096763 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38031c70 con 0x7fea4415d5d0 2023-11-24T08:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.953+0000 7fea49fac640 1 -- 172.21.15.64:0/1941096763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea4415d5d0 msgr2=0x7fea440ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.953+0000 7fea49fac640 1 --2- 172.21.15.64:0/1941096763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea4415d5d0 0x7fea440ce900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea380097a0 tx=0x7fea3802b820 comp rx=0 tx=0).stop 2023-11-24T08:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.953+0000 7fea49fac640 1 -- 172.21.15.64:0/1941096763 shutdown_connections 2023-11-24T08:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.953+0000 7fea49fac640 1 --2- 172.21.15.64:0/1941096763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440cf710 0x7fea440cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.953+0000 7fea49fac640 1 --2- 172.21.15.64:0/1941096763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea4415d5d0 0x7fea440ce900 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.953+0000 7fea49fac640 1 -- 172.21.15.64:0/1941096763 >> 172.21.15.64:0/1941096763 conn(0x7fea4409c3d0 msgr2=0x7fea440d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.953+0000 7fea49fac640 1 -- 172.21.15.64:0/1941096763 shutdown_connections 2023-11-24T08:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.953+0000 7fea49fac640 1 -- 172.21.15.64:0/1941096763 wait complete. 2023-11-24T08:04:53.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.955+0000 7fea49fac640 1 Processor -- start 2023-11-24T08:04:53.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.955+0000 7fea49fac640 1 -- start start 2023-11-24T08:04:53.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.956+0000 7fea49fac640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea440cf710 0x7fea4415f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.956+0000 7fea49fac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440d1670 0x7fea440ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.956+0000 7fea49fac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4415d5d0 0x7fea4415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.956+0000 7fea49fac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4415c450 con 0x7fea4415d5d0 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.956+0000 7fea49fac640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea440cd130 con 0x7fea440cf710 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.956+0000 7fea49fac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea440cce50 con 0x7fea440d1670 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea497ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4415d5d0 0x7fea4415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea497ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4415d5d0 0x7fea4415bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56866/0 (socket says 172.21.15.64:56866) 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea497ab640 1 -- 172.21.15.64:0/3092111328 learned_addr learned my addr 172.21.15.64:0/3092111328 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea43fff640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440d1670 0x7fea440ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea48faa640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea440cf710 0x7fea4415f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea497ab640 1 -- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440d1670 msgr2=0x7fea440ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea497ab640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440d1670 0x7fea440ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea497ab640 1 -- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea440cf710 msgr2=0x7fea4415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea497ab640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea440cf710 0x7fea4415f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea497ab640 1 -- 172.21.15.64:0/3092111328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea340094d0 con 0x7fea4415d5d0 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea48faa640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea440cf710 0x7fea4415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.957+0000 7fea43fff640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440d1670 0x7fea440ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.958+0000 7fea497ab640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4415d5d0 0x7fea4415bf10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c6b0 tx=0x7fea3c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.958+0000 7fea417fa640 1 -- 172.21.15.64:0/3092111328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c003b70 con 0x7fea4415d5d0 2023-11-24T08:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.958+0000 7fea417fa640 1 -- 172.21.15.64:0/3092111328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c003cf0 con 0x7fea4415d5d0 2023-11-24T08:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.958+0000 7fea417fa640 1 -- 172.21.15.64:0/3092111328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c003ea0 con 0x7fea4415d5d0 2023-11-24T08:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.958+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea38009400 con 0x7fea4415d5d0 2023-11-24T08:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.958+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4415ec10 con 0x7fea4415d5d0 2023-11-24T08:04:53.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.959+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4404e840 con 0x7fea4415d5d0 2023-11-24T08:04:53.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.960+0000 7fea417fa640 1 -- 172.21.15.64:0/3092111328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea3c005310 con 0x7fea4415d5d0 2023-11-24T08:04:53.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.961+0000 7fea417fa640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea3006ae70 0x7fea3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:53.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.961+0000 7fea417fa640 1 -- 172.21.15.64:0/3092111328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fea3c089430 con 0x7fea4415d5d0 2023-11-24T08:04:53.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.962+0000 7fea48faa640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea3006ae70 0x7fea3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.963+0000 7fea48faa640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea3006ae70 0x7fea3006d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea38036000 tx=0x7fea3803d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:53.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:53.965+0000 7fea417fa640 1 -- 172.21.15.64:0/3092111328 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea3c054440 con 0x7fea4415d5d0 2023-11-24T08:04:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.179+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fea440678d0 con 0x7fea4415d5d0 2023-11-24T08:04:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.179+0000 7fea417fa640 1 -- 172.21.15.64:0/3092111328 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fea3c057a60 con 0x7fea4415d5d0 2023-11-24T08:04:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:04:54.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.182+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea3006ae70 msgr2=0x7fea3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.182+0000 7fea49fac640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea3006ae70 0x7fea3006d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea38036000 tx=0x7fea3803d040 comp rx=0 tx=0).stop 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.182+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4415d5d0 msgr2=0x7fea4415bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.182+0000 7fea49fac640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4415d5d0 0x7fea4415bf10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c6b0 tx=0x7fea3c00cb70 comp rx=0 tx=0).stop 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.182+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 shutdown_connections 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.182+0000 7fea49fac640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea3006ae70 0x7fea3006d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.182+0000 7fea49fac640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4415d5d0 0x7fea4415bf10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.183+0000 7fea49fac640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea440d1670 0x7fea440ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.183+0000 7fea49fac640 1 --2- 172.21.15.64:0/3092111328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea440cf710 0x7fea4415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.183+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 >> 172.21.15.64:0/3092111328 conn(0x7fea4409c3d0 msgr2=0x7fea440c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.183+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 shutdown_connections 2023-11-24T08:04:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.183+0000 7fea49fac640 1 -- 172.21.15.64:0/3092111328 wait complete. 2023-11-24T08:04:54.205 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:04:54.205 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:54.258 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:54.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.486+0000 7f6a590ea640 1 Processor -- start 2023-11-24T08:04:54.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.486+0000 7f6a590ea640 1 -- start start 2023-11-24T08:04:54.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.488+0000 7f6a590ea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a54157770 0x7f6a5406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.488+0000 7f6a590ea640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6a5406d080 0x7f6a5415d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.488+0000 7f6a590ea640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a54161670 con 0x7f6a5415ddb0 2023-11-24T08:04:54.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.488+0000 7f6a590ea640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5406d500 con 0x7f6a5406d080 2023-11-24T08:04:54.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.488+0000 7f6a590ea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a54071730 con 0x7f6a54157770 2023-11-24T08:04:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.488+0000 7f6a537fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6a5406d080 0x7f6a5415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.488+0000 7f6a588e9640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6a5415ddb0 0x7f6a540ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42734/0 (socket says 172.21.15.64:42734) 2023-11-24T08:04:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.488+0000 7f6a588e9640 1 -- 172.21.15.64:0/164358880 learned_addr learned my addr 172.21.15.64:0/164358880 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.489+0000 7f6a53fff640 1 --2- 172.21.15.64:0/164358880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a54157770 0x7f6a5406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.489+0000 7f6a537fe640 1 -- 172.21.15.64:0/164358880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a54157770 msgr2=0x7f6a5406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.489+0000 7f6a537fe640 1 --2- 172.21.15.64:0/164358880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a54157770 0x7f6a5406cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.489+0000 7f6a537fe640 1 -- 172.21.15.64:0/164358880 >> v1:172.21.15.64:6789/0 conn(0x7f6a5415ddb0 legacy=0x7f6a540ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.489+0000 7f6a52ffd640 1 -- 172.21.15.64:0/164358880 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995838951 0 0) 0x7f6a54161670 con 0x7f6a5415ddb0 2023-11-24T08:04:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.489+0000 7f6a537fe640 1 -- 172.21.15.64:0/164358880 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5415e610 con 0x7f6a5406d080 2023-11-24T08:04:54.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.490+0000 7f6a537fe640 1 --2- 172.21.15.64:0/164358880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a5406d080 0x7f6a5415d870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a480097a0 tx=0x7f6a4802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93eca7915c2a47d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:54.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a52ffd640 1 -- 172.21.15.64:0/164358880 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48031830 con 0x7f6a5406d080 2023-11-24T08:04:54.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a52ffd640 1 -- 172.21.15.64:0/164358880 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48031990 con 0x7f6a5406d080 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a52ffd640 1 -- 172.21.15.64:0/164358880 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48031d20 con 0x7f6a5406d080 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a590ea640 1 -- 172.21.15.64:0/164358880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a5406d080 msgr2=0x7f6a5415d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a590ea640 1 --2- 172.21.15.64:0/164358880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a5406d080 0x7f6a5415d870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a480097a0 tx=0x7f6a4802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a590ea640 1 -- 172.21.15.64:0/164358880 shutdown_connections 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a590ea640 1 --2- 172.21.15.64:0/164358880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a5406d080 0x7f6a5415d870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a590ea640 1 --2- 172.21.15.64:0/164358880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a54157770 0x7f6a5406cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a590ea640 1 -- 172.21.15.64:0/164358880 >> 172.21.15.64:0/164358880 conn(0x7f6a5406a860 msgr2=0x7f6a5406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a590ea640 1 -- 172.21.15.64:0/164358880 shutdown_connections 2023-11-24T08:04:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.491+0000 7f6a590ea640 1 -- 172.21.15.64:0/164358880 wait complete. 2023-11-24T08:04:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.493+0000 7f6a590ea640 1 Processor -- start 2023-11-24T08:04:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.493+0000 7f6a590ea640 1 -- start start 2023-11-24T08:04:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.494+0000 7f6a590ea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a5406d080 0x7f6a541f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.494+0000 7f6a590ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a54157770 0x7f6a540cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.494+0000 7f6a590ea640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a5415ddb0 0x7f6a5414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.494+0000 7f6a590ea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540caef0 con 0x7f6a54157770 2023-11-24T08:04:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.494+0000 7f6a590ea640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5414ed90 con 0x7f6a5415ddb0 2023-11-24T08:04:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.494+0000 7f6a590ea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540b3bf0 con 0x7f6a5406d080 2023-11-24T08:04:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.494+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a54157770 0x7f6a540cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a54157770 0x7f6a540cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56890/0 (socket says 172.21.15.64:56890) 2023-11-24T08:04:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a537fe640 1 -- 172.21.15.64:0/1406914504 learned_addr learned my addr 172.21.15.64:0/1406914504 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a588e9640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a5415ddb0 0x7f6a5414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a537fe640 1 -- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a5406d080 msgr2=0x7f6a541f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a53fff640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a5406d080 0x7f6a541f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a537fe640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a5406d080 0x7f6a541f2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a537fe640 1 -- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a5415ddb0 msgr2=0x7f6a5414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a537fe640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a5415ddb0 0x7f6a5414e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a537fe640 1 -- 172.21.15.64:0/1406914504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a48009400 con 0x7f6a54157770 2023-11-24T08:04:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a588e9640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a5415ddb0 0x7f6a5414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.495+0000 7f6a537fe640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a54157770 0x7f6a540cd270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009890 tx=0x7f6a480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.496+0000 7f6a50ff9640 1 -- 172.21.15.64:0/1406914504 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4803b980 con 0x7f6a54157770 2023-11-24T08:04:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.496+0000 7f6a50ff9640 1 -- 172.21.15.64:0/1406914504 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4803bb00 con 0x7f6a54157770 2023-11-24T08:04:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.496+0000 7f6a50ff9640 1 -- 172.21.15.64:0/1406914504 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48045400 con 0x7f6a54157770 2023-11-24T08:04:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.496+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a541f3230 con 0x7f6a54157770 2023-11-24T08:04:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.496+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a541f37f0 con 0x7f6a54157770 2023-11-24T08:04:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.497+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a540cb030 con 0x7f6a54157770 2023-11-24T08:04:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.498+0000 7f6a50ff9640 1 -- 172.21.15.64:0/1406914504 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a4804e440 con 0x7f6a54157770 2023-11-24T08:04:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.499+0000 7f6a50ff9640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a4006ae90 0x7f6a4006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.499+0000 7f6a50ff9640 1 -- 172.21.15.64:0/1406914504 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6a480ab850 con 0x7f6a54157770 2023-11-24T08:04:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.500+0000 7f6a53fff640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a4006ae90 0x7f6a4006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.501+0000 7f6a53fff640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a4006ae90 0x7f6a4006d350 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a540b3430 tx=0x7f6a44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:54.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.503+0000 7f6a50ff9640 1 -- 172.21.15.64:0/1406914504 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a540cb030 con 0x7f6a54157770 2023-11-24T08:04:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.698+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a540cb030 con 0x7f6a54157770 2023-11-24T08:04:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.701+0000 7f6a50ff9640 1 -- 172.21.15.64:0/1406914504 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7f6a540cb030 con 0x7f6a54157770 2023-11-24T08:04:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:54.704 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:54.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a4006ae90 msgr2=0x7f6a4006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a4006ae90 0x7f6a4006d350 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a540b3430 tx=0x7f6a44008040 comp rx=0 tx=0).stop 2023-11-24T08:04:54.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a54157770 msgr2=0x7f6a540cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a54157770 0x7f6a540cd270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009890 tx=0x7f6a480046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 shutdown_connections 2023-11-24T08:04:54.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a4006ae90 0x7f6a4006d350 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6a5415ddb0 0x7f6a5414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a54157770 0x7f6a540cd270 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 --2- 172.21.15.64:0/1406914504 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6a5406d080 0x7f6a541f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 >> 172.21.15.64:0/1406914504 conn(0x7f6a5406a860 msgr2=0x7f6a540b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 shutdown_connections 2023-11-24T08:04:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.704+0000 7f6a590ea640 1 -- 172.21.15.64:0/1406914504 wait complete. 2023-11-24T08:04:54.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:54.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.954+0000 7fb021fef640 1 Processor -- start 2023-11-24T08:04:54.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.955+0000 7fb021fef640 1 -- start start 2023-11-24T08:04:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.956+0000 7fb021fef640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.956+0000 7fb021fef640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb01c15f0a0 0x7fb01c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.956+0000 7fb021fef640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c15f930 con 0x7fb01c0b35a0 2023-11-24T08:04:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.956+0000 7fb021fef640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c0b43b0 con 0x7fb01c15f0a0 2023-11-24T08:04:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.956+0000 7fb021fef640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c15f530 con 0x7fb01c144770 2023-11-24T08:04:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.957+0000 7fb0217ee640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb01c15f0a0 0x7fb01c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.957+0000 7fb020fed640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.957+0000 7fb0217ee640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb01c15f0a0 0x7fb01c0ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:60502/0 (socket says 172.21.15.64:60502) 2023-11-24T08:04:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.957+0000 7fb0217ee640 1 -- 172.21.15.64:0/1431839451 learned_addr learned my addr 172.21.15.64:0/1431839451 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.958+0000 7fb0217ee640 1 -- 172.21.15.64:0/1431839451 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 msgr2=0x7fb01c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:54.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.958+0000 7fb0217ee640 1 --2- 172.21.15.64:0/1431839451 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.958+0000 7fb0217ee640 1 -- 172.21.15.64:0/1431839451 >> v1:172.21.15.91:6789/0 conn(0x7fb01c0b35a0 legacy=0x7fb01c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.958+0000 7fb0217ee640 1 -- 172.21.15.64:0/1431839451 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c0ce9f0 con 0x7fb01c15f0a0 2023-11-24T08:04:54.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.959+0000 7fb0217ee640 1 --2- 172.21.15.64:0/1431839451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c15f0a0 0x7fb01c0ce4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb014009e80 tx=0x7fb01402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=913dc8bb19961794 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:54.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.959+0000 7fb01b7fe640 1 -- 172.21.15.64:0/1431839451 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01402caf0 con 0x7fb01c15f0a0 2023-11-24T08:04:54.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.959+0000 7fb01b7fe640 1 -- 172.21.15.64:0/1431839451 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01402cc50 con 0x7fb01c15f0a0 2023-11-24T08:04:54.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.959+0000 7fb01b7fe640 1 -- 172.21.15.64:0/1431839451 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140343f0 con 0x7fb01c15f0a0 2023-11-24T08:04:54.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.960+0000 7fb021fef640 1 -- 172.21.15.64:0/1431839451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c15f0a0 msgr2=0x7fb01c0ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.960+0000 7fb021fef640 1 --2- 172.21.15.64:0/1431839451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c15f0a0 0x7fb01c0ce4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb014009e80 tx=0x7fb01402bf50 comp rx=0 tx=0).stop 2023-11-24T08:04:54.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.960+0000 7fb021fef640 1 -- 172.21.15.64:0/1431839451 shutdown_connections 2023-11-24T08:04:54.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.960+0000 7fb021fef640 1 --2- 172.21.15.64:0/1431839451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c15f0a0 0x7fb01c0ce4b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.960+0000 7fb021fef640 1 --2- 172.21.15.64:0/1431839451 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.960+0000 7fb021fef640 1 -- 172.21.15.64:0/1431839451 >> 172.21.15.64:0/1431839451 conn(0x7fb01c0cec10 msgr2=0x7fb01c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:54.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.960+0000 7fb021fef640 1 -- 172.21.15.64:0/1431839451 shutdown_connections 2023-11-24T08:04:54.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.960+0000 7fb021fef640 1 -- 172.21.15.64:0/1431839451 wait complete. 2023-11-24T08:04:54.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.962+0000 7fb021fef640 1 Processor -- start 2023-11-24T08:04:54.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.962+0000 7fb021fef640 1 -- start start 2023-11-24T08:04:54.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb021fef640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c0b35a0 0x7fb01c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb021fef640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c0a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb021fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c15f0a0 0x7fb01c13d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb021fef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0af190 con 0x7fb01c15f0a0 2023-11-24T08:04:54.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb021fef640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0ae2e0 con 0x7fb01c0b35a0 2023-11-24T08:04:54.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb021fef640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0ae510 con 0x7fb01c144770 2023-11-24T08:04:54.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb0217ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c15f0a0 0x7fb01c13d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb0217ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c15f0a0 0x7fb01c13d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56900/0 (socket says 172.21.15.64:56900) 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.963+0000 7fb0217ee640 1 -- 172.21.15.64:0/2417937701 learned_addr learned my addr 172.21.15.64:0/2417937701 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb020fed640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c0b35a0 0x7fb01c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb01bfff640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c0a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb0217ee640 1 -- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 msgr2=0x7fb01c0a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb0217ee640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c0a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb0217ee640 1 -- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c0b35a0 msgr2=0x7fb01c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb0217ee640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c0b35a0 0x7fb01c0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb0217ee640 1 -- 172.21.15.64:0/2417937701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb014009b30 con 0x7fb01c15f0a0 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb01bfff640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c0a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:54.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb0217ee640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c15f0a0 0x7fb01c13d840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb014009fa0 tx=0x7fb014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb0197fa640 1 -- 172.21.15.64:0/2417937701 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014009510 con 0x7fb01c15f0a0 2023-11-24T08:04:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.964+0000 7fb0197fa640 1 -- 172.21.15.64:0/2417937701 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01403e450 con 0x7fb01c15f0a0 2023-11-24T08:04:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.965+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c0ae7f0 con 0x7fb01c15f0a0 2023-11-24T08:04:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.965+0000 7fb0197fa640 1 -- 172.21.15.64:0/2417937701 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01403e7e0 con 0x7fb01c15f0a0 2023-11-24T08:04:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.965+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c0aedb0 con 0x7fb01c15f0a0 2023-11-24T08:04:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.966+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c0af2d0 con 0x7fb01c15f0a0 2023-11-24T08:04:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.967+0000 7fb0197fa640 1 -- 172.21.15.64:0/2417937701 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb01403e940 con 0x7fb01c15f0a0 2023-11-24T08:04:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.968+0000 7fb0197fa640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00006ae40 0x7fb00006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.968+0000 7fb0197fa640 1 -- 172.21.15.64:0/2417937701 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb0140ac660 con 0x7fb01c15f0a0 2023-11-24T08:04:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.968+0000 7fb020fed640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00006ae40 0x7fb00006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:54.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.969+0000 7fb020fed640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00006ae40 0x7fb00006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb010009ad0 tx=0x7fb010008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:54.972+0000 7fb0197fa640 1 -- 172.21.15.64:0/2417937701 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb01c0af2d0 con 0x7fb01c15f0a0 2023-11-24T08:04:55.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.167+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb01c0af2d0 con 0x7fb01c15f0a0 2023-11-24T08:04:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.169+0000 7fb0197fa640 1 -- 172.21.15.64:0/2417937701 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb01c0af2d0 con 0x7fb01c15f0a0 2023-11-24T08:04:55.171 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.171+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00006ae40 msgr2=0x7fb00006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.171+0000 7fb021fef640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00006ae40 0x7fb00006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb010009ad0 tx=0x7fb010008040 comp rx=0 tx=0).stop 2023-11-24T08:04:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.171+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c15f0a0 msgr2=0x7fb01c13d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.171+0000 7fb021fef640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c15f0a0 0x7fb01c13d840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb014009fa0 tx=0x7fb014004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.172+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 shutdown_connections 2023-11-24T08:04:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.172+0000 7fb021fef640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00006ae40 0x7fb00006d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.172+0000 7fb021fef640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c15f0a0 0x7fb01c13d840 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.172+0000 7fb021fef640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb01c144770 0x7fb01c0a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.172+0000 7fb021fef640 1 --2- 172.21.15.64:0/2417937701 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb01c0b35a0 0x7fb01c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.172+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 >> 172.21.15.64:0/2417937701 conn(0x7fb01c0cec10 msgr2=0x7fb01c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.172+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 shutdown_connections 2023-11-24T08:04:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.172+0000 7fb021fef640 1 -- 172.21.15.64:0/2417937701 wait complete. 2023-11-24T08:04:55.194 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.414+0000 7f6aab090640 1 Processor -- start 2023-11-24T08:04:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.414+0000 7f6aab090640 1 -- start start 2023-11-24T08:04:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.415+0000 7f6aab090640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa40ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.415+0000 7f6aab090640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6aa4070490 0x7f6aa4070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.415+0000 7f6aab090640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa406d7a0 con 0x7f6aa40acd70 2023-11-24T08:04:55.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.415+0000 7f6aab090640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa415a5b0 con 0x7f6aa4070490 2023-11-24T08:04:55.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.415+0000 7f6aab090640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40ae9d0 con 0x7f6aa406dc30 2023-11-24T08:04:55.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.416+0000 7f6aa3fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa40ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.416+0000 7f6aa3fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa40ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:51738/0 (socket says 172.21.15.64:51738) 2023-11-24T08:04:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.416+0000 7f6aa962d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6aa4070490 0x7f6aa4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.416+0000 7f6aa3fff640 1 -- 172.21.15.64:0/3934026483 learned_addr learned my addr 172.21.15.64:0/3934026483 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.417+0000 7f6aa962d640 1 -- 172.21.15.64:0/3934026483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 msgr2=0x7f6aa40ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.417+0000 7f6aa962d640 1 --2- 172.21.15.64:0/3934026483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa40ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.417+0000 7f6aa962d640 1 -- 172.21.15.64:0/3934026483 >> v1:172.21.15.91:6789/0 conn(0x7f6aa40acd70 legacy=0x7f6aa40ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.417+0000 7f6aa962d640 1 -- 172.21.15.64:0/3934026483 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa406ce70 con 0x7f6aa4070490 2023-11-24T08:04:55.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.418+0000 7f6aa962d640 1 --2- 172.21.15.64:0/3934026483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 0x7f6aa4070920 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009d90 tx=0x7f6a9c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41f43a650a7bda72 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.419+0000 7f6aa37fe640 1 -- 172.21.15.64:0/3934026483 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c02cc50 con 0x7f6aa4070490 2023-11-24T08:04:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.419+0000 7f6aa37fe640 1 -- 172.21.15.64:0/3934026483 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c02cdb0 con 0x7f6aa4070490 2023-11-24T08:04:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.419+0000 7f6aa37fe640 1 -- 172.21.15.64:0/3934026483 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c034660 con 0x7f6aa4070490 2023-11-24T08:04:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.420+0000 7f6aab090640 1 -- 172.21.15.64:0/3934026483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 msgr2=0x7f6aa4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.420+0000 7f6aab090640 1 --2- 172.21.15.64:0/3934026483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 0x7f6aa4070920 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009d90 tx=0x7f6a9c02bea0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.420+0000 7f6aab090640 1 -- 172.21.15.64:0/3934026483 shutdown_connections 2023-11-24T08:04:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.420+0000 7f6aab090640 1 --2- 172.21.15.64:0/3934026483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 0x7f6aa4070920 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.420+0000 7f6aab090640 1 --2- 172.21.15.64:0/3934026483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa40ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.420+0000 7f6aab090640 1 -- 172.21.15.64:0/3934026483 >> 172.21.15.64:0/3934026483 conn(0x7f6aa40b0ba0 msgr2=0x7f6aa40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.420+0000 7f6aab090640 1 -- 172.21.15.64:0/3934026483 shutdown_connections 2023-11-24T08:04:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.420+0000 7f6aab090640 1 -- 172.21.15.64:0/3934026483 wait complete. 2023-11-24T08:04:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.422+0000 7f6aab090640 1 Processor -- start 2023-11-24T08:04:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.422+0000 7f6aab090640 1 -- start start 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aab090640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa414c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aab090640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 0x7f6aa4070870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aab090640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40acd70 0x7f6aa4070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aab090640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40cae90 con 0x7f6aa40acd70 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aab090640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa414ed30 con 0x7f6aa4070490 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aab090640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa414e1b0 con 0x7f6aa406dc30 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aa962d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40acd70 0x7f6aa4070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aa962d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40acd70 0x7f6aa4070db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56912/0 (socket says 172.21.15.64:56912) 2023-11-24T08:04:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aa962d640 1 -- 172.21.15.64:0/1979488485 learned_addr learned my addr 172.21.15.64:0/1979488485 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aa8e2c640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa414c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.423+0000 7f6aa3fff640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 0x7f6aa4070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa962d640 1 -- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 msgr2=0x7f6aa414c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa962d640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa414c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa962d640 1 -- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 msgr2=0x7f6aa4070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa962d640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 0x7f6aa4070870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa962d640 1 -- 172.21.15.64:0/1979488485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c009a70 con 0x7f6aa40acd70 2023-11-24T08:04:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa962d640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40acd70 0x7f6aa4070db0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009eb0 tx=0x7f6a9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa17fa640 1 -- 172.21.15.64:0/1979488485 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c03e610 con 0x7f6aa40acd70 2023-11-24T08:04:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa17fa640 1 -- 172.21.15.64:0/1979488485 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c03e770 con 0x7f6aa40acd70 2023-11-24T08:04:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa414e490 con 0x7f6aa40acd70 2023-11-24T08:04:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.424+0000 7f6aa17fa640 1 -- 172.21.15.64:0/1979488485 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c03eb20 con 0x7f6aa40acd70 2023-11-24T08:04:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.425+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa414ea50 con 0x7f6aa40acd70 2023-11-24T08:04:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.426+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa415a6f0 con 0x7f6aa40acd70 2023-11-24T08:04:55.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.427+0000 7f6aa17fa640 1 -- 172.21.15.64:0/1979488485 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a9c03b050 con 0x7f6aa40acd70 2023-11-24T08:04:55.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.428+0000 7f6aa17fa640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a9006ae40 0x7f6a9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.428+0000 7f6aa17fa640 1 -- 172.21.15.64:0/1979488485 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6a9c0abf90 con 0x7f6aa40acd70 2023-11-24T08:04:55.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.428+0000 7f6aa8e2c640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a9006ae40 0x7f6a9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.429+0000 7f6aa8e2c640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a9006ae40 0x7f6a9006d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a98007fd0 tx=0x7f6a98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.432+0000 7f6aa17fa640 1 -- 172.21.15.64:0/1979488485 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6aa415a6f0 con 0x7f6aa40acd70 2023-11-24T08:04:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.629+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6aa415a6f0 con 0x7f6aa40acd70 2023-11-24T08:04:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.630+0000 7f6aa17fa640 1 -- 172.21.15.64:0/1979488485 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6aa415a6f0 con 0x7f6aa40acd70 2023-11-24T08:04:55.631 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:55.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a9006ae40 msgr2=0x7f6a9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a9006ae40 0x7f6a9006d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a98007fd0 tx=0x7f6a98009040 comp rx=0 tx=0).stop 2023-11-24T08:04:55.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40acd70 msgr2=0x7f6aa4070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40acd70 0x7f6aa4070db0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009eb0 tx=0x7f6a9c004a40 comp rx=0 tx=0).stop 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 shutdown_connections 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6a9006ae40 0x7f6a9006d300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40acd70 0x7f6aa4070db0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6aa4070490 0x7f6aa4070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.632+0000 7f6aab090640 1 --2- 172.21.15.64:0/1979488485 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6aa406dc30 0x7f6aa414c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.633+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 >> 172.21.15.64:0/1979488485 conn(0x7f6aa40b0ba0 msgr2=0x7f6aa40af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.633+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 shutdown_connections 2023-11-24T08:04:55.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.633+0000 7f6aab090640 1 -- 172.21.15.64:0/1979488485 wait complete. 2023-11-24T08:04:55.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:55.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.883+0000 7fcaf0b2e640 1 Processor -- start 2023-11-24T08:04:55.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.883+0000 7fcaf0b2e640 1 -- start start 2023-11-24T08:04:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcaf0b2e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaec1435d0 0x7fcaec071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcaf0b2e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcaec1439b0 0x7fcaec0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcaf0b2e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaec070d40 con 0x7fcaec0b3f70 2023-11-24T08:04:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcaf0b2e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaec0b43e0 con 0x7fcaec1435d0 2023-11-24T08:04:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcaf0b2e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaec071020 con 0x7fcaec1439b0 2023-11-24T08:04:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcaeb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaec1435d0 0x7fcaec071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcaeb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaec1435d0 0x7fcaec071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56914/0 (socket says 172.21.15.64:56914) 2023-11-24T08:04:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/961898523 learned_addr learned my addr 172.21.15.64:0/961898523 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.885+0000 7fcae3fff640 1 --2- 172.21.15.64:0/961898523 >> v2:172.21.15.91:3300/0 conn(0x7fcaec1439b0 0x7fcaec0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.886+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/961898523 >> v2:172.21.15.91:3300/0 conn(0x7fcaec1439b0 msgr2=0x7fcaec0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.886+0000 7fcaeb7fe640 1 --2- 172.21.15.64:0/961898523 >> v2:172.21.15.91:3300/0 conn(0x7fcaec1439b0 0x7fcaec0b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.886+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/961898523 >> v1:172.21.15.91:6789/0 conn(0x7fcaec0b3f70 legacy=0x7fcaec070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.886+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/961898523 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec071300 con 0x7fcaec1435d0 2023-11-24T08:04:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.887+0000 7fcaeb7fe640 1 --2- 172.21.15.64:0/961898523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec1435d0 0x7fcaec071de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0cbe80 tx=0x7fcadc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6180f708adb26e37 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:55.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.887+0000 7fcaeaffd640 1 -- 172.21.15.64:0/961898523 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0317e0 con 0x7fcaec1435d0 2023-11-24T08:04:55.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.887+0000 7fcaeaffd640 1 -- 172.21.15.64:0/961898523 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc031940 con 0x7fcaec1435d0 2023-11-24T08:04:55.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.887+0000 7fcaeaffd640 1 -- 172.21.15.64:0/961898523 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc031cd0 con 0x7fcaec1435d0 2023-11-24T08:04:55.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.888+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/961898523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec1435d0 msgr2=0x7fcaec071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.888+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/961898523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec1435d0 0x7fcaec071de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0cbe80 tx=0x7fcadc02b880 comp rx=0 tx=0).stop 2023-11-24T08:04:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.888+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/961898523 shutdown_connections 2023-11-24T08:04:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.889+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/961898523 >> v2:172.21.15.91:3300/0 conn(0x7fcaec1439b0 0x7fcaec0b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.889+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/961898523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec1435d0 0x7fcaec071de0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.889+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/961898523 >> 172.21.15.64:0/961898523 conn(0x7fcaec0cec10 msgr2=0x7fcaec0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.889+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/961898523 shutdown_connections 2023-11-24T08:04:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.889+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/961898523 wait complete. 2023-11-24T08:04:55.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.891+0000 7fcaf0b2e640 1 Processor -- start 2023-11-24T08:04:55.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.891+0000 7fcaf0b2e640 1 -- start start 2023-11-24T08:04:55.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaf0b2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0b3f70 0x7fcaec162850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaf0b2e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaec1435d0 0x7fcaec165240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaf0b2e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec1439b0 0x7fcaec165780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaf0b2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec0ad280 con 0x7fcaec0b3f70 2023-11-24T08:04:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaf0b2e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec15c860 con 0x7fcaec1439b0 2023-11-24T08:04:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaf0b2e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec0ad4d0 con 0x7fcaec1435d0 2023-11-24T08:04:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaeb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0b3f70 0x7fcaec162850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaeb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0b3f70 0x7fcaec162850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56928/0 (socket says 172.21.15.64:56928) 2023-11-24T08:04:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.892+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/217119883 learned_addr learned my addr 172.21.15.64:0/217119883 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.893+0000 7fcae3fff640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaec1435d0 0x7fcaec165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.893+0000 7fcaebfff640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec1439b0 0x7fcaec165780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.893+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaec1435d0 msgr2=0x7fcaec165240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.893+0000 7fcaeb7fe640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaec1435d0 0x7fcaec165240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.893+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec1439b0 msgr2=0x7fcaec165780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.893+0000 7fcaeb7fe640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec1439b0 0x7fcaec165780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.893+0000 7fcaeb7fe640 1 -- 172.21.15.64:0/217119883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc0094d0 con 0x7fcaec0b3f70 2023-11-24T08:04:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.894+0000 7fcaeb7fe640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0b3f70 0x7fcaec162850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcadc02b8c0 tx=0x7fcadc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.894+0000 7fcae8ff9640 1 -- 172.21.15.64:0/217119883 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc032880 con 0x7fcaec0b3f70 2023-11-24T08:04:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.894+0000 7fcae8ff9640 1 -- 172.21.15.64:0/217119883 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc032a20 con 0x7fcaec0b3f70 2023-11-24T08:04:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.894+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec0ad7b0 con 0x7fcaec0b3f70 2023-11-24T08:04:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.894+0000 7fcae8ff9640 1 -- 172.21.15.64:0/217119883 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc032dd0 con 0x7fcaec0b3f70 2023-11-24T08:04:55.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.894+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaec0add70 con 0x7fcaec0b3f70 2023-11-24T08:04:55.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.895+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaec04e840 con 0x7fcaec0b3f70 2023-11-24T08:04:55.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.896+0000 7fcae8ff9640 1 -- 172.21.15.64:0/217119883 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcadc04e450 con 0x7fcaec0b3f70 2023-11-24T08:04:55.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.897+0000 7fcae8ff9640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcacc06ae40 0x7fcacc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:55.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.897+0000 7fcae8ff9640 1 -- 172.21.15.64:0/217119883 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fcadc0ab5e0 con 0x7fcaec0b3f70 2023-11-24T08:04:55.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.898+0000 7fcae3fff640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcacc06ae40 0x7fcacc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:55.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.899+0000 7fcae3fff640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcacc06ae40 0x7fcacc06d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0b2db0 tx=0x7fcad4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:55.902+0000 7fcae8ff9640 1 -- 172.21.15.64:0/217119883 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcadc076740 con 0x7fcaec0b3f70 2023-11-24T08:04:56.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.101+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcaec0b3130 con 0x7fcaec0b3f70 2023-11-24T08:04:56.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.101+0000 7fcae8ff9640 1 -- 172.21.15.64:0/217119883 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcadc079d60 con 0x7fcaec0b3f70 2023-11-24T08:04:56.103 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:56.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcacc06ae40 msgr2=0x7fcacc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcacc06ae40 0x7fcacc06d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0b2db0 tx=0x7fcad4008040 comp rx=0 tx=0).stop 2023-11-24T08:04:56.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0b3f70 msgr2=0x7fcaec162850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0b3f70 0x7fcaec162850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcadc02b8c0 tx=0x7fcadc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 shutdown_connections 2023-11-24T08:04:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcacc06ae40 0x7fcacc06d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaec1439b0 0x7fcaec165780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaec1435d0 0x7fcaec165240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 --2- 172.21.15.64:0/217119883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaec0b3f70 0x7fcaec162850 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 >> 172.21.15.64:0/217119883 conn(0x7fcaec0cec10 msgr2=0x7fcaec0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 shutdown_connections 2023-11-24T08:04:56.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.104+0000 7fcaf0b2e640 1 -- 172.21.15.64:0/217119883 wait complete. 2023-11-24T08:04:56.125 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:04:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.344+0000 7f6553fff640 1 Processor -- start 2023-11-24T08:04:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.344+0000 7f6553fff640 1 -- start start 2023-11-24T08:04:56.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.346+0000 7f6553fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f65541416f0 0x7f655415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.346+0000 7f6553fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6554070bf0 con 0x7f6554141e80 2023-11-24T08:04:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.346+0000 7f6553fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655406d850 con 0x7f655406d230 2023-11-24T08:04:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.346+0000 7f6553fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655406db30 con 0x7f65541416f0 2023-11-24T08:04:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.346+0000 7f65527fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6554141e80 0x7f655406cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42748/0 (socket says 172.21.15.64:42748) 2023-11-24T08:04:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.346+0000 7f65527fc640 1 -- 172.21.15.64:0/409991761 learned_addr learned my addr 172.21.15.64:0/409991761 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6552ffd640 1 --2- 172.21.15.64:0/409991761 >> v2:172.21.15.91:3300/0 conn(0x7f65541416f0 0x7f655415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893098272 0 0) 0x7f6554070bf0 con 0x7f6554141e80 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f653c003520 con 0x7f6554141e80 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2028041673 0 0) 0x7f653c003520 con 0x7f6554141e80 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 >> v2:172.21.15.91:3300/0 conn(0x7f65541416f0 msgr2=0x7f655415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 --2- 172.21.15.64:0/409991761 >> v2:172.21.15.91:3300/0 conn(0x7f65541416f0 0x7f655415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 >> v1:172.21.15.91:6789/0 conn(0x7f655406d230 legacy=0x7f65540cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540705a0 con 0x7f6554141e80 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f6548001670 con 0x7f6554141e80 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 >> v1:172.21.15.64:6789/0 conn(0x7f6554141e80 legacy=0x7f655406cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c003a30 0x7f653c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0047c0 0x7f653c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.347+0000 7f6551ffb640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f653c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655406db30 con 0x7f653c0051d0 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c00c100 con 0x7f653c0047c0 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c00c2c0 con 0x7f653c003a30 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f6552ffd640 1 --2- 172.21.15.64:0/409991761 >> v2:172.21.15.91:3300/0 conn(0x7f65541416f0 0x7f655415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f65537fe640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f653c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f65527fc640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0047c0 0x7f653c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f65537fe640 1 -- 172.21.15.64:0/409991761 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c003a30 msgr2=0x7f653c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f65537fe640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f653c003a30 0x7f653c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f65537fe640 1 -- 172.21.15.64:0/409991761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0047c0 msgr2=0x7f653c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f65537fe640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f653c0047c0 0x7f653c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.348+0000 7f65537fe640 1 -- 172.21.15.64:0/409991761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c003520 con 0x7f653c0051d0 2023-11-24T08:04:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.349+0000 7f65537fe640 1 -- 172.21.15.64:0/409991761 reap_dead start 2023-11-24T08:04:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.349+0000 7f65537fe640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f653c00b980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f654c001e50 tx=0x7f653c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adb4243564c0bfd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c003b70 con 0x7f653c0051d0 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c003d10 con 0x7f653c0051d0 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6551ffb640 1 -- 172.21.15.64:0/409991761 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c005310 con 0x7f653c0051d0 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6553fff640 1 -- 172.21.15.64:0/409991761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 msgr2=0x7f653c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6553fff640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f653c00b980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f654c001e50 tx=0x7f653c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6553fff640 1 -- 172.21.15.64:0/409991761 shutdown_connections 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6553fff640 1 --2- 172.21.15.64:0/409991761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f653c00b980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6553fff640 1 -- 172.21.15.64:0/409991761 >> 172.21.15.64:0/409991761 conn(0x7f65540b0840 msgr2=0x7f65540b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6553fff640 1 -- 172.21.15.64:0/409991761 shutdown_connections 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.350+0000 7f6553fff640 1 -- 172.21.15.64:0/409991761 wait complete. 2023-11-24T08:04:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.352+0000 7f6553fff640 1 Processor -- start 2023-11-24T08:04:56.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.352+0000 7f6553fff640 1 -- start start 2023-11-24T08:04:56.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.353+0000 7f6553fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f65540b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.353+0000 7f6553fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f655415bfe0 0x7f655415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.353+0000 7f6553fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f655415c460 0x7f65541361a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.353+0000 7f6553fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65540b3ca0 con 0x7f653c0051d0 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.353+0000 7f6553fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65540b32d0 con 0x7f655415bfe0 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.353+0000 7f6553fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65540b26c0 con 0x7f655415c460 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f65540b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f65540b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56942/0 (socket says 172.21.15.64:56942) 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 -- 172.21.15.64:0/336088531 learned_addr learned my addr 172.21.15.64:0/336088531 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:56.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f65537fe640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f655415c460 0x7f65541361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f65527fc640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f655415bfe0 0x7f655415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 -- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f655415c460 msgr2=0x7f65541361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f655415c460 0x7f65541361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 -- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f655415bfe0 msgr2=0x7f655415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f655415bfe0 0x7f655415ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 -- 172.21.15.64:0/336088531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540b29a0 con 0x7f653c0051d0 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f65527fc640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f655415bfe0 0x7f655415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.354+0000 7f6552ffd640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f65540b3760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f654400e930 tx=0x7f654400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.355+0000 7f653b7fe640 1 -- 172.21.15.64:0/336088531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400ca00 con 0x7f653c0051d0 2023-11-24T08:04:56.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.355+0000 7f653b7fe640 1 -- 172.21.15.64:0/336088531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654400cb80 con 0x7f653c0051d0 2023-11-24T08:04:56.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.355+0000 7f653b7fe640 1 -- 172.21.15.64:0/336088531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544004030 con 0x7f653c0051d0 2023-11-24T08:04:56.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.355+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65540b2c80 con 0x7f653c0051d0 2023-11-24T08:04:56.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.355+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65540b2370 con 0x7f653c0051d0 2023-11-24T08:04:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.356+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65540cc990 con 0x7f653c0051d0 2023-11-24T08:04:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.357+0000 7f653b7fe640 1 -- 172.21.15.64:0/336088531 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6544004190 con 0x7f653c0051d0 2023-11-24T08:04:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.358+0000 7f653b7fe640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f654006ae40 0x7f654006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.358+0000 7f653b7fe640 1 -- 172.21.15.64:0/336088531 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f65440896c0 con 0x7f653c0051d0 2023-11-24T08:04:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.359+0000 7f65527fc640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f654006ae40 0x7f654006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.360+0000 7f65527fc640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f654006ae40 0x7f654006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65541445e0 tx=0x7f6548000f90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:56.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.363+0000 7f653b7fe640 1 -- 172.21.15.64:0/336088531 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6544054770 con 0x7f653c0051d0 2023-11-24T08:04:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.568+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f655404e840 con 0x7f653c0051d0 2023-11-24T08:04:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.569+0000 7f653b7fe640 1 -- 172.21.15.64:0/336088531 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6544057d90 con 0x7f653c0051d0 2023-11-24T08:04:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f654006ae40 msgr2=0x7f654006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f654006ae40 0x7f654006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65541445e0 tx=0x7f6548000f90 comp rx=0 tx=0).stop 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 msgr2=0x7f65540b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f65540b3760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f654400e930 tx=0x7f654400edf0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 shutdown_connections 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f654006ae40 0x7f654006d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f655415c460 0x7f65541361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f655415bfe0 0x7f655415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 --2- 172.21.15.64:0/336088531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f653c0051d0 0x7f65540b3760 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 >> 172.21.15.64:0/336088531 conn(0x7f65540b0840 msgr2=0x7f65540ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 shutdown_connections 2023-11-24T08:04:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:56.572+0000 7f6553fff640 1 -- 172.21.15.64:0/336088531 wait complete. 2023-11-24T08:04:56.592 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T08:04:56.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:56.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-24T08:04:56.694 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:04:56.694 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:04:56.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:04:56.694 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T08:04:56.750 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:04:56.750 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:56.810 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:04:56.810 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:56.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:57.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.097+0000 7f9116039640 1 Processor -- start 2023-11-24T08:04:57.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.098+0000 7f9116039640 1 -- start start 2023-11-24T08:04:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.099+0000 7f9116039640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100d1900 0x7f91100b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.099+0000 7f9116039640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91100cf0c0 con 0x7f911015d5b0 2023-11-24T08:04:57.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.099+0000 7f9116039640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91100b01b0 con 0x7f91100d14a0 2023-11-24T08:04:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.099+0000 7f9116039640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911015d990 con 0x7f91100d1900 2023-11-24T08:04:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9115037640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f911015d5b0 0x7f91100d3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58348/0 (socket says 172.21.15.64:58348) 2023-11-24T08:04:57.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9115037640 1 -- 172.21.15.64:0/420439469 learned_addr learned my addr 172.21.15.64:0/420439469 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510028973 0 0) 0x7f91100cf0c0 con 0x7f911015d5b0 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90fc003520 con 0x7f911015d5b0 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9115838640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100d1900 0x7f91100b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1761334457 0 0) 0x7f90fc003520 con 0x7f911015d5b0 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100d1900 msgr2=0x7f91100b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100d1900 0x7f91100b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 >> v1:172.21.15.91:6789/0 conn(0x7f91100d14a0 legacy=0x7f91100b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100ceb00 con 0x7f911015d5b0 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f910c001670 con 0x7f911015d5b0 2023-11-24T08:04:57.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.100+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 >> v1:172.21.15.64:6789/0 conn(0x7f911015d5b0 legacy=0x7f91100d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9107fff640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90fc003a10 0x7f90fc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9107fff640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0047c0 0x7f90fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9107fff640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90fc0051d0 0x7f90fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911015d990 con 0x7f90fc0051d0 2023-11-24T08:04:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc00c100 con 0x7f90fc0047c0 2023-11-24T08:04:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc00c2c0 con 0x7f90fc003a10 2023-11-24T08:04:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9115838640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100d1900 0x7f91100b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:04:57.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9115838640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90fc0051d0 0x7f90fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9115037640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90fc003a10 0x7f90fc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9114836640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0047c0 0x7f90fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9115838640 1 -- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90fc003a10 msgr2=0x7f90fc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9115838640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90fc003a10 0x7f90fc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9115838640 1 -- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0047c0 msgr2=0x7f90fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9115838640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0047c0 0x7f90fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.101+0000 7f9115838640 1 -- 172.21.15.64:0/420439469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc003520 con 0x7f90fc0051d0 2023-11-24T08:04:57.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.103+0000 7f9115838640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90fc0051d0 0x7f90fc00b980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9108009f00 tx=0x7f90fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9774813465cd9d93 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:57.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.103+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910802cc70 con 0x7f90fc0051d0 2023-11-24T08:04:57.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.103+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910802cdd0 con 0x7f90fc0051d0 2023-11-24T08:04:57.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.103+0000 7f9107fff640 1 -- 172.21.15.64:0/420439469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108003bb0 con 0x7f90fc0051d0 2023-11-24T08:04:57.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.103+0000 7f9116039640 1 -- 172.21.15.64:0/420439469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90fc0051d0 msgr2=0x7f90fc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.103+0000 7f9116039640 1 --2- 172.21.15.64:0/420439469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90fc0051d0 0x7f90fc00b980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9108009f00 tx=0x7f90fc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.103+0000 7f9115838640 1 -- 172.21.15.64:0/420439469 reap_dead start 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.104+0000 7f9116039640 1 -- 172.21.15.64:0/420439469 shutdown_connections 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.104+0000 7f9116039640 1 -- 172.21.15.64:0/420439469 >> 172.21.15.64:0/420439469 conn(0x7f911009c3b0 msgr2=0x7f91100d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.104+0000 7f9116039640 1 -- 172.21.15.64:0/420439469 shutdown_connections 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.104+0000 7f9116039640 1 -- 172.21.15.64:0/420439469 wait complete. 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.105+0000 7f9116039640 1 Processor -- start 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.106+0000 7f9116039640 1 -- start start 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9116039640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0051d0 0x7f91100b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9116039640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100b5020 0x7f91101523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9116039640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91100c8770 0x7f91100c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9116039640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100c9130 con 0x7f91100c8770 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9116039640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100b4230 con 0x7f90fc0051d0 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9116039640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100b36b0 con 0x7f91100b5020 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9115838640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91100c8770 0x7f91100c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9115838640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91100c8770 0x7f91100c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53322/0 (socket says 172.21.15.64:53322) 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9115838640 1 -- 172.21.15.64:0/4087996592 learned_addr learned my addr 172.21.15.64:0/4087996592 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9115037640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0051d0 0x7f91100b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.107+0000 7f9114836640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100b5020 0x7f91101523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9115838640 1 -- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100b5020 msgr2=0x7f91101523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9115838640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100b5020 0x7f91101523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9115838640 1 -- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0051d0 msgr2=0x7f91100b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9115838640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0051d0 0x7f91100b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9115838640 1 -- 172.21.15.64:0/4087996592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100b3990 con 0x7f91100c8770 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9115838640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91100c8770 0x7f91100c8bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9108007360 tx=0x7f9108004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9105ffb640 1 -- 172.21.15.64:0/4087996592 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910803c070 con 0x7f91100c8770 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9105ffb640 1 -- 172.21.15.64:0/4087996592 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9108003e30 con 0x7f91100c8770 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9105ffb640 1 -- 172.21.15.64:0/4087996592 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108004470 con 0x7f91100c8770 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.108+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91100b3c70 con 0x7f91100c8770 2023-11-24T08:04:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.109+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91100b54a0 con 0x7f91100c8770 2023-11-24T08:04:57.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.110+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911004e840 con 0x7f91100c8770 2023-11-24T08:04:57.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.110+0000 7f9105ffb640 1 -- 172.21.15.64:0/4087996592 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f91080045d0 con 0x7f91100c8770 2023-11-24T08:04:57.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.111+0000 7f9105ffb640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f90f406ae40 0x7f90f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.112+0000 7f9105ffb640 1 -- 172.21.15.64:0/4087996592 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f91080aaa20 con 0x7f91100c8770 2023-11-24T08:04:57.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.112+0000 7f9115037640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f90f406ae40 0x7f90f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.113+0000 7f9115037640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f90f406ae40 0x7f90f406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f910c001bc0 tx=0x7f910c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.116+0000 7f9105ffb640 1 -- 172.21.15.64:0/4087996592 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9108075a30 con 0x7f91100c8770 2023-11-24T08:04:57.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.314+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91100678d0 con 0x7f91100c8770 2023-11-24T08:04:57.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.315+0000 7f9105ffb640 1 -- 172.21.15.64:0/4087996592 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7f9108045090 con 0x7f91100c8770 2023-11-24T08:04:57.318 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:57.318 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:57.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.318+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f90f406ae40 msgr2=0x7f90f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.318+0000 7f9116039640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f90f406ae40 0x7f90f406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f910c001bc0 tx=0x7f910c009040 comp rx=0 tx=0).stop 2023-11-24T08:04:57.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.318+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91100c8770 msgr2=0x7f91100c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.318+0000 7f9116039640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91100c8770 0x7f91100c8bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9108007360 tx=0x7f9108004ab0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.319+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 shutdown_connections 2023-11-24T08:04:57.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.319+0000 7f9116039640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f90f406ae40 0x7f90f406d300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.319+0000 7f9116039640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91100c8770 0x7f91100c8bf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.319+0000 7f9116039640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f91100b5020 0x7f91101523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.319+0000 7f9116039640 1 --2- 172.21.15.64:0/4087996592 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90fc0051d0 0x7f91100b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.319+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 >> 172.21.15.64:0/4087996592 conn(0x7f911009c3b0 msgr2=0x7f911015dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:57.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.319+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 shutdown_connections 2023-11-24T08:04:57.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.319+0000 7f9116039640 1 -- 172.21.15.64:0/4087996592 wait complete. 2023-11-24T08:04:57.340 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:57.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.574+0000 7fae349db640 1 Processor -- start 2023-11-24T08:04:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.574+0000 7fae349db640 1 -- start start 2023-11-24T08:04:57.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.576+0000 7fae349db640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae301435d0 0x7fae30071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.576+0000 7fae349db640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 0x7fae300b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.576+0000 7fae349db640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fae300b3f70 0x7fae30070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.576+0000 7fae349db640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae300b4400 con 0x7fae301435d0 2023-11-24T08:04:57.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.576+0000 7fae349db640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae30070d40 con 0x7fae300b3f70 2023-11-24T08:04:57.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.576+0000 7fae349db640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae30071020 con 0x7fae301439b0 2023-11-24T08:04:57.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.576+0000 7fae2f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae301435d0 0x7fae30071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae301435d0 0x7fae30071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53334/0 (socket says 172.21.15.64:53334) 2023-11-24T08:04:57.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2f7fe640 1 -- 172.21.15.64:0/1657305705 learned_addr learned my addr 172.21.15.64:0/1657305705 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:57.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2effd640 1 --2- 172.21.15.64:0/1657305705 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 0x7fae300b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2ffff640 1 --2- 172.21.15.64:0/1657305705 >> v2:172.21.15.91:3300/0 conn(0x7fae300b3f70 0x7fae30070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2f7fe640 1 -- 172.21.15.64:0/1657305705 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 msgr2=0x7fae300b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:57.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2f7fe640 1 --2- 172.21.15.64:0/1657305705 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 0x7fae300b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2f7fe640 1 -- 172.21.15.64:0/1657305705 >> v2:172.21.15.91:3300/0 conn(0x7fae300b3f70 msgr2=0x7fae30070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2f7fe640 1 --2- 172.21.15.64:0/1657305705 >> v2:172.21.15.91:3300/0 conn(0x7fae300b3f70 0x7fae30070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.577+0000 7fae2f7fe640 1 -- 172.21.15.64:0/1657305705 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30071300 con 0x7fae301435d0 2023-11-24T08:04:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.578+0000 7fae2f7fe640 1 --2- 172.21.15.64:0/1657305705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae30071de0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fae20009870 tx=0x7fae2002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e8877d728f1a26 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.579+0000 7fae2e7fc640 1 -- 172.21.15.64:0/1657305705 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20031640 con 0x7fae301435d0 2023-11-24T08:04:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.579+0000 7fae2e7fc640 1 -- 172.21.15.64:0/1657305705 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae200317a0 con 0x7fae301435d0 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.579+0000 7fae2e7fc640 1 -- 172.21.15.64:0/1657305705 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20031b30 con 0x7fae301435d0 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.579+0000 7fae349db640 1 -- 172.21.15.64:0/1657305705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 msgr2=0x7fae30071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.579+0000 7fae349db640 1 --2- 172.21.15.64:0/1657305705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae30071de0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fae20009870 tx=0x7fae2002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.580+0000 7fae349db640 1 -- 172.21.15.64:0/1657305705 shutdown_connections 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.580+0000 7fae349db640 1 --2- 172.21.15.64:0/1657305705 >> v2:172.21.15.91:3300/0 conn(0x7fae300b3f70 0x7fae30070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.580+0000 7fae349db640 1 --2- 172.21.15.64:0/1657305705 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 0x7fae300b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.580+0000 7fae349db640 1 --2- 172.21.15.64:0/1657305705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae30071de0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.580+0000 7fae349db640 1 -- 172.21.15.64:0/1657305705 >> 172.21.15.64:0/1657305705 conn(0x7fae300cec10 msgr2=0x7fae300cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.580+0000 7fae349db640 1 -- 172.21.15.64:0/1657305705 shutdown_connections 2023-11-24T08:04:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.580+0000 7fae349db640 1 -- 172.21.15.64:0/1657305705 wait complete. 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.582+0000 7fae349db640 1 Processor -- start 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.582+0000 7fae349db640 1 -- start start 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae349db640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae300b3f70 0x7fae30142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae349db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae3014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae349db640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 0x7fae3014ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae349db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3014ace0 con 0x7fae301435d0 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae349db640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3014a0d0 con 0x7fae300b3f70 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae349db640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3014a3b0 con 0x7fae301439b0 2023-11-24T08:04:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae2effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae3014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae2effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae3014b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53350/0 (socket says 172.21.15.64:53350) 2023-11-24T08:04:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae2effd640 1 -- 172.21.15.64:0/3873360554 learned_addr learned my addr 172.21.15.64:0/3873360554 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae2effd640 1 -- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 msgr2=0x7fae3014ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae2effd640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 0x7fae3014ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae2effd640 1 -- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae300b3f70 msgr2=0x7fae30142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.584+0000 7fae2f7fe640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae300b3f70 0x7fae30142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.585+0000 7fae2effd640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae300b3f70 0x7fae30142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.585+0000 7fae2effd640 1 -- 172.21.15.64:0/3873360554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200094d0 con 0x7fae301435d0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.585+0000 7fae2effd640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae3014b500 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fae2400d5f0 tx=0x7fae2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.585+0000 7fae17fff640 1 -- 172.21.15.64:0/3873360554 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240125f0 con 0x7fae301435d0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.585+0000 7fae17fff640 1 -- 172.21.15.64:0/3873360554 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24012750 con 0x7fae301435d0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.585+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3014a690 con 0x7fae301435d0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.585+0000 7fae17fff640 1 -- 172.21.15.64:0/3873360554 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24012b00 con 0x7fae301435d0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.585+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3014bf80 con 0x7fae301435d0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.587+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3004e840 con 0x7fae301435d0 2023-11-24T08:04:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.588+0000 7fae17fff640 1 -- 172.21.15.64:0/3873360554 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae24012c60 con 0x7fae301435d0 2023-11-24T08:04:57.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.589+0000 7fae17fff640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae1c06ae40 0x7fae1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:57.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.589+0000 7fae17fff640 1 -- 172.21.15.64:0/3873360554 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fae24089b90 con 0x7fae301435d0 2023-11-24T08:04:57.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.590+0000 7fae2f7fe640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae1c06ae40 0x7fae1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.591+0000 7fae2f7fe640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae1c06ae40 0x7fae1c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fae20036000 tx=0x7fae2003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:57.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.593+0000 7fae17fff640 1 -- 172.21.15.64:0/3873360554 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae24054bb0 con 0x7fae301435d0 2023-11-24T08:04:57.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.789+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae3014ae20 con 0x7fae301435d0 2023-11-24T08:04:57.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.790+0000 7fae17fff640 1 -- 172.21.15.64:0/3873360554 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae3014ae20 con 0x7fae301435d0 2023-11-24T08:04:57.792 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:04:57.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae1c06ae40 msgr2=0x7fae1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae1c06ae40 0x7fae1c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fae20036000 tx=0x7fae2003c040 comp rx=0 tx=0).stop 2023-11-24T08:04:57.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 msgr2=0x7fae3014b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:57.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae3014b500 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fae2400d5f0 tx=0x7fae2400dab0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 shutdown_connections 2023-11-24T08:04:57.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae1c06ae40 0x7fae1c06d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae301439b0 0x7fae3014ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae301435d0 0x7fae3014b500 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 --2- 172.21.15.64:0/3873360554 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae300b3f70 0x7fae30142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 >> 172.21.15.64:0/3873360554 conn(0x7fae300cec10 msgr2=0x7fae300b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 shutdown_connections 2023-11-24T08:04:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:57.793+0000 7fae349db640 1 -- 172.21.15.64:0/3873360554 wait complete. 2023-11-24T08:04:57.815 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:04:58.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.037+0000 7f6855cd1640 1 Processor -- start 2023-11-24T08:04:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.037+0000 7f6855cd1640 1 -- start start 2023-11-24T08:04:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.039+0000 7f6855cd1640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6850144770 0x7f685015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.039+0000 7f6855cd1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68500b35a0 0x7f685015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.039+0000 7f6855cd1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f685006c9d0 0x7f685006ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.039+0000 7f6855cd1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500b43b0 con 0x7f685006c9d0 2023-11-24T08:04:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.039+0000 7f6855cd1640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685015f930 con 0x7f6850144770 2023-11-24T08:04:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.039+0000 7f6855cd1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500b3b10 con 0x7f68500b35a0 2023-11-24T08:04:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f68554d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f685006c9d0 0x7f685006ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f68554d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f685006c9d0 0x7f685006ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53352/0 (socket says 172.21.15.64:53352) 2023-11-24T08:04:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f68554d0640 1 -- 172.21.15.64:0/1469534494 learned_addr learned my addr 172.21.15.64:0/1469534494 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f6854ccf640 1 --2- 172.21.15.64:0/1469534494 >> v2:172.21.15.91:3300/0 conn(0x7f6850144770 0x7f685015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f684ffff640 1 --2- 172.21.15.64:0/1469534494 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68500b35a0 0x7f685015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f68554d0640 1 -- 172.21.15.64:0/1469534494 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68500b35a0 msgr2=0x7f685015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f68554d0640 1 --2- 172.21.15.64:0/1469534494 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68500b35a0 0x7f685015f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f68554d0640 1 -- 172.21.15.64:0/1469534494 >> v2:172.21.15.91:3300/0 conn(0x7f6850144770 msgr2=0x7f685015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f68554d0640 1 --2- 172.21.15.64:0/1469534494 >> v2:172.21.15.91:3300/0 conn(0x7f6850144770 0x7f685015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f68554d0640 1 -- 172.21.15.64:0/1469534494 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500b3df0 con 0x7f685006c9d0 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f684ffff640 1 --2- 172.21.15.64:0/1469534494 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68500b35a0 0x7f685015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.040+0000 7f6854ccf640 1 --2- 172.21.15.64:0/1469534494 >> v2:172.21.15.91:3300/0 conn(0x7f6850144770 0x7f685015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:58.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.041+0000 7f68554d0640 1 --2- 172.21.15.64:0/1469534494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685006c9d0 0x7f685006ce60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6848009870 tx=0x7f684802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f96f352e0283e3c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f684f7fe640 1 -- 172.21.15.64:0/1469534494 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848031640 con 0x7f685006c9d0 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f684f7fe640 1 -- 172.21.15.64:0/1469534494 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68480317a0 con 0x7f685006c9d0 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f684f7fe640 1 -- 172.21.15.64:0/1469534494 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848031b30 con 0x7f685006c9d0 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 -- 172.21.15.64:0/1469534494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685006c9d0 msgr2=0x7f685006ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 --2- 172.21.15.64:0/1469534494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685006c9d0 0x7f685006ce60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6848009870 tx=0x7f684802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 -- 172.21.15.64:0/1469534494 shutdown_connections 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 --2- 172.21.15.64:0/1469534494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685006c9d0 0x7f685006ce60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 --2- 172.21.15.64:0/1469534494 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68500b35a0 0x7f685015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 --2- 172.21.15.64:0/1469534494 >> v2:172.21.15.91:3300/0 conn(0x7f6850144770 0x7f685015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 -- 172.21.15.64:0/1469534494 >> 172.21.15.64:0/1469534494 conn(0x7f68500cec10 msgr2=0x7f68500cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 -- 172.21.15.64:0/1469534494 shutdown_connections 2023-11-24T08:04:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.042+0000 7f6855cd1640 1 -- 172.21.15.64:0/1469534494 wait complete. 2023-11-24T08:04:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.044+0000 7f6855cd1640 1 Processor -- start 2023-11-24T08:04:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.044+0000 7f6855cd1640 1 -- start start 2023-11-24T08:04:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.045+0000 7f6855cd1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685006c9d0 0x7f68501f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.045+0000 7f6855cd1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f68500b35a0 0x7f685015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.045+0000 7f6855cd1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6850144770 0x7f68500ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.045+0000 7f6855cd1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685015c600 con 0x7f6850144770 2023-11-24T08:04:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.045+0000 7f6855cd1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501f3340 con 0x7f68500b35a0 2023-11-24T08:04:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.045+0000 7f6855cd1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68501f3620 con 0x7f685006c9d0 2023-11-24T08:04:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f68554d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6850144770 0x7f68500ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f68554d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6850144770 0x7f68500ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53366/0 (socket says 172.21.15.64:53366) 2023-11-24T08:04:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f68554d0640 1 -- 172.21.15.64:0/3229657741 learned_addr learned my addr 172.21.15.64:0/3229657741 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f6854ccf640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685006c9d0 0x7f68501f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f684ffff640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f68500b35a0 0x7f685015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f68554d0640 1 -- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685006c9d0 msgr2=0x7f68501f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f68554d0640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685006c9d0 0x7f68501f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f68554d0640 1 -- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f68500b35a0 msgr2=0x7f685015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f68554d0640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f68500b35a0 0x7f685015de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.046+0000 7f68554d0640 1 -- 172.21.15.64:0/3229657741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480094d0 con 0x7f6850144770 2023-11-24T08:04:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.047+0000 7f68554d0640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6850144770 0x7f68500ac870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6848009870 tx=0x7f68480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.047+0000 7f684d7fa640 1 -- 172.21.15.64:0/3229657741 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684803ac60 con 0x7f6850144770 2023-11-24T08:04:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.047+0000 7f684d7fa640 1 -- 172.21.15.64:0/3229657741 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684803ade0 con 0x7f6850144770 2023-11-24T08:04:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.047+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68501f3900 con 0x7f6850144770 2023-11-24T08:04:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.047+0000 7f684d7fa640 1 -- 172.21.15.64:0/3229657741 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848044630 con 0x7f6850144770 2023-11-24T08:04:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.047+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68501f3be0 con 0x7f6850144770 2023-11-24T08:04:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.048+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685004e840 con 0x7f6850144770 2023-11-24T08:04:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.049+0000 7f684d7fa640 1 -- 172.21.15.64:0/3229657741 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f684802f050 con 0x7f6850144770 2023-11-24T08:04:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.050+0000 7f684d7fa640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683006ae40 0x7f683006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.050+0000 7f684d7fa640 1 -- 172.21.15.64:0/3229657741 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f68480ab720 con 0x7f6850144770 2023-11-24T08:04:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.051+0000 7f6854ccf640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683006ae40 0x7f683006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.052+0000 7f6854ccf640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683006ae40 0x7f683006d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6840007130 tx=0x7f6840007060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.055+0000 7f684d7fa640 1 -- 172.21.15.64:0/3229657741 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6848076740 con 0x7f6850144770 2023-11-24T08:04:58.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.260+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68500b2460 con 0x7f6850144770 2023-11-24T08:04:58.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.261+0000 7f684d7fa640 1 -- 172.21.15.64:0/3229657741 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6848079d60 con 0x7f6850144770 2023-11-24T08:04:58.262 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:04:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.263+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683006ae40 msgr2=0x7f683006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.263+0000 7f6855cd1640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683006ae40 0x7f683006d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6840007130 tx=0x7f6840007060 comp rx=0 tx=0).stop 2023-11-24T08:04:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.263+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6850144770 msgr2=0x7f68500ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.263+0000 7f6855cd1640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6850144770 0x7f68500ac870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6848009870 tx=0x7f68480046c0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.264+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 shutdown_connections 2023-11-24T08:04:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.264+0000 7f6855cd1640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683006ae40 0x7f683006d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.264+0000 7f6855cd1640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6850144770 0x7f68500ac870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.264+0000 7f6855cd1640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f68500b35a0 0x7f685015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.264+0000 7f6855cd1640 1 --2- 172.21.15.64:0/3229657741 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685006c9d0 0x7f68501f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.264+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 >> 172.21.15.64:0/3229657741 conn(0x7f68500cec10 msgr2=0x7f685013dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:58.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.264+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 shutdown_connections 2023-11-24T08:04:58.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.264+0000 7f6855cd1640 1 -- 172.21.15.64:0/3229657741 wait complete. 2023-11-24T08:04:58.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:04:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.514+0000 7fcb29209640 1 Processor -- start 2023-11-24T08:04:58.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.514+0000 7fcb29209640 1 -- start start 2023-11-24T08:04:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb29209640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb240b48d0 0x7fcb240b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb29209640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b3210 0x7fcb24167290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb29209640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb240b23e0 con 0x7fcb24160bd0 2023-11-24T08:04:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb29209640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb240cdc70 con 0x7fcb240b48d0 2023-11-24T08:04:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb29209640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240b2870 con 0x7fcb240b3210 2023-11-24T08:04:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb1a59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb240b48d0 0x7fcb240b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb1a59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb240b48d0 0x7fcb240b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53380/0 (socket says 172.21.15.64:53380) 2023-11-24T08:04:58.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb1a59c640 1 -- 172.21.15.64:0/1281922011 learned_addr learned my addr 172.21.15.64:0/1281922011 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb2359e640 1 --2- 172.21.15.64:0/1281922011 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b3210 0x7fcb24167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.516+0000 7fcb1a59c640 1 -- 172.21.15.64:0/1281922011 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b3210 msgr2=0x7fcb24167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.517+0000 7fcb1a59c640 1 --2- 172.21.15.64:0/1281922011 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b3210 0x7fcb24167290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.517+0000 7fcb1a59c640 1 -- 172.21.15.64:0/1281922011 >> v1:172.21.15.91:6789/0 conn(0x7fcb24160bd0 legacy=0x7fcb240b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.517+0000 7fcb1a59c640 1 -- 172.21.15.64:0/1281922011 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240d41b0 con 0x7fcb240b48d0 2023-11-24T08:04:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.518+0000 7fcb1a59c640 1 --2- 172.21.15.64:0/1281922011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b48d0 0x7fcb240b1ea0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009870 tx=0x7fcb0c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3503a1f92ef0384d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.518+0000 7fcb2259c640 1 -- 172.21.15.64:0/1281922011 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0317c0 con 0x7fcb240b48d0 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.518+0000 7fcb2259c640 1 -- 172.21.15.64:0/1281922011 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c031920 con 0x7fcb240b48d0 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.518+0000 7fcb2259c640 1 -- 172.21.15.64:0/1281922011 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c031cb0 con 0x7fcb240b48d0 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.519+0000 7fcb29209640 1 -- 172.21.15.64:0/1281922011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b48d0 msgr2=0x7fcb240b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.519+0000 7fcb29209640 1 --2- 172.21.15.64:0/1281922011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b48d0 0x7fcb240b1ea0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009870 tx=0x7fcb0c02b860 comp rx=0 tx=0).stop 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.519+0000 7fcb29209640 1 -- 172.21.15.64:0/1281922011 shutdown_connections 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.519+0000 7fcb29209640 1 --2- 172.21.15.64:0/1281922011 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b3210 0x7fcb24167290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.519+0000 7fcb29209640 1 --2- 172.21.15.64:0/1281922011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b48d0 0x7fcb240b1ea0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.519+0000 7fcb29209640 1 -- 172.21.15.64:0/1281922011 >> 172.21.15.64:0/1281922011 conn(0x7fcb2409c5a0 msgr2=0x7fcb240d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.519+0000 7fcb29209640 1 -- 172.21.15.64:0/1281922011 shutdown_connections 2023-11-24T08:04:58.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.519+0000 7fcb29209640 1 -- 172.21.15.64:0/1281922011 wait complete. 2023-11-24T08:04:58.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.521+0000 7fcb29209640 1 Processor -- start 2023-11-24T08:04:58.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.521+0000 7fcb29209640 1 -- start start 2023-11-24T08:04:58.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb29209640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b3210 0x7fcb240c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb29209640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b48d0 0x7fcb241f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb29209640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb24160bd0 0x7fcb2415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb29209640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2415cbf0 con 0x7fcb240b3210 2023-11-24T08:04:58.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb29209640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240c66a0 con 0x7fcb24160bd0 2023-11-24T08:04:58.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb29209640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2415c570 con 0x7fcb240b48d0 2023-11-24T08:04:58.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b3210 0x7fcb240c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b3210 0x7fcb240c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53396/0 (socket says 172.21.15.64:53396) 2023-11-24T08:04:58.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.522+0000 7fcb22d9d640 1 -- 172.21.15.64:0/1897825710 learned_addr learned my addr 172.21.15.64:0/1897825710 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:58.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb22d9d640 1 -- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b48d0 msgr2=0x7fcb241f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb22d9d640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b48d0 0x7fcb241f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb22d9d640 1 -- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb24160bd0 msgr2=0x7fcb2415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb22d9d640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb24160bd0 0x7fcb2415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb22d9d640 1 -- 172.21.15.64:0/1897825710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c0094d0 con 0x7fcb240b3210 2023-11-24T08:04:58.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb22d9d640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b3210 0x7fcb240c6160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400ba50 tx=0x7fcb1400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb1bfff640 1 -- 172.21.15.64:0/1897825710 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1400cb80 con 0x7fcb240b3210 2023-11-24T08:04:58.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb1bfff640 1 -- 172.21.15.64:0/1897825710 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1400cd20 con 0x7fcb240b3210 2023-11-24T08:04:58.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb1bfff640 1 -- 172.21.15.64:0/1897825710 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14014680 con 0x7fcb240b3210 2023-11-24T08:04:58.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb240b6ec0 con 0x7fcb240b3210 2023-11-24T08:04:58.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.523+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2415c810 con 0x7fcb240b3210 2023-11-24T08:04:58.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.524+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2415c3d0 con 0x7fcb240b3210 2023-11-24T08:04:58.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.525+0000 7fcb1bfff640 1 -- 172.21.15.64:0/1897825710 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb140147e0 con 0x7fcb240b3210 2023-11-24T08:04:58.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.526+0000 7fcb1bfff640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406ae40 0x7fcb0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.526+0000 7fcb1bfff640 1 -- 172.21.15.64:0/1897825710 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fcb14089570 con 0x7fcb240b3210 2023-11-24T08:04:58.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.527+0000 7fcb1a59c640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406ae40 0x7fcb0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.528+0000 7fcb1a59c640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406ae40 0x7fcb0406d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c036000 tx=0x7fcb0c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.531+0000 7fcb1bfff640 1 -- 172.21.15.64:0/1897825710 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb2415c3d0 con 0x7fcb240b3210 2023-11-24T08:04:58.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.725+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb2415c3d0 con 0x7fcb240b3210 2023-11-24T08:04:58.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.726+0000 7fcb1bfff640 1 -- 172.21.15.64:0/1897825710 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb14019020 con 0x7fcb240b3210 2023-11-24T08:04:58.728 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:04:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406ae40 msgr2=0x7fcb0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406ae40 0x7fcb0406d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c036000 tx=0x7fcb0c030000 comp rx=0 tx=0).stop 2023-11-24T08:04:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b3210 msgr2=0x7fcb240c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b3210 0x7fcb240c6160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400ba50 tx=0x7fcb1400bf10 comp rx=0 tx=0).stop 2023-11-24T08:04:58.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 shutdown_connections 2023-11-24T08:04:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406ae40 0x7fcb0406d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb24160bd0 0x7fcb2415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb240b48d0 0x7fcb241f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 --2- 172.21.15.64:0/1897825710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb240b3210 0x7fcb240c6160 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 >> 172.21.15.64:0/1897825710 conn(0x7fcb2409c5a0 msgr2=0x7fcb240d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:58.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 shutdown_connections 2023-11-24T08:04:58.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.729+0000 7fcb29209640 1 -- 172.21.15.64:0/1897825710 wait complete. 2023-11-24T08:04:58.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:04:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.976+0000 7f94b5967640 1 Processor -- start 2023-11-24T08:04:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.976+0000 7f94b5967640 1 -- start start 2023-11-24T08:04:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.978+0000 7f94b5967640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f94b0070550 0x7f94b0160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.978+0000 7f94b5967640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b0161940 con 0x7f94b00b2550 2023-11-24T08:04:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.978+0000 7f94b5967640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b00b3040 con 0x7f94b0161530 2023-11-24T08:04:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.978+0000 7f94b5967640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b0070cf0 con 0x7f94b0070550 2023-11-24T08:04:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.978+0000 7f94b4965640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f94b00b2550 0x7f94b00b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58354/0 (socket says 172.21.15.64:58354) 2023-11-24T08:04:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.978+0000 7f94b4965640 1 -- 172.21.15.64:0/1302240304 learned_addr learned my addr 172.21.15.64:0/1302240304 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94affff640 1 --2- 172.21.15.64:0/1302240304 >> v2:172.21.15.91:3300/0 conn(0x7f94b0070550 0x7f94b0160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570266055 0 0) 0x7f94b0161940 con 0x7f94b00b2550 2023-11-24T08:04:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a0003520 con 0x7f94b00b2550 2023-11-24T08:04:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 271774792 0 0) 0x7f94a0003520 con 0x7f94b00b2550 2023-11-24T08:04:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 >> v2:172.21.15.91:3300/0 conn(0x7f94b0070550 msgr2=0x7f94b0160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 --2- 172.21.15.64:0/1302240304 >> v2:172.21.15.91:3300/0 conn(0x7f94b0070550 0x7f94b0160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 >> v1:172.21.15.91:6789/0 conn(0x7f94b0161530 legacy=0x7f94b00ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0070fd0 con 0x7f94b00b2550 2023-11-24T08:04:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f94a4001670 con 0x7f94b00b2550 2023-11-24T08:04:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.979+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 >> v1:172.21.15.64:6789/0 conn(0x7f94b00b2550 legacy=0x7f94b00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94af7fe640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94a00039e0 0x7f94a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94af7fe640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94a00047c0 0x7f94a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94af7fe640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b0070cf0 con 0x7f94a0005180 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a000c0b0 con 0x7f94a00039e0 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a000c270 con 0x7f94a00047c0 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94b5166640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94affff640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94a00047c0 0x7f94a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94b4965640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94a00039e0 0x7f94a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94b5166640 1 -- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94a00047c0 msgr2=0x7f94a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94b5166640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94a00047c0 0x7f94a0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94b5166640 1 -- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94a00039e0 msgr2=0x7f94a0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.980+0000 7f94b5166640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94a00039e0 0x7f94a0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.981+0000 7f94b5166640 1 -- 172.21.15.64:0/1302240304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0003520 con 0x7f94a0005180 2023-11-24T08:04:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.981+0000 7f94b4965640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94a00039e0 0x7f94a0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.981+0000 7f94b5166640 1 -- 172.21.15.64:0/1302240304 reap_dead start 2023-11-24T08:04:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.982+0000 7f94b5166640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94a000b980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f94a8001e50 tx=0x7f94a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71886f11a249d98 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.982+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8003b70 con 0x7f94a0005180 2023-11-24T08:04:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.982+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a8003cf0 con 0x7f94a0005180 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.982+0000 7f94af7fe640 1 -- 172.21.15.64:0/1302240304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8005310 con 0x7f94a0005180 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.982+0000 7f94b5967640 1 -- 172.21.15.64:0/1302240304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 msgr2=0x7f94a000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.982+0000 7f94b5967640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94a000b980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f94a8001e50 tx=0x7f94a00057f0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.983+0000 7f94b5967640 1 -- 172.21.15.64:0/1302240304 shutdown_connections 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.983+0000 7f94b5967640 1 --2- 172.21.15.64:0/1302240304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94a000b980 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.983+0000 7f94b5967640 1 -- 172.21.15.64:0/1302240304 >> 172.21.15.64:0/1302240304 conn(0x7f94b00b0840 msgr2=0x7f94b00b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.983+0000 7f94b5967640 1 -- 172.21.15.64:0/1302240304 shutdown_connections 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.983+0000 7f94b5967640 1 -- 172.21.15.64:0/1302240304 wait complete. 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.984+0000 7f94b5967640 1 Processor -- start 2023-11-24T08:04:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.985+0000 7f94b5967640 1 -- start start 2023-11-24T08:04:58.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b5967640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94b00aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b5967640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94b0159210 0x7f94b01f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b5967640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b015b520 0x7f94b015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b5967640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b015bee0 con 0x7f94a0005180 2023-11-24T08:04:58.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b5967640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b00af530 con 0x7f94b015b520 2023-11-24T08:04:58.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b5967640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b01f3d30 con 0x7f94b0159210 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b4965640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94b00aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b4965640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94b00aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53410/0 (socket says 172.21.15.64:53410) 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b4965640 1 -- 172.21.15.64:0/2459294388 learned_addr learned my addr 172.21.15.64:0/2459294388 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94affff640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94b0159210 0x7f94b01f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b4965640 1 -- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94b0159210 msgr2=0x7f94b01f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b4965640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94b0159210 0x7f94b01f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b4965640 1 -- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b015b520 msgr2=0x7f94b015b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b4965640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b015b520 0x7f94b015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:58.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.986+0000 7f94b4965640 1 -- 172.21.15.64:0/2459294388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0159780 con 0x7f94a0005180 2023-11-24T08:04:58.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.987+0000 7f94b4965640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94b00aeff0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f94a400d7e0 tx=0x7f94a400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.987+0000 7f94ad7fa640 1 -- 172.21.15.64:0/2459294388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40074e0 con 0x7f94a0005180 2023-11-24T08:04:58.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.987+0000 7f94ad7fa640 1 -- 172.21.15.64:0/2459294388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a4007640 con 0x7f94a0005180 2023-11-24T08:04:58.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.987+0000 7f94ad7fa640 1 -- 172.21.15.64:0/2459294388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40049e0 con 0x7f94a0005180 2023-11-24T08:04:58.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.987+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b0159a60 con 0x7f94a0005180 2023-11-24T08:04:58.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.987+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b015a020 con 0x7f94a0005180 2023-11-24T08:04:58.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.988+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b015a240 con 0x7f94a0005180 2023-11-24T08:04:58.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.989+0000 7f94ad7fa640 1 -- 172.21.15.64:0/2459294388 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94a4004b40 con 0x7f94a0005180 2023-11-24T08:04:58.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.990+0000 7f94ad7fa640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f949406ae40 0x7f949406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:58.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.990+0000 7f94ad7fa640 1 -- 172.21.15.64:0/2459294388 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f94a4089300 con 0x7f94a0005180 2023-11-24T08:04:58.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.991+0000 7f94affff640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f949406ae40 0x7f949406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:58.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.992+0000 7f94affff640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f949406ae40 0x7f949406d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f94b0160c70 tx=0x7f949c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:58.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:58.994+0000 7f94ad7fa640 1 -- 172.21.15.64:0/2459294388 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94a4054330 con 0x7f94a0005180 2023-11-24T08:04:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.210+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f94b0066e50 con 0x7f94a0005180 2023-11-24T08:04:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.211+0000 7f94ad7fa640 1 -- 172.21.15.64:0/2459294388 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f94a4057950 con 0x7f94a0005180 2023-11-24T08:04:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.213+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f949406ae40 msgr2=0x7f949406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.213+0000 7f94b5967640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f949406ae40 0x7f949406d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f94b0160c70 tx=0x7f949c008040 comp rx=0 tx=0).stop 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.213+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 msgr2=0x7f94b00aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.213+0000 7f94b5967640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94b00aeff0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f94a400d7e0 tx=0x7f94a400dca0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.214+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 shutdown_connections 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.214+0000 7f94b5967640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f949406ae40 0x7f949406d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.214+0000 7f94b5967640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f94b015b520 0x7f94b015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.214+0000 7f94b5967640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94b0159210 0x7f94b01f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.214+0000 7f94b5967640 1 --2- 172.21.15.64:0/2459294388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a0005180 0x7f94b00aeff0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.214+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 >> 172.21.15.64:0/2459294388 conn(0x7f94b00b0840 msgr2=0x7f94b00cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:59.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.214+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 shutdown_connections 2023-11-24T08:04:59.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.214+0000 7f94b5967640 1 -- 172.21.15.64:0/2459294388 wait complete. 2023-11-24T08:04:59.234 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:04:59.234 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:04:59.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:04:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.523+0000 7f836deda640 1 Processor -- start 2023-11-24T08:04:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.523+0000 7f836deda640 1 -- start start 2023-11-24T08:04:59.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.524+0000 7f836deda640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f83680b4c80 0x7f83680b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:59.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.524+0000 7f836deda640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f83680d2920 0x7f83680d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.524+0000 7f836deda640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83680b55a0 con 0x7f83680b1d20 2023-11-24T08:04:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.524+0000 7f836deda640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83680d1d50 con 0x7f83680d2920 2023-11-24T08:04:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.524+0000 7f836deda640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83680d2030 con 0x7f83680b4c80 2023-11-24T08:04:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f8367fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f83680b1d20 0x7f83680b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58368/0 (socket says 172.21.15.64:58368) 2023-11-24T08:04:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f8367fff640 1 -- 172.21.15.64:0/483517799 learned_addr learned my addr 172.21.15.64:0/483517799 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f8366ffd640 1 --2- 172.21.15.64:0/483517799 >> v2:172.21.15.64:3300/0 conn(0x7f83680d2920 0x7f83680d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f83677fe640 1 --2- 172.21.15.64:0/483517799 >> v2:172.21.15.91:3300/0 conn(0x7f83680b4c80 0x7f83680b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f83667fc640 1 -- 172.21.15.64:0/483517799 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336195969 0 0) 0x7f83680b55a0 con 0x7f83680b1d20 2023-11-24T08:04:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f83667fc640 1 -- 172.21.15.64:0/483517799 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8354003520 con 0x7f83680b1d20 2023-11-24T08:04:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f8366ffd640 1 -- 172.21.15.64:0/483517799 >> v2:172.21.15.91:3300/0 conn(0x7f83680b4c80 msgr2=0x7f83680b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f8366ffd640 1 --2- 172.21.15.64:0/483517799 >> v2:172.21.15.91:3300/0 conn(0x7f83680b4c80 0x7f83680b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f8366ffd640 1 -- 172.21.15.64:0/483517799 >> v1:172.21.15.64:6789/0 conn(0x7f83680b1d20 legacy=0x7f83680b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.525+0000 7f8366ffd640 1 -- 172.21.15.64:0/483517799 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83680d2310 con 0x7f83680d2920 2023-11-24T08:04:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.526+0000 7f8366ffd640 1 --2- 172.21.15.64:0/483517799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f83680d13e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f83680b5d70 tx=0x7f835002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01182b45af5ee10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.527+0000 7f83667fc640 1 -- 172.21.15.64:0/483517799 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835002e070 con 0x7f83680d2920 2023-11-24T08:04:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.527+0000 7f83667fc640 1 -- 172.21.15.64:0/483517799 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83500315a0 con 0x7f83680d2920 2023-11-24T08:04:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.527+0000 7f83667fc640 1 -- 172.21.15.64:0/483517799 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350031930 con 0x7f83680d2920 2023-11-24T08:04:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.527+0000 7f836deda640 1 -- 172.21.15.64:0/483517799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 msgr2=0x7f83680d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.527+0000 7f836deda640 1 --2- 172.21.15.64:0/483517799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f83680d13e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f83680b5d70 tx=0x7f835002b840 comp rx=0 tx=0).stop 2023-11-24T08:04:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.528+0000 7f836deda640 1 -- 172.21.15.64:0/483517799 shutdown_connections 2023-11-24T08:04:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.528+0000 7f836deda640 1 --2- 172.21.15.64:0/483517799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f83680d13e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.528+0000 7f836deda640 1 --2- 172.21.15.64:0/483517799 >> v2:172.21.15.91:3300/0 conn(0x7f83680b4c80 0x7f83680b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.528+0000 7f836deda640 1 -- 172.21.15.64:0/483517799 >> 172.21.15.64:0/483517799 conn(0x7f83680721a0 msgr2=0x7f83680cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.528+0000 7f836deda640 1 -- 172.21.15.64:0/483517799 shutdown_connections 2023-11-24T08:04:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.528+0000 7f836deda640 1 -- 172.21.15.64:0/483517799 wait complete. 2023-11-24T08:04:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.529+0000 7f836deda640 1 Processor -- start 2023-11-24T08:04:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.530+0000 7f836deda640 1 -- start start 2023-11-24T08:04:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.530+0000 7f836deda640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83680b1d20 0x7f83680cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.530+0000 7f836deda640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83680b4c80 0x7f836815f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f836deda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f836815f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f836deda640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83680ccfa0 con 0x7f83680d2920 2023-11-24T08:04:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f836deda640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83681f6d70 con 0x7f83680b1d20 2023-11-24T08:04:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f836deda640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8368157f90 con 0x7f83680b4c80 2023-11-24T08:04:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8367fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f836815f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8367fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f836815f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53428/0 (socket says 172.21.15.64:53428) 2023-11-24T08:04:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8367fff640 1 -- 172.21.15.64:0/3269332773 learned_addr learned my addr 172.21.15.64:0/3269332773 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:04:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f83677fe640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83680b1d20 0x7f83680cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8367fff640 1 -- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83680b4c80 msgr2=0x7f836815f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8366ffd640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83680b4c80 0x7f836815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8367fff640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83680b4c80 0x7f836815f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8367fff640 1 -- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83680b1d20 msgr2=0x7f83680cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8367fff640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83680b1d20 0x7f83680cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8367fff640 1 -- 172.21.15.64:0/3269332773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350009400 con 0x7f83680d2920 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f83677fe640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83680b1d20 0x7f83680cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.531+0000 7f8366ffd640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83680b4c80 0x7f836815f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.532+0000 7f8367fff640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f836815f890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f836000c390 tx=0x7f836000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.532+0000 7f835ffff640 1 -- 172.21.15.64:0/3269332773 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360005ce0 con 0x7f83680d2920 2023-11-24T08:04:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.532+0000 7f835ffff640 1 -- 172.21.15.64:0/3269332773 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8360005e40 con 0x7f83680d2920 2023-11-24T08:04:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.532+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83681f6440 con 0x7f83680d2920 2023-11-24T08:04:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.532+0000 7f835ffff640 1 -- 172.21.15.64:0/3269332773 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360007310 con 0x7f83680d2920 2023-11-24T08:04:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.532+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83681f6a00 con 0x7f83680d2920 2023-11-24T08:04:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.533+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83681f6f00 con 0x7f83680d2920 2023-11-24T08:04:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.534+0000 7f835ffff640 1 -- 172.21.15.64:0/3269332773 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8360007470 con 0x7f83680d2920 2023-11-24T08:04:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.535+0000 7f835ffff640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f834806ae40 0x7f834806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.535+0000 7f835ffff640 1 -- 172.21.15.64:0/3269332773 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8360088ca0 con 0x7f83680d2920 2023-11-24T08:04:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.536+0000 7f83677fe640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f834806ae40 0x7f834806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.537+0000 7f83677fe640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f834806ae40 0x7f834806d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8358007330 tx=0x7f835800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:59.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.540+0000 7f835ffff640 1 -- 172.21.15.64:0/3269332773 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8360053cb0 con 0x7f83680d2920 2023-11-24T08:04:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.734+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8368074770 con 0x7f83680d2920 2023-11-24T08:04:59.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.736+0000 7f835ffff640 1 -- 172.21.15.64:0/3269332773 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7f83600572d0 con 0x7f83680d2920 2023-11-24T08:04:59.738 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:04:59.738 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:04:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f834806ae40 msgr2=0x7f834806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f834806ae40 0x7f834806d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8358007330 tx=0x7f835800b040 comp rx=0 tx=0).stop 2023-11-24T08:04:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 msgr2=0x7f836815f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f836815f890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f836000c390 tx=0x7f836000c850 comp rx=0 tx=0).stop 2023-11-24T08:04:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 shutdown_connections 2023-11-24T08:04:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f834806ae40 0x7f834806d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83680d2920 0x7f836815f890 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f83680b4c80 0x7f836815f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.738+0000 7f836deda640 1 --2- 172.21.15.64:0/3269332773 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f83680b1d20 0x7f83680cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.739+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 >> 172.21.15.64:0/3269332773 conn(0x7f83680721a0 msgr2=0x7f836815cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.739+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 shutdown_connections 2023-11-24T08:04:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.739+0000 7f836deda640 1 -- 172.21.15.64:0/3269332773 wait complete. 2023-11-24T08:04:59.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:04:59.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:04:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.989+0000 7f8311883640 1 Processor -- start 2023-11-24T08:04:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.989+0000 7f8311883640 1 -- start start 2023-11-24T08:04:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.991+0000 7f8311883640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f830c15d3b0 0x7f830c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:04:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.991+0000 7f8311883640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c0c6bf0 con 0x7f830c0d0fa0 2023-11-24T08:04:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.991+0000 7f8311883640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c0d1430 con 0x7f830c0d1d90 2023-11-24T08:04:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.991+0000 7f8311883640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c0cedc0 con 0x7f830c15d3b0 2023-11-24T08:04:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.991+0000 7f830bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f830c0d0fa0 0x7f830c0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58372/0 (socket says 172.21.15.64:58372) 2023-11-24T08:04:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.991+0000 7f830bfff640 1 -- 172.21.15.64:0/1046366050 learned_addr learned my addr 172.21.15.64:0/1046366050 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:04:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.991+0000 7f8310881640 1 --2- 172.21.15.64:0/1046366050 >> v2:172.21.15.64:3300/0 conn(0x7f830c15d3b0 0x7f830c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:04:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.992+0000 7f830b7fe640 1 -- 172.21.15.64:0/1046366050 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481702561 0 0) 0x7f830c0c6bf0 con 0x7f830c0d0fa0 2023-11-24T08:04:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.992+0000 7f830b7fe640 1 -- 172.21.15.64:0/1046366050 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82fc003520 con 0x7f830c0d0fa0 2023-11-24T08:04:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.992+0000 7f8310881640 1 -- 172.21.15.64:0/1046366050 >> v1:172.21.15.91:6789/0 conn(0x7f830c0d1d90 legacy=0x7f830c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:04:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.992+0000 7f8310881640 1 -- 172.21.15.64:0/1046366050 >> v1:172.21.15.64:6789/0 conn(0x7f830c0d0fa0 legacy=0x7f830c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.992+0000 7f8310881640 1 -- 172.21.15.64:0/1046366050 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c0c6e40 con 0x7f830c15d3b0 2023-11-24T08:04:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.993+0000 7f8310881640 1 --2- 172.21.15.64:0/1046366050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c15d3b0 0x7f830c0ce6c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f83000097a0 tx=0x7f830002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc152030ed97d367 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:04:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.993+0000 7f830b7fe640 1 -- 172.21.15.64:0/1046366050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830002e070 con 0x7f830c15d3b0 2023-11-24T08:04:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.993+0000 7f830b7fe640 1 -- 172.21.15.64:0/1046366050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83000315a0 con 0x7f830c15d3b0 2023-11-24T08:04:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.993+0000 7f830b7fe640 1 -- 172.21.15.64:0/1046366050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300031930 con 0x7f830c15d3b0 2023-11-24T08:04:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.994+0000 7f8311883640 1 -- 172.21.15.64:0/1046366050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c15d3b0 msgr2=0x7f830c0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:04:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.994+0000 7f8311883640 1 --2- 172.21.15.64:0/1046366050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c15d3b0 0x7f830c0ce6c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f83000097a0 tx=0x7f830002b840 comp rx=0 tx=0).stop 2023-11-24T08:04:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.994+0000 7f8311883640 1 -- 172.21.15.64:0/1046366050 shutdown_connections 2023-11-24T08:04:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.994+0000 7f8311883640 1 --2- 172.21.15.64:0/1046366050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c15d3b0 0x7f830c0ce6c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:04:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.994+0000 7f8311883640 1 -- 172.21.15.64:0/1046366050 >> 172.21.15.64:0/1046366050 conn(0x7f830c0d30a0 msgr2=0x7f830c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:04:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.994+0000 7f8311883640 1 -- 172.21.15.64:0/1046366050 shutdown_connections 2023-11-24T08:04:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.994+0000 7f8311883640 1 -- 172.21.15.64:0/1046366050 wait complete. 2023-11-24T08:04:59.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.996+0000 7f8311883640 1 Processor -- start 2023-11-24T08:04:59.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.996+0000 7f8311883640 1 -- start start 2023-11-24T08:05:00.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8311883640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c0d0fa0 0x7f830c0c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8311883640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f830c0d1d90 0x7f830c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8311883640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f830c15d3b0 0x7f830c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8311883640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c15e970 con 0x7f830c0d0fa0 2023-11-24T08:05:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8311883640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c15ec50 con 0x7f830c15d3b0 2023-11-24T08:05:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8311883640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c15ef30 con 0x7f830c0d1d90 2023-11-24T08:05:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8310881640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c0d0fa0 0x7f830c0c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8310881640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c0d0fa0 0x7f830c0c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53440/0 (socket says 172.21.15.64:53440) 2023-11-24T08:05:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.997+0000 7f8310881640 1 -- 172.21.15.64:0/3936302319 learned_addr learned my addr 172.21.15.64:0/3936302319 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.998+0000 7f8310881640 1 -- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f830c0d1d90 msgr2=0x7f830c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.998+0000 7f8311082640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f830c15d3b0 0x7f830c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.998+0000 7f8310881640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f830c0d1d90 0x7f830c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.998+0000 7f8310881640 1 -- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f830c15d3b0 msgr2=0x7f830c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.998+0000 7f8310881640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f830c15d3b0 0x7f830c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.998+0000 7f8310881640 1 -- 172.21.15.64:0/3936302319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300009400 con 0x7f830c0d0fa0 2023-11-24T08:05:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.998+0000 7f8311082640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f830c15d3b0 0x7f830c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.998+0000 7f8310881640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c0d0fa0 0x7f830c0c6580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8300034040 tx=0x7f83000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.999+0000 7f83097fa640 1 -- 172.21.15.64:0/3936302319 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300032840 con 0x7f830c0d0fa0 2023-11-24T08:05:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.999+0000 7f83097fa640 1 -- 172.21.15.64:0/3936302319 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83000329a0 con 0x7f830c0d0fa0 2023-11-24T08:05:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.999+0000 7f83097fa640 1 -- 172.21.15.64:0/3936302319 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300032d50 con 0x7f830c0d0fa0 2023-11-24T08:05:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.999+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c15f210 con 0x7f830c0d0fa0 2023-11-24T08:05:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:04:59.999+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c15f7d0 con 0x7f830c0d0fa0 2023-11-24T08:05:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.000+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c04e840 con 0x7f830c0d0fa0 2023-11-24T08:05:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.001+0000 7f83097fa640 1 -- 172.21.15.64:0/3936302319 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f830003b3f0 con 0x7f830c0d0fa0 2023-11-24T08:05:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.002+0000 7f83097fa640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f82f006ae40 0x7f82f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.002+0000 7f83097fa640 1 -- 172.21.15.64:0/3936302319 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f83000ab5a0 con 0x7f830c0d0fa0 2023-11-24T08:05:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.003+0000 7f830bfff640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f82f006ae40 0x7f82f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.004+0000 7f830bfff640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f82f006ae40 0x7f82f006d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82f8000f60 tx=0x7f82f8007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:00.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.006+0000 7f83097fa640 1 -- 172.21.15.64:0/3936302319 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83000765c0 con 0x7f830c0d0fa0 2023-11-24T08:05:00.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.203+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f830c06b170 con 0x7f830c0d0fa0 2023-11-24T08:05:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.204+0000 7f83097fa640 1 -- 172.21.15.64:0/3936302319 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83000a9e60 con 0x7f830c0d0fa0 2023-11-24T08:05:00.206 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:05:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f82f006ae40 msgr2=0x7f82f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f82f006ae40 0x7f82f006d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82f8000f60 tx=0x7f82f8007970 comp rx=0 tx=0).stop 2023-11-24T08:05:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c0d0fa0 msgr2=0x7f830c0c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c0d0fa0 0x7f830c0c6580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8300034040 tx=0x7f83000046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 shutdown_connections 2023-11-24T08:05:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f82f006ae40 0x7f82f006d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f830c15d3b0 0x7f830c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f830c0d1d90 0x7f830c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 --2- 172.21.15.64:0/3936302319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c0d0fa0 0x7f830c0c6580 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.207+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 >> 172.21.15.64:0/3936302319 conn(0x7f830c0d30a0 msgr2=0x7f830c15e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.208+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 shutdown_connections 2023-11-24T08:05:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.208+0000 7f8311883640 1 -- 172.21.15.64:0/3936302319 wait complete. 2023-11-24T08:05:00.227 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:05:00.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.450+0000 7fcec25cf640 1 Processor -- start 2023-11-24T08:05:00.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.450+0000 7fcec25cf640 1 -- start start 2023-11-24T08:05:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec25cf640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcebc156eb0 0x7fcebc0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec25cf640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 0x7fcebc06cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec25cf640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcebc0b2bf0 con 0x7fcebc15a840 2023-11-24T08:05:00.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec25cf640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcebc071000 con 0x7fcebc156eb0 2023-11-24T08:05:00.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec25cf640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0b2630 con 0x7fcebc157290 2023-11-24T08:05:00.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec0dcc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcebc15a840 0x7fcebc0708f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58376/0 (socket says 172.21.15.64:58376) 2023-11-24T08:05:00.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec0dcc640 1 -- 172.21.15.64:0/3744703724 learned_addr learned my addr 172.21.15.64:0/3744703724 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:00.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415733934 0 0) 0x7fcebc0b2bf0 con 0x7fcebc15a840 2023-11-24T08:05:00.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec1dce640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 0x7fcebc06cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.452+0000 7fcec15cd640 1 --2- 172.21.15.64:0/3744703724 >> v2:172.21.15.91:3300/0 conn(0x7fcebc156eb0 0x7fcebc0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea8003520 con 0x7fcebc15a840 2023-11-24T08:05:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440905503 0 0) 0x7fcea8003520 con 0x7fcebc15a840 2023-11-24T08:05:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 msgr2=0x7fcebc06cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 0x7fcebc06cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 >> v2:172.21.15.91:3300/0 conn(0x7fcebc156eb0 msgr2=0x7fcebc0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3744703724 >> v2:172.21.15.91:3300/0 conn(0x7fcebc156eb0 0x7fcebc0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc0b2910 con 0x7fcebc15a840 2023-11-24T08:05:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fceac001670 con 0x7fcebc15a840 2023-11-24T08:05:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fcec1dce640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 0x7fcebc06cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 >> v1:172.21.15.64:6789/0 conn(0x7fcebc15a840 legacy=0x7fcebc0708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcea80039e0 0x7fcea8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 0x7fcea80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea8004d00 0x7fcea8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc071000 con 0x7fcea8004d00 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0b2630 con 0x7fcea80039e0 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea800b9b0 con 0x7fcebc157290 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.453+0000 7fcec1dce640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea8004d00 0x7fcea8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.454+0000 7fcec15cd640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcea80039e0 0x7fcea8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.454+0000 7fcec0dcc640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 0x7fcea80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.454+0000 7fcec1dce640 1 -- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 msgr2=0x7fcea80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.454+0000 7fcec1dce640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcebc157290 0x7fcea80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.454+0000 7fcec1dce640 1 -- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcea80039e0 msgr2=0x7fcea8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.454+0000 7fcec1dce640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcea80039e0 0x7fcea8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.454+0000 7fcec1dce640 1 -- 172.21.15.64:0/3744703724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8003520 con 0x7fcea8004d00 2023-11-24T08:05:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.454+0000 7fcec15cd640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcea80039e0 0x7fcea8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.455+0000 7fcec1dce640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea8004d00 0x7fcea8005180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009c40 tx=0x7fcea80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61cdb7508a7aa86 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.455+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4035600 con 0x7fcea8004d00 2023-11-24T08:05:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.455+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4031050 con 0x7fcea8004d00 2023-11-24T08:05:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.455+0000 7fceb3fff640 1 -- 172.21.15.64:0/3744703724 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4035960 con 0x7fcea8004d00 2023-11-24T08:05:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.456+0000 7fcec25cf640 1 -- 172.21.15.64:0/3744703724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea8004d00 msgr2=0x7fcea8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.456+0000 7fcec25cf640 1 --2- 172.21.15.64:0/3744703724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea8004d00 0x7fcea8005180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009c40 tx=0x7fcea80057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.456+0000 7fcec1dce640 1 -- 172.21.15.64:0/3744703724 reap_dead start 2023-11-24T08:05:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.456+0000 7fcec25cf640 1 -- 172.21.15.64:0/3744703724 shutdown_connections 2023-11-24T08:05:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.456+0000 7fcec25cf640 1 -- 172.21.15.64:0/3744703724 >> 172.21.15.64:0/3744703724 conn(0x7fcebc0cec10 msgr2=0x7fcebc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.456+0000 7fcec25cf640 1 -- 172.21.15.64:0/3744703724 shutdown_connections 2023-11-24T08:05:00.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.456+0000 7fcec25cf640 1 -- 172.21.15.64:0/3744703724 wait complete. 2023-11-24T08:05:00.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.458+0000 7fcec25cf640 1 Processor -- start 2023-11-24T08:05:00.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.458+0000 7fcec25cf640 1 -- start start 2023-11-24T08:05:00.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.459+0000 7fcec25cf640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcea8004d00 0x7fcebc155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.459+0000 7fcec25cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcebc0cd7a0 0x7fcebc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.459+0000 7fcec25cf640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc130cf0 0x7fcebc131170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.459+0000 7fcec25cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0ce160 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.459+0000 7fcec25cf640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc1327f0 con 0x7fcebc130cf0 2023-11-24T08:05:00.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.459+0000 7fcec25cf640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc131be0 con 0x7fcea8004d00 2023-11-24T08:05:00.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec0dcc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcebc0cd7a0 0x7fcebc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec0dcc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcebc0cd7a0 0x7fcebc0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53460/0 (socket says 172.21.15.64:53460) 2023-11-24T08:05:00.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec0dcc640 1 -- 172.21.15.64:0/1783908048 learned_addr learned my addr 172.21.15.64:0/1783908048 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:00.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec15cd640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcea8004d00 0x7fcebc155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec1dce640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc130cf0 0x7fcebc131170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec0dcc640 1 -- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcea8004d00 msgr2=0x7fcebc155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec0dcc640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcea8004d00 0x7fcebc155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec0dcc640 1 -- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc130cf0 msgr2=0x7fcebc131170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec0dcc640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc130cf0 0x7fcebc131170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec0dcc640 1 -- 172.21.15.64:0/1783908048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc131ec0 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.460+0000 7fcec15cd640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcea8004d00 0x7fcebc155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:00.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.461+0000 7fcec0dcc640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcebc0cd7a0 0x7fcebc0cdc20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fceac00e840 tx=0x7fceac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:00.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.461+0000 7fceb1ffb640 1 -- 172.21.15.64:0/1783908048 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac00c8d0 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.461+0000 7fceb1ffb640 1 -- 172.21.15.64:0/1783908048 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac00ca50 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.461+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcebc1321a0 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.461+0000 7fceb1ffb640 1 -- 172.21.15.64:0/1783908048 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac00ce00 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.461+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc1316b0 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.462+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcebc04e840 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.463+0000 7fceb1ffb640 1 -- 172.21.15.64:0/1783908048 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fceac01d460 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.465+0000 7fceb1ffb640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcea006ae70 0x7fcea006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.465+0000 7fceb1ffb640 1 -- 172.21.15.64:0/1783908048 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fceac0896f0 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.465+0000 7fcec15cd640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcea006ae70 0x7fcea006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.466+0000 7fcec15cd640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcea006ae70 0x7fcea006d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009a00 tx=0x7fceb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:00.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.469+0000 7fceb1ffb640 1 -- 172.21.15.64:0/1783908048 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fceac054710 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.663+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcebc0b3990 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.664+0000 7fceb1ffb640 1 -- 172.21.15.64:0/1783908048 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fceac057d30 con 0x7fcebc0cd7a0 2023-11-24T08:05:00.666 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcea006ae70 msgr2=0x7fcea006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcea006ae70 0x7fcea006d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009a00 tx=0x7fceb8008040 comp rx=0 tx=0).stop 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcebc0cd7a0 msgr2=0x7fcebc0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcebc0cd7a0 0x7fcebc0cdc20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fceac00e840 tx=0x7fceac00ed00 comp rx=0 tx=0).stop 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 shutdown_connections 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcea006ae70 0x7fcea006d330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcebc130cf0 0x7fcebc131170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcebc0cd7a0 0x7fcebc0cdc20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 --2- 172.21.15.64:0/1783908048 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcea8004d00 0x7fcebc155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 >> 172.21.15.64:0/1783908048 conn(0x7fcebc0cec10 msgr2=0x7fcebc0caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:00.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 shutdown_connections 2023-11-24T08:05:00.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.667+0000 7fcec25cf640 1 -- 172.21.15.64:0/1783908048 wait complete. 2023-11-24T08:05:00.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:05:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.919+0000 7fec04ebb640 1 Processor -- start 2023-11-24T08:05:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.919+0000 7fec04ebb640 1 -- start start 2023-11-24T08:05:00.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.921+0000 7fec04ebb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec000b4140 0x7fec00071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.921+0000 7fec04ebb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fec001439b0 0x7fec00070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.921+0000 7fec04ebb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec00070d40 con 0x7fec001435d0 2023-11-24T08:05:00.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.921+0000 7fec04ebb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec00071020 con 0x7fec000b4140 2023-11-24T08:05:00.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.921+0000 7fec04ebb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec00071300 con 0x7fec001439b0 2023-11-24T08:05:00.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.922+0000 7febf7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec000b4140 0x7fec00071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.922+0000 7febf7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec000b4140 0x7fec00071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53466/0 (socket says 172.21.15.64:53466) 2023-11-24T08:05:00.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.922+0000 7febf7fff640 1 -- 172.21.15.64:0/171564828 learned_addr learned my addr 172.21.15.64:0/171564828 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:00.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.922+0000 7febfffff640 1 --2- 172.21.15.64:0/171564828 >> v2:172.21.15.91:3300/0 conn(0x7fec001439b0 0x7fec00070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.922+0000 7febf7fff640 1 -- 172.21.15.64:0/171564828 >> v2:172.21.15.91:3300/0 conn(0x7fec001439b0 msgr2=0x7fec00070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:00.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.922+0000 7febf7fff640 1 --2- 172.21.15.64:0/171564828 >> v2:172.21.15.91:3300/0 conn(0x7fec001439b0 0x7fec00070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.922+0000 7febf7fff640 1 -- 172.21.15.64:0/171564828 >> v1:172.21.15.91:6789/0 conn(0x7fec001435d0 legacy=0x7fec000b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.922+0000 7febf7fff640 1 -- 172.21.15.64:0/171564828 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec000715e0 con 0x7fec000b4140 2023-11-24T08:05:00.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.926+0000 7febf7fff640 1 --2- 172.21.15.64:0/171564828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec000b4140 0x7fec00071de0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec000cbba0 tx=0x7febe802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be83bc247b5de0a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:00.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.926+0000 7febfeffd640 1 -- 172.21.15.64:0/171564828 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8031870 con 0x7fec000b4140 2023-11-24T08:05:00.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.926+0000 7febfeffd640 1 -- 172.21.15.64:0/171564828 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe80319d0 con 0x7fec000b4140 2023-11-24T08:05:00.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.926+0000 7febfeffd640 1 -- 172.21.15.64:0/171564828 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8031d60 con 0x7fec000b4140 2023-11-24T08:05:00.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.927+0000 7fec04ebb640 1 -- 172.21.15.64:0/171564828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec000b4140 msgr2=0x7fec00071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.927+0000 7fec04ebb640 1 --2- 172.21.15.64:0/171564828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec000b4140 0x7fec00071de0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec000cbba0 tx=0x7febe802b910 comp rx=0 tx=0).stop 2023-11-24T08:05:00.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.927+0000 7fec04ebb640 1 -- 172.21.15.64:0/171564828 shutdown_connections 2023-11-24T08:05:00.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.927+0000 7fec04ebb640 1 --2- 172.21.15.64:0/171564828 >> v2:172.21.15.91:3300/0 conn(0x7fec001439b0 0x7fec00070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.927+0000 7fec04ebb640 1 --2- 172.21.15.64:0/171564828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec000b4140 0x7fec00071de0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.927+0000 7fec04ebb640 1 -- 172.21.15.64:0/171564828 >> 172.21.15.64:0/171564828 conn(0x7fec000cec10 msgr2=0x7fec000cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:00.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.927+0000 7fec04ebb640 1 -- 172.21.15.64:0/171564828 shutdown_connections 2023-11-24T08:05:00.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.927+0000 7fec04ebb640 1 -- 172.21.15.64:0/171564828 wait complete. 2023-11-24T08:05:00.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.929+0000 7fec04ebb640 1 Processor -- start 2023-11-24T08:05:00.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.929+0000 7fec04ebb640 1 -- start start 2023-11-24T08:05:00.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.930+0000 7fec04ebb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec000b4140 0x7fec0015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.930+0000 7fec04ebb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fec001435d0 0x7fec0013c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.930+0000 7fec04ebb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec001439b0 0x7fec0013f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7fec04ebb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec001f4110 con 0x7fec001439b0 2023-11-24T08:05:00.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7fec04ebb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec001f3590 con 0x7fec000b4140 2023-11-24T08:05:00.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7fec04ebb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec001f3870 con 0x7fec001435d0 2023-11-24T08:05:00.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febfffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec001439b0 0x7fec0013f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febfffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec001439b0 0x7fec0013f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53472/0 (socket says 172.21.15.64:53472) 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febfffff640 1 -- 172.21.15.64:0/1346611439 learned_addr learned my addr 172.21.15.64:0/1346611439 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febff7fe640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec000b4140 0x7fec0015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febf7fff640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fec001435d0 0x7fec0013c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febfffff640 1 -- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fec001435d0 msgr2=0x7fec0013c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febfffff640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fec001435d0 0x7fec0013c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febfffff640 1 -- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec000b4140 msgr2=0x7fec0015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febfffff640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec000b4140 0x7fec0015bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febfffff640 1 -- 172.21.15.64:0/1346611439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80094d0 con 0x7fec001439b0 2023-11-24T08:05:00.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.931+0000 7febf7fff640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fec001435d0 0x7fec0013c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:00.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.932+0000 7febfffff640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec001439b0 0x7fec0013f810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7febf800efc0 tx=0x7febf800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:00.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.932+0000 7febfcff9640 1 -- 172.21.15.64:0/1346611439 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8009b20 con 0x7fec001439b0 2023-11-24T08:05:00.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.932+0000 7febfcff9640 1 -- 172.21.15.64:0/1346611439 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf800f040 con 0x7fec001439b0 2023-11-24T08:05:00.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.932+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec001f3b50 con 0x7fec001439b0 2023-11-24T08:05:00.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.932+0000 7febfcff9640 1 -- 172.21.15.64:0/1346611439 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf80146c0 con 0x7fec001439b0 2023-11-24T08:05:00.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.932+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0015c4d0 con 0x7fec001439b0 2023-11-24T08:05:00.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.934+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0015c6c0 con 0x7fec001439b0 2023-11-24T08:05:00.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.935+0000 7febfcff9640 1 -- 172.21.15.64:0/1346611439 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febf8014820 con 0x7fec001439b0 2023-11-24T08:05:00.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.936+0000 7febfcff9640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febe006ae70 0x7febe006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:00.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.936+0000 7febfcff9640 1 -- 172.21.15.64:0/1346611439 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7febf808a300 con 0x7fec001439b0 2023-11-24T08:05:00.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.937+0000 7febff7fe640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febe006ae70 0x7febe006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:00.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.938+0000 7febff7fe640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febe006ae70 0x7febe006d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f60 tx=0x7febf0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:00.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:00.944+0000 7febfcff9640 1 -- 172.21.15.64:0/1346611439 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febf80553b0 con 0x7fec001439b0 2023-11-24T08:05:01.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.141+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec0004e840 con 0x7fec001439b0 2023-11-24T08:05:01.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.142+0000 7febfcff9640 1 -- 172.21.15.64:0/1346611439 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7febf8009e60 con 0x7fec001439b0 2023-11-24T08:05:01.144 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:05:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febe006ae70 msgr2=0x7febe006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febe006ae70 0x7febe006d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f60 tx=0x7febf0009040 comp rx=0 tx=0).stop 2023-11-24T08:05:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec001439b0 msgr2=0x7fec0013f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec001439b0 0x7fec0013f810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7febf800efc0 tx=0x7febf800c490 comp rx=0 tx=0).stop 2023-11-24T08:05:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 shutdown_connections 2023-11-24T08:05:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febe006ae70 0x7febe006d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec001439b0 0x7fec0013f810 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fec001435d0 0x7fec0013c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 --2- 172.21.15.64:0/1346611439 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fec000b4140 0x7fec0015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 >> 172.21.15.64:0/1346611439 conn(0x7fec000cec10 msgr2=0x7fec000ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 shutdown_connections 2023-11-24T08:05:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.145+0000 7fec04ebb640 1 -- 172.21.15.64:0/1346611439 wait complete. 2023-11-24T08:05:01.168 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:05:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.398+0000 7f3c69623640 1 Processor -- start 2023-11-24T08:05:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.398+0000 7f3c69623640 1 -- start start 2023-11-24T08:05:01.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.400+0000 7f3c69623640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c6406d520 0x7f3c6406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:01.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.400+0000 7f3c69623640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3c640acb50 0x7f3c640719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:01.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.400+0000 7f3c69623640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c64070dc0 con 0x7f3c64165430 2023-11-24T08:05:01.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.400+0000 7f3c69623640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c64161d00 con 0x7f3c640acb50 2023-11-24T08:05:01.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.400+0000 7f3c69623640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6406de40 con 0x7f3c6406d520 2023-11-24T08:05:01.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.401+0000 7f3c5bfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3c640acb50 0x7f3c640719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:01.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.401+0000 7f3c62ffd640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c6406d520 0x7f3c6406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:01.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.401+0000 7f3c637fe640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f3c64165430 0x7f3c641658e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:46842/0 (socket says 172.21.15.64:46842) 2023-11-24T08:05:01.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.401+0000 7f3c637fe640 1 -- 172.21.15.64:0/248535621 learned_addr learned my addr 172.21.15.64:0/248535621 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:01.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.401+0000 7f3c62ffd640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c6406d520 0x7f3c6406d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:57202/0 (socket says 172.21.15.64:57202) 2023-11-24T08:05:01.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.401+0000 7f3c627fc640 1 -- 172.21.15.64:0/248535621 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702161824 0 0) 0x7f3c64070dc0 con 0x7f3c64165430 2023-11-24T08:05:01.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.402+0000 7f3c627fc640 1 -- 172.21.15.64:0/248535621 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c50003520 con 0x7f3c64165430 2023-11-24T08:05:01.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.402+0000 7f3c62ffd640 1 -- 172.21.15.64:0/248535621 >> v2:172.21.15.91:3300/0 conn(0x7f3c640acb50 msgr2=0x7f3c640719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:01.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.402+0000 7f3c62ffd640 1 --2- 172.21.15.64:0/248535621 >> v2:172.21.15.91:3300/0 conn(0x7f3c640acb50 0x7f3c640719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.402+0000 7f3c62ffd640 1 -- 172.21.15.64:0/248535621 >> v1:172.21.15.91:6789/0 conn(0x7f3c64165430 legacy=0x7f3c641658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:01.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.402+0000 7f3c62ffd640 1 -- 172.21.15.64:0/248535621 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c640cedb0 con 0x7f3c6406d520 2023-11-24T08:05:01.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.403+0000 7f3c62ffd640 1 --2- 172.21.15.64:0/248535621 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c6406d520 0x7f3c6406d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c540097a0 tx=0x7f3c5402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8fb1ab2574187f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:01.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.403+0000 7f3c627fc640 1 -- 172.21.15.64:0/248535621 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5402d040 con 0x7f3c6406d520 2023-11-24T08:05:01.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.403+0000 7f3c627fc640 1 -- 172.21.15.64:0/248535621 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c54031810 con 0x7f3c6406d520 2023-11-24T08:05:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.403+0000 7f3c627fc640 1 -- 172.21.15.64:0/248535621 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54031ba0 con 0x7f3c6406d520 2023-11-24T08:05:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.404+0000 7f3c69623640 1 -- 172.21.15.64:0/248535621 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c6406d520 msgr2=0x7f3c6406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.404+0000 7f3c69623640 1 --2- 172.21.15.64:0/248535621 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c6406d520 0x7f3c6406d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c540097a0 tx=0x7f3c5402b8b0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.404+0000 7f3c69623640 1 -- 172.21.15.64:0/248535621 shutdown_connections 2023-11-24T08:05:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.404+0000 7f3c69623640 1 --2- 172.21.15.64:0/248535621 >> v2:172.21.15.91:3300/0 conn(0x7f3c640acb50 0x7f3c640719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.404+0000 7f3c69623640 1 --2- 172.21.15.64:0/248535621 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c6406d520 0x7f3c6406d900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.404+0000 7f3c69623640 1 -- 172.21.15.64:0/248535621 >> 172.21.15.64:0/248535621 conn(0x7f3c640b0d80 msgr2=0x7f3c640b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:01.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.404+0000 7f3c69623640 1 -- 172.21.15.64:0/248535621 shutdown_connections 2023-11-24T08:05:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.404+0000 7f3c69623640 1 -- 172.21.15.64:0/248535621 wait complete. 2023-11-24T08:05:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.406+0000 7f3c69623640 1 Processor -- start 2023-11-24T08:05:01.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.406+0000 7f3c69623640 1 -- start start 2023-11-24T08:05:01.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c69623640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c6406d520 0x7f3c6409a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c69623640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c640acb50 0x7f3c6409b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c69623640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c64165430 0x7f3c6409bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c69623640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6415e760 con 0x7f3c640acb50 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c69623640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6409abe0 con 0x7f3c6406d520 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c69623640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6415db50 con 0x7f3c64165430 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c640acb50 0x7f3c6409b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c640acb50 0x7f3c6409b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53476/0 (socket says 172.21.15.64:53476) 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.407+0000 7f3c5bfff640 1 -- 172.21.15.64:0/487569727 learned_addr learned my addr 172.21.15.64:0/487569727 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c62ffd640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c6406d520 0x7f3c6409a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:01.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c637fe640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c64165430 0x7f3c6409bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c5bfff640 1 -- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c64165430 msgr2=0x7f3c6409bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c5bfff640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c64165430 0x7f3c6409bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c5bfff640 1 -- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c6406d520 msgr2=0x7f3c6409a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c5bfff640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c6406d520 0x7f3c6409a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c5bfff640 1 -- 172.21.15.64:0/487569727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54009400 con 0x7f3c640acb50 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c62ffd640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c6406d520 0x7f3c6409a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c637fe640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c64165430 0x7f3c6409bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c5bfff640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c640acb50 0x7f3c6409b800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00e930 tx=0x7f3c4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.408+0000 7f3c5b7fe640 1 -- 172.21.15.64:0/487569727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c009690 con 0x7f3c640acb50 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.409+0000 7f3c5b7fe640 1 -- 172.21.15.64:0/487569727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c00ca00 con 0x7f3c640acb50 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.409+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6415de30 con 0x7f3c640acb50 2023-11-24T08:05:01.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.409+0000 7f3c5b7fe640 1 -- 172.21.15.64:0/487569727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00cdb0 con 0x7f3c640acb50 2023-11-24T08:05:01.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.409+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6415e3f0 con 0x7f3c640acb50 2023-11-24T08:05:01.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.410+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c64074770 con 0x7f3c640acb50 2023-11-24T08:05:01.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.411+0000 7f3c5b7fe640 1 -- 172.21.15.64:0/487569727 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c4c013470 con 0x7f3c640acb50 2023-11-24T08:05:01.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.412+0000 7f3c5b7fe640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:01.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.412+0000 7f3c5b7fe640 1 -- 172.21.15.64:0/487569727 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3c4c089980 con 0x7f3c640acb50 2023-11-24T08:05:01.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.412+0000 7f3c62ffd640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:01.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.414+0000 7f3c62ffd640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c4406ae40 0x7f3c4406d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c54036000 tx=0x7f3c5403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:01.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.417+0000 7f3c5b7fe640 1 -- 172.21.15.64:0/487569727 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c4c054a60 con 0x7f3c640acb50 2023-11-24T08:05:01.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.626+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3c6406e320 con 0x7f3c640acb50 2023-11-24T08:05:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.627+0000 7f3c5b7fe640 1 -- 172.21.15.64:0/487569727 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3c4c017080 con 0x7f3c640acb50 2023-11-24T08:05:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:05:01.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c4406ae40 msgr2=0x7f3c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:01.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c4406ae40 0x7f3c4406d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c54036000 tx=0x7f3c5403c040 comp rx=0 tx=0).stop 2023-11-24T08:05:01.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c640acb50 msgr2=0x7f3c6409b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c640acb50 0x7f3c6409b800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00e930 tx=0x7f3c4c00edf0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 shutdown_connections 2023-11-24T08:05:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c64165430 0x7f3c6409bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c640acb50 0x7f3c6409b800 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 --2- 172.21.15.64:0/487569727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c6406d520 0x7f3c6409a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 >> 172.21.15.64:0/487569727 conn(0x7f3c640b0d80 msgr2=0x7f3c6415fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 shutdown_connections 2023-11-24T08:05:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:01.630+0000 7f3c69623640 1 -- 172.21.15.64:0/487569727 wait complete. 2023-11-24T08:05:01.651 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi091.front.sepia.ceph.com... 2023-11-24T08:05:01.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:01.652 DEBUG:teuthology.orchestra.run.smithi091:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-24T08:05:01.759 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:01.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:05:01.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:01.760 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T08:05:01.809 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:05:01.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.042+0000 7f65f5e1c640 1 Processor -- start 2023-11-24T08:05:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.042+0000 7f65f5e1c640 1 -- start start 2023-11-24T08:05:02.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.043+0000 7f65f5e1c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f65f00d17d0 0x7f65f015c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65f5e1c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f00cdf70 0x7f65f00b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65f5e1c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00cf410 con 0x7f65f00d1bb0 2023-11-24T08:05:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65f5e1c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f00ce7f0 con 0x7f65f00d17d0 2023-11-24T08:05:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65f5e1c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015c6c0 con 0x7f65f00cdf70 2023-11-24T08:05:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65eeffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f65f00d1bb0 0x7f65f00cf8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58378/0 (socket says 172.21.15.64:58378) 2023-11-24T08:05:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65eeffd640 1 -- 172.21.15.64:0/3223578285 learned_addr learned my addr 172.21.15.64:0/3223578285 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65ef7fe640 1 --2- 172.21.15.64:0/3223578285 >> v2:172.21.15.91:3300/0 conn(0x7f65f00d17d0 0x7f65f015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65effff640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f00cdf70 0x7f65f00b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.044+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870876636 0 0) 0x7f65f00cf410 con 0x7f65f00d1bb0 2023-11-24T08:05:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65dc003520 con 0x7f65f00d1bb0 2023-11-24T08:05:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342041906 0 0) 0x7f65dc003520 con 0x7f65f00d1bb0 2023-11-24T08:05:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f00cdf70 msgr2=0x7f65f00b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f00cdf70 0x7f65f00b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 >> v2:172.21.15.91:3300/0 conn(0x7f65f00d17d0 msgr2=0x7f65f015c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 --2- 172.21.15.64:0/3223578285 >> v2:172.21.15.91:3300/0 conn(0x7f65f00d17d0 0x7f65f015c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00ce510 con 0x7f65f00d1bb0 2023-11-24T08:05:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f65d8001670 con 0x7f65f00d1bb0 2023-11-24T08:05:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 >> v1:172.21.15.64:6789/0 conn(0x7f65f00d1bb0 legacy=0x7f65f00cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f00cdf70 0x7f65dc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65dc003ef0 0x7f65dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015c6c0 con 0x7f65dc004d00 2023-11-24T08:05:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00b980 con 0x7f65dc003ef0 2023-11-24T08:05:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00bac0 con 0x7f65f00cdf70 2023-11-24T08:05:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.045+0000 7f65effff640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.046+0000 7f65eeffd640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65dc003ef0 0x7f65dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.046+0000 7f65ef7fe640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f00cdf70 0x7f65dc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.046+0000 7f65effff640 1 -- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f00cdf70 msgr2=0x7f65dc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.046+0000 7f65effff640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f00cdf70 0x7f65dc003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.046+0000 7f65effff640 1 -- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65dc003ef0 msgr2=0x7f65dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.046+0000 7f65effff640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65dc003ef0 0x7f65dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.046+0000 7f65effff640 1 -- 172.21.15.64:0/3223578285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc003520 con 0x7f65dc004d00 2023-11-24T08:05:02.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.046+0000 7f65eeffd640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65dc003ef0 0x7f65dc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:02.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.047+0000 7f65effff640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65dc005180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009ac0 tx=0x7f65dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd0a42976542690 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:02.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.047+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e802c6b0 con 0x7f65dc004d00 2023-11-24T08:05:02.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.047+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e802c810 con 0x7f65dc004d00 2023-11-24T08:05:02.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.047+0000 7f65ee7fc640 1 -- 172.21.15.64:0/3223578285 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e802cba0 con 0x7f65dc004d00 2023-11-24T08:05:02.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.048+0000 7f65f5e1c640 1 -- 172.21.15.64:0/3223578285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 msgr2=0x7f65dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.048+0000 7f65f5e1c640 1 --2- 172.21.15.64:0/3223578285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65dc005180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009ac0 tx=0x7f65dc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.048+0000 7f65effff640 1 -- 172.21.15.64:0/3223578285 reap_dead start 2023-11-24T08:05:02.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.048+0000 7f65f5e1c640 1 -- 172.21.15.64:0/3223578285 shutdown_connections 2023-11-24T08:05:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.048+0000 7f65f5e1c640 1 -- 172.21.15.64:0/3223578285 >> 172.21.15.64:0/3223578285 conn(0x7f65f00661f0 msgr2=0x7f65f0066600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.048+0000 7f65f5e1c640 1 -- 172.21.15.64:0/3223578285 shutdown_connections 2023-11-24T08:05:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.048+0000 7f65f5e1c640 1 -- 172.21.15.64:0/3223578285 wait complete. 2023-11-24T08:05:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.050+0000 7f65f5e1c640 1 Processor -- start 2023-11-24T08:05:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.050+0000 7f65f5e1c640 1 -- start start 2023-11-24T08:05:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65f5e1c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65f00b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65f5e1c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f01511c0 0x7f65f00b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65f5e1c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65f015a1d0 0x7f65f015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65f5e1c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015ab70 con 0x7f65dc004d00 2023-11-24T08:05:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65f5e1c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f006bf20 con 0x7f65f015a1d0 2023-11-24T08:05:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65f5e1c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f00b4eb0 con 0x7f65f01511c0 2023-11-24T08:05:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65f00b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65f00b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53504/0 (socket says 172.21.15.64:53504) 2023-11-24T08:05:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.051+0000 7f65ef7fe640 1 -- 172.21.15.64:0/752133268 learned_addr learned my addr 172.21.15.64:0/752133268 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65eeffd640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f01511c0 0x7f65f00b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65effff640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65f015a1d0 0x7f65f015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65ef7fe640 1 -- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f01511c0 msgr2=0x7f65f00b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65ef7fe640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f01511c0 0x7f65f00b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65ef7fe640 1 -- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65f015a1d0 msgr2=0x7f65f015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65ef7fe640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65f015a1d0 0x7f65f015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65ef7fe640 1 -- 172.21.15.64:0/752133268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f006b5f0 con 0x7f65dc004d00 2023-11-24T08:05:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65effff640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65f015a1d0 0x7f65f015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65ef7fe640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65f00b5690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f65e000f3d0 tx=0x7f65e000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.052+0000 7f65e7fff640 1 -- 172.21.15.64:0/752133268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0016620 con 0x7f65dc004d00 2023-11-24T08:05:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.053+0000 7f65e7fff640 1 -- 172.21.15.64:0/752133268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e00167a0 con 0x7f65dc004d00 2023-11-24T08:05:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.053+0000 7f65e7fff640 1 -- 172.21.15.64:0/752133268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0016b50 con 0x7f65dc004d00 2023-11-24T08:05:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.053+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f006b8d0 con 0x7f65dc004d00 2023-11-24T08:05:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.053+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f00b4af0 con 0x7f65dc004d00 2023-11-24T08:05:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.054+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f0074770 con 0x7f65dc004d00 2023-11-24T08:05:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.055+0000 7f65e7fff640 1 -- 172.21.15.64:0/752133268 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f65e0016cb0 con 0x7f65dc004d00 2023-11-24T08:05:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.056+0000 7f65e7fff640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f65d006ae40 0x7f65d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.056+0000 7f65e7fff640 1 -- 172.21.15.64:0/752133268 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f65e008a610 con 0x7f65dc004d00 2023-11-24T08:05:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.057+0000 7f65eeffd640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f65d006ae40 0x7f65d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.058+0000 7f65eeffd640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f65d006ae40 0x7f65d006d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65d80017a0 tx=0x7f65d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.061+0000 7f65e7fff640 1 -- 172.21.15.64:0/752133268 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65e00556c0 con 0x7f65dc004d00 2023-11-24T08:05:02.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.299+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65f00b4c80 con 0x7f65dc004d00 2023-11-24T08:05:02.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.301+0000 7f65e7fff640 1 -- 172.21.15.64:0/752133268 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f65e0058ce0 con 0x7f65dc004d00 2023-11-24T08:05:02.303 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:02.303 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:03:55.699656+0000","modified":"2023-11-24T08:03:58.759467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:05:02.303 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T08:05:02.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f65d006ae40 msgr2=0x7f65d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f65d006ae40 0x7f65d006d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65d80017a0 tx=0x7f65d8009040 comp rx=0 tx=0).stop 2023-11-24T08:05:02.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 msgr2=0x7f65f00b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65f00b5690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f65e000f3d0 tx=0x7f65e000f890 comp rx=0 tx=0).stop 2023-11-24T08:05:02.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 shutdown_connections 2023-11-24T08:05:02.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f65d006ae40 0x7f65d006d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f65f015a1d0 0x7f65f015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65f01511c0 0x7f65f00b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 --2- 172.21.15.64:0/752133268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65dc004d00 0x7f65f00b5690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 >> 172.21.15.64:0/752133268 conn(0x7f65f00661f0 msgr2=0x7f65f015f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:02.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 shutdown_connections 2023-11-24T08:05:02.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.304+0000 7f65f5e1c640 1 -- 172.21.15.64:0/752133268 wait complete. 2023-11-24T08:05:02.327 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:02.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.552+0000 7fdafd321640 1 Processor -- start 2023-11-24T08:05:02.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.552+0000 7fdafd321640 1 -- start start 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.553+0000 7fdafd321640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdaf815d5b0 0x7fdaf80d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.553+0000 7fdafd321640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf80d14a0 0x7fdaf80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.553+0000 7fdafd321640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdaf80cf7a0 0x7fdaf80cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.553+0000 7fdafd321640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf80cf0c0 con 0x7fdaf815d5b0 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdafd321640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf80d1e40 con 0x7fdaf80cf7a0 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdafd321640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80b01b0 con 0x7fdaf80d14a0 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdaf7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdaf815d5b0 0x7fdaf80d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdaf7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdaf815d5b0 0x7fdaf80d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53510/0 (socket says 172.21.15.64:53510) 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdaf7fff640 1 -- 172.21.15.64:0/3192962189 learned_addr learned my addr 172.21.15.64:0/3192962189 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdaf7fff640 1 -- 172.21.15.64:0/3192962189 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf80d14a0 msgr2=0x7fdaf80d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:02.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdaf7fff640 1 --2- 172.21.15.64:0/3192962189 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf80d14a0 0x7fdaf80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdaf7fff640 1 -- 172.21.15.64:0/3192962189 >> v2:172.21.15.91:3300/0 conn(0x7fdaf80cf7a0 msgr2=0x7fdaf80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:02.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdaf7fff640 1 --2- 172.21.15.64:0/3192962189 >> v2:172.21.15.91:3300/0 conn(0x7fdaf80cf7a0 0x7fdaf80cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.554+0000 7fdaf7fff640 1 -- 172.21.15.64:0/3192962189 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf815d990 con 0x7fdaf815d5b0 2023-11-24T08:05:02.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.555+0000 7fdaf7fff640 1 --2- 172.21.15.64:0/3192962189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf815d5b0 0x7fdaf80d3990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80d2d60 tx=0x7fdae802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf59531ca588578 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:02.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdaf77fe640 1 -- 172.21.15.64:0/3192962189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8031640 con 0x7fdaf815d5b0 2023-11-24T08:05:02.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdaf77fe640 1 -- 172.21.15.64:0/3192962189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae80317a0 con 0x7fdaf815d5b0 2023-11-24T08:05:02.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdaf77fe640 1 -- 172.21.15.64:0/3192962189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8031b30 con 0x7fdaf815d5b0 2023-11-24T08:05:02.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 -- 172.21.15.64:0/3192962189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf815d5b0 msgr2=0x7fdaf80d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 --2- 172.21.15.64:0/3192962189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf815d5b0 0x7fdaf80d3990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80d2d60 tx=0x7fdae802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 -- 172.21.15.64:0/3192962189 shutdown_connections 2023-11-24T08:05:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 --2- 172.21.15.64:0/3192962189 >> v2:172.21.15.91:3300/0 conn(0x7fdaf80cf7a0 0x7fdaf80cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 --2- 172.21.15.64:0/3192962189 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf80d14a0 0x7fdaf80d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 --2- 172.21.15.64:0/3192962189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf815d5b0 0x7fdaf80d3990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 -- 172.21.15.64:0/3192962189 >> 172.21.15.64:0/3192962189 conn(0x7fdaf809c3b0 msgr2=0x7fdaf80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 -- 172.21.15.64:0/3192962189 shutdown_connections 2023-11-24T08:05:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.556+0000 7fdafd321640 1 -- 172.21.15.64:0/3192962189 wait complete. 2023-11-24T08:05:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.558+0000 7fdafd321640 1 Processor -- start 2023-11-24T08:05:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.558+0000 7fdafd321640 1 -- start start 2023-11-24T08:05:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.559+0000 7fdafd321640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaf80cf7a0 0x7fdaf815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.559+0000 7fdafd321640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf80d14a0 0x7fdaf80cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.559+0000 7fdafd321640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf815d5b0 0x7fdaf80cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.559+0000 7fdafd321640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80c9a40 con 0x7fdaf80d14a0 2023-11-24T08:05:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.559+0000 7fdafd321640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80c8dc0 con 0x7fdaf80cf7a0 2023-11-24T08:05:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.559+0000 7fdafd321640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80c90a0 con 0x7fdaf815d5b0 2023-11-24T08:05:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaeffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf80d14a0 0x7fdaf80cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaeffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf80d14a0 0x7fdaf80cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53514/0 (socket says 172.21.15.64:53514) 2023-11-24T08:05:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaeffff640 1 -- 172.21.15.64:0/1577547756 learned_addr learned my addr 172.21.15.64:0/1577547756 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaf7fff640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaf80cf7a0 0x7fdaf815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdafcb20640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf815d5b0 0x7fdaf80cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaeffff640 1 -- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf815d5b0 msgr2=0x7fdaf80cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaeffff640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf815d5b0 0x7fdaf80cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaeffff640 1 -- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaf80cf7a0 msgr2=0x7fdaf815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaeffff640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaf80cf7a0 0x7fdaf815a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.560+0000 7fdaeffff640 1 -- 172.21.15.64:0/1577547756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae80094d0 con 0x7fdaf80d14a0 2023-11-24T08:05:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.561+0000 7fdaeffff640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf80d14a0 0x7fdaf80cc310 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdae000b4a0 tx=0x7fdae000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.561+0000 7fdaf57fa640 1 -- 172.21.15.64:0/1577547756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0010610 con 0x7fdaf80d14a0 2023-11-24T08:05:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.561+0000 7fdaf57fa640 1 -- 172.21.15.64:0/1577547756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0010770 con 0x7fdaf80d14a0 2023-11-24T08:05:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.561+0000 7fdaf57fa640 1 -- 172.21.15.64:0/1577547756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0010b20 con 0x7fdaf80d14a0 2023-11-24T08:05:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.561+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf80c9380 con 0x7fdaf80d14a0 2023-11-24T08:05:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.561+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf815ad50 con 0x7fdaf80d14a0 2023-11-24T08:05:02.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.562+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf80b2330 con 0x7fdaf80d14a0 2023-11-24T08:05:02.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.563+0000 7fdaf57fa640 1 -- 172.21.15.64:0/1577547756 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdae0010c80 con 0x7fdaf80d14a0 2023-11-24T08:05:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.564+0000 7fdaf57fa640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae40 0x7fdad806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.564+0000 7fdaf57fa640 1 -- 172.21.15.64:0/1577547756 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdae0089320 con 0x7fdaf80d14a0 2023-11-24T08:05:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.565+0000 7fdaf7fff640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae40 0x7fdad806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.566+0000 7fdaf7fff640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae40 0x7fdad806d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdae8036000 tx=0x7fdae803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.569+0000 7fdaf57fa640 1 -- 172.21.15.64:0/1577547756 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdae0054330 con 0x7fdaf80d14a0 2023-11-24T08:05:02.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.767+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaf80c9b80 con 0x7fdaf80d14a0 2023-11-24T08:05:02.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.770+0000 7fdaf57fa640 1 -- 172.21.15.64:0/1577547756 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7fdae0057950 con 0x7fdaf80d14a0 2023-11-24T08:05:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:02.772 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae40 msgr2=0x7fdad806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae40 0x7fdad806d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdae8036000 tx=0x7fdae803c040 comp rx=0 tx=0).stop 2023-11-24T08:05:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf80d14a0 msgr2=0x7fdaf80cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf80d14a0 0x7fdaf80cc310 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdae000b4a0 tx=0x7fdae000b960 comp rx=0 tx=0).stop 2023-11-24T08:05:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 shutdown_connections 2023-11-24T08:05:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae40 0x7fdad806d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaf815d5b0 0x7fdaf80cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf80d14a0 0x7fdaf80cc310 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 --2- 172.21.15.64:0/1577547756 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaf80cf7a0 0x7fdaf815a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 >> 172.21.15.64:0/1577547756 conn(0x7fdaf809c3b0 msgr2=0x7fdaf815eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 shutdown_connections 2023-11-24T08:05:02.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:02.773+0000 7fdafd321640 1 -- 172.21.15.64:0/1577547756 wait complete. 2023-11-24T08:05:02.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:03:53.626355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:03:54.726508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:05:02.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:05:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.012+0000 7fac655dc640 1 Processor -- start 2023-11-24T08:05:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.012+0000 7fac655dc640 1 -- start start 2023-11-24T08:05:03.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.013+0000 7fac655dc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fac600acd70 0x7fac6006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.013+0000 7fac655dc640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fac60070490 0x7fac60070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.013+0000 7fac655dc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6006d7a0 con 0x7fac600ae490 2023-11-24T08:05:03.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac655dc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6015a5b0 con 0x7fac600acd70 2023-11-24T08:05:03.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac655dc640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6006ce70 con 0x7fac60070490 2023-11-24T08:05:03.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fac600acd70 0x7fac6006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5e7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fac600ae490 0x7fac600ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58386/0 (socket says 172.21.15.64:58386) 2023-11-24T08:05:03.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5e7fc640 1 -- 172.21.15.64:0/2973313170 learned_addr learned my addr 172.21.15.64:0/2973313170 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:03.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5f7fe640 1 --2- 172.21.15.64:0/2973313170 >> v2:172.21.15.91:3300/0 conn(0x7fac60070490 0x7fac60070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5effd640 1 -- 172.21.15.64:0/2973313170 >> v2:172.21.15.91:3300/0 conn(0x7fac60070490 msgr2=0x7fac60070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:03.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5effd640 1 --2- 172.21.15.64:0/2973313170 >> v2:172.21.15.91:3300/0 conn(0x7fac60070490 0x7fac60070900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5effd640 1 -- 172.21.15.64:0/2973313170 >> v1:172.21.15.64:6789/0 conn(0x7fac600ae490 legacy=0x7fac600ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5effd640 1 -- 172.21.15.64:0/2973313170 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6006d150 con 0x7fac600acd70 2023-11-24T08:05:03.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5f7fe640 1 --2- 172.21.15.64:0/2973313170 >> v2:172.21.15.91:3300/0 conn(0x7fac60070490 0x7fac60070900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:03.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.014+0000 7fac5dffb640 1 -- 172.21.15.64:0/2973313170 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388679899 0 0) 0x7fac6006d7a0 con 0x7fac600ae490 2023-11-24T08:05:03.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.016+0000 7fac5effd640 1 --2- 172.21.15.64:0/2973313170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6006dc30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fac60067e70 tx=0x7fac5002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fb27fec013200a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.016+0000 7fac5dffb640 1 -- 172.21.15.64:0/2973313170 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5002cc70 con 0x7fac600acd70 2023-11-24T08:05:03.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.016+0000 7fac5dffb640 1 -- 172.21.15.64:0/2973313170 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5002cdd0 con 0x7fac600acd70 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.016+0000 7fac5dffb640 1 -- 172.21.15.64:0/2973313170 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50034660 con 0x7fac600acd70 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.016+0000 7fac655dc640 1 -- 172.21.15.64:0/2973313170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 msgr2=0x7fac6006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.016+0000 7fac655dc640 1 --2- 172.21.15.64:0/2973313170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6006dc30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fac60067e70 tx=0x7fac5002bec0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.017+0000 7fac655dc640 1 -- 172.21.15.64:0/2973313170 shutdown_connections 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.017+0000 7fac655dc640 1 --2- 172.21.15.64:0/2973313170 >> v2:172.21.15.91:3300/0 conn(0x7fac60070490 0x7fac60070900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.017+0000 7fac655dc640 1 --2- 172.21.15.64:0/2973313170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6006dc30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.017+0000 7fac655dc640 1 -- 172.21.15.64:0/2973313170 >> 172.21.15.64:0/2973313170 conn(0x7fac600b0ba0 msgr2=0x7fac600b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.017+0000 7fac655dc640 1 -- 172.21.15.64:0/2973313170 shutdown_connections 2023-11-24T08:05:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.017+0000 7fac655dc640 1 -- 172.21.15.64:0/2973313170 wait complete. 2023-11-24T08:05:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.018+0000 7fac655dc640 1 Processor -- start 2023-11-24T08:05:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.019+0000 7fac655dc640 1 -- start start 2023-11-24T08:05:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.019+0000 7fac655dc640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac60070490 0x7fac6009ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.019+0000 7fac655dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6014df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.019+0000 7fac655dc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac600ae490 0x7fac600cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.019+0000 7fac655dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6009b240 con 0x7fac600acd70 2023-11-24T08:05:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.019+0000 7fac655dc640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6009c8e0 con 0x7fac60070490 2023-11-24T08:05:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.019+0000 7fac655dc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6009bfb0 con 0x7fac600ae490 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6014df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6014df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53534/0 (socket says 172.21.15.64:53534) 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5e7fc640 1 -- 172.21.15.64:0/581868589 learned_addr learned my addr 172.21.15.64:0/581868589 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5effd640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac60070490 0x7fac6009ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5f7fe640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac600ae490 0x7fac600cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5e7fc640 1 -- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac600ae490 msgr2=0x7fac600cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5e7fc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac600ae490 0x7fac600cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5e7fc640 1 -- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac60070490 msgr2=0x7fac6009ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5e7fc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac60070490 0x7fac6009ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5e7fc640 1 -- 172.21.15.64:0/581868589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac50009a70 con 0x7fac600acd70 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.020+0000 7fac5f7fe640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac600ae490 0x7fac600cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:03.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.021+0000 7fac5e7fc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6014df50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c500 tx=0x7fac5400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.021+0000 7fac477fe640 1 -- 172.21.15.64:0/581868589 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54007560 con 0x7fac600acd70 2023-11-24T08:05:03.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.021+0000 7fac477fe640 1 -- 172.21.15.64:0/581868589 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac540047e0 con 0x7fac600acd70 2023-11-24T08:05:03.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.021+0000 7fac477fe640 1 -- 172.21.15.64:0/581868589 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54004b90 con 0x7fac600acd70 2023-11-24T08:05:03.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.021+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6009c290 con 0x7fac600acd70 2023-11-24T08:05:03.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.021+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6014e490 con 0x7fac600acd70 2023-11-24T08:05:03.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.022+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6009d200 con 0x7fac600acd70 2023-11-24T08:05:03.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.023+0000 7fac477fe640 1 -- 172.21.15.64:0/581868589 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac54004cf0 con 0x7fac600acd70 2023-11-24T08:05:03.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.024+0000 7fac477fe640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac4c06ae40 0x7fac4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.024+0000 7fac477fe640 1 -- 172.21.15.64:0/581868589 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fac54088870 con 0x7fac600acd70 2023-11-24T08:05:03.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.025+0000 7fac5effd640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac4c06ae40 0x7fac4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.026+0000 7fac5effd640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac4c06ae40 0x7fac4c06d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fac50036000 tx=0x7fac50030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.029+0000 7fac477fe640 1 -- 172.21.15.64:0/581868589 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac54053880 con 0x7fac600acd70 2023-11-24T08:05:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.223+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac6004e920 con 0x7fac600acd70 2023-11-24T08:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.224+0000 7fac477fe640 1 -- 172.21.15.64:0/581868589 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac540180a0 con 0x7fac600acd70 2023-11-24T08:05:03.227 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:05:03.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac4c06ae40 msgr2=0x7fac4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac4c06ae40 0x7fac4c06d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fac50036000 tx=0x7fac50030000 comp rx=0 tx=0).stop 2023-11-24T08:05:03.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 msgr2=0x7fac6014df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6014df50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c500 tx=0x7fac5400c9c0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 shutdown_connections 2023-11-24T08:05:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac4c06ae40 0x7fac4c06d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac600ae490 0x7fac600cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac600acd70 0x7fac6014df50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 --2- 172.21.15.64:0/581868589 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac60070490 0x7fac6009ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 >> 172.21.15.64:0/581868589 conn(0x7fac600b0ba0 msgr2=0x7fac600b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:03.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.226+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 shutdown_connections 2023-11-24T08:05:03.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.227+0000 7fac655dc640 1 -- 172.21.15.64:0/581868589 wait complete. 2023-11-24T08:05:03.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:05:03.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.475+0000 7f8ff88bb640 1 Processor -- start 2023-11-24T08:05:03.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.475+0000 7f8ff88bb640 1 -- start start 2023-11-24T08:05:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.476+0000 7f8ff88bb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ff415d3b0 0x7f8ff40ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.477+0000 7f8ff88bb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ff40d1d90 0x7f8ff40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.477+0000 7f8ff88bb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff40c6bf0 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.477+0000 7f8ff88bb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff40d1430 con 0x7f8ff40d1d90 2023-11-24T08:05:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.477+0000 7f8ff88bb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff40cedc0 con 0x7f8ff415d3b0 2023-11-24T08:05:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.477+0000 7f8ff2ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8ff40d0fa0 0x7f8ff40d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58392/0 (socket says 172.21.15.64:58392) 2023-11-24T08:05:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.477+0000 7f8ff2ffd640 1 -- 172.21.15.64:0/468919169 learned_addr learned my addr 172.21.15.64:0/468919169 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.477+0000 7f8ff3fff640 1 --2- 172.21.15.64:0/468919169 >> v2:172.21.15.64:3300/0 conn(0x7f8ff40d1d90 0x7f8ff40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.477+0000 7f8ff37fe640 1 --2- 172.21.15.64:0/468919169 >> v2:172.21.15.91:3300/0 conn(0x7f8ff415d3b0 0x7f8ff40ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.478+0000 7f8ff3fff640 1 -- 172.21.15.64:0/468919169 >> v2:172.21.15.91:3300/0 conn(0x7f8ff415d3b0 msgr2=0x7f8ff40ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.478+0000 7f8ff3fff640 1 --2- 172.21.15.64:0/468919169 >> v2:172.21.15.91:3300/0 conn(0x7f8ff415d3b0 0x7f8ff40ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.478+0000 7f8ff3fff640 1 -- 172.21.15.64:0/468919169 >> v1:172.21.15.64:6789/0 conn(0x7f8ff40d0fa0 legacy=0x7f8ff40d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.478+0000 7f8ff3fff640 1 -- 172.21.15.64:0/468919169 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40c6e40 con 0x7f8ff40d1d90 2023-11-24T08:05:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.478+0000 7f8ff27fc640 1 -- 172.21.15.64:0/468919169 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983730753 0 0) 0x7f8ff40c6bf0 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.479+0000 7f8ff3fff640 1 --2- 172.21.15.64:0/468919169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d1d90 0x7f8ff40b3ca0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ff408b2a0 tx=0x7f8fec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b89a7d9c6e4f91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.479+0000 7f8ff27fc640 1 -- 172.21.15.64:0/468919169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec02e070 con 0x7f8ff40d1d90 2023-11-24T08:05:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.479+0000 7f8ff27fc640 1 -- 172.21.15.64:0/468919169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec0315a0 con 0x7f8ff40d1d90 2023-11-24T08:05:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.479+0000 7f8ff27fc640 1 -- 172.21.15.64:0/468919169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec031930 con 0x7f8ff40d1d90 2023-11-24T08:05:03.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.480+0000 7f8ff88bb640 1 -- 172.21.15.64:0/468919169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d1d90 msgr2=0x7f8ff40b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.480+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/468919169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d1d90 0x7f8ff40b3ca0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ff408b2a0 tx=0x7f8fec02b840 comp rx=0 tx=0).stop 2023-11-24T08:05:03.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.480+0000 7f8ff88bb640 1 -- 172.21.15.64:0/468919169 shutdown_connections 2023-11-24T08:05:03.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.480+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/468919169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d1d90 0x7f8ff40b3ca0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.480+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/468919169 >> v2:172.21.15.91:3300/0 conn(0x7f8ff415d3b0 0x7f8ff40ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.480+0000 7f8ff88bb640 1 -- 172.21.15.64:0/468919169 >> 172.21.15.64:0/468919169 conn(0x7f8ff40d30a0 msgr2=0x7f8ff40d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:03.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.480+0000 7f8ff88bb640 1 -- 172.21.15.64:0/468919169 shutdown_connections 2023-11-24T08:05:03.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.480+0000 7f8ff88bb640 1 -- 172.21.15.64:0/468919169 wait complete. 2023-11-24T08:05:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.482+0000 7f8ff88bb640 1 Processor -- start 2023-11-24T08:05:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.482+0000 7f8ff88bb640 1 -- start start 2023-11-24T08:05:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff88bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d0fa0 0x7f8ff414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff88bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff40d1d90 0x7f8ff414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff88bb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff415d3b0 0x7f8ff40a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff88bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff414d7e0 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff88bb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff414e140 con 0x7f8ff415d3b0 2023-11-24T08:05:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff88bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff414cb60 con 0x7f8ff40d1d90 2023-11-24T08:05:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d0fa0 0x7f8ff414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d0fa0 0x7f8ff414dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53552/0 (socket says 172.21.15.64:53552) 2023-11-24T08:05:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff37fe640 1 -- 172.21.15.64:0/2667261252 learned_addr learned my addr 172.21.15.64:0/2667261252 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff37fe640 1 -- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff40d1d90 msgr2=0x7f8ff414c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff37fe640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff40d1d90 0x7f8ff414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff3fff640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff415d3b0 0x7f8ff40a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff37fe640 1 -- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff415d3b0 msgr2=0x7f8ff40a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.483+0000 7f8ff37fe640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff415d3b0 0x7f8ff40a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.484+0000 7f8ff37fe640 1 -- 172.21.15.64:0/2667261252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec009400 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.484+0000 7f8ff37fe640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d0fa0 0x7f8ff414dc00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4007e40 tx=0x7f8fe400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.484+0000 7f8febfff640 1 -- 172.21.15.64:0/2667261252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4014620 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.484+0000 7f8febfff640 1 -- 172.21.15.64:0/2667261252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400ee90 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.484+0000 7f8febfff640 1 -- 172.21.15.64:0/2667261252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40149c0 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.484+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff414ce40 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.484+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff414d400 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.485+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff414da50 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.486+0000 7f8febfff640 1 -- 172.21.15.64:0/2667261252 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fe4014b20 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.487+0000 7f8febfff640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8fd406ae40 0x7f8fd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.487+0000 7f8febfff640 1 -- 172.21.15.64:0/2667261252 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8fe408a070 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.488+0000 7f8ff2ffd640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8fd406ae40 0x7f8fd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.489+0000 7f8ff2ffd640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8fd406ae40 0x7f8fd406d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0011e0 tx=0x7f8fdc007940 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.492+0000 7f8febfff640 1 -- 172.21.15.64:0/2667261252 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8fe40bce90 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.686+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ff404e840 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.687+0000 7f8febfff640 1 -- 172.21.15.64:0/2667261252 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8fe4054ff0 con 0x7f8ff40d0fa0 2023-11-24T08:05:03.689 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:05:03.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.689+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8fd406ae40 msgr2=0x7f8fd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.689+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8fd406ae40 0x7f8fd406d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0011e0 tx=0x7f8fdc007940 comp rx=0 tx=0).stop 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.689+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d0fa0 msgr2=0x7f8ff414dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.689+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d0fa0 0x7f8ff414dc00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4007e40 tx=0x7f8fe400e4b0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.690+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 shutdown_connections 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.690+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8fd406ae40 0x7f8fd406d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.690+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ff415d3b0 0x7f8ff40a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.690+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ff40d1d90 0x7f8ff414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.690+0000 7f8ff88bb640 1 --2- 172.21.15.64:0/2667261252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff40d0fa0 0x7f8ff414dc00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.690+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 >> 172.21.15.64:0/2667261252 conn(0x7f8ff40d30a0 msgr2=0x7f8ff40c8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.690+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 shutdown_connections 2023-11-24T08:05:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.690+0000 7f8ff88bb640 1 -- 172.21.15.64:0/2667261252 wait complete. 2023-11-24T08:05:03.710 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:05:03.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.940+0000 7f8c8759e640 1 Processor -- start 2023-11-24T08:05:03.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.940+0000 7f8c8759e640 1 -- start start 2023-11-24T08:05:03.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.942+0000 7f8c8759e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8c880b4140 0x7f8c88071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.942+0000 7f8c8759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c881439b0 0x7f8c88070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.942+0000 7f8c8759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c88070d40 con 0x7f8c881435d0 2023-11-24T08:05:03.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.942+0000 7f8c8759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c88071020 con 0x7f8c881439b0 2023-11-24T08:05:03.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.942+0000 7f8c8759e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c88071300 con 0x7f8c880b4140 2023-11-24T08:05:03.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.942+0000 7f8c8659c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8c881435d0 0x7f8c880b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58394/0 (socket says 172.21.15.64:58394) 2023-11-24T08:05:03.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.942+0000 7f8c8659c640 1 -- 172.21.15.64:0/4154310323 learned_addr learned my addr 172.21.15.64:0/4154310323 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:03.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.942+0000 7f8c86d9d640 1 --2- 172.21.15.64:0/4154310323 >> v2:172.21.15.64:3300/0 conn(0x7f8c881439b0 0x7f8c88070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.943+0000 7f8c7dd9b640 1 --2- 172.21.15.64:0/4154310323 >> v2:172.21.15.91:3300/0 conn(0x7f8c880b4140 0x7f8c88071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.943+0000 7f8c86d9d640 1 -- 172.21.15.64:0/4154310323 >> v2:172.21.15.91:3300/0 conn(0x7f8c880b4140 msgr2=0x7f8c88071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.943+0000 7f8c86d9d640 1 --2- 172.21.15.64:0/4154310323 >> v2:172.21.15.91:3300/0 conn(0x7f8c880b4140 0x7f8c88071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.943+0000 7f8c86d9d640 1 -- 172.21.15.64:0/4154310323 >> v1:172.21.15.64:6789/0 conn(0x7f8c881435d0 legacy=0x7f8c880b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.943+0000 7f8c85d9b640 1 -- 172.21.15.64:0/4154310323 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807463642 0 0) 0x7f8c88070d40 con 0x7f8c881435d0 2023-11-24T08:05:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.943+0000 7f8c86d9d640 1 -- 172.21.15.64:0/4154310323 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880715e0 con 0x7f8c881439b0 2023-11-24T08:05:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.943+0000 7f8c7dd9b640 1 --2- 172.21.15.64:0/4154310323 >> v2:172.21.15.91:3300/0 conn(0x7f8c880b4140 0x7f8c88071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.944+0000 7f8c86d9d640 1 --2- 172.21.15.64:0/4154310323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881439b0 0x7f8c88070050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c880cbeb0 tx=0x7f8c8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea472d5e24f11b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.944+0000 7f8c85d9b640 1 -- 172.21.15.64:0/4154310323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8002e070 con 0x7f8c881439b0 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.944+0000 7f8c85d9b640 1 -- 172.21.15.64:0/4154310323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c800315a0 con 0x7f8c881439b0 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.944+0000 7f8c85d9b640 1 -- 172.21.15.64:0/4154310323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80031930 con 0x7f8c881439b0 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.945+0000 7f8c8759e640 1 -- 172.21.15.64:0/4154310323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881439b0 msgr2=0x7f8c88070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.945+0000 7f8c8759e640 1 --2- 172.21.15.64:0/4154310323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881439b0 0x7f8c88070050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c880cbeb0 tx=0x7f8c8002b840 comp rx=0 tx=0).stop 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.945+0000 7f8c8759e640 1 -- 172.21.15.64:0/4154310323 shutdown_connections 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.945+0000 7f8c8759e640 1 --2- 172.21.15.64:0/4154310323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881439b0 0x7f8c88070050 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.945+0000 7f8c8759e640 1 --2- 172.21.15.64:0/4154310323 >> v2:172.21.15.91:3300/0 conn(0x7f8c880b4140 0x7f8c88071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.945+0000 7f8c8759e640 1 -- 172.21.15.64:0/4154310323 >> 172.21.15.64:0/4154310323 conn(0x7f8c880cec10 msgr2=0x7f8c880cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.945+0000 7f8c8759e640 1 -- 172.21.15.64:0/4154310323 shutdown_connections 2023-11-24T08:05:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.945+0000 7f8c8759e640 1 -- 172.21.15.64:0/4154310323 wait complete. 2023-11-24T08:05:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.947+0000 7f8c8759e640 1 Processor -- start 2023-11-24T08:05:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.947+0000 7f8c8759e640 1 -- start start 2023-11-24T08:05:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c880b4140 0x7f8c880cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8759e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c881435d0 0x7f8c880cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c881439b0 0x7f8c88151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880ade30 con 0x7f8c880b4140 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8759e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c881f2e50 con 0x7f8c881435d0 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c881f3130 con 0x7f8c881439b0 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c880b4140 0x7f8c880cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c880b4140 0x7f8c880cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53568/0 (socket says 172.21.15.64:53568) 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.948+0000 7f8c8659c640 1 -- 172.21.15.64:0/2284332005 learned_addr learned my addr 172.21.15.64:0/2284332005 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c8659c640 1 -- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c881439b0 msgr2=0x7f8c88151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c7dd9b640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c881435d0 0x7f8c880cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c8659c640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c881439b0 0x7f8c88151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c8659c640 1 -- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c881435d0 msgr2=0x7f8c880cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c8659c640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c881435d0 0x7f8c880cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c8659c640 1 -- 172.21.15.64:0/2284332005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80009400 con 0x7f8c880b4140 2023-11-24T08:05:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c8659c640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c880b4140 0x7f8c880cd350 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c890 tx=0x7f8c7800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c7f7fe640 1 -- 172.21.15.64:0/2284332005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800a640 con 0x7f8c880b4140 2023-11-24T08:05:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c7f7fe640 1 -- 172.21.15.64:0/2284332005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800a7c0 con 0x7f8c880b4140 2023-11-24T08:05:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c7f7fe640 1 -- 172.21.15.64:0/2284332005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800ab70 con 0x7f8c880b4140 2023-11-24T08:05:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c881f3410 con 0x7f8c880b4140 2023-11-24T08:05:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.949+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c881f39d0 con 0x7f8c880b4140 2023-11-24T08:05:03.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.951+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8804e840 con 0x7f8c880b4140 2023-11-24T08:05:03.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.952+0000 7f8c7f7fe640 1 -- 172.21.15.64:0/2284332005 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c7800acd0 con 0x7f8c880b4140 2023-11-24T08:05:03.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.952+0000 7f8c7f7fe640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c6806ae70 0x7f8c6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:03.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.953+0000 7f8c7f7fe640 1 -- 172.21.15.64:0/2284332005 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8c78088c30 con 0x7f8c880b4140 2023-11-24T08:05:03.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.953+0000 7f8c7dd9b640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c6806ae70 0x7f8c6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:03.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.954+0000 7f8c7dd9b640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c6806ae70 0x7f8c6806d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c8815d020 tx=0x7f8c70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:03.957+0000 7f8c7f7fe640 1 -- 172.21.15.64:0/2284332005 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c78053bb0 con 0x7f8c880b4140 2023-11-24T08:05:04.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.151+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c8806d680 con 0x7f8c880b4140 2023-11-24T08:05:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.152+0000 7f8c7f7fe640 1 -- 172.21.15.64:0/2284332005 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c78018020 con 0x7f8c880b4140 2023-11-24T08:05:04.154 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c6806ae70 msgr2=0x7f8c6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c6806ae70 0x7f8c6806d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c8815d020 tx=0x7f8c70008040 comp rx=0 tx=0).stop 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c880b4140 msgr2=0x7f8c880cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c880b4140 0x7f8c880cd350 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c890 tx=0x7f8c7800cd50 comp rx=0 tx=0).stop 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 shutdown_connections 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c6806ae70 0x7f8c6806d330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c881439b0 0x7f8c88151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c881435d0 0x7f8c880cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 --2- 172.21.15.64:0/2284332005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c880b4140 0x7f8c880cd350 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 >> 172.21.15.64:0/2284332005 conn(0x7f8c880cec10 msgr2=0x7f8c881402c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 shutdown_connections 2023-11-24T08:05:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.155+0000 7f8c8759e640 1 -- 172.21.15.64:0/2284332005 wait complete. 2023-11-24T08:05:04.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:04.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.403+0000 7f7acc813640 1 Processor -- start 2023-11-24T08:05:04.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.403+0000 7f7acc813640 1 -- start start 2023-11-24T08:05:04.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.405+0000 7f7acc813640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7ac815d460 0x7f7ac80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.405+0000 7f7acc813640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 0x7f7ac80cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.405+0000 7f7acc813640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac80cbf60 con 0x7f7ac815d840 2023-11-24T08:05:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.405+0000 7f7acc813640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac80cc520 con 0x7f7ac815d460 2023-11-24T08:05:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.405+0000 7f7acc813640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80d03b0 con 0x7f7ac806be70 2023-11-24T08:05:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.405+0000 7f7ac77fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7ac815d460 0x7f7ac80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.405+0000 7f7ac7fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 0x7f7ac80cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.405+0000 7f7ac77fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7ac815d460 0x7f7ac80b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:60296/0 (socket says 172.21.15.64:60296) 2023-11-24T08:05:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.406+0000 7f7ac77fe640 1 -- 172.21.15.64:0/1691194793 learned_addr learned my addr 172.21.15.64:0/1691194793 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.406+0000 7f7ac77fe640 1 -- 172.21.15.64:0/1691194793 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 msgr2=0x7f7ac80cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.406+0000 7f7ac77fe640 1 --2- 172.21.15.64:0/1691194793 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 0x7f7ac80cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.406+0000 7f7ac77fe640 1 -- 172.21.15.64:0/1691194793 >> v1:172.21.15.91:6789/0 conn(0x7f7ac815d840 legacy=0x7f7ac806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.406+0000 7f7ac77fe640 1 -- 172.21.15.64:0/1691194793 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac806c3c0 con 0x7f7ac815d460 2023-11-24T08:05:04.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.408+0000 7f7ac77fe640 1 --2- 172.21.15.64:0/1691194793 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 0x7f7ac80b2100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009870 tx=0x7f7ab802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9359e27b96e30ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:04.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7ac6ffd640 1 -- 172.21.15.64:0/1691194793 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8030840 con 0x7f7ac815d460 2023-11-24T08:05:04.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7ac6ffd640 1 -- 172.21.15.64:0/1691194793 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab80309a0 con 0x7f7ac815d460 2023-11-24T08:05:04.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7ac6ffd640 1 -- 172.21.15.64:0/1691194793 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8030d30 con 0x7f7ac815d460 2023-11-24T08:05:04.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7acc813640 1 -- 172.21.15.64:0/1691194793 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 msgr2=0x7f7ac80b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7acc813640 1 --2- 172.21.15.64:0/1691194793 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 0x7f7ac80b2100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009870 tx=0x7f7ab802b980 comp rx=0 tx=0).stop 2023-11-24T08:05:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7acc813640 1 -- 172.21.15.64:0/1691194793 shutdown_connections 2023-11-24T08:05:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7acc813640 1 --2- 172.21.15.64:0/1691194793 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 0x7f7ac80cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7acc813640 1 --2- 172.21.15.64:0/1691194793 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 0x7f7ac80b2100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.409+0000 7f7acc813640 1 -- 172.21.15.64:0/1691194793 >> 172.21.15.64:0/1691194793 conn(0x7f7ac80d0f00 msgr2=0x7f7ac80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.410+0000 7f7acc813640 1 -- 172.21.15.64:0/1691194793 shutdown_connections 2023-11-24T08:05:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.410+0000 7f7acc813640 1 -- 172.21.15.64:0/1691194793 wait complete. 2023-11-24T08:05:04.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.411+0000 7f7acc813640 1 Processor -- start 2023-11-24T08:05:04.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.411+0000 7f7acc813640 1 -- start start 2023-11-24T08:05:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.412+0000 7f7acc813640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 0x7f7ac814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.412+0000 7f7acc813640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 0x7f7ac80cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.412+0000 7f7acc813640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ac815d840 0x7f7ac80ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.412+0000 7f7acc813640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8159a30 con 0x7f7ac815d840 2023-11-24T08:05:04.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.412+0000 7f7acc813640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8159d10 con 0x7f7ac815d460 2023-11-24T08:05:04.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.412+0000 7f7acc813640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8159ff0 con 0x7f7ac806be70 2023-11-24T08:05:04.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ac815d840 0x7f7ac80ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ac815d840 0x7f7ac80ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53582/0 (socket says 172.21.15.64:53582) 2023-11-24T08:05:04.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac7fff640 1 -- 172.21.15.64:0/4087250452 learned_addr learned my addr 172.21.15.64:0/4087250452 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:04.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac77fe640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 0x7f7ac814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7abffff640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 0x7f7ac80cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac7fff640 1 -- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 msgr2=0x7f7ac814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac7fff640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 0x7f7ac814d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac7fff640 1 -- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 msgr2=0x7f7ac80cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac7fff640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 0x7f7ac80cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7ac7fff640 1 -- 172.21.15.64:0/4087250452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80094d0 con 0x7f7ac815d840 2023-11-24T08:05:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.413+0000 7f7abffff640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 0x7f7ac80cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.414+0000 7f7ac7fff640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ac815d840 0x7f7ac80ccef0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000efc0 tx=0x7f7ac000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.414+0000 7f7ac4ff9640 1 -- 172.21.15.64:0/4087250452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0009ba0 con 0x7f7ac815d840 2023-11-24T08:05:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.414+0000 7f7ac4ff9640 1 -- 172.21.15.64:0/4087250452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac000f040 con 0x7f7ac815d840 2023-11-24T08:05:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.414+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac8157440 con 0x7f7ac815d840 2023-11-24T08:05:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.414+0000 7f7ac4ff9640 1 -- 172.21.15.64:0/4087250452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00146c0 con 0x7f7ac815d840 2023-11-24T08:05:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.414+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac81569c0 con 0x7f7ac815d840 2023-11-24T08:05:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.415+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac81572a0 con 0x7f7ac815d840 2023-11-24T08:05:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.416+0000 7f7ac4ff9640 1 -- 172.21.15.64:0/4087250452 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ac0014820 con 0x7f7ac815d840 2023-11-24T08:05:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.417+0000 7f7ac4ff9640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7aa806ae40 0x7f7aa806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.417+0000 7f7ac4ff9640 1 -- 172.21.15.64:0/4087250452 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7ac008a220 con 0x7f7ac815d840 2023-11-24T08:05:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.418+0000 7f7ac77fe640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7aa806ae40 0x7f7aa806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.419+0000 7f7ac77fe640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7aa806ae40 0x7f7aa806d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8036000 tx=0x7f7ab803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.422+0000 7f7ac4ff9640 1 -- 172.21.15.64:0/4087250452 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7ac81572a0 con 0x7f7ac815d840 2023-11-24T08:05:04.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.654+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ac8157960 con 0x7f7ac815d840 2023-11-24T08:05:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.656+0000 7f7ac4ff9640 1 -- 172.21.15.64:0/4087250452 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f7ac8157960 con 0x7f7ac815d840 2023-11-24T08:05:04.658 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:04.658 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:03:55.699656+0000","modified":"2023-11-24T08:03:58.759467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:05:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.658+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7aa806ae40 msgr2=0x7f7aa806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.658+0000 7f7acc813640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7aa806ae40 0x7f7aa806d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8036000 tx=0x7f7ab803b040 comp rx=0 tx=0).stop 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.658+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ac815d840 msgr2=0x7f7ac80ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.658+0000 7f7acc813640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ac815d840 0x7f7ac80ccef0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000efc0 tx=0x7f7ac000c490 comp rx=0 tx=0).stop 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.659+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 shutdown_connections 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.659+0000 7f7acc813640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7aa806ae40 0x7f7aa806d300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.659+0000 7f7acc813640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ac815d840 0x7f7ac80ccef0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.659+0000 7f7acc813640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7ac815d460 0x7f7ac80cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.659+0000 7f7acc813640 1 --2- 172.21.15.64:0/4087250452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7ac806be70 0x7f7ac814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.659+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 >> 172.21.15.64:0/4087250452 conn(0x7f7ac80d0f00 msgr2=0x7f7ac806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.659+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 shutdown_connections 2023-11-24T08:05:04.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.659+0000 7f7acc813640 1 -- 172.21.15.64:0/4087250452 wait complete. 2023-11-24T08:05:04.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:04.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.908+0000 7fd50e5bc640 1 Processor -- start 2023-11-24T08:05:04.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.908+0000 7fd50e5bc640 1 -- start start 2023-11-24T08:05:04.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.909+0000 7fd50e5bc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd50813dfd0 0x7fd50815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.909+0000 7fd50e5bc640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd5080cf050 0x7fd5080cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.910+0000 7fd50e5bc640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5080ce7c0 con 0x7fd50806c130 2023-11-24T08:05:04.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.910+0000 7fd50e5bc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5080cfa40 con 0x7fd50813dfd0 2023-11-24T08:05:04.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.910+0000 7fd50e5bc640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd508140900 con 0x7fd5080cf050 2023-11-24T08:05:04.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.910+0000 7fd50d5ba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd50813dfd0 0x7fd50815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.910+0000 7fd50d5ba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd50813dfd0 0x7fd50815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53598/0 (socket says 172.21.15.64:53598) 2023-11-24T08:05:04.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.910+0000 7fd50d5ba640 1 -- 172.21.15.64:0/4172873824 learned_addr learned my addr 172.21.15.64:0/4172873824 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:04.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.910+0000 7fd50cdb9640 1 --2- 172.21.15.64:0/4172873824 >> v2:172.21.15.91:3300/0 conn(0x7fd5080cf050 0x7fd5080cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.911+0000 7fd50d5ba640 1 -- 172.21.15.64:0/4172873824 >> v2:172.21.15.91:3300/0 conn(0x7fd5080cf050 msgr2=0x7fd5080cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:04.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.911+0000 7fd50d5ba640 1 --2- 172.21.15.64:0/4172873824 >> v2:172.21.15.91:3300/0 conn(0x7fd5080cf050 0x7fd5080cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.911+0000 7fd50d5ba640 1 -- 172.21.15.64:0/4172873824 >> v1:172.21.15.91:6789/0 conn(0x7fd50806c130 legacy=0x7fd50806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:04.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.911+0000 7fd50d5ba640 1 -- 172.21.15.64:0/4172873824 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50815fa70 con 0x7fd50813dfd0 2023-11-24T08:05:04.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.912+0000 7fd50d5ba640 1 --2- 172.21.15.64:0/4172873824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd50815be70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd50806e070 tx=0x7fd50402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b079140b78447008 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:04.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.912+0000 7fd4fffff640 1 -- 172.21.15.64:0/4172873824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504031870 con 0x7fd50813dfd0 2023-11-24T08:05:04.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.912+0000 7fd4fffff640 1 -- 172.21.15.64:0/4172873824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5040319d0 con 0x7fd50813dfd0 2023-11-24T08:05:04.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.912+0000 7fd4fffff640 1 -- 172.21.15.64:0/4172873824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504031d60 con 0x7fd50813dfd0 2023-11-24T08:05:04.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.913+0000 7fd50e5bc640 1 -- 172.21.15.64:0/4172873824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 msgr2=0x7fd50815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.913+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/4172873824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd50815be70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd50806e070 tx=0x7fd50402b910 comp rx=0 tx=0).stop 2023-11-24T08:05:04.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.913+0000 7fd50e5bc640 1 -- 172.21.15.64:0/4172873824 shutdown_connections 2023-11-24T08:05:04.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.913+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/4172873824 >> v2:172.21.15.91:3300/0 conn(0x7fd5080cf050 0x7fd5080cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.913+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/4172873824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd50815be70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.913+0000 7fd50e5bc640 1 -- 172.21.15.64:0/4172873824 >> 172.21.15.64:0/4172873824 conn(0x7fd50806a860 msgr2=0x7fd50806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:04.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.913+0000 7fd50e5bc640 1 -- 172.21.15.64:0/4172873824 shutdown_connections 2023-11-24T08:05:04.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.913+0000 7fd50e5bc640 1 -- 172.21.15.64:0/4172873824 wait complete. 2023-11-24T08:05:04.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.915+0000 7fd50e5bc640 1 Processor -- start 2023-11-24T08:05:04.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.915+0000 7fd50e5bc640 1 -- start start 2023-11-24T08:05:04.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50e5bc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd50806c130 0x7fd5080b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50e5bc640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5080cf050 0x7fd50812f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50e5bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd5080b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50e5bc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5080cace0 con 0x7fd50813dfd0 2023-11-24T08:05:04.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50e5bc640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5080b2380 con 0x7fd5080cf050 2023-11-24T08:05:04.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50e5bc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5080b2620 con 0x7fd50806c130 2023-11-24T08:05:04.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50ddbb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd5080b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50ddbb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd5080b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53610/0 (socket says 172.21.15.64:53610) 2023-11-24T08:05:04.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.916+0000 7fd50ddbb640 1 -- 172.21.15.64:0/3615788502 learned_addr learned my addr 172.21.15.64:0/3615788502 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50ddbb640 1 -- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd50806c130 msgr2=0x7fd5080b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50ddbb640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd50806c130 0x7fd5080b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50ddbb640 1 -- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5080cf050 msgr2=0x7fd50812f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50cdb9640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5080cf050 0x7fd50812f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50ddbb640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5080cf050 0x7fd50812f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50ddbb640 1 -- 172.21.15.64:0/3615788502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040094d0 con 0x7fd50813dfd0 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50ddbb640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd5080b33a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd50000c780 tx=0x7fd50000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd4fdffb640 1 -- 172.21.15.64:0/3615788502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500003d80 con 0x7fd50813dfd0 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd4fdffb640 1 -- 172.21.15.64:0/3615788502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50000ce90 con 0x7fd50813dfd0 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd4fdffb640 1 -- 172.21.15.64:0/3615788502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5000052b0 con 0x7fd50813dfd0 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5080b0ee0 con 0x7fd50813dfd0 2023-11-24T08:05:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.917+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5080b2ba0 con 0x7fd50813dfd0 2023-11-24T08:05:04.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.919+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50804e840 con 0x7fd50813dfd0 2023-11-24T08:05:04.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.920+0000 7fd4fdffb640 1 -- 172.21.15.64:0/3615788502 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd500005410 con 0x7fd50813dfd0 2023-11-24T08:05:04.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.921+0000 7fd4fdffb640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd4ec06ae40 0x7fd4ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:04.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.921+0000 7fd4fdffb640 1 -- 172.21.15.64:0/3615788502 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd500088b00 con 0x7fd50813dfd0 2023-11-24T08:05:04.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.922+0000 7fd50d5ba640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd4ec06ae40 0x7fd4ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:04.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.923+0000 7fd50d5ba640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd4ec06ae40 0x7fd4ec06d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd504036000 tx=0x7fd504030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:04.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:04.926+0000 7fd4fdffb640 1 -- 172.21.15.64:0/3615788502 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd500053b20 con 0x7fd50813dfd0 2023-11-24T08:05:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.120+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5080ce9a0 con 0x7fd50813dfd0 2023-11-24T08:05:05.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.121+0000 7fd4fdffb640 1 -- 172.21.15.64:0/3615788502 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7fd500057140 con 0x7fd50813dfd0 2023-11-24T08:05:05.124 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:05.124 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:05.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd4ec06ae40 msgr2=0x7fd4ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd4ec06ae40 0x7fd4ec06d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd504036000 tx=0x7fd504030000 comp rx=0 tx=0).stop 2023-11-24T08:05:05.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 msgr2=0x7fd5080b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd5080b33a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd50000c780 tx=0x7fd50000cc40 comp rx=0 tx=0).stop 2023-11-24T08:05:05.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 shutdown_connections 2023-11-24T08:05:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd4ec06ae40 0x7fd4ec06d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd50813dfd0 0x7fd5080b33a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5080cf050 0x7fd50812f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 --2- 172.21.15.64:0/3615788502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd50806c130 0x7fd5080b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.124+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 >> 172.21.15.64:0/3615788502 conn(0x7fd50806a860 msgr2=0x7fd50813f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:05.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.125+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 shutdown_connections 2023-11-24T08:05:05.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.125+0000 7fd50e5bc640 1 -- 172.21.15.64:0/3615788502 wait complete. 2023-11-24T08:05:05.146 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:05:05.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.372+0000 7f4c4f027640 1 Processor -- start 2023-11-24T08:05:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.373+0000 7f4c4f027640 1 -- start start 2023-11-24T08:05:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.374+0000 7f4c4f027640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c480d14a0 0x7f4c480d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.374+0000 7f4c4f027640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480b0640 0x7f4c480b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.374+0000 7f4c4f027640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480cf0c0 con 0x7f4c4815d5b0 2023-11-24T08:05:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.374+0000 7f4c4f027640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c480b01b0 con 0x7f4c480d14a0 2023-11-24T08:05:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.374+0000 7f4c4f027640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815d990 con 0x7f4c480b0640 2023-11-24T08:05:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.375+0000 7f4c4d824640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c480d14a0 0x7f4c480d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.375+0000 7f4c4d824640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c480d14a0 0x7f4c480d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53618/0 (socket says 172.21.15.64:53618) 2023-11-24T08:05:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.375+0000 7f4c4d824640 1 -- 172.21.15.64:0/3351034940 learned_addr learned my addr 172.21.15.64:0/3351034940 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:05.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.375+0000 7f4c4e826640 1 --2- 172.21.15.64:0/3351034940 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480b0640 0x7f4c480b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.375+0000 7f4c4d824640 1 -- 172.21.15.64:0/3351034940 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480b0640 msgr2=0x7f4c480b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:05.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.375+0000 7f4c4d824640 1 --2- 172.21.15.64:0/3351034940 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480b0640 0x7f4c480b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.375+0000 7f4c4d824640 1 -- 172.21.15.64:0/3351034940 >> v1:172.21.15.91:6789/0 conn(0x7f4c4815d5b0 legacy=0x7f4c480d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:05.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.375+0000 7f4c4d824640 1 -- 172.21.15.64:0/3351034940 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480d1e40 con 0x7f4c480d14a0 2023-11-24T08:05:05.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.376+0000 7f4c4d824640 1 --2- 172.21.15.64:0/3351034940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c480d14a0 0x7f4c480d1900 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009870 tx=0x7f4c3802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0b87e11fe07cd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4d023640 1 -- 172.21.15.64:0/3351034940 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38031640 con 0x7f4c480d14a0 2023-11-24T08:05:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4d023640 1 -- 172.21.15.64:0/3351034940 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c380317a0 con 0x7f4c480d14a0 2023-11-24T08:05:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4d023640 1 -- 172.21.15.64:0/3351034940 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38031b30 con 0x7f4c480d14a0 2023-11-24T08:05:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4f027640 1 -- 172.21.15.64:0/3351034940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c480d14a0 msgr2=0x7f4c480d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4f027640 1 --2- 172.21.15.64:0/3351034940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c480d14a0 0x7f4c480d1900 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009870 tx=0x7f4c3802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4f027640 1 -- 172.21.15.64:0/3351034940 shutdown_connections 2023-11-24T08:05:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4f027640 1 --2- 172.21.15.64:0/3351034940 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480b0640 0x7f4c480b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4f027640 1 --2- 172.21.15.64:0/3351034940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c480d14a0 0x7f4c480d1900 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4f027640 1 -- 172.21.15.64:0/3351034940 >> 172.21.15.64:0/3351034940 conn(0x7f4c4809c3b0 msgr2=0x7f4c480d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4f027640 1 -- 172.21.15.64:0/3351034940 shutdown_connections 2023-11-24T08:05:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.377+0000 7f4c4f027640 1 -- 172.21.15.64:0/3351034940 wait complete. 2023-11-24T08:05:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.379+0000 7f4c4f027640 1 Processor -- start 2023-11-24T08:05:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.379+0000 7f4c4f027640 1 -- start start 2023-11-24T08:05:05.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.380+0000 7f4c4f027640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c480b0640 0x7f4c481558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.380+0000 7f4c4f027640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480d14a0 0x7f4c480c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.380+0000 7f4c4f027640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c4815d5b0 0x7f4c480d3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.380+0000 7f4c4f027640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815f650 con 0x7f4c4815d5b0 2023-11-24T08:05:05.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.380+0000 7f4c4f027640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c480c80a0 con 0x7f4c480b0640 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.380+0000 7f4c4f027640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815ef90 con 0x7f4c480d14a0 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c4815d5b0 0x7f4c480d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c4815d5b0 0x7f4c480d3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53624/0 (socket says 172.21.15.64:53624) 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 -- 172.21.15.64:0/2558585051 learned_addr learned my addr 172.21.15.64:0/2558585051 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4d824640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480d14a0 0x7f4c480c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e025640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c480b0640 0x7f4c481558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 -- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480d14a0 msgr2=0x7f4c480c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480d14a0 0x7f4c480c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 -- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c480b0640 msgr2=0x7f4c481558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c480b0640 0x7f4c481558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 -- 172.21.15.64:0/2558585051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380094d0 con 0x7f4c4815d5b0 2023-11-24T08:05:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.381+0000 7f4c4e826640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c4815d5b0 0x7f4c480d3070 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000cc30 tx=0x7f4c4000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:05.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.382+0000 7f4c3effd640 1 -- 172.21.15.64:0/2558585051 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000ce40 con 0x7f4c4815d5b0 2023-11-24T08:05:05.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.382+0000 7f4c3effd640 1 -- 172.21.15.64:0/2558585051 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c40003990 con 0x7f4c4815d5b0 2023-11-24T08:05:05.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.382+0000 7f4c3effd640 1 -- 172.21.15.64:0/2558585051 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40003d20 con 0x7f4c4815d5b0 2023-11-24T08:05:05.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.382+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4815f270 con 0x7f4c4815d5b0 2023-11-24T08:05:05.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.382+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c480c7e70 con 0x7f4c4815d5b0 2023-11-24T08:05:05.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.383+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4815edf0 con 0x7f4c4815d5b0 2023-11-24T08:05:05.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.384+0000 7f4c3effd640 1 -- 172.21.15.64:0/2558585051 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c40003e80 con 0x7f4c4815d5b0 2023-11-24T08:05:05.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.385+0000 7f4c3effd640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4c2c06ae40 0x7f4c2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.385+0000 7f4c3effd640 1 -- 172.21.15.64:0/2558585051 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4c40089a10 con 0x7f4c4815d5b0 2023-11-24T08:05:05.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.386+0000 7f4c4e025640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4c2c06ae40 0x7f4c2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.387+0000 7f4c4e025640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4c2c06ae40 0x7f4c2c06d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c44003d90 tx=0x7f4c44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:05.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.389+0000 7f4c3effd640 1 -- 172.21.15.64:0/2558585051 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c4815edf0 con 0x7f4c4815d5b0 2023-11-24T08:05:05.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.621+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c4815f790 con 0x7f4c4815d5b0 2023-11-24T08:05:05.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.622+0000 7f4c3effd640 1 -- 172.21.15.64:0/2558585051 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f4c4815f790 con 0x7f4c4815d5b0 2023-11-24T08:05:05.624 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:05.624 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:03:55.699656+0000","modified":"2023-11-24T08:03:58.759467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:05:05.624 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T08:05:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.624+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4c2c06ae40 msgr2=0x7f4c2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.624+0000 7f4c4f027640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4c2c06ae40 0x7f4c2c06d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c44003d90 tx=0x7f4c44009040 comp rx=0 tx=0).stop 2023-11-24T08:05:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.624+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c4815d5b0 msgr2=0x7f4c480d3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.624+0000 7f4c4f027640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c4815d5b0 0x7f4c480d3070 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000cc30 tx=0x7f4c4000a5c0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.625+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 shutdown_connections 2023-11-24T08:05:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.625+0000 7f4c4f027640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4c2c06ae40 0x7f4c2c06d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.625+0000 7f4c4f027640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c4815d5b0 0x7f4c480d3070 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.625+0000 7f4c4f027640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4c480d14a0 0x7f4c480c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.625+0000 7f4c4f027640 1 --2- 172.21.15.64:0/2558585051 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4c480b0640 0x7f4c481558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.625+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 >> 172.21.15.64:0/2558585051 conn(0x7f4c4809c3b0 msgr2=0x7f4c4806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.625+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 shutdown_connections 2023-11-24T08:05:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.625+0000 7f4c4f027640 1 -- 172.21.15.64:0/2558585051 wait complete. 2023-11-24T08:05:05.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:05.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.871+0000 7f19b7fff640 1 Processor -- start 2023-11-24T08:05:05.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.871+0000 7f19b7fff640 1 -- start start 2023-11-24T08:05:05.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.872+0000 7f19b7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19b815d5b0 0x7f19b80d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.872+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.872+0000 7f19b7fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b80cf0c0 con 0x7f19b80d14a0 2023-11-24T08:05:05.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.872+0000 7f19b7fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b80b01b0 con 0x7f19b815d5b0 2023-11-24T08:05:05.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.872+0000 7f19b7fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b815d990 con 0x7f19b80b1650 2023-11-24T08:05:05.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19b815d5b0 0x7f19b80d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19b815d5b0 0x7f19b80d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53630/0 (socket says 172.21.15.64:53630) 2023-11-24T08:05:05.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b6ffd640 1 -- 172.21.15.64:0/2632003831 learned_addr learned my addr 172.21.15.64:0/2632003831 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:05.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b77fe640 1 --2- 172.21.15.64:0/2632003831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b6ffd640 1 -- 172.21.15.64:0/2632003831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 msgr2=0x7f19b80d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:05.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b6ffd640 1 --2- 172.21.15.64:0/2632003831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b80d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b6ffd640 1 -- 172.21.15.64:0/2632003831 >> v1:172.21.15.91:6789/0 conn(0x7f19b80d14a0 legacy=0x7f19b80cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:05.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b6ffd640 1 -- 172.21.15.64:0/2632003831 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b80d1e40 con 0x7f19b815d5b0 2023-11-24T08:05:05.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.873+0000 7f19b77fe640 1 --2- 172.21.15.64:0/2632003831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b80d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:05.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.874+0000 7f19b6ffd640 1 --2- 172.21.15.64:0/2632003831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b815d5b0 0x7f19b80d3990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009870 tx=0x7f19a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1b09b7c9cbc0c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:05.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b5ffb640 1 -- 172.21.15.64:0/2632003831 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8031640 con 0x7f19b815d5b0 2023-11-24T08:05:05.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b5ffb640 1 -- 172.21.15.64:0/2632003831 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a80317a0 con 0x7f19b815d5b0 2023-11-24T08:05:05.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b5ffb640 1 -- 172.21.15.64:0/2632003831 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8031b30 con 0x7f19b815d5b0 2023-11-24T08:05:05.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b7fff640 1 -- 172.21.15.64:0/2632003831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b815d5b0 msgr2=0x7f19b80d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b7fff640 1 --2- 172.21.15.64:0/2632003831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b815d5b0 0x7f19b80d3990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009870 tx=0x7f19a802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b7fff640 1 -- 172.21.15.64:0/2632003831 shutdown_connections 2023-11-24T08:05:05.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b7fff640 1 --2- 172.21.15.64:0/2632003831 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b80d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b7fff640 1 --2- 172.21.15.64:0/2632003831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b815d5b0 0x7f19b80d3990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b7fff640 1 -- 172.21.15.64:0/2632003831 >> 172.21.15.64:0/2632003831 conn(0x7f19b809c3b0 msgr2=0x7f19b80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:05.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.875+0000 7f19b7fff640 1 -- 172.21.15.64:0/2632003831 shutdown_connections 2023-11-24T08:05:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.876+0000 7f19b7fff640 1 -- 172.21.15.64:0/2632003831 wait complete. 2023-11-24T08:05:05.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.877+0000 7f19b7fff640 1 Processor -- start 2023-11-24T08:05:05.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.877+0000 7f19b7fff640 1 -- start start 2023-11-24T08:05:05.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.878+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b8153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.878+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b80d14a0 0x7f19b80cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.878+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19b815d5b0 0x7f19b80c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.878+0000 7f19b7fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80cc820 con 0x7f19b80d14a0 2023-11-24T08:05:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.878+0000 7f19b7fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80cb780 con 0x7f19b815d5b0 2023-11-24T08:05:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.878+0000 7f19b7fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80cba60 con 0x7f19b80b1650 2023-11-24T08:05:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b80d14a0 0x7f19b80cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b80d14a0 0x7f19b80cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53632/0 (socket says 172.21.15.64:53632) 2023-11-24T08:05:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 -- 172.21.15.64:0/1399368272 learned_addr learned my addr 172.21.15.64:0/1399368272 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b77fe640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19b815d5b0 0x7f19b80c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b6ffd640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b8153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 -- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 msgr2=0x7f19b8153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b8153ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 -- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19b815d5b0 msgr2=0x7f19b80c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:05.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19b815d5b0 0x7f19b80c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:05.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 -- 172.21.15.64:0/1399368272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a80094d0 con 0x7f19b80d14a0 2023-11-24T08:05:05.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b77fe640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19b815d5b0 0x7f19b80c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:05.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.879+0000 7f19b67fc640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b80d14a0 0x7f19b80cb110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f19a000c7c0 tx=0x7f19a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:05.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.880+0000 7f19af7fe640 1 -- 172.21.15.64:0/1399368272 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000aa50 con 0x7f19b80d14a0 2023-11-24T08:05:05.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.880+0000 7f19af7fe640 1 -- 172.21.15.64:0/1399368272 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a000abb0 con 0x7f19b80d14a0 2023-11-24T08:05:05.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.880+0000 7f19af7fe640 1 -- 172.21.15.64:0/1399368272 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0012420 con 0x7f19b80d14a0 2023-11-24T08:05:05.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.880+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b80cbd40 con 0x7f19b80d14a0 2023-11-24T08:05:05.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.880+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b80cc300 con 0x7f19b80d14a0 2023-11-24T08:05:05.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.881+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b80cedc0 con 0x7f19b80d14a0 2023-11-24T08:05:05.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.882+0000 7f19af7fe640 1 -- 172.21.15.64:0/1399368272 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f19a0012580 con 0x7f19b80d14a0 2023-11-24T08:05:05.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.883+0000 7f19af7fe640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f199806ae90 0x7f199806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.884+0000 7f19af7fe640 1 -- 172.21.15.64:0/1399368272 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f19a00895d0 con 0x7f19b80d14a0 2023-11-24T08:05:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.884+0000 7f19b6ffd640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f199806ae90 0x7f199806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:05.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.885+0000 7f19b6ffd640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f199806ae90 0x7f199806d350 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19a8036000 tx=0x7f19a803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:05.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:05.888+0000 7f19af7fe640 1 -- 172.21.15.64:0/1399368272 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19b80cedc0 con 0x7f19b80d14a0 2023-11-24T08:05:06.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.121+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19b80cedc0 con 0x7f19b80d14a0 2023-11-24T08:05:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.122+0000 7f19af7fe640 1 -- 172.21.15.64:0/1399368272 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f19b80cedc0 con 0x7f19b80d14a0 2023-11-24T08:05:06.124 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:06.125 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4265680293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4265680293},{"type":"v1","addr":"172.21.15.64:6833","nonce":4265680293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2312478409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2312478409},{"type":"v1","addr":"172.21.15.64:6835","nonce":2312478409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/4053830240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":4053830240},{"type":"v1","addr":"172.21.15.91:6836","nonce":4053830240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:03:55.699656+0000","modified":"2023-11-24T08:03:58.759467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.91:6837/3953784254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":3953784254},{"type":"v1","addr":"172.21.15.91:6837","nonce":3953784254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:05:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-11-24T08:05:06.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f199806ae90 msgr2=0x7f199806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f199806ae90 0x7f199806d350 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19a8036000 tx=0x7f19a803c040 comp rx=0 tx=0).stop 2023-11-24T08:05:06.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b80d14a0 msgr2=0x7f19b80cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b80d14a0 0x7f19b80cb110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f19a000c7c0 tx=0x7f19a000cc80 comp rx=0 tx=0).stop 2023-11-24T08:05:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 shutdown_connections 2023-11-24T08:05:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f199806ae90 0x7f199806d350 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19b815d5b0 0x7f19b80c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b80d14a0 0x7f19b80cb110 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 --2- 172.21.15.64:0/1399368272 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19b80b1650 0x7f19b8153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 >> 172.21.15.64:0/1399368272 conn(0x7f19b809c3b0 msgr2=0x7f19b815dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 shutdown_connections 2023-11-24T08:05:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.125+0000 7f19b7fff640 1 -- 172.21.15.64:0/1399368272 wait complete. 2023-11-24T08:05:06.148 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:06.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.373+0000 7f8dae8c6640 1 Processor -- start 2023-11-24T08:05:06.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.374+0000 7f8dae8c6640 1 -- start start 2023-11-24T08:05:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.375+0000 7f8dae8c6640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8da80ce260 0x7f8da80d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.375+0000 7f8dae8c6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8da80ce6a0 0x7f8da80d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.375+0000 7f8dae8c6640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da815d8b0 con 0x7f8da815d4d0 2023-11-24T08:05:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.375+0000 7f8dae8c6640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80b91b0 con 0x7f8da80ce6a0 2023-11-24T08:05:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.375+0000 7f8dae8c6640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80b2400 con 0x7f8da80ce260 2023-11-24T08:05:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.376+0000 7f8dace63640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8da80ce6a0 0x7f8da80d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.376+0000 7f8dace63640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8da80ce6a0 0x7f8da80d1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53644/0 (socket says 172.21.15.64:53644) 2023-11-24T08:05:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.376+0000 7f8dace63640 1 -- 172.21.15.64:0/4413756 learned_addr learned my addr 172.21.15.64:0/4413756 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.376+0000 7f8d9f7fe640 1 --2- 172.21.15.64:0/4413756 >> v2:172.21.15.91:3300/0 conn(0x7f8da80ce260 0x7f8da80d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.376+0000 7f8dace63640 1 -- 172.21.15.64:0/4413756 >> v2:172.21.15.91:3300/0 conn(0x7f8da80ce260 msgr2=0x7f8da80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.376+0000 7f8dace63640 1 --2- 172.21.15.64:0/4413756 >> v2:172.21.15.91:3300/0 conn(0x7f8da80ce260 0x7f8da80d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.376+0000 7f8dace63640 1 -- 172.21.15.64:0/4413756 >> v1:172.21.15.91:6789/0 conn(0x7f8da815d4d0 legacy=0x7f8da80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.376+0000 7f8dace63640 1 -- 172.21.15.64:0/4413756 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80b26e0 con 0x7f8da80ce6a0 2023-11-24T08:05:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.378+0000 7f8dace63640 1 --2- 172.21.15.64:0/4413756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce6a0 0x7f8da80d1d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8da806d2a0 tx=0x7f8da002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbad4465ea19cbc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.378+0000 7f8da77fe640 1 -- 172.21.15.64:0/4413756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0030870 con 0x7f8da80ce6a0 2023-11-24T08:05:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.378+0000 7f8da77fe640 1 -- 172.21.15.64:0/4413756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da00309d0 con 0x7f8da80ce6a0 2023-11-24T08:05:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.378+0000 7f8da77fe640 1 -- 172.21.15.64:0/4413756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0030d60 con 0x7f8da80ce6a0 2023-11-24T08:05:06.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.379+0000 7f8dae8c6640 1 -- 172.21.15.64:0/4413756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce6a0 msgr2=0x7f8da80d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.379+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/4413756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce6a0 0x7f8da80d1d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8da806d2a0 tx=0x7f8da002b9b0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.379+0000 7f8dae8c6640 1 -- 172.21.15.64:0/4413756 shutdown_connections 2023-11-24T08:05:06.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.379+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/4413756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce6a0 0x7f8da80d1d90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.379+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/4413756 >> v2:172.21.15.91:3300/0 conn(0x7f8da80ce260 0x7f8da80d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.379+0000 7f8dae8c6640 1 -- 172.21.15.64:0/4413756 >> 172.21.15.64:0/4413756 conn(0x7f8da8071f90 msgr2=0x7f8da80723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:06.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.379+0000 7f8dae8c6640 1 -- 172.21.15.64:0/4413756 shutdown_connections 2023-11-24T08:05:06.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.379+0000 7f8dae8c6640 1 -- 172.21.15.64:0/4413756 wait complete. 2023-11-24T08:05:06.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.381+0000 7f8dae8c6640 1 Processor -- start 2023-11-24T08:05:06.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.381+0000 7f8dae8c6640 1 -- start start 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dae8c6640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da80ce260 0x7f8da8154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dae8c6640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8da80ce6a0 0x7f8da8155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dae8c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da815d4d0 0x7f8da80cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dae8c6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da815da40 con 0x7f8da815d4d0 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dae8c6640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81f4030 con 0x7f8da80ce260 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dae8c6640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8154c60 con 0x7f8da80ce6a0 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dace63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da815d4d0 0x7f8da80cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dace63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da815d4d0 0x7f8da80cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53660/0 (socket says 172.21.15.64:53660) 2023-11-24T08:05:06.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8dace63640 1 -- 172.21.15.64:0/564914941 learned_addr learned my addr 172.21.15.64:0/564914941 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.384+0000 7f8d9f7fe640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8da80ce6a0 0x7f8da8155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.383+0000 7f8da7fff640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da80ce260 0x7f8da8154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.384+0000 7f8dace63640 1 -- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8da80ce6a0 msgr2=0x7f8da8155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.384+0000 7f8dace63640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8da80ce6a0 0x7f8da8155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.384+0000 7f8dace63640 1 -- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da80ce260 msgr2=0x7f8da8154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.384+0000 7f8dace63640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da80ce260 0x7f8da8154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.384+0000 7f8dace63640 1 -- 172.21.15.64:0/564914941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da00094d0 con 0x7f8da815d4d0 2023-11-24T08:05:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.384+0000 7f8dace63640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da815d4d0 0x7f8da80cbd40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009960 tx=0x7f8da0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:06.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.385+0000 7f8da57fa640 1 -- 172.21.15.64:0/564914941 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0039c90 con 0x7f8da815d4d0 2023-11-24T08:05:06.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.385+0000 7f8da57fa640 1 -- 172.21.15.64:0/564914941 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da0039e10 con 0x7f8da815d4d0 2023-11-24T08:05:06.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.385+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da815dc50 con 0x7f8da815d4d0 2023-11-24T08:05:06.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.385+0000 7f8da57fa640 1 -- 172.21.15.64:0/564914941 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da00446c0 con 0x7f8da815d4d0 2023-11-24T08:05:06.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.385+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da815e210 con 0x7f8da815d4d0 2023-11-24T08:05:06.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.386+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8074770 con 0x7f8da815d4d0 2023-11-24T08:05:06.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.387+0000 7f8da57fa640 1 -- 172.21.15.64:0/564914941 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8da002f3c0 con 0x7f8da815d4d0 2023-11-24T08:05:06.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.388+0000 7f8da57fa640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8d8806ae40 0x7f8d8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.389+0000 7f8da57fa640 1 -- 172.21.15.64:0/564914941 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8da003e070 con 0x7f8da815d4d0 2023-11-24T08:05:06.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.389+0000 7f8da7fff640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8d8806ae40 0x7f8d8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.390+0000 7f8da7fff640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8d8806ae40 0x7f8d8806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8da815bd10 tx=0x7f8d98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:06.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.393+0000 7f8da57fa640 1 -- 172.21.15.64:0/564914941 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8da0077290 con 0x7f8da815d4d0 2023-11-24T08:05:06.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.587+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8da80b5a20 con 0x7f8da815d4d0 2023-11-24T08:05:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.590+0000 7f8da57fa640 1 -- 172.21.15.64:0/564914941 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16753 (secure 0 0 0) 0x7f8da007a8b0 con 0x7f8da815d4d0 2023-11-24T08:05:06.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:06.592 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:04:37.991251+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:03:53.626355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:03:54.726508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:06.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.592+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8d8806ae40 msgr2=0x7f8d8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8d8806ae40 0x7f8d8806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8da815bd10 tx=0x7f8d98009040 comp rx=0 tx=0).stop 2023-11-24T08:05:06.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da815d4d0 msgr2=0x7f8da80cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da815d4d0 0x7f8da80cbd40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009960 tx=0x7f8da0004730 comp rx=0 tx=0).stop 2023-11-24T08:05:06.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 shutdown_connections 2023-11-24T08:05:06.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8d8806ae40 0x7f8d8806d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da815d4d0 0x7f8da80cbd40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8da80ce6a0 0x7f8da8155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 --2- 172.21.15.64:0/564914941 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8da80ce260 0x7f8da8154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 >> 172.21.15.64:0/564914941 conn(0x7f8da8071f90 msgr2=0x7f8da8158650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:06.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 shutdown_connections 2023-11-24T08:05:06.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.593+0000 7f8dae8c6640 1 -- 172.21.15.64:0/564914941 wait complete. 2023-11-24T08:05:06.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:05:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.835+0000 7f87da1f4640 1 Processor -- start 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.836+0000 7f87da1f4640 1 -- start start 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.837+0000 7f87da1f4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d41416f0 0x7f87d415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.837+0000 7f87da1f4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87d406d230 0x7f87d415f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.837+0000 7f87da1f4640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d4070bf0 con 0x7f87d4141e80 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.837+0000 7f87da1f4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d406db30 con 0x7f87d406d230 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.837+0000 7f87da1f4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d406d850 con 0x7f87d41416f0 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.837+0000 7f87d99f3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87d406d230 0x7f87d415f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.837+0000 7f87d99f3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87d406d230 0x7f87d415f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53666/0 (socket says 172.21.15.64:53666) 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.837+0000 7f87d99f3640 1 -- 172.21.15.64:0/2279805146 learned_addr learned my addr 172.21.15.64:0/2279805146 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.838+0000 7f87d91f2640 1 --2- 172.21.15.64:0/2279805146 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d41416f0 0x7f87d415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.838+0000 7f87d99f3640 1 -- 172.21.15.64:0/2279805146 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d41416f0 msgr2=0x7f87d415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.838+0000 7f87d99f3640 1 --2- 172.21.15.64:0/2279805146 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d41416f0 0x7f87d415ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.838+0000 7f87d99f3640 1 -- 172.21.15.64:0/2279805146 >> v1:172.21.15.91:6789/0 conn(0x7f87d4141e80 legacy=0x7f87d406cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.838+0000 7f87d99f3640 1 -- 172.21.15.64:0/2279805146 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40705a0 con 0x7f87d406d230 2023-11-24T08:05:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.839+0000 7f87d99f3640 1 --2- 172.21.15.64:0/2279805146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d415f290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009870 tx=0x7f87cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535a2cf43d137846 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.839+0000 7f87cbfff640 1 -- 172.21.15.64:0/2279805146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc031640 con 0x7f87d406d230 2023-11-24T08:05:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.839+0000 7f87cbfff640 1 -- 172.21.15.64:0/2279805146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0317a0 con 0x7f87d406d230 2023-11-24T08:05:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.839+0000 7f87cbfff640 1 -- 172.21.15.64:0/2279805146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc031b30 con 0x7f87d406d230 2023-11-24T08:05:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.840+0000 7f87da1f4640 1 -- 172.21.15.64:0/2279805146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 msgr2=0x7f87d415f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.840+0000 7f87da1f4640 1 --2- 172.21.15.64:0/2279805146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d415f290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009870 tx=0x7f87cc02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.840+0000 7f87da1f4640 1 -- 172.21.15.64:0/2279805146 shutdown_connections 2023-11-24T08:05:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.840+0000 7f87da1f4640 1 --2- 172.21.15.64:0/2279805146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d415f290 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.840+0000 7f87da1f4640 1 --2- 172.21.15.64:0/2279805146 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d41416f0 0x7f87d415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.840+0000 7f87da1f4640 1 -- 172.21.15.64:0/2279805146 >> 172.21.15.64:0/2279805146 conn(0x7f87d40b0840 msgr2=0x7f87d40b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.840+0000 7f87da1f4640 1 -- 172.21.15.64:0/2279805146 shutdown_connections 2023-11-24T08:05:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.840+0000 7f87da1f4640 1 -- 172.21.15.64:0/2279805146 wait complete. 2023-11-24T08:05:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.842+0000 7f87da1f4640 1 Processor -- start 2023-11-24T08:05:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.842+0000 7f87da1f4640 1 -- start start 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87da1f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d40ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87da1f4640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87d41416f0 0x7f87d40ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87da1f4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d4141e80 0x7f87d415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87da1f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4130870 con 0x7f87d406d230 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87da1f4640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4130b10 con 0x7f87d41416f0 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87da1f4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4130df0 con 0x7f87d4141e80 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87d91f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d40ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87d91f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d40ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53670/0 (socket says 172.21.15.64:53670) 2023-11-24T08:05:06.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.843+0000 7f87d91f2640 1 -- 172.21.15.64:0/1877255914 learned_addr learned my addr 172.21.15.64:0/1877255914 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:06.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87d91f2640 1 -- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d4141e80 msgr2=0x7f87d415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87d91f2640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d4141e80 0x7f87d415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87d91f2640 1 -- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87d41416f0 msgr2=0x7f87d40ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87d91f2640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87d41416f0 0x7f87d40ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87d91f2640 1 -- 172.21.15.64:0/1877255914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0094d0 con 0x7f87d406d230 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87d91f2640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d40ad0a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f87d000ce10 tx=0x7f87d000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87c9ffb640 1 -- 172.21.15.64:0/1877255914 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0003b90 con 0x7f87d406d230 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87c9ffb640 1 -- 172.21.15.64:0/1877255914 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d0003cf0 con 0x7f87d406d230 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d41310d0 con 0x7f87d406d230 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87c9ffb640 1 -- 172.21.15.64:0/1877255914 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0003ea0 con 0x7f87d406d230 2023-11-24T08:05:06.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.844+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d4131690 con 0x7f87d406d230 2023-11-24T08:05:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.846+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d40cca10 con 0x7f87d406d230 2023-11-24T08:05:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.846+0000 7f87c9ffb640 1 -- 172.21.15.64:0/1877255914 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87d0004240 con 0x7f87d406d230 2023-11-24T08:05:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.847+0000 7f87c9ffb640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.847+0000 7f87c9ffb640 1 -- 172.21.15.64:0/1877255914 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f87d0012070 con 0x7f87d406d230 2023-11-24T08:05:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.848+0000 7f87d89f1640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.849+0000 7f87d89f1640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f87b806ae40 0x7f87b806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87c4006fd0 tx=0x7f87c4006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:06.852+0000 7f87c9ffb640 1 -- 172.21.15.64:0/1877255914 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87d00550e0 con 0x7f87d406d230 2023-11-24T08:05:07.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.088+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f87d40cc7b0 con 0x7f87d406d230 2023-11-24T08:05:07.091 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-11-24T08:05:07.089+0000 7fb13dee4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:05:07.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.093+0000 7f87c9ffb640 1 -- 172.21.15.64:0/1877255914 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f87d0058700 con 0x7f87d406d230 2023-11-24T08:05:07.094 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:05:07.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f87b806ae40 msgr2=0x7f87b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:07.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f87b806ae40 0x7f87b806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87c4006fd0 tx=0x7f87c4006bc0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 msgr2=0x7f87d40ad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d40ad0a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f87d000ce10 tx=0x7f87d000a660 comp rx=0 tx=0).stop 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 shutdown_connections 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87d4141e80 0x7f87d415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87d41416f0 0x7f87d40ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 --2- 172.21.15.64:0/1877255914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87d406d230 0x7f87d40ad0a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.095+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 >> 172.21.15.64:0/1877255914 conn(0x7f87d40b0840 msgr2=0x7f87d40af270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.096+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 shutdown_connections 2023-11-24T08:05:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.096+0000 7f87da1f4640 1 -- 172.21.15.64:0/1877255914 wait complete. 2023-11-24T08:05:07.116 INFO:tasks.ceph.mds.b.smithi091.stderr: -5006> 2023-11-24T08:03:50.545+0000 7fb6b342f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-24T08:05:07.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:05:07.125 INFO:tasks.ceph.mds.b.smithi091.stderr: -3787> 2023-11-24T08:04:01.549+0000 7fb6ad423640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:05:07.225 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2023-11-24T08:05:07.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.346+0000 7f748a31a640 1 Processor -- start 2023-11-24T08:05:07.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.347+0000 7f748a31a640 1 -- start start 2023-11-24T08:05:07.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.348+0000 7f748a31a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74840d1d90 0x7f74840ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:07.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.348+0000 7f748a31a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f74840d0fa0 0x7f74840b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.348+0000 7f748a31a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74840c6bf0 con 0x7f748415d3b0 2023-11-24T08:05:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.348+0000 7f748a31a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840d13d0 con 0x7f74840d0fa0 2023-11-24T08:05:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.348+0000 7f748a31a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74840cedc0 con 0x7f74840d1d90 2023-11-24T08:05:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.349+0000 7f7489318640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f748415d3b0 0x7f74840d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46326/0 (socket says 172.21.15.64:46326) 2023-11-24T08:05:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.349+0000 7f7489318640 1 -- 172.21.15.64:0/3601871047 learned_addr learned my addr 172.21.15.64:0/3601871047 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:07.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.349+0000 7f7489b19640 1 --2- 172.21.15.64:0/3601871047 >> v2:172.21.15.64:3300/0 conn(0x7f74840d0fa0 0x7f74840b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.349+0000 7f7488b17640 1 --2- 172.21.15.64:0/3601871047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74840d1d90 0x7f74840ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.349+0000 7f747bfff640 1 -- 172.21.15.64:0/3601871047 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442095377 0 0) 0x7f74840c6bf0 con 0x7f748415d3b0 2023-11-24T08:05:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.349+0000 7f747bfff640 1 -- 172.21.15.64:0/3601871047 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7470003520 con 0x7f748415d3b0 2023-11-24T08:05:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.349+0000 7f7489b19640 1 -- 172.21.15.64:0/3601871047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74840d1d90 msgr2=0x7f74840ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.350+0000 7f7489b19640 1 --2- 172.21.15.64:0/3601871047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74840d1d90 0x7f74840ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.350+0000 7f7489b19640 1 -- 172.21.15.64:0/3601871047 >> v1:172.21.15.64:6789/0 conn(0x7f748415d3b0 legacy=0x7f74840d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.350+0000 7f7489b19640 1 -- 172.21.15.64:0/3601871047 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840c6e40 con 0x7f74840d0fa0 2023-11-24T08:05:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.350+0000 7f7488b17640 1 --2- 172.21.15.64:0/3601871047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74840d1d90 0x7f74840ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.351+0000 7f7489b19640 1 --2- 172.21.15.64:0/3601871047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b3ca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f747c009d90 tx=0x7f747c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9366ad07263afbc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.351+0000 7f747bfff640 1 -- 172.21.15.64:0/3601871047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c02cbc0 con 0x7f74840d0fa0 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.351+0000 7f747bfff640 1 -- 172.21.15.64:0/3601871047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c02cd20 con 0x7f74840d0fa0 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.351+0000 7f747bfff640 1 -- 172.21.15.64:0/3601871047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c034640 con 0x7f74840d0fa0 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.352+0000 7f748a31a640 1 -- 172.21.15.64:0/3601871047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 msgr2=0x7f74840b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.352+0000 7f748a31a640 1 --2- 172.21.15.64:0/3601871047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b3ca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f747c009d90 tx=0x7f747c02be10 comp rx=0 tx=0).stop 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.352+0000 7f748a31a640 1 -- 172.21.15.64:0/3601871047 shutdown_connections 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.352+0000 7f748a31a640 1 --2- 172.21.15.64:0/3601871047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b3ca0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.352+0000 7f748a31a640 1 --2- 172.21.15.64:0/3601871047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f74840d1d90 0x7f74840ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.352+0000 7f748a31a640 1 -- 172.21.15.64:0/3601871047 >> 172.21.15.64:0/3601871047 conn(0x7f74840d30a0 msgr2=0x7f74840d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.352+0000 7f748a31a640 1 -- 172.21.15.64:0/3601871047 shutdown_connections 2023-11-24T08:05:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.352+0000 7f748a31a640 1 -- 172.21.15.64:0/3601871047 wait complete. 2023-11-24T08:05:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.354+0000 7f748a31a640 1 Processor -- start 2023-11-24T08:05:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.354+0000 7f748a31a640 1 -- start start 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f748a31a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f748a31a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74840d1d90 0x7f74841f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f748a31a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f748415d3b0 0x7f74841f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f748a31a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840b61d0 con 0x7f74840d0fa0 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f748a31a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840b64b0 con 0x7f74840d1d90 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f748a31a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74840b6790 con 0x7f748415d3b0 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f7489318640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f7489318640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48010/0 (socket says 172.21.15.64:48010) 2023-11-24T08:05:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.355+0000 7f7489318640 1 -- 172.21.15.64:0/1504938777 learned_addr learned my addr 172.21.15.64:0/1504938777 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7488b17640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74840d1d90 0x7f74841f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7489318640 1 -- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f748415d3b0 msgr2=0x7f74841f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7489318640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f748415d3b0 0x7f74841f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7489318640 1 -- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74840d1d90 msgr2=0x7f74841f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7489318640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74840d1d90 0x7f74841f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7489318640 1 -- 172.21.15.64:0/1504938777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c009a70 con 0x7f74840d0fa0 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7488b17640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74840d1d90 0x7f74841f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7489318640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b8510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f748000c390 tx=0x7f748000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7479ffb640 1 -- 172.21.15.64:0/1504938777 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480005ce0 con 0x7f74840d0fa0 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7479ffb640 1 -- 172.21.15.64:0/1504938777 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7480005e40 con 0x7f74840d0fa0 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74840b6a70 con 0x7f74840d0fa0 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f7479ffb640 1 -- 172.21.15.64:0/1504938777 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74800072f0 con 0x7f74840d0fa0 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.356+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748415b300 con 0x7f74840d0fa0 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.357+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74840b6f90 con 0x7f74840d0fa0 2023-11-24T08:05:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.359+0000 7f7479ffb640 1 -- 172.21.15.64:0/1504938777 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7480007450 con 0x7f74840d0fa0 2023-11-24T08:05:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.359+0000 7f7479ffb640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f746806ae40 0x7f746806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.360+0000 7f7479ffb640 1 -- 172.21.15.64:0/1504938777 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f7480088dc0 con 0x7f74840d0fa0 2023-11-24T08:05:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.360+0000 7f7488b17640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f746806ae40 0x7f746806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.361+0000 7f7488b17640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f746806ae40 0x7f746806d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7474006fd0 tx=0x7f7474008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:07.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.364+0000 7f7479ffb640 1 -- 172.21.15.64:0/1504938777 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74840b6f90 con 0x7f74840d0fa0 2023-11-24T08:05:07.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:07.600+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f74840b6f90 con 0x7f74840d0fa0 2023-11-24T08:05:08.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.190+0000 7f7479ffb640 1 -- 172.21.15.64:0/1504938777 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f74840b6f90 con 0x7f74840d0fa0 2023-11-24T08:05:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.193+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f746806ae40 msgr2=0x7f746806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.193+0000 7f748a31a640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f746806ae40 0x7f746806d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7474006fd0 tx=0x7f7474008040 comp rx=0 tx=0).stop 2023-11-24T08:05:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.193+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 msgr2=0x7f74840b8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.193+0000 7f748a31a640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b8510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f748000c390 tx=0x7f748000c850 comp rx=0 tx=0).stop 2023-11-24T08:05:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.194+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 shutdown_connections 2023-11-24T08:05:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.194+0000 7f748a31a640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f746806ae40 0x7f746806d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.194+0000 7f748a31a640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f748415d3b0 0x7f74841f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.194+0000 7f748a31a640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f74840d1d90 0x7f74841f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.194+0000 7f748a31a640 1 --2- 172.21.15.64:0/1504938777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74840d0fa0 0x7f74840b8510 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.194+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 >> 172.21.15.64:0/1504938777 conn(0x7f74840d30a0 msgr2=0x7f74840b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.194+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 shutdown_connections 2023-11-24T08:05:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.194+0000 7f748a31a640 1 -- 172.21.15.64:0/1504938777 wait complete. 2023-11-24T08:05:08.216 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:05:08.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.442+0000 7f679c9fc640 1 Processor -- start 2023-11-24T08:05:08.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.442+0000 7f679c9fc640 1 -- start start 2023-11-24T08:05:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.444+0000 7f679c9fc640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f679815b350 0x7f67980abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.444+0000 7f679c9fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f679809cb30 0x7f67980b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.444+0000 7f679c9fc640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679815b730 con 0x7f67980adb60 2023-11-24T08:05:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.444+0000 7f679c9fc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67980ac740 con 0x7f679809cb30 2023-11-24T08:05:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.444+0000 7f679c9fc640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67980ac460 con 0x7f679815b350 2023-11-24T08:05:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.444+0000 7f678ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f679809cb30 0x7f67980b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.444+0000 7f678ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f679809cb30 0x7f67980b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48026/0 (socket says 172.21.15.64:48026) 2023-11-24T08:05:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.445+0000 7f678ffff640 1 -- 172.21.15.64:0/2475000675 learned_addr learned my addr 172.21.15.64:0/2475000675 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.445+0000 7f679659c640 1 --2- 172.21.15.64:0/2475000675 >> v2:172.21.15.91:3300/0 conn(0x7f679815b350 0x7f67980abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.445+0000 7f678ffff640 1 -- 172.21.15.64:0/2475000675 >> v2:172.21.15.91:3300/0 conn(0x7f679815b350 msgr2=0x7f67980abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.445+0000 7f678ffff640 1 --2- 172.21.15.64:0/2475000675 >> v2:172.21.15.91:3300/0 conn(0x7f679815b350 0x7f67980abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.445+0000 7f678ffff640 1 -- 172.21.15.64:0/2475000675 >> v1:172.21.15.91:6789/0 conn(0x7f67980adb60 legacy=0x7f67980adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.445+0000 7f678ffff640 1 -- 172.21.15.64:0/2475000675 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67980b2a50 con 0x7f679809cb30 2023-11-24T08:05:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.446+0000 7f678ffff640 1 --2- 172.21.15.64:0/2475000675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679809cb30 0x7f67980b3530 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f67980b5100 tx=0x7f678002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562dfbff713ff55a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f6795d9b640 1 -- 172.21.15.64:0/2475000675 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780031870 con 0x7f679809cb30 2023-11-24T08:05:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f6795d9b640 1 -- 172.21.15.64:0/2475000675 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67800319d0 con 0x7f679809cb30 2023-11-24T08:05:08.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f6795d9b640 1 -- 172.21.15.64:0/2475000675 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780031d60 con 0x7f679809cb30 2023-11-24T08:05:08.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f679c9fc640 1 -- 172.21.15.64:0/2475000675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679809cb30 msgr2=0x7f67980b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:08.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f679c9fc640 1 --2- 172.21.15.64:0/2475000675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679809cb30 0x7f67980b3530 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f67980b5100 tx=0x7f678002b910 comp rx=0 tx=0).stop 2023-11-24T08:05:08.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f679c9fc640 1 -- 172.21.15.64:0/2475000675 shutdown_connections 2023-11-24T08:05:08.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f679c9fc640 1 --2- 172.21.15.64:0/2475000675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679809cb30 0x7f67980b3530 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f679c9fc640 1 --2- 172.21.15.64:0/2475000675 >> v2:172.21.15.91:3300/0 conn(0x7f679815b350 0x7f67980abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f679c9fc640 1 -- 172.21.15.64:0/2475000675 >> 172.21.15.64:0/2475000675 conn(0x7f67980b0ba0 msgr2=0x7f67980b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:08.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.447+0000 7f679c9fc640 1 -- 172.21.15.64:0/2475000675 shutdown_connections 2023-11-24T08:05:08.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.448+0000 7f679c9fc640 1 -- 172.21.15.64:0/2475000675 wait complete. 2023-11-24T08:05:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.449+0000 7f679c9fc640 1 Processor -- start 2023-11-24T08:05:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.449+0000 7f679c9fc640 1 -- start start 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.450+0000 7f679c9fc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f679809cb30 0x7f67981576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.450+0000 7f679c9fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67980adb60 0x7f679813da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.450+0000 7f679c9fc640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f679815b350 0x7f679815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.450+0000 7f679c9fc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679815e4c0 con 0x7f67980adb60 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.450+0000 7f679c9fc640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679813dfd0 con 0x7f679815b350 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.450+0000 7f679c9fc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679815d840 con 0x7f679809cb30 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f678ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67980adb60 0x7f679813da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f678ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67980adb60 0x7f679813da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48036/0 (socket says 172.21.15.64:48036) 2023-11-24T08:05:08.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f678ffff640 1 -- 172.21.15.64:0/3004406627 learned_addr learned my addr 172.21.15.64:0/3004406627 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f6796d9d640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f679815b350 0x7f679815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f679659c640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f679809cb30 0x7f67981576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f678ffff640 1 -- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f679809cb30 msgr2=0x7f67981576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f678ffff640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f679809cb30 0x7f67981576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f678ffff640 1 -- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f679815b350 msgr2=0x7f679815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f678ffff640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f679815b350 0x7f679815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f678ffff640 1 -- 172.21.15.64:0/3004406627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67800094d0 con 0x7f67980adb60 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f679659c640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f679809cb30 0x7f67981576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.451+0000 7f6796d9d640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f679815b350 0x7f679815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.452+0000 7f678ffff640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67980adb60 0x7f679813da90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f678002b950 tx=0x7f67800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.452+0000 7f678effd640 1 -- 172.21.15.64:0/3004406627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678003abe0 con 0x7f67980adb60 2023-11-24T08:05:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.452+0000 7f678effd640 1 -- 172.21.15.64:0/3004406627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678003ad40 con 0x7f67980adb60 2023-11-24T08:05:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.452+0000 7f678effd640 1 -- 172.21.15.64:0/3004406627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780044610 con 0x7f67980adb60 2023-11-24T08:05:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.452+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679815db20 con 0x7f67980adb60 2023-11-24T08:05:08.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.452+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679815e0e0 con 0x7f67980adb60 2023-11-24T08:05:08.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.453+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679809d200 con 0x7f67980adb60 2023-11-24T08:05:08.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.454+0000 7f678effd640 1 -- 172.21.15.64:0/3004406627 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6780044770 con 0x7f67980adb60 2023-11-24T08:05:08.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.456+0000 7f678effd640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677806ae40 0x7f677806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:08.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.456+0000 7f678effd640 1 -- 172.21.15.64:0/3004406627 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f678002d040 con 0x7f67980adb60 2023-11-24T08:05:08.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.456+0000 7f679659c640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677806ae40 0x7f677806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:08.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.457+0000 7f679659c640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677806ae40 0x7f677806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6788006fd0 tx=0x7f6788008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:08.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.459+0000 7f678effd640 1 -- 172.21.15.64:0/3004406627 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6780076b10 con 0x7f67980adb60 2023-11-24T08:05:08.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:08.655+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f679814f690 con 0x7f67980adb60 2023-11-24T08:05:09.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.197+0000 7f678effd640 1 -- 172.21.15.64:0/3004406627 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6780035020 con 0x7f67980adb60 2023-11-24T08:05:09.199 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:05:09.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.200+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677806ae40 msgr2=0x7f677806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:09.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.200+0000 7f679c9fc640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677806ae40 0x7f677806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6788006fd0 tx=0x7f6788008040 comp rx=0 tx=0).stop 2023-11-24T08:05:09.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67980adb60 msgr2=0x7f679813da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:09.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67980adb60 0x7f679813da90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f678002b950 tx=0x7f67800046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 shutdown_connections 2023-11-24T08:05:09.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f677806ae40 0x7f677806d300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f679815b350 0x7f679815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67980adb60 0x7f679813da90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 --2- 172.21.15.64:0/3004406627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f679809cb30 0x7f67981576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 >> 172.21.15.64:0/3004406627 conn(0x7f67980b0ba0 msgr2=0x7f6798153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:09.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 shutdown_connections 2023-11-24T08:05:09.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.201+0000 7f679c9fc640 1 -- 172.21.15.64:0/3004406627 wait complete. 2023-11-24T08:05:09.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:05:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.458+0000 7f879bfff640 1 Processor -- start 2023-11-24T08:05:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.459+0000 7f879bfff640 1 -- start start 2023-11-24T08:05:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.460+0000 7f879bfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f879c15d3b0 0x7f879c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.460+0000 7f879bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c0d1d90 0x7f879c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.460+0000 7f879bfff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c0c6bf0 con 0x7f879c0d0fa0 2023-11-24T08:05:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.460+0000 7f879bfff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879c0d1430 con 0x7f879c15d3b0 2023-11-24T08:05:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.460+0000 7f879bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0cedc0 con 0x7f879c0d1d90 2023-11-24T08:05:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.461+0000 7f879a7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f879c0d0fa0 0x7f879c0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46332/0 (socket says 172.21.15.64:46332) 2023-11-24T08:05:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.461+0000 7f879a7fc640 1 -- 172.21.15.64:0/51940977 learned_addr learned my addr 172.21.15.64:0/51940977 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.461+0000 7f879affd640 1 --2- 172.21.15.64:0/51940977 >> v2:172.21.15.91:3300/0 conn(0x7f879c15d3b0 0x7f879c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.461+0000 7f879b7fe640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c0d1d90 0x7f879c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.461+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364877431 0 0) 0x7f879c0c6bf0 con 0x7f879c0d0fa0 2023-11-24T08:05:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.461+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8784003520 con 0x7f879c0d0fa0 2023-11-24T08:05:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483134545 0 0) 0x7f8784003520 con 0x7f879c0d0fa0 2023-11-24T08:05:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c0d1d90 msgr2=0x7f879c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c0d1d90 0x7f879c0b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 >> v2:172.21.15.91:3300/0 conn(0x7f879c15d3b0 msgr2=0x7f879c0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 --2- 172.21.15.64:0/51940977 >> v2:172.21.15.91:3300/0 conn(0x7f879c15d3b0 0x7f879c0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0c6e40 con 0x7f879c0d0fa0 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f8790001670 con 0x7f879c0d0fa0 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 >> v1:172.21.15.64:6789/0 conn(0x7f879c0d0fa0 legacy=0x7f879c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87840039e0 0x7f8784003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c0d1d90 0x7f87840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8784004d00 0x7f8784005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0d1430 con 0x7f8784004d00 2023-11-24T08:05:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0cedc0 con 0x7f87840039e0 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878400ba60 con 0x7f879c0d1d90 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.462+0000 7f879b7fe640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8784004d00 0x7f8784005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.463+0000 7f879affd640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87840039e0 0x7f8784003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.463+0000 7f879a7fc640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c0d1d90 0x7f87840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.463+0000 7f879b7fe640 1 -- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c0d1d90 msgr2=0x7f87840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.463+0000 7f879b7fe640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c0d1d90 0x7f87840047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.463+0000 7f879b7fe640 1 -- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87840039e0 msgr2=0x7f8784003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.463+0000 7f879b7fe640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87840039e0 0x7f8784003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.463+0000 7f879b7fe640 1 -- 172.21.15.64:0/51940977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784003520 con 0x7f8784004d00 2023-11-24T08:05:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.463+0000 7f879affd640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87840039e0 0x7f8784003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:09.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.464+0000 7f879b7fe640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8784004d00 0x7f8784005180 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f879c0ca120 tx=0x7f87840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d2aac7684a7eb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:09.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.464+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879402cb90 con 0x7f8784004d00 2023-11-24T08:05:09.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.464+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879400f040 con 0x7f8784004d00 2023-11-24T08:05:09.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.464+0000 7f8799ffb640 1 -- 172.21.15.64:0/51940977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794034440 con 0x7f8784004d00 2023-11-24T08:05:09.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.465+0000 7f879bfff640 1 -- 172.21.15.64:0/51940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8784004d00 msgr2=0x7f8784005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:09.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.465+0000 7f879bfff640 1 --2- 172.21.15.64:0/51940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8784004d00 0x7f8784005180 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f879c0ca120 tx=0x7f87840057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.465+0000 7f879b7fe640 1 -- 172.21.15.64:0/51940977 reap_dead start 2023-11-24T08:05:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.465+0000 7f879bfff640 1 -- 172.21.15.64:0/51940977 shutdown_connections 2023-11-24T08:05:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.465+0000 7f879bfff640 1 -- 172.21.15.64:0/51940977 >> 172.21.15.64:0/51940977 conn(0x7f879c0d30a0 msgr2=0x7f879c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.465+0000 7f879bfff640 1 -- 172.21.15.64:0/51940977 shutdown_connections 2023-11-24T08:05:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.465+0000 7f879bfff640 1 -- 172.21.15.64:0/51940977 wait complete. 2023-11-24T08:05:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.467+0000 7f879bfff640 1 Processor -- start 2023-11-24T08:05:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.467+0000 7f879bfff640 1 -- start start 2023-11-24T08:05:09.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c153bc0 0x7f879c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c153fa0 0x7f879c0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f879c0ca6b0 0x7f879c0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c15df40 con 0x7f879c153fa0 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c15d910 con 0x7f879c0ca6b0 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c1546b0 con 0x7f879c153bc0 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c153fa0 0x7f879c0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c153fa0 0x7f879c0ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48048/0 (socket says 172.21.15.64:48048) 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.468+0000 7f879a7fc640 1 -- 172.21.15.64:0/3995955776 learned_addr learned my addr 172.21.15.64:0/3995955776 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879affd640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c153bc0 0x7f879c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879a7fc640 1 -- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c153bc0 msgr2=0x7f879c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879b7fe640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f879c0ca6b0 0x7f879c0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879a7fc640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c153bc0 0x7f879c0ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879a7fc640 1 -- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f879c0ca6b0 msgr2=0x7f879c0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879a7fc640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f879c0ca6b0 0x7f879c0cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879a7fc640 1 -- 172.21.15.64:0/3995955776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0b8040 con 0x7f879c153fa0 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879affd640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c153bc0 0x7f879c0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879a7fc640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c153fa0 0x7f879c0ca170 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f879000d7a0 tx=0x7f879000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f87837fe640 1 -- 172.21.15.64:0/3995955776 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790004cb0 con 0x7f879c153fa0 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f87837fe640 1 -- 172.21.15.64:0/3995955776 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8790004e10 con 0x7f879c153fa0 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f87837fe640 1 -- 172.21.15.64:0/3995955776 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790007310 con 0x7f879c153fa0 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.469+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c0b8320 con 0x7f879c153fa0 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.470+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c0b6ea0 con 0x7f879c153fa0 2023-11-24T08:05:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.471+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c15e080 con 0x7f879c153fa0 2023-11-24T08:05:09.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.472+0000 7f87837fe640 1 -- 172.21.15.64:0/3995955776 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8790007470 con 0x7f879c153fa0 2023-11-24T08:05:09.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.473+0000 7f87837fe640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f878806ae40 0x7f878806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:09.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.473+0000 7f87837fe640 1 -- 172.21.15.64:0/3995955776 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f8790088d30 con 0x7f879c153fa0 2023-11-24T08:05:09.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.473+0000 7f879affd640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f878806ae40 0x7f878806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:09.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.474+0000 7f879affd640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f878806ae40 0x7f878806d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f878c009b20 tx=0x7f878c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:09.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.477+0000 7f87837fe640 1 -- 172.21.15.64:0/3995955776 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f879c15e080 con 0x7f879c153fa0 2023-11-24T08:05:09.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:09.670+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f879c15e080 con 0x7f879c153fa0 2023-11-24T08:05:10.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.216+0000 7f87837fe640 1 -- 172.21.15.64:0/3995955776 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f879c15e080 con 0x7f879c153fa0 2023-11-24T08:05:10.218 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-11-24T08:05:10.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f878806ae40 msgr2=0x7f878806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:10.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f878806ae40 0x7f878806d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f878c009b20 tx=0x7f878c008040 comp rx=0 tx=0).stop 2023-11-24T08:05:10.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c153fa0 msgr2=0x7f879c0ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:10.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c153fa0 0x7f879c0ca170 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f879000d7a0 tx=0x7f879000dc60 comp rx=0 tx=0).stop 2023-11-24T08:05:10.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 shutdown_connections 2023-11-24T08:05:10.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f878806ae40 0x7f878806d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f879c0ca6b0 0x7f879c0cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f879c153fa0 0x7f879c0ca170 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 --2- 172.21.15.64:0/3995955776 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f879c153bc0 0x7f879c0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.220+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 >> 172.21.15.64:0/3995955776 conn(0x7f879c0d30a0 msgr2=0x7f879c06af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:10.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.221+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 shutdown_connections 2023-11-24T08:05:10.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.221+0000 7f879bfff640 1 -- 172.21.15.64:0/3995955776 wait complete. 2023-11-24T08:05:10.248 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:05:10.248 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:05:10.248 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:05:10.248 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:10.248 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:05:10.248 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:05:10.249 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:05:10.249 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:10.249 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:05:10.249 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:05:10.249 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:05:10.250 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:10.250 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:05:10.250 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:05:10.250 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:05:10.251 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:10.268 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:05:10.268 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:05:10.270 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:05:10.271 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:05:10.275 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:05:10.276 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:05:10.279 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:05:10.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:05:10.319 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:05:10.319 DEBUG:teuthology.parallel:result is None 2023-11-24T08:05:10.326 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:05:10.326 DEBUG:teuthology.parallel:result is None 2023-11-24T08:05:10.328 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:05:10.328 DEBUG:teuthology.parallel:result is None 2023-11-24T08:05:10.335 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:05:10.336 DEBUG:teuthology.parallel:result is None 2023-11-24T08:05:10.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:05:10.480 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2023-11-24T08:05:10.483 INFO:tasks.ceph.mds.d.smithi091.stdout:starting mds.d at 2023-11-24T08:05:10.492 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:05:10.496 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T08:05:10.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.531+0000 7f1421a5c640 1 Processor -- start 2023-11-24T08:05:10.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.532+0000 7f1421a5c640 1 -- start start 2023-11-24T08:05:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f1421a5c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f141c0b3b60 0x7f141c06d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f1421a5c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c06da60 0x7f141c071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f1421a5c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0ae8a0 con 0x7f141c0715d0 2023-11-24T08:05:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f1421a5c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141c06dee0 con 0x7f141c0b3b60 2023-11-24T08:05:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f1421a5c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0ad400 con 0x7f141c06da60 2023-11-24T08:05:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f141b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f141c0715d0 0x7f141c161000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46354/0 (socket says 172.21.15.64:46354) 2023-11-24T08:05:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f141b7fe640 1 -- 172.21.15.64:0/4038892397 learned_addr learned my addr 172.21.15.64:0/4038892397 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:10.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f141affd640 1 --2- 172.21.15.64:0/4038892397 >> v2:172.21.15.64:3300/0 conn(0x7f141c0b3b60 0x7f141c06d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:10.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.533+0000 7f141a7fc640 1 --2- 172.21.15.64:0/4038892397 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c06da60 0x7f141c071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:10.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.534+0000 7f141affd640 1 -- 172.21.15.64:0/4038892397 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c06da60 msgr2=0x7f141c071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:10.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.534+0000 7f141affd640 1 --2- 172.21.15.64:0/4038892397 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c06da60 0x7f141c071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.534+0000 7f141affd640 1 -- 172.21.15.64:0/4038892397 >> v1:172.21.15.64:6789/0 conn(0x7f141c0715d0 legacy=0x7f141c161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:10.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.534+0000 7f141affd640 1 -- 172.21.15.64:0/4038892397 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0acb50 con 0x7f141c0b3b60 2023-11-24T08:05:10.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.534+0000 7f1419ffb640 1 -- 172.21.15.64:0/4038892397 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648960981 0 0) 0x7f141c0ae8a0 con 0x7f141c0715d0 2023-11-24T08:05:10.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.535+0000 7f141affd640 1 --2- 172.21.15.64:0/4038892397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c0b3b60 0x7f141c06d520 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f140c0097a0 tx=0x7f140c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdf7ddad404444f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:10.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.535+0000 7f1419ffb640 1 -- 172.21.15.64:0/4038892397 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c031830 con 0x7f141c0b3b60 2023-11-24T08:05:10.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.535+0000 7f1419ffb640 1 -- 172.21.15.64:0/4038892397 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c031990 con 0x7f141c0b3b60 2023-11-24T08:05:10.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.535+0000 7f1419ffb640 1 -- 172.21.15.64:0/4038892397 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c031d20 con 0x7f141c0b3b60 2023-11-24T08:05:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.536+0000 7f1421a5c640 1 -- 172.21.15.64:0/4038892397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c0b3b60 msgr2=0x7f141c06d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.536+0000 7f1421a5c640 1 --2- 172.21.15.64:0/4038892397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c0b3b60 0x7f141c06d520 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f140c0097a0 tx=0x7f140c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.536+0000 7f1421a5c640 1 -- 172.21.15.64:0/4038892397 shutdown_connections 2023-11-24T08:05:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.536+0000 7f1421a5c640 1 --2- 172.21.15.64:0/4038892397 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c06da60 0x7f141c071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.536+0000 7f1421a5c640 1 --2- 172.21.15.64:0/4038892397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c0b3b60 0x7f141c06d520 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.536+0000 7f1421a5c640 1 -- 172.21.15.64:0/4038892397 >> 172.21.15.64:0/4038892397 conn(0x7f141c0b0d80 msgr2=0x7f141c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.536+0000 7f1421a5c640 1 -- 172.21.15.64:0/4038892397 shutdown_connections 2023-11-24T08:05:10.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.536+0000 7f1421a5c640 1 -- 172.21.15.64:0/4038892397 wait complete. 2023-11-24T08:05:10.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.538+0000 7f1421a5c640 1 Processor -- start 2023-11-24T08:05:10.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.538+0000 7f1421a5c640 1 -- start start 2023-11-24T08:05:10.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f1421a5c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c06da60 0x7f141c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:10.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f1421a5c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c0715d0 0x7f141c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:10.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f1421a5c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f141c0c9880 0x7f141c0ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f1421a5c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0cbae0 con 0x7f141c06da60 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f1421a5c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0cbdc0 con 0x7f141c0c9880 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f1421a5c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0cc0a0 con 0x7f141c0715d0 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f141affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c06da60 0x7f141c1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f141affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c06da60 0x7f141c1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48088/0 (socket says 172.21.15.64:48088) 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.539+0000 7f141affd640 1 -- 172.21.15.64:0/1314636275 learned_addr learned my addr 172.21.15.64:0/1314636275 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f141a7fc640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c0715d0 0x7f141c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f141affd640 1 -- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c0715d0 msgr2=0x7f141c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f141affd640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c0715d0 0x7f141c0c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f141affd640 1 -- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f141c0c9880 msgr2=0x7f141c0ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f141affd640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f141c0c9880 0x7f141c0ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f141affd640 1 -- 172.21.15.64:0/1314636275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c009400 con 0x7f141c06da60 2023-11-24T08:05:10.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f141affd640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c06da60 0x7f141c1582e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f140c0098c0 tx=0x7f140c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f1403fff640 1 -- 172.21.15.64:0/1314636275 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0317b0 con 0x7f141c06da60 2023-11-24T08:05:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f1403fff640 1 -- 172.21.15.64:0/1314636275 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c03b9f0 con 0x7f141c06da60 2023-11-24T08:05:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f1403fff640 1 -- 172.21.15.64:0/1314636275 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c03bda0 con 0x7f141c06da60 2023-11-24T08:05:10.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c0cc380 con 0x7f141c06da60 2023-11-24T08:05:10.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.540+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c14c720 con 0x7f141c06da60 2023-11-24T08:05:10.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.542+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c0cc8a0 con 0x7f141c06da60 2023-11-24T08:05:10.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.543+0000 7f1403fff640 1 -- 172.21.15.64:0/1314636275 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f140c04f3a0 con 0x7f141c06da60 2023-11-24T08:05:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.543+0000 7f1403fff640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f140806ae40 0x7f140806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.543+0000 7f1403fff640 1 -- 172.21.15.64:0/1314636275 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f140c0ab920 con 0x7f141c06da60 2023-11-24T08:05:10.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.544+0000 7f141a7fc640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f140806ae40 0x7f140806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:10.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.545+0000 7f141a7fc640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f140806ae40 0x7f140806d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f141c15ac70 tx=0x7f1410008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:10.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.548+0000 7f1403fff640 1 -- 172.21.15.64:0/1314636275 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f141c0cc8a0 con 0x7f141c06da60 2023-11-24T08:05:10.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:10.753+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f141c0cc8a0 con 0x7f141c06da60 2023-11-24T08:05:12.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.005+0000 7f1403fff640 1 -- 172.21.15.64:0/1314636275 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f141c0cc8a0 con 0x7f141c06da60 2023-11-24T08:05:12.008 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.009+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f140806ae40 msgr2=0x7f140806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.009+0000 7f1421a5c640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f140806ae40 0x7f140806d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f141c15ac70 tx=0x7f1410008040 comp rx=0 tx=0).stop 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.009+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c06da60 msgr2=0x7f141c1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.009+0000 7f1421a5c640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c06da60 0x7f141c1582e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f140c0098c0 tx=0x7f140c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.010+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 shutdown_connections 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.010+0000 7f1421a5c640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f140806ae40 0x7f140806d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.010+0000 7f1421a5c640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f141c0c9880 0x7f141c0ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.010+0000 7f1421a5c640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f141c0715d0 0x7f141c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.010+0000 7f1421a5c640 1 --2- 172.21.15.64:0/1314636275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f141c06da60 0x7f141c1582e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.010+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 >> 172.21.15.64:0/1314636275 conn(0x7f141c0b0d80 msgr2=0x7f141c0ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:12.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.010+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 shutdown_connections 2023-11-24T08:05:12.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.010+0000 7f1421a5c640 1 -- 172.21.15.64:0/1314636275 wait complete. 2023-11-24T08:05:12.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:05:12.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.247+0000 7fb483883640 1 Processor -- start 2023-11-24T08:05:12.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.248+0000 7fb483883640 1 -- start start 2023-11-24T08:05:12.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.249+0000 7fb483883640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb47c0ad720 0x7fb47c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.249+0000 7fb483883640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c0abff0 0x7fb47c0b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.249+0000 7fb483883640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47c15c360 con 0x7fb47c06d520 2023-11-24T08:05:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.249+0000 7fb483883640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47c0b41a0 con 0x7fb47c0ad720 2023-11-24T08:05:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.249+0000 7fb483883640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c15ba30 con 0x7fb47c0abff0 2023-11-24T08:05:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.250+0000 7fb480e1e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb47c0ad720 0x7fb47c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.250+0000 7fb480e1e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb47c0ad720 0x7fb47c0b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48098/0 (socket says 172.21.15.64:48098) 2023-11-24T08:05:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.250+0000 7fb480e1e640 1 -- 172.21.15.64:0/3645988099 learned_addr learned my addr 172.21.15.64:0/3645988099 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.250+0000 7fb481e20640 1 --2- 172.21.15.64:0/3645988099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c0abff0 0x7fb47c0b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.250+0000 7fb480e1e640 1 -- 172.21.15.64:0/3645988099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c0abff0 msgr2=0x7fb47c0b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.250+0000 7fb480e1e640 1 --2- 172.21.15.64:0/3645988099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c0abff0 0x7fb47c0b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.250+0000 7fb480e1e640 1 -- 172.21.15.64:0/3645988099 >> v1:172.21.15.91:6789/0 conn(0x7fb47c06d520 legacy=0x7fb47c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.250+0000 7fb480e1e640 1 -- 172.21.15.64:0/3645988099 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c15bd10 con 0x7fb47c0ad720 2023-11-24T08:05:12.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.251+0000 7fb480e1e640 1 --2- 172.21.15.64:0/3645988099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0ad720 0x7fb47c0b2080 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009870 tx=0x7fb46c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69be66107f45a6fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.252+0000 7fb473fff640 1 -- 172.21.15.64:0/3645988099 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c030840 con 0x7fb47c0ad720 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.252+0000 7fb473fff640 1 -- 172.21.15.64:0/3645988099 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c0309a0 con 0x7fb47c0ad720 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.252+0000 7fb473fff640 1 -- 172.21.15.64:0/3645988099 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c030d30 con 0x7fb47c0ad720 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.252+0000 7fb483883640 1 -- 172.21.15.64:0/3645988099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0ad720 msgr2=0x7fb47c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.252+0000 7fb483883640 1 --2- 172.21.15.64:0/3645988099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0ad720 0x7fb47c0b2080 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009870 tx=0x7fb46c02b980 comp rx=0 tx=0).stop 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.253+0000 7fb483883640 1 -- 172.21.15.64:0/3645988099 shutdown_connections 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.253+0000 7fb483883640 1 --2- 172.21.15.64:0/3645988099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c0abff0 0x7fb47c0b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.253+0000 7fb483883640 1 --2- 172.21.15.64:0/3645988099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0ad720 0x7fb47c0b2080 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.253+0000 7fb483883640 1 -- 172.21.15.64:0/3645988099 >> 172.21.15.64:0/3645988099 conn(0x7fb47c0b0d80 msgr2=0x7fb47c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.253+0000 7fb483883640 1 -- 172.21.15.64:0/3645988099 shutdown_connections 2023-11-24T08:05:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.253+0000 7fb483883640 1 -- 172.21.15.64:0/3645988099 wait complete. 2023-11-24T08:05:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.254+0000 7fb483883640 1 Processor -- start 2023-11-24T08:05:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.255+0000 7fb483883640 1 -- start start 2023-11-24T08:05:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb483883640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c06d520 0x7fb47c155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb483883640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0abff0 0x7fb47c147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb483883640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb47c0cdcb0 0x7fb47c071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb483883640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0c9320 con 0x7fb47c0abff0 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb483883640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c155aa0 con 0x7fb47c0cdcb0 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb483883640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0c8400 con 0x7fb47c06d520 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb480e1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0abff0 0x7fb47c147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb480e1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0abff0 0x7fb47c147180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48112/0 (socket says 172.21.15.64:48112) 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb480e1e640 1 -- 172.21.15.64:0/1326287498 learned_addr learned my addr 172.21.15.64:0/1326287498 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb48161f640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c06d520 0x7fb47c155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb480e1e640 1 -- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c06d520 msgr2=0x7fb47c155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb480e1e640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c06d520 0x7fb47c155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb480e1e640 1 -- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb47c0cdcb0 msgr2=0x7fb47c071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb481e20640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb47c0cdcb0 0x7fb47c071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb480e1e640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb47c0cdcb0 0x7fb47c071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb480e1e640 1 -- 172.21.15.64:0/1326287498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0094d0 con 0x7fb47c0abff0 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.256+0000 7fb481e20640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb47c0cdcb0 0x7fb47c071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.257+0000 7fb480e1e640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0abff0 0x7fb47c147180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0090b0 tx=0x7fb46c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.257+0000 7fb471ffb640 1 -- 172.21.15.64:0/1326287498 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c039c90 con 0x7fb47c0abff0 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.257+0000 7fb471ffb640 1 -- 172.21.15.64:0/1326287498 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c039e10 con 0x7fb47c0abff0 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.257+0000 7fb471ffb640 1 -- 172.21.15.64:0/1326287498 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c004260 con 0x7fb47c0abff0 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.257+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47c0c86a0 con 0x7fb47c0abff0 2023-11-24T08:05:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.257+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47c0c8c60 con 0x7fb47c0abff0 2023-11-24T08:05:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.258+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47c074770 con 0x7fb47c0abff0 2023-11-24T08:05:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.259+0000 7fb471ffb640 1 -- 172.21.15.64:0/1326287498 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb46c0043c0 con 0x7fb47c0abff0 2023-11-24T08:05:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.260+0000 7fb471ffb640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb46006ae40 0x7fb46006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.260+0000 7fb471ffb640 1 -- 172.21.15.64:0/1326287498 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb46c03e070 con 0x7fb47c0abff0 2023-11-24T08:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.261+0000 7fb48161f640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb46006ae40 0x7fb46006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.262+0000 7fb48161f640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb46006ae40 0x7fb46006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb478000f60 tx=0x7fb478009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.265+0000 7fb471ffb640 1 -- 172.21.15.64:0/1326287498 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb46c076b70 con 0x7fb47c0abff0 2023-11-24T08:05:12.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.526+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb47c15b6b0 con 0x7fb47c0abff0 2023-11-24T08:05:12.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.527+0000 7fb471ffb640 1 -- 172.21.15.64:0/1326287498 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fb46c07a190 con 0x7fb47c0abff0 2023-11-24T08:05:12.529 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:12.529 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllV/IJxAAMEWwBM8H9uSLw5Bd93bScA==", 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBl35hoEBAADvW80b1cRhhgBRkYgNaftg==", 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBlZ1rGORAAjjOrPcL663guE4Q9MKA+6g==", 2023-11-24T08:05:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlqV+1IRAAaLczF6+eDIuMM6X5Jgni/A==", 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:05:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCuWGBlsrGpIBAAVTJlhEtf7rPd1J4wUZRWTg==", 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCxWGBl+u/pLBAAwh85K8evI69aY9Nij5dBRQ==", 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC1WGBl1kGECxAAtoQ9UI319UpAU7OwZehzew==", 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:05:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC4WGBlSlE2JRAAjPT1020nD4Dnrw1hrB2bKQ==", 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-11-24T08:05:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC/WGBlrRSGKRAAaiWZZvvd7O8PuipXF1oAUQ==", 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDDWGBls+mkBxAAJZXdXFq1+8QC5dBE14+YJA==", 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDGWGBl71+WIBAA/NNCMwpi0ycNJvXZFHKuIA==", 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJWGBl11BPKxAAnXhK4EqlIyvbHCpHzPCDUg==", 2023-11-24T08:05:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA==", 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:05:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlsyr3DBAASDVvIWUkg06i7YARod5spw==", 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlhsnfABAA5sZEDm1Uyin3dIOEoMX+gg==", 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlT9iOFxAAiupeG8QnYzYA8pJYftfxNA==", 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCnWGBlKY7iFBAAmkZ+pYWFmZ7aPH06BiE0+w==", 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:05:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mds", 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDSWGBlwijoNhAAelRkwAr6tC8zvnz8c0WK+w==", 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDSWGBl6zLoNhAAxGxCa9F2azeIr0hjQaj2ew==", 2023-11-24T08:05:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-osd", 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDSWGBlBz7oNhAA5ItKKFFW6DCRGZF//ArppA==", 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDSWGBlF0foNhAAaEB/H+sXBe5jq3IfVRR9YQ==", 2023-11-24T08:05:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDSWGBlUFDoNhAAqW+uVrnzzeikMIkS2UhBtA==", 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T08:05:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDSWGBljFnoNhAAD80KgH9rXyxeeblH9/bsDA==", 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllqymGhAA+wKOSJeuBn60R4gZJ3edoA==", 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBloKAEDhAAFSBzeCcgrTtmsfXrxKqCRQ==", 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T08:05:12.546 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb46006ae40 msgr2=0x7fb46006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb46006ae40 0x7fb46006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb478000f60 tx=0x7fb478009040 comp rx=0 tx=0).stop 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0abff0 msgr2=0x7fb47c147180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0abff0 0x7fb47c147180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0090b0 tx=0x7fb46c004730 comp rx=0 tx=0).stop 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 shutdown_connections 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb46006ae40 0x7fb46006d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb47c0cdcb0 0x7fb47c071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb47c0abff0 0x7fb47c147180 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 --2- 172.21.15.64:0/1326287498 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb47c06d520 0x7fb47c155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 >> 172.21.15.64:0/1326287498 conn(0x7fb47c0b0d80 msgr2=0x7fb47c15ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:12.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 shutdown_connections 2023-11-24T08:05:12.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.530+0000 7fb483883640 1 -- 172.21.15.64:0/1326287498 wait complete. 2023-11-24T08:05:12.551 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T08:05:12.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.777+0000 7fd24f0d1640 1 Processor -- start 2023-11-24T08:05:12.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.778+0000 7fd24f0d1640 1 -- start start 2023-11-24T08:05:12.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.779+0000 7fd24f0d1640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd248141e80 0x7fd248071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.779+0000 7fd24f0d1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd248071650 0x7fd24806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.779+0000 7fd24f0d1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24806cae0 con 0x7fd24815ed20 2023-11-24T08:05:12.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.779+0000 7fd24f0d1640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2481422e0 con 0x7fd248141e80 2023-11-24T08:05:12.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.779+0000 7fd24f0d1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd248071e40 con 0x7fd248071650 2023-11-24T08:05:12.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.779+0000 7fd24e0cf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd24815ed20 0x7fd24815f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46358/0 (socket says 172.21.15.64:46358) 2023-11-24T08:05:12.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.779+0000 7fd24e0cf640 1 -- 172.21.15.64:0/2459953710 learned_addr learned my addr 172.21.15.64:0/2459953710 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724467405 0 0) 0x7fd24806cae0 con 0x7fd24815ed20 2023-11-24T08:05:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24e8d0640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd248071650 0x7fd24806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd234003520 con 0x7fd24815ed20 2023-11-24T08:05:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d8ce640 1 --2- 172.21.15.64:0/2459953710 >> v2:172.21.15.91:3300/0 conn(0x7fd248141e80 0x7fd248071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041709406 0 0) 0x7fd234003520 con 0x7fd24815ed20 2023-11-24T08:05:12.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd248071650 msgr2=0x7fd24806cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd248071650 0x7fd24806cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 >> v2:172.21.15.91:3300/0 conn(0x7fd248141e80 msgr2=0x7fd248071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 --2- 172.21.15.64:0/2459953710 >> v2:172.21.15.91:3300/0 conn(0x7fd248141e80 0x7fd248071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24806c1b0 con 0x7fd24815ed20 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fd244001670 con 0x7fd24815ed20 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d8ce640 1 --2- 172.21.15.64:0/2459953710 >> v2:172.21.15.91:3300/0 conn(0x7fd248141e80 0x7fd248071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 >> v1:172.21.15.64:6789/0 conn(0x7fd24815ed20 legacy=0x7fd24815f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2340039e0 0x7fd234003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd248071650 0x7fd2340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd234004d00 0x7fd234005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd248071e40 con 0x7fd2340039e0 2023-11-24T08:05:12.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23400b9b0 con 0x7fd234004d00 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.780+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23400bb70 con 0x7fd248071650 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24e0cf640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2340039e0 0x7fd234003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24d8ce640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd248071650 0x7fd2340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24e8d0640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd234004d00 0x7fd234005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24e0cf640 1 -- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd248071650 msgr2=0x7fd2340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24e0cf640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd248071650 0x7fd2340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24e0cf640 1 -- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd234004d00 msgr2=0x7fd234005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24e0cf640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd234004d00 0x7fd234005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24e0cf640 1 -- 172.21.15.64:0/2459953710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd234003520 con 0x7fd2340039e0 2023-11-24T08:05:12.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.781+0000 7fd24e8d0640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd234004d00 0x7fd234005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.782+0000 7fd24e0cf640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2340039e0 0x7fd234003e00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd24400c240 tx=0x7fd2340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b403cce3faf4abbf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.782+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24402cb00 con 0x7fd2340039e0 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.782+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24402cc60 con 0x7fd2340039e0 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.782+0000 7fd24d0cd640 1 -- 172.21.15.64:0/2459953710 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244005af0 con 0x7fd2340039e0 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.783+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2459953710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2340039e0 msgr2=0x7fd234003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.783+0000 7fd24f0d1640 1 --2- 172.21.15.64:0/2459953710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2340039e0 0x7fd234003e00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd24400c240 tx=0x7fd2340057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.783+0000 7fd24e8d0640 1 -- 172.21.15.64:0/2459953710 reap_dead start 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.783+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2459953710 shutdown_connections 2023-11-24T08:05:12.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.783+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2459953710 >> 172.21.15.64:0/2459953710 conn(0x7fd2480b0840 msgr2=0x7fd2480b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:12.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.783+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2459953710 shutdown_connections 2023-11-24T08:05:12.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.783+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2459953710 wait complete. 2023-11-24T08:05:12.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.785+0000 7fd24f0d1640 1 Processor -- start 2023-11-24T08:05:12.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.785+0000 7fd24f0d1640 1 -- start start 2023-11-24T08:05:12.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.786+0000 7fd24f0d1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2340039e0 0x7fd248156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.786+0000 7fd24f0d1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2480a9470 0x7fd2480ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.786+0000 7fd24f0d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2480aec60 0x7fd2480a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.786+0000 7fd24f0d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480af110 con 0x7fd2480aec60 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.786+0000 7fd24f0d1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd248140aa0 con 0x7fd2480a9470 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.786+0000 7fd24f0d1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480a8460 con 0x7fd2340039e0 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2480aec60 0x7fd2480a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2480aec60 0x7fd2480a98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48132/0 (socket says 172.21.15.64:48132) 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 -- 172.21.15.64:0/2829865451 learned_addr learned my addr 172.21.15.64:0/2829865451 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e0cf640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2340039e0 0x7fd248156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24d8ce640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2480a9470 0x7fd2480ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 -- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2340039e0 msgr2=0x7fd248156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2340039e0 0x7fd248156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 -- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2480a9470 msgr2=0x7fd2480ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2480a9470 0x7fd2480ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 -- 172.21.15.64:0/2829865451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd248156fb0 con 0x7fd2480aec60 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.787+0000 7fd24e8d0640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2480aec60 0x7fd2480a98d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd248072210 tx=0x7fd24000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.788+0000 7fd23effd640 1 -- 172.21.15.64:0/2829865451 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240017400 con 0x7fd2480aec60 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.788+0000 7fd23effd640 1 -- 172.21.15.64:0/2829865451 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd240011040 con 0x7fd2480aec60 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.788+0000 7fd23effd640 1 -- 172.21.15.64:0/2829865451 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240017780 con 0x7fd2480aec60 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.788+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2480a86b0 con 0x7fd2480aec60 2023-11-24T08:05:12.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.788+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2480a8c70 con 0x7fd2480aec60 2023-11-24T08:05:12.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.789+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd248067d10 con 0x7fd2480aec60 2023-11-24T08:05:12.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.790+0000 7fd23effd640 1 -- 172.21.15.64:0/2829865451 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2400203b0 con 0x7fd2480aec60 2023-11-24T08:05:12.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.791+0000 7fd23effd640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd22c06ae70 0x7fd22c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:12.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.791+0000 7fd23effd640 1 -- 172.21.15.64:0/2829865451 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd240089690 con 0x7fd2480aec60 2023-11-24T08:05:12.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.792+0000 7fd24e0cf640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd22c06ae70 0x7fd22c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:12.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.793+0000 7fd24e0cf640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd22c06ae70 0x7fd22c06d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd244034000 tx=0x7fd244007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:12.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:12.796+0000 7fd23effd640 1 -- 172.21.15.64:0/2829865451 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd240054a80 con 0x7fd2480aec60 2023-11-24T08:05:13.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.067+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fd2480cd2d0 con 0x7fd2480aec60 2023-11-24T08:05:13.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.071+0000 7fd23effd640 1 -- 172.21.15.64:0/2829865451 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd2400580a0 con 0x7fd2480aec60 2023-11-24T08:05:13.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.074+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd22c06ae70 msgr2=0x7fd22c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.074+0000 7fd24f0d1640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd22c06ae70 0x7fd22c06d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd244034000 tx=0x7fd244007000 comp rx=0 tx=0).stop 2023-11-24T08:05:13.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.074+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2480aec60 msgr2=0x7fd2480a98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.074+0000 7fd24f0d1640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2480aec60 0x7fd2480a98d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd248072210 tx=0x7fd24000f4a0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.075+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 shutdown_connections 2023-11-24T08:05:13.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.075+0000 7fd24f0d1640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd22c06ae70 0x7fd22c06d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.075+0000 7fd24f0d1640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2480aec60 0x7fd2480a98d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.075+0000 7fd24f0d1640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd2480a9470 0x7fd2480ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.075+0000 7fd24f0d1640 1 --2- 172.21.15.64:0/2829865451 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd2340039e0 0x7fd248156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.075+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 >> 172.21.15.64:0/2829865451 conn(0x7fd2480b0840 msgr2=0x7fd2480cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:13.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.075+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 shutdown_connections 2023-11-24T08:05:13.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.075+0000 7fd24f0d1640 1 -- 172.21.15.64:0/2829865451 wait complete. 2023-11-24T08:05:13.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T08:05:13.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.320+0000 7fc216044640 1 Processor -- start 2023-11-24T08:05:13.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.320+0000 7fc216044640 1 -- start start 2023-11-24T08:05:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc216044640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2101407e0 0x7fc21006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc216044640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 0x7fc2100b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc216044640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc210140bc0 con 0x7fc210143b30 2023-11-24T08:05:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc216044640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2100b1be0 con 0x7fc2101407e0 2023-11-24T08:05:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc216044640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100b1ec0 con 0x7fc21015e3d0 2023-11-24T08:05:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc214841640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc210143b30 0x7fc21015dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46370/0 (socket says 172.21.15.64:46370) 2023-11-24T08:05:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc214841640 1 -- 172.21.15.64:0/3227302425 learned_addr learned my addr 172.21.15.64:0/3227302425 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc215042640 1 --2- 172.21.15.64:0/3227302425 >> v2:172.21.15.64:3300/0 conn(0x7fc2101407e0 0x7fc21006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.322+0000 7fc215843640 1 --2- 172.21.15.64:0/3227302425 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 0x7fc2100b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.323+0000 7fc207fff640 1 -- 172.21.15.64:0/3227302425 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123472392 0 0) 0x7fc210140bc0 con 0x7fc210143b30 2023-11-24T08:05:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.323+0000 7fc207fff640 1 -- 172.21.15.64:0/3227302425 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1fc005730 con 0x7fc210143b30 2023-11-24T08:05:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.323+0000 7fc215042640 1 -- 172.21.15.64:0/3227302425 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 msgr2=0x7fc2100b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.323+0000 7fc215042640 1 --2- 172.21.15.64:0/3227302425 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 0x7fc2100b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.323+0000 7fc215042640 1 -- 172.21.15.64:0/3227302425 >> v1:172.21.15.64:6789/0 conn(0x7fc210143b30 legacy=0x7fc21015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.323+0000 7fc215042640 1 -- 172.21.15.64:0/3227302425 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100b21a0 con 0x7fc2101407e0 2023-11-24T08:05:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.323+0000 7fc215843640 1 --2- 172.21.15.64:0/3227302425 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 0x7fc2100b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.324+0000 7fc215042640 1 --2- 172.21.15.64:0/3227302425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc21006c320 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0097a0 tx=0x7fc20c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5821f3b4da1cb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.324+0000 7fc207fff640 1 -- 172.21.15.64:0/3227302425 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c031830 con 0x7fc2101407e0 2023-11-24T08:05:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.324+0000 7fc207fff640 1 -- 172.21.15.64:0/3227302425 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c031990 con 0x7fc2101407e0 2023-11-24T08:05:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.324+0000 7fc207fff640 1 -- 172.21.15.64:0/3227302425 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c031d20 con 0x7fc2101407e0 2023-11-24T08:05:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.325+0000 7fc216044640 1 -- 172.21.15.64:0/3227302425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 msgr2=0x7fc21006c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.325+0000 7fc216044640 1 --2- 172.21.15.64:0/3227302425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc21006c320 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0097a0 tx=0x7fc20c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.325+0000 7fc216044640 1 -- 172.21.15.64:0/3227302425 shutdown_connections 2023-11-24T08:05:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.325+0000 7fc216044640 1 --2- 172.21.15.64:0/3227302425 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 0x7fc2100b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.325+0000 7fc216044640 1 --2- 172.21.15.64:0/3227302425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc21006c320 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.325+0000 7fc216044640 1 -- 172.21.15.64:0/3227302425 >> 172.21.15.64:0/3227302425 conn(0x7fc2100b0840 msgr2=0x7fc2100b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.325+0000 7fc216044640 1 -- 172.21.15.64:0/3227302425 shutdown_connections 2023-11-24T08:05:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.325+0000 7fc216044640 1 -- 172.21.15.64:0/3227302425 wait complete. 2023-11-24T08:05:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.327+0000 7fc216044640 1 Processor -- start 2023-11-24T08:05:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.327+0000 7fc216044640 1 -- start start 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc216044640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc210158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc216044640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc210143b30 0x7fc2100b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc216044640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 0x7fc2100b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc216044640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100ae130 con 0x7fc2101407e0 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc216044640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21015a7c0 con 0x7fc210143b30 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc216044640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21015aaa0 con 0x7fc21015e3d0 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc215042640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc210158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc215042640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc210158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48146/0 (socket says 172.21.15.64:48146) 2023-11-24T08:05:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.328+0000 7fc215042640 1 -- 172.21.15.64:0/706215512 learned_addr learned my addr 172.21.15.64:0/706215512 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc215042640 1 -- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 msgr2=0x7fc2100b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc215042640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 0x7fc2100b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc215042640 1 -- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc210143b30 msgr2=0x7fc2100b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc215042640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc210143b30 0x7fc2100b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc215042640 1 -- 172.21.15.64:0/706215512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c009400 con 0x7fc2101407e0 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc215042640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc210158f20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc20c034040 tx=0x7fc20c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc205ffb640 1 -- 172.21.15.64:0/706215512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c03a8f0 con 0x7fc2101407e0 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc205ffb640 1 -- 172.21.15.64:0/706215512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c03aa70 con 0x7fc2101407e0 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc205ffb640 1 -- 172.21.15.64:0/706215512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c03ae20 con 0x7fc2101407e0 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.329+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21015ad80 con 0x7fc2101407e0 2023-11-24T08:05:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.330+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc210159460 con 0x7fc2101407e0 2023-11-24T08:05:13.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.331+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21004e840 con 0x7fc2101407e0 2023-11-24T08:05:13.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.331+0000 7fc205ffb640 1 -- 172.21.15.64:0/706215512 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc20c04d440 con 0x7fc2101407e0 2023-11-24T08:05:13.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.332+0000 7fc205ffb640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc1f406ae40 0x7fc1f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.332+0000 7fc205ffb640 1 -- 172.21.15.64:0/706215512 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc20c0ab470 con 0x7fc2101407e0 2023-11-24T08:05:13.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.333+0000 7fc214841640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc1f406ae40 0x7fc1f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.334+0000 7fc214841640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc1f406ae40 0x7fc1f406d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2100ae430 tx=0x7fc200007930 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.337+0000 7fc205ffb640 1 -- 172.21.15.64:0/706215512 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc20c0767d0 con 0x7fc2101407e0 2023-11-24T08:05:13.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.599+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc2100b3de0 con 0x7fc2101407e0 2023-11-24T08:05:13.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.603+0000 7fc205ffb640 1 -- 172.21.15.64:0/706215512 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc20c079df0 con 0x7fc2101407e0 2023-11-24T08:05:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc1f406ae40 msgr2=0x7fc1f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc1f406ae40 0x7fc1f406d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2100ae430 tx=0x7fc200007930 comp rx=0 tx=0).stop 2023-11-24T08:05:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 msgr2=0x7fc210158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc210158f20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc20c034040 tx=0x7fc20c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 shutdown_connections 2023-11-24T08:05:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc1f406ae40 0x7fc1f406d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc21015e3d0 0x7fc2100b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc210143b30 0x7fc2100b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 --2- 172.21.15.64:0/706215512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2101407e0 0x7fc210158f20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 >> 172.21.15.64:0/706215512 conn(0x7fc2100b0840 msgr2=0x7fc21013f050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:13.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 shutdown_connections 2023-11-24T08:05:13.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.606+0000 7fc216044640 1 -- 172.21.15.64:0/706215512 wait complete. 2023-11-24T08:05:13.629 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T08:05:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.853+0000 7f4030a9f640 1 Processor -- start 2023-11-24T08:05:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.854+0000 7f4030a9f640 1 -- start start 2023-11-24T08:05:13.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.855+0000 7f4030a9f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f402c15d540 0x7f402c070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.855+0000 7f4030a9f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c0cb190 con 0x7f402c0cad30 2023-11-24T08:05:13.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.856+0000 7f4030a9f640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c071bc0 con 0x7f402c161630 2023-11-24T08:05:13.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.856+0000 7f4030a9f640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c161d80 con 0x7f402c15d540 2023-11-24T08:05:13.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.856+0000 7f402affd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f402c0cad30 0x7f402c160f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46380/0 (socket says 172.21.15.64:46380) 2023-11-24T08:05:13.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.856+0000 7f402affd640 1 -- 172.21.15.64:0/423972341 learned_addr learned my addr 172.21.15.64:0/423972341 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.856+0000 7f402b7fe640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f402c15d540 0x7f402c070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.856+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125835152 0 0) 0x7f402c0cb190 con 0x7f402c0cad30 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4018003520 con 0x7f402c0cad30 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849343619 0 0) 0x7f402c071bc0 con 0x7f402c161630 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f402c0cb190 con 0x7f402c161630 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1015162011 0 0) 0x7f4018003520 con 0x7f402c0cad30 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f402c15d540 msgr2=0x7f402c070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f402c15d540 0x7f402c070fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 >> v1:172.21.15.91:6789/0 conn(0x7f402c161630 legacy=0x7f402c06cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c071600 con 0x7f402c0cad30 2023-11-24T08:05:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f4014001670 con 0x7f402c0cad30 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 >> v1:172.21.15.64:6789/0 conn(0x7f402c0cad30 legacy=0x7f402c160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4018003ba0 0x7f4018003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40180047c0 0x7f4018004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f401800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c161d80 con 0x7f4018005140 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401800bfd0 con 0x7f4018003ba0 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.857+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401800c190 con 0x7f40180047c0 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.858+0000 7f402bfff640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f401800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.858+0000 7f402affd640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40180047c0 0x7f4018004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.858+0000 7f402b7fe640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4018003ba0 0x7f4018003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.858+0000 7f402bfff640 1 -- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40180047c0 msgr2=0x7f4018004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.858+0000 7f402bfff640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f40180047c0 0x7f4018004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.858+0000 7f402bfff640 1 -- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4018003ba0 msgr2=0x7f4018003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:13.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.858+0000 7f402bfff640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4018003ba0 0x7f4018003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.858+0000 7f402bfff640 1 -- 172.21.15.64:0/423972341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018003520 con 0x7f4018005140 2023-11-24T08:05:13.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.859+0000 7f402bfff640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f401800b980 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f402c06e580 tx=0x7f40180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9eca560370bafd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:13.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.859+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240052d0 con 0x7f4018005140 2023-11-24T08:05:13.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4024005430 con 0x7f4018005140 2023-11-24T08:05:13.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f402a7fc640 1 -- 172.21.15.64:0/423972341 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024003810 con 0x7f4018005140 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f4030a9f640 1 -- 172.21.15.64:0/423972341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 msgr2=0x7f401800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f4030a9f640 1 --2- 172.21.15.64:0/423972341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f401800b980 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f402c06e580 tx=0x7f40180057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f402bfff640 1 -- 172.21.15.64:0/423972341 reap_dead start 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f4030a9f640 1 -- 172.21.15.64:0/423972341 shutdown_connections 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f4030a9f640 1 -- 172.21.15.64:0/423972341 >> 172.21.15.64:0/423972341 conn(0x7f402c0b0840 msgr2=0x7f402c0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f4030a9f640 1 -- 172.21.15.64:0/423972341 shutdown_connections 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.860+0000 7f4030a9f640 1 -- 172.21.15.64:0/423972341 wait complete. 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.862+0000 7f4030a9f640 1 Processor -- start 2023-11-24T08:05:13.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.862+0000 7f4030a9f640 1 -- start start 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.863+0000 7f4030a9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f402c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.863+0000 7f4030a9f640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f402c14d800 0x7f402c15fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.863+0000 7f4030a9f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f402c15da60 0x7f402c15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f4030a9f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c160360 con 0x7f4018005140 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f4030a9f640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1511e0 con 0x7f402c14d800 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f4030a9f640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1514c0 con 0x7f402c15da60 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f402c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f402c150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48164/0 (socket says 172.21.15.64:48164) 2023-11-24T08:05:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402b7fe640 1 -- 172.21.15.64:0/3890975916 learned_addr learned my addr 172.21.15.64:0/3890975916 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402affd640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f402c14d800 0x7f402c15fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402b7fe640 1 -- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f402c15da60 msgr2=0x7f402c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402b7fe640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f402c15da60 0x7f402c15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402b7fe640 1 -- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f402c14d800 msgr2=0x7f402c15fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402b7fe640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f402c14d800 0x7f402c15fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.864+0000 7f402b7fe640 1 -- 172.21.15.64:0/3890975916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c1517a0 con 0x7f4018005140 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.865+0000 7f402b7fe640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f402c150bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f401c007f30 tx=0x7f401c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.865+0000 7f4023fff640 1 -- 172.21.15.64:0/3890975916 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c00e030 con 0x7f4018005140 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.865+0000 7f4023fff640 1 -- 172.21.15.64:0/3890975916 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c016470 con 0x7f4018005140 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.865+0000 7f4023fff640 1 -- 172.21.15.64:0/3890975916 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c016820 con 0x7f4018005140 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.865+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c151a80 con 0x7f4018005140 2023-11-24T08:05:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.865+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c14dc80 con 0x7f4018005140 2023-11-24T08:05:13.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.866+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c04e840 con 0x7f4018005140 2023-11-24T08:05:13.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.867+0000 7f4023fff640 1 -- 172.21.15.64:0/3890975916 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f401c016980 con 0x7f4018005140 2023-11-24T08:05:13.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.868+0000 7f4023fff640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f400c06ae40 0x7f400c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:13.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.868+0000 7f4023fff640 1 -- 172.21.15.64:0/3890975916 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f401c00f070 con 0x7f4018005140 2023-11-24T08:05:13.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.869+0000 7f402affd640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f400c06ae40 0x7f400c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:13.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.870+0000 7f402affd640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f400c06ae40 0x7f400c06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4014001b60 tx=0x7f4014009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:13.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:13.872+0000 7f4023fff640 1 -- 172.21.15.64:0/3890975916 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f401c055640 con 0x7f4018005140 2023-11-24T08:05:14.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.138+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f402c158640 con 0x7f4018005140 2023-11-24T08:05:14.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.143+0000 7f4023fff640 1 -- 172.21.15.64:0/3890975916 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f401c058c60 con 0x7f4018005140 2023-11-24T08:05:14.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f400c06ae40 msgr2=0x7f400c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f400c06ae40 0x7f400c06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4014001b60 tx=0x7f4014009040 comp rx=0 tx=0).stop 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 msgr2=0x7f402c150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f402c150bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f401c007f30 tx=0x7f401c011660 comp rx=0 tx=0).stop 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 shutdown_connections 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f400c06ae40 0x7f400c06d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f402c15da60 0x7f402c15dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f402c14d800 0x7f402c15fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 --2- 172.21.15.64:0/3890975916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4018005140 0x7f402c150bd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 >> 172.21.15.64:0/3890975916 conn(0x7f402c0b0840 msgr2=0x7f402c0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.146+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 shutdown_connections 2023-11-24T08:05:14.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.147+0000 7f4030a9f640 1 -- 172.21.15.64:0/3890975916 wait complete. 2023-11-24T08:05:14.167 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T08:05:14.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.370+0000 7f3ea3d08640 1 Processor -- start 2023-11-24T08:05:14.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.370+0000 7f3ea3d08640 1 -- start start 2023-11-24T08:05:14.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.372+0000 7f3ea3d08640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3e9c1469e0 0x7f3e9c15d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.372+0000 7f3ea3d08640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3e9c15de10 0x7f3e9c06c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.372+0000 7f3ea3d08640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c06cd00 con 0x7f3e9c06c900 2023-11-24T08:05:14.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.372+0000 7f3ea3d08640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0d13b0 con 0x7f3e9c15de10 2023-11-24T08:05:14.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.372+0000 7f3ea3d08640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c070b30 con 0x7f3e9c1469e0 2023-11-24T08:05:14.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.373+0000 7f3ea22a5640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3e9c06c900 0x7f3e9c0b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46382/0 (socket says 172.21.15.64:46382) 2023-11-24T08:05:14.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.373+0000 7f3ea22a5640 1 -- 172.21.15.64:0/2968318878 learned_addr learned my addr 172.21.15.64:0/2968318878 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.373+0000 7f3ea12a3640 1 --2- 172.21.15.64:0/2968318878 >> v2:172.21.15.64:3300/0 conn(0x7f3e9c15de10 0x7f3e9c06c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.373+0000 7f3ea1aa4640 1 --2- 172.21.15.64:0/2968318878 >> v2:172.21.15.91:3300/0 conn(0x7f3e9c1469e0 0x7f3e9c15d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.373+0000 7f3ea12a3640 1 -- 172.21.15.64:0/2968318878 >> v2:172.21.15.91:3300/0 conn(0x7f3e9c1469e0 msgr2=0x7f3e9c15d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.374+0000 7f3ea12a3640 1 --2- 172.21.15.64:0/2968318878 >> v2:172.21.15.91:3300/0 conn(0x7f3e9c1469e0 0x7f3e9c15d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.373+0000 7f3ea0aa2640 1 -- 172.21.15.64:0/2968318878 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908445165 0 0) 0x7f3e9c06cd00 con 0x7f3e9c06c900 2023-11-24T08:05:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.374+0000 7f3ea12a3640 1 -- 172.21.15.64:0/2968318878 >> v1:172.21.15.64:6789/0 conn(0x7f3e9c06c900 legacy=0x7f3e9c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.374+0000 7f3ea12a3640 1 -- 172.21.15.64:0/2968318878 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c070e10 con 0x7f3e9c15de10 2023-11-24T08:05:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.375+0000 7f3ea12a3640 1 --2- 172.21.15.64:0/2968318878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c06c3c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c06dab0 tx=0x7f3e8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff83458e888e2091 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:14.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.375+0000 7f3ea0aa2640 1 -- 172.21.15.64:0/2968318878 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c031830 con 0x7f3e9c15de10 2023-11-24T08:05:14.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.375+0000 7f3ea0aa2640 1 -- 172.21.15.64:0/2968318878 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c031990 con 0x7f3e9c15de10 2023-11-24T08:05:14.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.375+0000 7f3ea0aa2640 1 -- 172.21.15.64:0/2968318878 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c031d20 con 0x7f3e9c15de10 2023-11-24T08:05:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.376+0000 7f3ea3d08640 1 -- 172.21.15.64:0/2968318878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 msgr2=0x7f3e9c06c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.376+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/2968318878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c06c3c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c06dab0 tx=0x7f3e8c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.376+0000 7f3ea3d08640 1 -- 172.21.15.64:0/2968318878 shutdown_connections 2023-11-24T08:05:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.376+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/2968318878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c06c3c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.376+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/2968318878 >> v2:172.21.15.91:3300/0 conn(0x7f3e9c1469e0 0x7f3e9c15d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.376+0000 7f3ea3d08640 1 -- 172.21.15.64:0/2968318878 >> 172.21.15.64:0/2968318878 conn(0x7f3e9c0b2e00 msgr2=0x7f3e9c0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.376+0000 7f3ea3d08640 1 -- 172.21.15.64:0/2968318878 shutdown_connections 2023-11-24T08:05:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.376+0000 7f3ea3d08640 1 -- 172.21.15.64:0/2968318878 wait complete. 2023-11-24T08:05:14.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.378+0000 7f3ea3d08640 1 Processor -- start 2023-11-24T08:05:14.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.378+0000 7f3ea3d08640 1 -- start start 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea3d08640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e9c06c900 0x7f3e9c0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea3d08640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c1469e0 0x7f3e9c1f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea3d08640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea3d08640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c133900 con 0x7f3e9c15de10 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea3d08640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1f3a80 con 0x7f3e9c1469e0 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea3d08640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1f31e0 con 0x7f3e9c06c900 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea22a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea22a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48176/0 (socket says 172.21.15.64:48176) 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea22a5640 1 -- 172.21.15.64:0/260974369 learned_addr learned my addr 172.21.15.64:0/260974369 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea12a3640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c1469e0 0x7f3e9c1f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea1aa4640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e9c06c900 0x7f3e9c0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea22a5640 1 -- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e9c06c900 msgr2=0x7f3e9c0afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea22a5640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e9c06c900 0x7f3e9c0afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea22a5640 1 -- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c1469e0 msgr2=0x7f3e9c1f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea22a5640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c1469e0 0x7f3e9c1f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.380+0000 7f3ea22a5640 1 -- 172.21.15.64:0/260974369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c009400 con 0x7f3e9c15de10 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.381+0000 7f3ea22a5640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c143e00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c780 tx=0x7f3e9400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.381+0000 7f3e927fc640 1 -- 172.21.15.64:0/260974369 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400ce90 con 0x7f3e9c15de10 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.381+0000 7f3e927fc640 1 -- 172.21.15.64:0/260974369 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9400a640 con 0x7f3e9c15de10 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.381+0000 7f3e927fc640 1 -- 172.21.15.64:0/260974369 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400a9d0 con 0x7f3e9c15de10 2023-11-24T08:05:14.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.381+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c1f34c0 con 0x7f3e9c15de10 2023-11-24T08:05:14.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.381+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c144340 con 0x7f3e9c15de10 2023-11-24T08:05:14.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.383+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c133a40 con 0x7f3e9c15de10 2023-11-24T08:05:14.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.385+0000 7f3e927fc640 1 -- 172.21.15.64:0/260974369 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e9400ab30 con 0x7f3e9c15de10 2023-11-24T08:05:14.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.386+0000 7f3e927fc640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3e8006ae40 0x7f3e8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.386+0000 7f3e927fc640 1 -- 172.21.15.64:0/260974369 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e94088750 con 0x7f3e9c15de10 2023-11-24T08:05:14.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.387+0000 7f3ea1aa4640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3e8006ae40 0x7f3e8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.388+0000 7f3ea1aa4640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3e8006ae40 0x7f3e8006d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ad0 tx=0x7f3e98008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:14.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.391+0000 7f3e927fc640 1 -- 172.21.15.64:0/260974369 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e94053be0 con 0x7f3e9c15de10 2023-11-24T08:05:14.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.659+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f3e9c074770 con 0x7f3e9c15de10 2023-11-24T08:05:14.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.663+0000 7f3e927fc640 1 -- 172.21.15.64:0/260974369 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3e94018020 con 0x7f3e9c15de10 2023-11-24T08:05:14.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.665+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3e8006ae40 msgr2=0x7f3e8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.665+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3e8006ae40 0x7f3e8006d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ad0 tx=0x7f3e98008040 comp rx=0 tx=0).stop 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.665+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 msgr2=0x7f3e9c143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.665+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c143e00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c780 tx=0x7f3e9400cc40 comp rx=0 tx=0).stop 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.665+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 shutdown_connections 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.665+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3e8006ae40 0x7f3e8006d300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.665+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c15de10 0x7f3e9c143e00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.666+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e9c1469e0 0x7f3e9c1f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.666+0000 7f3ea3d08640 1 --2- 172.21.15.64:0/260974369 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e9c06c900 0x7f3e9c0afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.666+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 >> 172.21.15.64:0/260974369 conn(0x7f3e9c0b2e00 msgr2=0x7f3e9c06ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.666+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 shutdown_connections 2023-11-24T08:05:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.666+0000 7f3ea3d08640 1 -- 172.21.15.64:0/260974369 wait complete. 2023-11-24T08:05:14.685 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T08:05:14.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.878+0000 7f1649505640 1 Processor -- start 2023-11-24T08:05:14.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.878+0000 7f1649505640 1 -- start start 2023-11-24T08:05:14.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.880+0000 7f1649505640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f164406dc30 0x7f16441660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.880+0000 7f1649505640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16440b4130 con 0x7f16440cbcb0 2023-11-24T08:05:14.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.880+0000 7f1649505640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16440b2080 con 0x7f164415b340 2023-11-24T08:05:14.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.880+0000 7f1649505640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16440b2360 con 0x7f164406dc30 2023-11-24T08:05:14.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.880+0000 7f16437fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f164406dc30 0x7f16441660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.880+0000 7f163bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f16440cbcb0 0x7f164406d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46394/0 (socket says 172.21.15.64:46394) 2023-11-24T08:05:14.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.880+0000 7f163bfff640 1 -- 172.21.15.64:0/1223919131 learned_addr learned my addr 172.21.15.64:0/1223919131 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:14.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.881+0000 7f16427fc640 1 -- 172.21.15.64:0/1223919131 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12902011 0 0) 0x7f16440b4130 con 0x7f16440cbcb0 2023-11-24T08:05:14.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.881+0000 7f16427fc640 1 -- 172.21.15.64:0/1223919131 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1630003520 con 0x7f16440cbcb0 2023-11-24T08:05:14.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.881+0000 7f16437fe640 1 -- 172.21.15.64:0/1223919131 >> v1:172.21.15.91:6789/0 conn(0x7f164415b340 legacy=0x7f16440abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.881+0000 7f16437fe640 1 -- 172.21.15.64:0/1223919131 >> v1:172.21.15.64:6789/0 conn(0x7f16440cbcb0 legacy=0x7f164406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.881+0000 7f16437fe640 1 -- 172.21.15.64:0/1223919131 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440b2640 con 0x7f164406dc30 2023-11-24T08:05:14.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.883+0000 7f16437fe640 1 --2- 172.21.15.64:0/1223919131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f16441660e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f163c009d90 tx=0x7f163c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56b63b46512f6f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:14.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.883+0000 7f16427fc640 1 -- 172.21.15.64:0/1223919131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c02cd00 con 0x7f164406dc30 2023-11-24T08:05:14.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.883+0000 7f16427fc640 1 -- 172.21.15.64:0/1223919131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c02ce60 con 0x7f164406dc30 2023-11-24T08:05:14.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.883+0000 7f16427fc640 1 -- 172.21.15.64:0/1223919131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c034660 con 0x7f164406dc30 2023-11-24T08:05:14.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.884+0000 7f1649505640 1 -- 172.21.15.64:0/1223919131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 msgr2=0x7f16441660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.884+0000 7f1649505640 1 --2- 172.21.15.64:0/1223919131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f16441660e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f163c009d90 tx=0x7f163c02bf50 comp rx=0 tx=0).stop 2023-11-24T08:05:14.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.884+0000 7f1649505640 1 -- 172.21.15.64:0/1223919131 shutdown_connections 2023-11-24T08:05:14.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.884+0000 7f1649505640 1 --2- 172.21.15.64:0/1223919131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f16441660e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.884+0000 7f1649505640 1 -- 172.21.15.64:0/1223919131 >> 172.21.15.64:0/1223919131 conn(0x7f16440b0d80 msgr2=0x7f16440b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:14.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.884+0000 7f1649505640 1 -- 172.21.15.64:0/1223919131 shutdown_connections 2023-11-24T08:05:14.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.884+0000 7f1649505640 1 -- 172.21.15.64:0/1223919131 wait complete. 2023-11-24T08:05:14.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.886+0000 7f1649505640 1 Processor -- start 2023-11-24T08:05:14.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.886+0000 7f1649505640 1 -- start start 2023-11-24T08:05:14.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1649505640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f1644152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1649505640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16440cbcb0 0x7f1644145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1649505640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f164415b340 0x7f1644157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1649505640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441533d0 con 0x7f164406dc30 2023-11-24T08:05:14.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1649505640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441536b0 con 0x7f16440cbcb0 2023-11-24T08:05:14.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1649505640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1644153990 con 0x7f164415b340 2023-11-24T08:05:14.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1642ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f1644152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1642ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f1644152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48186/0 (socket says 172.21.15.64:48186) 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.887+0000 7f1642ffd640 1 -- 172.21.15.64:0/1688540943 learned_addr learned my addr 172.21.15.64:0/1688540943 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.888+0000 7f163bfff640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16440cbcb0 0x7f1644145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.888+0000 7f1642ffd640 1 -- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f164415b340 msgr2=0x7f1644157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.888+0000 7f1642ffd640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f164415b340 0x7f1644157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.888+0000 7f1642ffd640 1 -- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16440cbcb0 msgr2=0x7f1644145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.888+0000 7f1642ffd640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16440cbcb0 0x7f1644145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.888+0000 7f1642ffd640 1 -- 172.21.15.64:0/1688540943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c009a70 con 0x7f164406dc30 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.888+0000 7f163bfff640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16440cbcb0 0x7f1644145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:14.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.888+0000 7f1642ffd640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f1644152df0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f163400c450 tx=0x7f163400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:14.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.889+0000 7f163b7fe640 1 -- 172.21.15.64:0/1688540943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16340052d0 con 0x7f164406dc30 2023-11-24T08:05:14.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.889+0000 7f163b7fe640 1 -- 172.21.15.64:0/1688540943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1634005430 con 0x7f164406dc30 2023-11-24T08:05:14.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.889+0000 7f163b7fe640 1 -- 172.21.15.64:0/1688540943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634012870 con 0x7f164406dc30 2023-11-24T08:05:14.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.889+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1644153c70 con 0x7f164406dc30 2023-11-24T08:05:14.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.889+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1644145760 con 0x7f164406dc30 2023-11-24T08:05:14.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.890+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1644154190 con 0x7f164406dc30 2023-11-24T08:05:14.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.891+0000 7f163b7fe640 1 -- 172.21.15.64:0/1688540943 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f16340129d0 con 0x7f164406dc30 2023-11-24T08:05:14.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.892+0000 7f163b7fe640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f162406ae40 0x7f162406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:14.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.893+0000 7f163b7fe640 1 -- 172.21.15.64:0/1688540943 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1634088e00 con 0x7f164406dc30 2023-11-24T08:05:14.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.893+0000 7f163bfff640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f162406ae40 0x7f162406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:14.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.894+0000 7f163bfff640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f162406ae40 0x7f162406d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1644150ef0 tx=0x7f162c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:14.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:14.896+0000 7f163b7fe640 1 -- 172.21.15.64:0/1688540943 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1634054200 con 0x7f164406dc30 2023-11-24T08:05:14.930 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:05:14.937 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:05:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.167+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f1644074770 con 0x7f164406dc30 2023-11-24T08:05:15.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.170+0000 7f163b7fe640 1 -- 172.21.15.64:0/1688540943 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1634057820 con 0x7f164406dc30 2023-11-24T08:05:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f162406ae40 msgr2=0x7f162406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f162406ae40 0x7f162406d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1644150ef0 tx=0x7f162c009040 comp rx=0 tx=0).stop 2023-11-24T08:05:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 msgr2=0x7f1644152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f1644152df0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f163400c450 tx=0x7f163400c910 comp rx=0 tx=0).stop 2023-11-24T08:05:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 shutdown_connections 2023-11-24T08:05:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f162406ae40 0x7f162406d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f164415b340 0x7f1644157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16440cbcb0 0x7f1644145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 --2- 172.21.15.64:0/1688540943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f164406dc30 0x7f1644152df0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 >> 172.21.15.64:0/1688540943 conn(0x7f16440b0d80 msgr2=0x7f16440cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:15.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 shutdown_connections 2023-11-24T08:05:15.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.174+0000 7f1649505640 1 -- 172.21.15.64:0/1688540943 wait complete. 2023-11-24T08:05:15.194 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T08:05:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.395+0000 7f91a23ac640 1 Processor -- start 2023-11-24T08:05:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.395+0000 7f91a23ac640 1 -- start start 2023-11-24T08:05:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.396+0000 7f91a23ac640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f919c15d430 0x7f919c0d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.396+0000 7f91a23ac640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f919c0cfaa0 0x7f919c1615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.396+0000 7f91a23ac640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919c15d810 con 0x7f919c161b20 2023-11-24T08:05:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.397+0000 7f91a23ac640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919c0d09d0 con 0x7f919c0cfaa0 2023-11-24T08:05:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.397+0000 7f91a23ac640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919c0d1030 con 0x7f919c15d430 2023-11-24T08:05:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.397+0000 7f91a0ba9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f919c0cfaa0 0x7f919c1615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.397+0000 7f91a0ba9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f919c0cfaa0 0x7f919c1615e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48202/0 (socket says 172.21.15.64:48202) 2023-11-24T08:05:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.397+0000 7f91a0ba9640 1 -- 172.21.15.64:0/1887399515 learned_addr learned my addr 172.21.15.64:0/1887399515 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.397+0000 7f91a13aa640 1 --2- 172.21.15.64:0/1887399515 >> v2:172.21.15.91:3300/0 conn(0x7f919c15d430 0x7f919c0d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.398+0000 7f91a0ba9640 1 -- 172.21.15.64:0/1887399515 >> v2:172.21.15.91:3300/0 conn(0x7f919c15d430 msgr2=0x7f919c0d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.398+0000 7f91a0ba9640 1 --2- 172.21.15.64:0/1887399515 >> v2:172.21.15.91:3300/0 conn(0x7f919c15d430 0x7f919c0d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.398+0000 7f91a0ba9640 1 -- 172.21.15.64:0/1887399515 >> v1:172.21.15.91:6789/0 conn(0x7f919c161b20 legacy=0x7f919c0d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.398+0000 7f91a0ba9640 1 -- 172.21.15.64:0/1887399515 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c0d0130 con 0x7f919c0cfaa0 2023-11-24T08:05:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.399+0000 7f91a0ba9640 1 --2- 172.21.15.64:0/1887399515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c0cfaa0 0x7f919c1615e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f919c0b2ca0 tx=0x7f918c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b633638af81778 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.399+0000 7f9193fff640 1 -- 172.21.15.64:0/1887399515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c0317e0 con 0x7f919c0cfaa0 2023-11-24T08:05:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.399+0000 7f9193fff640 1 -- 172.21.15.64:0/1887399515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c031940 con 0x7f919c0cfaa0 2023-11-24T08:05:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.399+0000 7f9193fff640 1 -- 172.21.15.64:0/1887399515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c031cd0 con 0x7f919c0cfaa0 2023-11-24T08:05:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.400+0000 7f91a23ac640 1 -- 172.21.15.64:0/1887399515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c0cfaa0 msgr2=0x7f919c1615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.400+0000 7f91a23ac640 1 --2- 172.21.15.64:0/1887399515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c0cfaa0 0x7f919c1615e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f919c0b2ca0 tx=0x7f918c02b880 comp rx=0 tx=0).stop 2023-11-24T08:05:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.400+0000 7f91a23ac640 1 -- 172.21.15.64:0/1887399515 shutdown_connections 2023-11-24T08:05:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.400+0000 7f91a23ac640 1 --2- 172.21.15.64:0/1887399515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c0cfaa0 0x7f919c1615e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.400+0000 7f91a23ac640 1 --2- 172.21.15.64:0/1887399515 >> v2:172.21.15.91:3300/0 conn(0x7f919c15d430 0x7f919c0d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.400+0000 7f91a23ac640 1 -- 172.21.15.64:0/1887399515 >> 172.21.15.64:0/1887399515 conn(0x7f919c08cb00 msgr2=0x7f919c09c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.400+0000 7f91a23ac640 1 -- 172.21.15.64:0/1887399515 shutdown_connections 2023-11-24T08:05:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.400+0000 7f91a23ac640 1 -- 172.21.15.64:0/1887399515 wait complete. 2023-11-24T08:05:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.402+0000 7f91a23ac640 1 Processor -- start 2023-11-24T08:05:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.402+0000 7f91a23ac640 1 -- start start 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a23ac640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f919c0cfaa0 0x7f919c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a23ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c15d430 0x7f919c152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a23ac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f919c161b20 0x7f919c0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a23ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c15ec10 con 0x7f919c15d430 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a23ac640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c0b10c0 con 0x7f919c0cfaa0 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a23ac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919c15e000 con 0x7f919c161b20 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a0ba9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c15d430 0x7f919c152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a0ba9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c15d430 0x7f919c152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48216/0 (socket says 172.21.15.64:48216) 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a0ba9640 1 -- 172.21.15.64:0/806398338 learned_addr learned my addr 172.21.15.64:0/806398338 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a13aa640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f919c0cfaa0 0x7f919c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a1bab640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f919c161b20 0x7f919c0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a0ba9640 1 -- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f919c161b20 msgr2=0x7f919c0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a0ba9640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f919c161b20 0x7f919c0b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a0ba9640 1 -- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f919c0cfaa0 msgr2=0x7f919c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a0ba9640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f919c0cfaa0 0x7f919c15da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.403+0000 7f91a0ba9640 1 -- 172.21.15.64:0/806398338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c0094d0 con 0x7f919c15d430 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.404+0000 7f91a0ba9640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c15d430 0x7f919c152180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f918c000c00 tx=0x7f918c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.404+0000 7f9191ffb640 1 -- 172.21.15.64:0/806398338 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c032880 con 0x7f919c15d430 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.404+0000 7f9191ffb640 1 -- 172.21.15.64:0/806398338 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c032a00 con 0x7f919c15d430 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.404+0000 7f9191ffb640 1 -- 172.21.15.64:0/806398338 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c032db0 con 0x7f919c15d430 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.404+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c15e2e0 con 0x7f919c15d430 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.404+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c15e8a0 con 0x7f919c15d430 2023-11-24T08:05:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.405+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c067960 con 0x7f919c15d430 2023-11-24T08:05:15.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.406+0000 7f9191ffb640 1 -- 172.21.15.64:0/806398338 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f918c04d3f0 con 0x7f919c15d430 2023-11-24T08:05:15.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.407+0000 7f9191ffb640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f918006ae40 0x7f918006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.407+0000 7f9191ffb640 1 -- 172.21.15.64:0/806398338 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f918c0aaff0 con 0x7f919c15d430 2023-11-24T08:05:15.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.408+0000 7f91a13aa640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f918006ae40 0x7f918006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.409+0000 7f91a13aa640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f918006ae40 0x7f918006d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9198006fd0 tx=0x7f9198008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.412+0000 7f9191ffb640 1 -- 172.21.15.64:0/806398338 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f918c0764f0 con 0x7f919c15d430 2023-11-24T08:05:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.672+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f919c04e840 con 0x7f919c15d430 2023-11-24T08:05:15.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.676+0000 7f9191ffb640 1 -- 172.21.15.64:0/806398338 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f918c079b10 con 0x7f919c15d430 2023-11-24T08:05:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f918006ae40 msgr2=0x7f918006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f918006ae40 0x7f918006d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9198006fd0 tx=0x7f9198008040 comp rx=0 tx=0).stop 2023-11-24T08:05:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c15d430 msgr2=0x7f919c152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c15d430 0x7f919c152180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f918c000c00 tx=0x7f918c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 shutdown_connections 2023-11-24T08:05:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f918006ae40 0x7f918006d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f919c161b20 0x7f919c0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919c15d430 0x7f919c152180 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 --2- 172.21.15.64:0/806398338 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f919c0cfaa0 0x7f919c15da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 >> 172.21.15.64:0/806398338 conn(0x7f919c08cb00 msgr2=0x7f919c0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 shutdown_connections 2023-11-24T08:05:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.679+0000 7f91a23ac640 1 -- 172.21.15.64:0/806398338 wait complete. 2023-11-24T08:05:15.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:15.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.893+0000 7f10bb59e640 1 Processor -- start 2023-11-24T08:05:15.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.893+0000 7f10bb59e640 1 -- start start 2023-11-24T08:05:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10bb59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f10bc143570 0x7f10bc0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10bb59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10bc143950 0x7f10bc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10bb59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc15e6b0 0x7f10bc15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10bb59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc071700 con 0x7f10bc143950 2023-11-24T08:05:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10bb59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc15b410 con 0x7f10bc143570 2023-11-24T08:05:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10bb59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc15b130 con 0x7f10bc15e6b0 2023-11-24T08:05:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10b9d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10bc143950 0x7f10bc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10b9d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10bc143950 0x7f10bc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48228/0 (socket says 172.21.15.64:48228) 2023-11-24T08:05:15.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.895+0000 7f10b9d9b640 1 -- 172.21.15.64:0/4220094160 learned_addr learned my addr 172.21.15.64:0/4220094160 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:15.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.896+0000 7f10ba59c640 1 --2- 172.21.15.64:0/4220094160 >> v2:172.21.15.91:3300/0 conn(0x7f10bc143570 0x7f10bc0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.896+0000 7f10bad9d640 1 --2- 172.21.15.64:0/4220094160 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc15e6b0 0x7f10bc15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.896+0000 7f10b9d9b640 1 -- 172.21.15.64:0/4220094160 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc15e6b0 msgr2=0x7f10bc15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:15.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.896+0000 7f10b9d9b640 1 --2- 172.21.15.64:0/4220094160 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc15e6b0 0x7f10bc15a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.896+0000 7f10b9d9b640 1 -- 172.21.15.64:0/4220094160 >> v2:172.21.15.91:3300/0 conn(0x7f10bc143570 msgr2=0x7f10bc0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:15.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.896+0000 7f10b9d9b640 1 --2- 172.21.15.64:0/4220094160 >> v2:172.21.15.91:3300/0 conn(0x7f10bc143570 0x7f10bc0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.896+0000 7f10b9d9b640 1 -- 172.21.15.64:0/4220094160 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc15dbd0 con 0x7f10bc143950 2023-11-24T08:05:15.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.897+0000 7f10b9d9b640 1 --2- 172.21.15.64:0/4220094160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cf590 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f10bc070c70 tx=0x7f10b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb0cc5619aed6c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:15.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.897+0000 7f10b959a640 1 -- 172.21.15.64:0/4220094160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00317e0 con 0x7f10bc143950 2023-11-24T08:05:15.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.897+0000 7f10b959a640 1 -- 172.21.15.64:0/4220094160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0031940 con 0x7f10bc143950 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.897+0000 7f10b959a640 1 -- 172.21.15.64:0/4220094160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0031cd0 con 0x7f10bc143950 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 -- 172.21.15.64:0/4220094160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 msgr2=0x7f10bc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 --2- 172.21.15.64:0/4220094160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cf590 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f10bc070c70 tx=0x7f10b002b880 comp rx=0 tx=0).stop 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 -- 172.21.15.64:0/4220094160 shutdown_connections 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 --2- 172.21.15.64:0/4220094160 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc15e6b0 0x7f10bc15a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 --2- 172.21.15.64:0/4220094160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cf590 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 --2- 172.21.15.64:0/4220094160 >> v2:172.21.15.91:3300/0 conn(0x7f10bc143570 0x7f10bc0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 -- 172.21.15.64:0/4220094160 >> 172.21.15.64:0/4220094160 conn(0x7f10bc06a860 msgr2=0x7f10bc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 -- 172.21.15.64:0/4220094160 shutdown_connections 2023-11-24T08:05:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.898+0000 7f10bb59e640 1 -- 172.21.15.64:0/4220094160 wait complete. 2023-11-24T08:05:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.900+0000 7f10bb59e640 1 Processor -- start 2023-11-24T08:05:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.900+0000 7f10bb59e640 1 -- start start 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.901+0000 7f10bb59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc143570 0x7f10bc0c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.901+0000 7f10bb59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.901+0000 7f10bb59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10bc15e6b0 0x7f10bc13bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.901+0000 7f10bb59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0c8b60 con 0x7f10bc143950 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.901+0000 7f10bb59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc13d910 con 0x7f10bc15e6b0 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.901+0000 7f10bb59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0c8d90 con 0x7f10bc143570 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48238/0 (socket says 172.21.15.64:48238) 2023-11-24T08:05:15.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 -- 172.21.15.64:0/891926254 learned_addr learned my addr 172.21.15.64:0/891926254 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10ba59c640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc143570 0x7f10bc0c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10bad9d640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10bc15e6b0 0x7f10bc13bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 -- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc143570 msgr2=0x7f10bc0c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc143570 0x7f10bc0c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 -- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10bc15e6b0 msgr2=0x7f10bc13bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10bc15e6b0 0x7f10bc13bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 -- 172.21.15.64:0/891926254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00094d0 con 0x7f10bc143950 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10ba59c640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc143570 0x7f10bc0c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10bad9d640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10bc15e6b0 0x7f10bc13bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.902+0000 7f10b9d9b640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cd7b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f10b0000c00 tx=0x7f10b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.903+0000 7f10aaffd640 1 -- 172.21.15.64:0/891926254 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0032880 con 0x7f10bc143950 2023-11-24T08:05:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.903+0000 7f10aaffd640 1 -- 172.21.15.64:0/891926254 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0032a20 con 0x7f10bc143950 2023-11-24T08:05:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.903+0000 7f10aaffd640 1 -- 172.21.15.64:0/891926254 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0032dd0 con 0x7f10bc143950 2023-11-24T08:05:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.903+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc0c9070 con 0x7f10bc143950 2023-11-24T08:05:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.903+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc13d5a0 con 0x7f10bc143950 2023-11-24T08:05:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.904+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc13daa0 con 0x7f10bc143950 2023-11-24T08:05:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.905+0000 7f10aaffd640 1 -- 172.21.15.64:0/891926254 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f10b004e450 con 0x7f10bc143950 2023-11-24T08:05:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.906+0000 7f10aaffd640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f109c06ae70 0x7f109c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.906+0000 7f10aaffd640 1 -- 172.21.15.64:0/891926254 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f10b00ab450 con 0x7f10bc143950 2023-11-24T08:05:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.907+0000 7f10ba59c640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f109c06ae70 0x7f109c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.908+0000 7f10ba59c640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f109c06ae70 0x7f109c06d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10ac006fd0 tx=0x7f10ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:15.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:15.910+0000 7f10aaffd640 1 -- 172.21.15.64:0/891926254 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10b0076880 con 0x7f10bc143950 2023-11-24T08:05:16.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.106+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10bc04e840 con 0x7f10bc143950 2023-11-24T08:05:16.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.108+0000 7f10aaffd640 1 -- 172.21.15.64:0/891926254 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12809 (secure 0 0 0) 0x7f10b0079ea0 con 0x7f10bc143950 2023-11-24T08:05:16.110 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:16.110 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":30,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:12.002955+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f109c06ae70 msgr2=0x7f109c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f109c06ae70 0x7f109c06d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10ac006fd0 tx=0x7f10ac008040 comp rx=0 tx=0).stop 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 msgr2=0x7f10bc0cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cd7b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f10b0000c00 tx=0x7f10b00046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 shutdown_connections 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f109c06ae70 0x7f109c06d330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10bc15e6b0 0x7f10bc13bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10bc143950 0x7f10bc0cd7b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 --2- 172.21.15.64:0/891926254 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10bc143570 0x7f10bc0c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 >> 172.21.15.64:0/891926254 conn(0x7f10bc06a860 msgr2=0x7f10bc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:16.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 shutdown_connections 2023-11-24T08:05:16.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.111+0000 7f10bb59e640 1 -- 172.21.15.64:0/891926254 wait complete. 2023-11-24T08:05:16.132 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:05:16.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:05:16.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.351+0000 7f710634e640 1 Processor -- start 2023-11-24T08:05:16.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.351+0000 7f710634e640 1 -- start start 2023-11-24T08:05:16.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.353+0000 7f710634e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100144770 0x7f710015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.353+0000 7f710634e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710015f930 con 0x7f710015f0a0 2023-11-24T08:05:16.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.353+0000 7f710634e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71000b3b10 con 0x7f71000b35a0 2023-11-24T08:05:16.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.353+0000 7f710634e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71000b43b0 con 0x7f7100144770 2023-11-24T08:05:16.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.353+0000 7f7105b4d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f710015f0a0 0x7f71000ce4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46398/0 (socket says 172.21.15.64:46398) 2023-11-24T08:05:16.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.353+0000 7f7105b4d640 1 -- 172.21.15.64:0/4026341502 learned_addr learned my addr 172.21.15.64:0/4026341502 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f710534c640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100144770 0x7f710015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502635197 0 0) 0x7f710015f930 con 0x7f710015f0a0 2023-11-24T08:05:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70ec003520 con 0x7f710015f0a0 2023-11-24T08:05:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4022259855 0 0) 0x7f70ec003520 con 0x7f710015f0a0 2023-11-24T08:05:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100144770 msgr2=0x7f710015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100144770 0x7f710015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 >> v1:172.21.15.91:6789/0 conn(0x7f71000b35a0 legacy=0x7f710006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000b3df0 con 0x7f710015f0a0 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f70f80019f0 con 0x7f710015f0a0 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.354+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 >> v1:172.21.15.64:6789/0 conn(0x7f710015f0a0 legacy=0x7f71000ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f70f7fff640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec003a10 0x7f70ec003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f70f7fff640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100144770 0x7f70ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f70f7fff640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000b43b0 con 0x7f70ec004d00 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec00ba90 con 0x7f70ec003a10 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec00bc50 con 0x7f7100144770 2023-11-24T08:05:16.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f7105b4d640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f710534c640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec003a10 0x7f70ec003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f7104b4b640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100144770 0x7f70ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f7105b4d640 1 -- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100144770 msgr2=0x7f70ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f7105b4d640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100144770 0x7f70ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f7105b4d640 1 -- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec003a10 msgr2=0x7f70ec003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f7105b4d640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec003a10 0x7f70ec003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.355+0000 7f7105b4d640 1 -- 172.21.15.64:0/4026341502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec003520 con 0x7f70ec004d00 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.356+0000 7f7105b4d640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f70f800a800 tx=0x7f70ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f009e5cc76043d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.356+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80072d0 con 0x7f70ec004d00 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f802f070 con 0x7f70ec004d00 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f70f7fff640 1 -- 172.21.15.64:0/4026341502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8007690 con 0x7f70ec004d00 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f710634e640 1 -- 172.21.15.64:0/4026341502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ec004d00 msgr2=0x7f70ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f710634e640 1 --2- 172.21.15.64:0/4026341502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f70f800a800 tx=0x7f70ec0057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f7105b4d640 1 -- 172.21.15.64:0/4026341502 reap_dead start 2023-11-24T08:05:16.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f710634e640 1 -- 172.21.15.64:0/4026341502 shutdown_connections 2023-11-24T08:05:16.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f710634e640 1 -- 172.21.15.64:0/4026341502 >> 172.21.15.64:0/4026341502 conn(0x7f71000cec10 msgr2=0x7f71000cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:16.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f710634e640 1 -- 172.21.15.64:0/4026341502 shutdown_connections 2023-11-24T08:05:16.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.357+0000 7f710634e640 1 -- 172.21.15.64:0/4026341502 wait complete. 2023-11-24T08:05:16.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.359+0000 7f710634e640 1 Processor -- start 2023-11-24T08:05:16.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.359+0000 7f710634e640 1 -- start start 2023-11-24T08:05:16.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f710634e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec004d00 0x7f710013e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f710634e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100139820 0x7f7100139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f710634e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7100152270 0x7f71001526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f710634e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710013a1e0 con 0x7f7100152270 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f710634e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71001f3000 con 0x7f70ec004d00 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f710634e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710013d6d0 con 0x7f7100139820 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f7105b4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7100152270 0x7f71001526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f7105b4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7100152270 0x7f71001526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48254/0 (socket says 172.21.15.64:48254) 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.360+0000 7f7105b4d640 1 -- 172.21.15.64:0/2229226328 learned_addr learned my addr 172.21.15.64:0/2229226328 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f710534c640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec004d00 0x7f710013e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f7104b4b640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100139820 0x7f7100139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f7105b4d640 1 -- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100139820 msgr2=0x7f7100139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f7105b4d640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100139820 0x7f7100139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f7105b4d640 1 -- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec004d00 msgr2=0x7f710013e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f7105b4d640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec004d00 0x7f710013e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f7105b4d640 1 -- 172.21.15.64:0/2229226328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710013a3e0 con 0x7f7100152270 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f710534c640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec004d00 0x7f710013e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f7104b4b640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100139820 0x7f7100139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f7105b4d640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7100152270 0x7f71001526b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f70f800a2c0 tx=0x7f70f8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f70f5ffb640 1 -- 172.21.15.64:0/2229226328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8004d60 con 0x7f7100152270 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.361+0000 7f70f5ffb640 1 -- 172.21.15.64:0/2229226328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f802ce90 con 0x7f7100152270 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.362+0000 7f70f5ffb640 1 -- 172.21.15.64:0/2229226328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80072d0 con 0x7f7100152270 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.362+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710013a6c0 con 0x7f7100152270 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.362+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710013ac80 con 0x7f7100152270 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.363+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71001f3170 con 0x7f7100152270 2023-11-24T08:05:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.364+0000 7f70f5ffb640 1 -- 172.21.15.64:0/2229226328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70f8031030 con 0x7f7100152270 2023-11-24T08:05:16.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.365+0000 7f70f5ffb640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70e406ae40 0x7f70e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.365+0000 7f70f5ffb640 1 -- 172.21.15.64:0/2229226328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f70f80aa180 con 0x7f7100152270 2023-11-24T08:05:16.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.365+0000 7f710534c640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70e406ae40 0x7f70e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.366+0000 7f710534c640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70e406ae40 0x7f70e406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009a00 tx=0x7f70fc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:16.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.369+0000 7f70f5ffb640 1 -- 172.21.15.64:0/2229226328 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71001f3170 con 0x7f7100152270 2023-11-24T08:05:16.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.556+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71001f3170 con 0x7f7100152270 2023-11-24T08:05:16.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.557+0000 7f70f5ffb640 1 -- 172.21.15.64:0/2229226328 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71001f3170 con 0x7f7100152270 2023-11-24T08:05:16.559 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:05:16.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70e406ae40 msgr2=0x7f70e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70e406ae40 0x7f70e406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009a00 tx=0x7f70fc008040 comp rx=0 tx=0).stop 2023-11-24T08:05:16.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7100152270 msgr2=0x7f71001526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7100152270 0x7f71001526b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f70f800a2c0 tx=0x7f70f8001d80 comp rx=0 tx=0).stop 2023-11-24T08:05:16.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 shutdown_connections 2023-11-24T08:05:16.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70e406ae40 0x7f70e406d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7100152270 0x7f71001526b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7100139820 0x7f7100139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 --2- 172.21.15.64:0/2229226328 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70ec004d00 0x7f710013e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 >> 172.21.15.64:0/2229226328 conn(0x7f71000cec10 msgr2=0x7f71000cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:16.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.559+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 shutdown_connections 2023-11-24T08:05:16.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.560+0000 7f710634e640 1 -- 172.21.15.64:0/2229226328 wait complete. 2023-11-24T08:05:16.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:05:16.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.802+0000 7f9bd2cd1640 1 Processor -- start 2023-11-24T08:05:16.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.802+0000 7f9bd2cd1640 1 -- start start 2023-11-24T08:05:16.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.803+0000 7f9bd2cd1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bcc15bfd0 0x7f9bcc0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.803+0000 7f9bd2cd1640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9bcc0b4120 0x7f9bcc0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.803+0000 7f9bd2cd1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc15c460 con 0x7f9bcc158af0 2023-11-24T08:05:16.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.803+0000 7f9bd2cd1640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b4af0 con 0x7f9bcc0b4120 2023-11-24T08:05:16.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.803+0000 7f9bd2cd1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc158ed0 con 0x7f9bcc15bfd0 2023-11-24T08:05:16.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.804+0000 7f9bd1ccf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9bcc158af0 0x7f9bcc15c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46406/0 (socket says 172.21.15.64:46406) 2023-11-24T08:05:16.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.804+0000 7f9bd1ccf640 1 -- 172.21.15.64:0/3532285508 learned_addr learned my addr 172.21.15.64:0/3532285508 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.804+0000 7f9bd14ce640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bcc15bfd0 0x7f9bcc0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.804+0000 7f9bd24d0640 1 --2- 172.21.15.64:0/3532285508 >> v2:172.21.15.91:3300/0 conn(0x7f9bcc0b4120 0x7f9bcc0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.804+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004593688 0 0) 0x7f9bcc15c460 con 0x7f9bcc158af0 2023-11-24T08:05:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.804+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb8003520 con 0x7f9bcc158af0 2023-11-24T08:05:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248438411 0 0) 0x7f9bb8003520 con 0x7f9bcc158af0 2023-11-24T08:05:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bcc15bfd0 msgr2=0x7f9bcc0cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bcc15bfd0 0x7f9bcc0cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 >> v2:172.21.15.91:3300/0 conn(0x7f9bcc0b4120 msgr2=0x7f9bcc0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 --2- 172.21.15.64:0/3532285508 >> v2:172.21.15.91:3300/0 conn(0x7f9bcc0b4120 0x7f9bcc0b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0cc7e0 con 0x7f9bcc158af0 2023-11-24T08:05:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f9bc8001670 con 0x7f9bcc158af0 2023-11-24T08:05:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 >> v1:172.21.15.64:6789/0 conn(0x7f9bcc158af0 legacy=0x7f9bcc15c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb80039e0 0x7f9bb8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80047c0 0x7f9bb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bcc15bfd0 0x7f9bb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b4af0 con 0x7f9bb80047c0 2023-11-24T08:05:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc158ed0 con 0x7f9bb80039e0 2023-11-24T08:05:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb800b980 con 0x7f9bcc15bfd0 2023-11-24T08:05:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd14ce640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80047c0 0x7f9bb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd1ccf640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb80039e0 0x7f9bb8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.805+0000 7f9bd24d0640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bcc15bfd0 0x7f9bb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.806+0000 7f9bd14ce640 1 -- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bcc15bfd0 msgr2=0x7f9bb8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.806+0000 7f9bd14ce640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bcc15bfd0 0x7f9bb8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.806+0000 7f9bd14ce640 1 -- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb80039e0 msgr2=0x7f9bb8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.806+0000 7f9bd14ce640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb80039e0 0x7f9bb8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.806+0000 7f9bd14ce640 1 -- 172.21.15.64:0/3532285508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8003520 con 0x7f9bb80047c0 2023-11-24T08:05:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.806+0000 7f9bd1ccf640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb80039e0 0x7f9bb8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.807+0000 7f9bd14ce640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80047c0 0x7f9bb8004c40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009af0 tx=0x7f9bb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc6ff95dc510819 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.807+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc02c6b0 con 0x7f9bb80047c0 2023-11-24T08:05:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.807+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc02c810 con 0x7f9bb80047c0 2023-11-24T08:05:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.807+0000 7f9bd0ccd640 1 -- 172.21.15.64:0/3532285508 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc02cba0 con 0x7f9bb80047c0 2023-11-24T08:05:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.808+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/3532285508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80047c0 msgr2=0x7f9bb8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.808+0000 7f9bd2cd1640 1 --2- 172.21.15.64:0/3532285508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80047c0 0x7f9bb8004c40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009af0 tx=0x7f9bb80057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.808+0000 7f9bd24d0640 1 -- 172.21.15.64:0/3532285508 reap_dead start 2023-11-24T08:05:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.808+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/3532285508 shutdown_connections 2023-11-24T08:05:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.808+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/3532285508 >> 172.21.15.64:0/3532285508 conn(0x7f9bcc0b0b10 msgr2=0x7f9bcc0b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.808+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/3532285508 shutdown_connections 2023-11-24T08:05:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.808+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/3532285508 wait complete. 2023-11-24T08:05:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.810+0000 7f9bd2cd1640 1 Processor -- start 2023-11-24T08:05:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.810+0000 7f9bd2cd1640 1 -- start start 2023-11-24T08:05:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd2cd1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80047c0 0x7f9bcc0cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd2cd1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bcc0c8190 0x7f9bcc141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd2cd1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bcc152620 0x7f9bcc152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd2cd1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1547b0 con 0x7f9bcc152620 2023-11-24T08:05:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd2cd1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc142290 con 0x7f9bcc0c8190 2023-11-24T08:05:16.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd2cd1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0c8620 con 0x7f9bb80047c0 2023-11-24T08:05:16.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd24d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bcc152620 0x7f9bcc152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd24d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bcc152620 0x7f9bcc152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48264/0 (socket says 172.21.15.64:48264) 2023-11-24T08:05:16.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd24d0640 1 -- 172.21.15.64:0/1399294627 learned_addr learned my addr 172.21.15.64:0/1399294627 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:16.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd1ccf640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80047c0 0x7f9bcc0cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.811+0000 7f9bd14ce640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bcc0c8190 0x7f9bcc141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bd24d0640 1 -- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80047c0 msgr2=0x7f9bcc0cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bd24d0640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80047c0 0x7f9bcc0cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bd24d0640 1 -- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bcc0c8190 msgr2=0x7f9bcc141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bd24d0640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bcc0c8190 0x7f9bcc141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bd24d0640 1 -- 172.21.15.64:0/1399294627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc154a00 con 0x7f9bcc152620 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bd24d0640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bcc152620 0x7f9bcc152ad0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0b1ae0 tx=0x7f9bc40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bc27fc640 1 -- 172.21.15.64:0/1399294627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400e070 con 0x7f9bcc152620 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bc27fc640 1 -- 172.21.15.64:0/1399294627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc4011ed0 con 0x7f9bcc152620 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc154ce0 con 0x7f9bcc152620 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bc27fc640 1 -- 172.21.15.64:0/1399294627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4003990 con 0x7f9bcc152620 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.812+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc153010 con 0x7f9bcc152620 2023-11-24T08:05:16.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.814+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc04e840 con 0x7f9bcc152620 2023-11-24T08:05:16.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.815+0000 7f9bc27fc640 1 -- 172.21.15.64:0/1399294627 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9bc4003b10 con 0x7f9bcc152620 2023-11-24T08:05:16.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.815+0000 7f9bc27fc640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9bb006ae70 0x7f9bb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:16.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.816+0000 7f9bc27fc640 1 -- 172.21.15.64:0/1399294627 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9bc4089930 con 0x7f9bcc152620 2023-11-24T08:05:16.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.816+0000 7f9bd1ccf640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9bb006ae70 0x7f9bb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:16.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.817+0000 7f9bd1ccf640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9bb006ae70 0x7f9bb006d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8001eb0 tx=0x7f9bc8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:16.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:16.820+0000 7f9bc27fc640 1 -- 172.21.15.64:0/1399294627 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bc4058850 con 0x7f9bcc152620 2023-11-24T08:05:17.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.016+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9bcc06d380 con 0x7f9bcc152620 2023-11-24T08:05:17.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.017+0000 7f9bc27fc640 1 -- 172.21.15.64:0/1399294627 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9bc40583e0 con 0x7f9bcc152620 2023-11-24T08:05:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:17.019 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:05:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-11-24T08:05:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-11-24T08:05:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:17.019 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:05:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9bb006ae70 msgr2=0x7f9bb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9bb006ae70 0x7f9bb006d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8001eb0 tx=0x7f9bc8009040 comp rx=0 tx=0).stop 2023-11-24T08:05:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bcc152620 msgr2=0x7f9bcc152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bcc152620 0x7f9bcc152ad0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0b1ae0 tx=0x7f9bc40115a0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 shutdown_connections 2023-11-24T08:05:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9bb006ae70 0x7f9bb006d330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bcc152620 0x7f9bcc152ad0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bcc0c8190 0x7f9bcc141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 --2- 172.21.15.64:0/1399294627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80047c0 0x7f9bcc0cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 >> 172.21.15.64:0/1399294627 conn(0x7f9bcc0b0b10 msgr2=0x7f9bcc0b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 shutdown_connections 2023-11-24T08:05:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.019+0000 7f9bd2cd1640 1 -- 172.21.15.64:0/1399294627 wait complete. 2023-11-24T08:05:17.041 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:05:17.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:05:17.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.271+0000 7fedd237a640 1 Processor -- start 2023-11-24T08:05:17.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.271+0000 7fedd237a640 1 -- start start 2023-11-24T08:05:17.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.272+0000 7fedd237a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fedcc0d1d90 0x7fedcc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:17.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.272+0000 7fedd237a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc0d0fa0 0x7fedcc0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:17.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.272+0000 7fedd237a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0c6bf0 con 0x7fedcc15d3b0 2023-11-24T08:05:17.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.272+0000 7fedd237a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0d13f0 con 0x7fedcc0d1d90 2023-11-24T08:05:17.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.272+0000 7fedd237a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0cedc0 con 0x7fedcc0d0fa0 2023-11-24T08:05:17.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.273+0000 7fedd1b79640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc0d0fa0 0x7fedcc0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:17.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.273+0000 7fedd0b77640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fedcc0d1d90 0x7fedcc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:17.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.273+0000 7fedd1378640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fedcc15d3b0 0x7fedcc0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:51188/0 (socket says 172.21.15.64:51188) 2023-11-24T08:05:17.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.273+0000 7fedd1378640 1 -- 172.21.15.64:0/1672804279 learned_addr learned my addr 172.21.15.64:0/1672804279 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:17.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.274+0000 7fedd1b79640 1 -- 172.21.15.64:0/1672804279 >> v2:172.21.15.91:3300/0 conn(0x7fedcc0d1d90 msgr2=0x7fedcc0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:17.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.274+0000 7fedd1b79640 1 --2- 172.21.15.64:0/1672804279 >> v2:172.21.15.91:3300/0 conn(0x7fedcc0d1d90 0x7fedcc0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.274+0000 7fedd1b79640 1 -- 172.21.15.64:0/1672804279 >> v1:172.21.15.91:6789/0 conn(0x7fedcc15d3b0 legacy=0x7fedcc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:17.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.274+0000 7fedd1b79640 1 -- 172.21.15.64:0/1672804279 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0c6e40 con 0x7fedcc0d0fa0 2023-11-24T08:05:17.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.274+0000 7fedd0b77640 1 --2- 172.21.15.64:0/1672804279 >> v2:172.21.15.91:3300/0 conn(0x7fedcc0d1d90 0x7fedcc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:05:17.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.275+0000 7fedd1b79640 1 --2- 172.21.15.64:0/1672804279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc0d0fa0 0x7fedcc0b3ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fedc40097a0 tx=0x7fedc402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a9ad8b9adaa932e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:17.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedcbfff640 1 -- 172.21.15.64:0/1672804279 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4031780 con 0x7fedcc0d0fa0 2023-11-24T08:05:17.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedcbfff640 1 -- 172.21.15.64:0/1672804279 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc40318e0 con 0x7fedcc0d0fa0 2023-11-24T08:05:17.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedcbfff640 1 -- 172.21.15.64:0/1672804279 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4031c70 con 0x7fedcc0d0fa0 2023-11-24T08:05:17.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedd237a640 1 -- 172.21.15.64:0/1672804279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc0d0fa0 msgr2=0x7fedcc0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:17.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedd237a640 1 --2- 172.21.15.64:0/1672804279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc0d0fa0 0x7fedcc0b3ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fedc40097a0 tx=0x7fedc402b820 comp rx=0 tx=0).stop 2023-11-24T08:05:17.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedd237a640 1 -- 172.21.15.64:0/1672804279 shutdown_connections 2023-11-24T08:05:17.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedd237a640 1 --2- 172.21.15.64:0/1672804279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc0d0fa0 0x7fedcc0b3ca0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedd237a640 1 --2- 172.21.15.64:0/1672804279 >> v2:172.21.15.91:3300/0 conn(0x7fedcc0d1d90 0x7fedcc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedd237a640 1 -- 172.21.15.64:0/1672804279 >> 172.21.15.64:0/1672804279 conn(0x7fedcc0d30a0 msgr2=0x7fedcc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:17.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedd237a640 1 -- 172.21.15.64:0/1672804279 shutdown_connections 2023-11-24T08:05:17.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.276+0000 7fedd237a640 1 -- 172.21.15.64:0/1672804279 wait complete. 2023-11-24T08:05:17.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.278+0000 7fedd237a640 1 Processor -- start 2023-11-24T08:05:17.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.278+0000 7fedd237a640 1 -- start start 2023-11-24T08:05:17.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.279+0000 7fedd237a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedcc0d0fa0 0x7fedcc0b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:17.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.279+0000 7fedd237a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc0d1d90 0x7fedcc159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:17.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.279+0000 7fedd237a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc15d3b0 0x7fedcc1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:17.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.279+0000 7fedd237a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0ca170 con 0x7fedcc0d1d90 2023-11-24T08:05:17.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.279+0000 7fedd237a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0ca450 con 0x7fedcc0d0fa0 2023-11-24T08:05:17.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.279+0000 7fedd237a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0ca730 con 0x7fedcc15d3b0 2023-11-24T08:05:17.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc0d1d90 0x7fedcc159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:17.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc0d1d90 0x7fedcc159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48316/0 (socket says 172.21.15.64:48316) 2023-11-24T08:05:17.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 -- 172.21.15.64:0/4157977788 learned_addr learned my addr 172.21.15.64:0/4157977788 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:17.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd1378640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedcc0d0fa0 0x7fedcc0b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:17.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd1b79640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc15d3b0 0x7fedcc1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:17.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 -- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc15d3b0 msgr2=0x7fedcc1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:17.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc15d3b0 0x7fedcc1573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 -- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedcc0d0fa0 msgr2=0x7fedcc0b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:17.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedcc0d0fa0 0x7fedcc0b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 -- 172.21.15.64:0/4157977788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009400 con 0x7fedcc0d1d90 2023-11-24T08:05:17.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.280+0000 7fedd0b77640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc0d1d90 0x7fedcc159c60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fedb800ebe0 tx=0x7fedb800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:17.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.281+0000 7fedc9ffb640 1 -- 172.21.15.64:0/4157977788 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8009760 con 0x7fedcc0d1d90 2023-11-24T08:05:17.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.281+0000 7fedc9ffb640 1 -- 172.21.15.64:0/4157977788 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800cc30 con 0x7fedcc0d1d90 2023-11-24T08:05:17.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.281+0000 7fedc9ffb640 1 -- 172.21.15.64:0/4157977788 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800eee0 con 0x7fedcc0d1d90 2023-11-24T08:05:17.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.281+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc4009400 con 0x7fedcc0d1d90 2023-11-24T08:05:17.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.281+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0cacf0 con 0x7fedcc0d1d90 2023-11-24T08:05:17.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.282+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc04e840 con 0x7fedcc0d1d90 2023-11-24T08:05:17.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.283+0000 7fedc9ffb640 1 -- 172.21.15.64:0/4157977788 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fedb8013470 con 0x7fedcc0d1d90 2023-11-24T08:05:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.284+0000 7fedc9ffb640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fedb006ae40 0x7fedb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.284+0000 7fedc9ffb640 1 -- 172.21.15.64:0/4157977788 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fedb8089580 con 0x7fedcc0d1d90 2023-11-24T08:05:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.285+0000 7fedd1378640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fedb006ae40 0x7fedb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.286+0000 7fedd1378640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fedb006ae40 0x7fedb006d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005c00 tx=0x7fedc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.289+0000 7fedc9ffb640 1 -- 172.21.15.64:0/4157977788 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fedb8054970 con 0x7fedcc0d1d90 2023-11-24T08:05:17.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.482+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fedcc15c780 con 0x7fedcc0d1d90 2023-11-24T08:05:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.978+0000 7fedc9ffb640 1 -- 172.21.15.64:0/4157977788 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7fedb8057f90 con 0x7fedcc0d1d90 2023-11-24T08:05:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-11-24T08:05:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.982+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fedb006ae40 msgr2=0x7fedb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.982+0000 7fedd237a640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fedb006ae40 0x7fedb006d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005c00 tx=0x7fedc0009040 comp rx=0 tx=0).stop 2023-11-24T08:05:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.982+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc0d1d90 msgr2=0x7fedcc159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.982+0000 7fedd237a640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc0d1d90 0x7fedcc159c60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fedb800ebe0 tx=0x7fedb800c520 comp rx=0 tx=0).stop 2023-11-24T08:05:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.983+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 shutdown_connections 2023-11-24T08:05:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.983+0000 7fedd237a640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fedb006ae40 0x7fedb006d300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.983+0000 7fedd237a640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedcc15d3b0 0x7fedcc1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.983+0000 7fedd237a640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc0d1d90 0x7fedcc159c60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.983+0000 7fedd237a640 1 --2- 172.21.15.64:0/4157977788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedcc0d0fa0 0x7fedcc0b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:17.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.983+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 >> 172.21.15.64:0/4157977788 conn(0x7fedcc0d30a0 msgr2=0x7fedcc0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:17.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.985+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 shutdown_connections 2023-11-24T08:05:17.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:17.986+0000 7fedd237a640 1 -- 172.21.15.64:0/4157977788 wait complete. 2023-11-24T08:05:18.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:05:18.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.227+0000 7fdaf08eb640 1 Processor -- start 2023-11-24T08:05:18.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.228+0000 7fdaf08eb640 1 -- start start 2023-11-24T08:05:18.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.229+0000 7fdaf08eb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdaec153ea0 0x7fdaec0ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:18.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.229+0000 7fdaf08eb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 0x7fdaec15d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:18.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.230+0000 7fdaf08eb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaec0ce650 con 0x7fdaec070850 2023-11-24T08:05:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.230+0000 7fdaf08eb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaec15ddd0 con 0x7fdaec153ea0 2023-11-24T08:05:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.230+0000 7fdaf08eb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec15e0a0 con 0x7fdaec154280 2023-11-24T08:05:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.230+0000 7fdaea59c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 0x7fdaec15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.230+0000 7fdaeb59e640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7fdaec070850 0x7fdaec071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:51192/0 (socket says 172.21.15.64:51192) 2023-11-24T08:05:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.230+0000 7fdaeb59e640 1 -- 172.21.15.64:0/442421225 learned_addr learned my addr 172.21.15.64:0/442421225 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.230+0000 7fdaead9d640 1 --2- 172.21.15.64:0/442421225 >> v2:172.21.15.91:3300/0 conn(0x7fdaec153ea0 0x7fdaec0ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.231+0000 7fdaead9d640 1 -- 172.21.15.64:0/442421225 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 msgr2=0x7fdaec15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.231+0000 7fdaead9d640 1 --2- 172.21.15.64:0/442421225 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 0x7fdaec15d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.231+0000 7fdaead9d640 1 -- 172.21.15.64:0/442421225 >> v1:172.21.15.91:6789/0 conn(0x7fdaec070850 legacy=0x7fdaec071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.231+0000 7fdaead9d640 1 -- 172.21.15.64:0/442421225 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec15e380 con 0x7fdaec153ea0 2023-11-24T08:05:18.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.232+0000 7fdaead9d640 1 --2- 172.21.15.64:0/442421225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 0x7fdaec0ad450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009d90 tx=0x7fdadc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26beceff5714fdc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:18.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.233+0000 7fdae9d9b640 1 -- 172.21.15.64:0/442421225 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc02cc50 con 0x7fdaec153ea0 2023-11-24T08:05:18.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.233+0000 7fdae9d9b640 1 -- 172.21.15.64:0/442421225 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc02cdb0 con 0x7fdaec153ea0 2023-11-24T08:05:18.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.233+0000 7fdae9d9b640 1 -- 172.21.15.64:0/442421225 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc034660 con 0x7fdaec153ea0 2023-11-24T08:05:18.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.233+0000 7fdaf08eb640 1 -- 172.21.15.64:0/442421225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 msgr2=0x7fdaec0ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:18.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.233+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/442421225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 0x7fdaec0ad450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009d90 tx=0x7fdadc02bea0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.234+0000 7fdaf08eb640 1 -- 172.21.15.64:0/442421225 shutdown_connections 2023-11-24T08:05:18.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.234+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/442421225 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 0x7fdaec15d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.234+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/442421225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 0x7fdaec0ad450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.234+0000 7fdaf08eb640 1 -- 172.21.15.64:0/442421225 >> 172.21.15.64:0/442421225 conn(0x7fdaec0cec10 msgr2=0x7fdaec0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:18.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.234+0000 7fdaf08eb640 1 -- 172.21.15.64:0/442421225 shutdown_connections 2023-11-24T08:05:18.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.234+0000 7fdaf08eb640 1 -- 172.21.15.64:0/442421225 wait complete. 2023-11-24T08:05:18.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.235+0000 7fdaf08eb640 1 Processor -- start 2023-11-24T08:05:18.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.236+0000 7fdaf08eb640 1 -- start start 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaf08eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec070850 0x7fdaec13f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaf08eb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 0x7fdaec13c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaf08eb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 0x7fdaec14e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaf08eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec14f550 con 0x7fdaec070850 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaf08eb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec13c8b0 con 0x7fdaec153ea0 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaf08eb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec14e940 con 0x7fdaec154280 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaead9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec070850 0x7fdaec13f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaead9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec070850 0x7fdaec13f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48318/0 (socket says 172.21.15.64:48318) 2023-11-24T08:05:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaead9d640 1 -- 172.21.15.64:0/3759890931 learned_addr learned my addr 172.21.15.64:0/3759890931 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaeb59e640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 0x7fdaec14e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaea59c640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 0x7fdaec13c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.237+0000 7fdaead9d640 1 -- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 msgr2=0x7fdaec14e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdaead9d640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 0x7fdaec14e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdaead9d640 1 -- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 msgr2=0x7fdaec13c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdaead9d640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 0x7fdaec13c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdaead9d640 1 -- 172.21.15.64:0/3759890931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc009a70 con 0x7fdaec070850 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdaead9d640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec070850 0x7fdaec13f790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009d90 tx=0x7fdadc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdad37fe640 1 -- 172.21.15.64:0/3759890931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc040650 con 0x7fdaec070850 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdad37fe640 1 -- 172.21.15.64:0/3759890931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc0407b0 con 0x7fdaec070850 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdad37fe640 1 -- 172.21.15.64:0/3759890931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc040b60 con 0x7fdaec070850 2023-11-24T08:05:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.238+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec14ec20 con 0x7fdaec070850 2023-11-24T08:05:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.239+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec14f1e0 con 0x7fdaec070850 2023-11-24T08:05:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.240+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaec04e840 con 0x7fdaec070850 2023-11-24T08:05:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.241+0000 7fdad37fe640 1 -- 172.21.15.64:0/3759890931 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdadc040cc0 con 0x7fdaec070850 2023-11-24T08:05:18.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.242+0000 7fdad37fe640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae70 0x7fdad806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:18.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.242+0000 7fdad37fe640 1 -- 172.21.15.64:0/3759890931 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fdadc0ac240 con 0x7fdaec070850 2023-11-24T08:05:18.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.242+0000 7fdaea59c640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae70 0x7fdad806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.244+0000 7fdaea59c640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae70 0x7fdad806d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdaec154d70 tx=0x7fdae0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.247+0000 7fdad37fe640 1 -- 172.21.15.64:0/3759890931 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdadc0775e0 con 0x7fdaec070850 2023-11-24T08:05:18.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.442+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdaec15f820 con 0x7fdaec070850 2023-11-24T08:05:18.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.992+0000 7fdad37fe640 1 -- 172.21.15.64:0/3759890931 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdadc07ac00 con 0x7fdaec070850 2023-11-24T08:05:18.994 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-11-24T08:05:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.996+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae70 msgr2=0x7fdad806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.996+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae70 0x7fdad806d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdaec154d70 tx=0x7fdae0008040 comp rx=0 tx=0).stop 2023-11-24T08:05:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.996+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec070850 msgr2=0x7fdaec13f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.996+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec070850 0x7fdaec13f790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009d90 tx=0x7fdadc004a40 comp rx=0 tx=0).stop 2023-11-24T08:05:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.997+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 shutdown_connections 2023-11-24T08:05:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.997+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdad806ae70 0x7fdad806d330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.997+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdaec154280 0x7fdaec14e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.997+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdaec153ea0 0x7fdaec13c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.997+0000 7fdaf08eb640 1 --2- 172.21.15.64:0/3759890931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaec070850 0x7fdaec13f790 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.997+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 >> 172.21.15.64:0/3759890931 conn(0x7fdaec0cec10 msgr2=0x7fdaec0ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:19.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.998+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 shutdown_connections 2023-11-24T08:05:19.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:18.998+0000 7fdaf08eb640 1 -- 172.21.15.64:0/3759890931 wait complete. 2023-11-24T08:05:19.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:05:19.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.224+0000 7f0b335be640 1 Processor -- start 2023-11-24T08:05:19.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.224+0000 7f0b335be640 1 -- start start 2023-11-24T08:05:19.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.226+0000 7f0b335be640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0b2c158f40 0x7f0b2c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:19.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.226+0000 7f0b335be640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c0cd2b0 con 0x7f0b2c0cf380 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.226+0000 7f0b335be640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c0d02e0 con 0x7f0b2c0cb9d0 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.226+0000 7f0b335be640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2c0cbe30 con 0x7f0b2c158f40 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.226+0000 7f0b31b5b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0b2c0cf380 0x7f0b2c15b830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32884/0 (socket says 172.21.15.64:32884) 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.226+0000 7f0b31b5b640 1 -- 172.21.15.64:0/3245094775 learned_addr learned my addr 172.21.15.64:0/3245094775 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531399799 0 0) 0x7f0b2c0cd2b0 con 0x7f0b2c0cf380 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b18003520 con 0x7f0b2c0cf380 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b3135a640 1 --2- 172.21.15.64:0/3245094775 >> v2:172.21.15.91:3300/0 conn(0x7f0b2c158f40 0x7f0b2c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 704145698 0 0) 0x7f0b18003520 con 0x7f0b2c0cf380 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 >> v2:172.21.15.91:3300/0 conn(0x7f0b2c158f40 msgr2=0x7f0b2c0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 --2- 172.21.15.64:0/3245094775 >> v2:172.21.15.91:3300/0 conn(0x7f0b2c158f40 0x7f0b2c0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 >> v1:172.21.15.91:6789/0 conn(0x7f0b2c0cb9d0 legacy=0x7f0b2c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c0cd4e0 con 0x7f0b2c0cf380 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0b240019f0 con 0x7f0b2c0cf380 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 >> v1:172.21.15.64:6789/0 conn(0x7f0b2c0cf380 legacy=0x7f0b2c15b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b18003970 0x7f0b18003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b18005180 0x7f0b1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c0cbe30 con 0x7f0b180047c0 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1800c020 con 0x7f0b18003970 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.227+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1800c1e0 con 0x7f0b18005180 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b30b59640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b3135a640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b18003970 0x7f0b18003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b31b5b640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b18005180 0x7f0b1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b30b59640 1 -- 172.21.15.64:0/3245094775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b18005180 msgr2=0x7f0b1800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b30b59640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b18005180 0x7f0b1800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b30b59640 1 -- 172.21.15.64:0/3245094775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b18003970 msgr2=0x7f0b18003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b30b59640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b18003970 0x7f0b18003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b30b59640 1 -- 172.21.15.64:0/3245094775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b18003520 con 0x7f0b180047c0 2023-11-24T08:05:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.228+0000 7f0b31b5b640 1 -- 172.21.15.64:0/3245094775 reap_dead start 2023-11-24T08:05:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.229+0000 7f0b30b59640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b18004c40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c001750 tx=0x7f0b180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68798e34671b6bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0052d0 con 0x7f0b180047c0 2023-11-24T08:05:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c02e050 con 0x7f0b180047c0 2023-11-24T08:05:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b23fff640 1 -- 172.21.15.64:0/3245094775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c003830 con 0x7f0b180047c0 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b335be640 1 -- 172.21.15.64:0/3245094775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 msgr2=0x7f0b18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b335be640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b18004c40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c001750 tx=0x7f0b180057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b335be640 1 -- 172.21.15.64:0/3245094775 shutdown_connections 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b335be640 1 --2- 172.21.15.64:0/3245094775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b18004c40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b335be640 1 -- 172.21.15.64:0/3245094775 >> 172.21.15.64:0/3245094775 conn(0x7f0b2c0cdb20 msgr2=0x7f0b2c0cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b335be640 1 -- 172.21.15.64:0/3245094775 shutdown_connections 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.230+0000 7f0b335be640 1 -- 172.21.15.64:0/3245094775 wait complete. 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.232+0000 7f0b335be640 1 Processor -- start 2023-11-24T08:05:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.232+0000 7f0b335be640 1 -- start start 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.233+0000 7f0b335be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b2c14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.233+0000 7f0b335be640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b2c0ae0d0 0x7f0b2c06b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.233+0000 7f0b335be640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b2c0ca9a0 0x7f0b2c06ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.233+0000 7f0b335be640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c0caf70 con 0x7f0b180047c0 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.233+0000 7f0b335be640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c158840 con 0x7f0b2c0ae0d0 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.233+0000 7f0b335be640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c14b5a0 con 0x7f0b2c0ca9a0 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b3135a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b2c14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b3135a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b2c14abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48326/0 (socket says 172.21.15.64:48326) 2023-11-24T08:05:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b3135a640 1 -- 172.21.15.64:0/3662823957 learned_addr learned my addr 172.21.15.64:0/3662823957 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:19.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b30b59640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b2c0ae0d0 0x7f0b2c06b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:19.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b31b5b640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b2c0ca9a0 0x7f0b2c06ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:19.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b3135a640 1 -- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b2c0ca9a0 msgr2=0x7f0b2c06ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:19.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b3135a640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b2c0ca9a0 0x7f0b2c06ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:19.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b3135a640 1 -- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b2c0ae0d0 msgr2=0x7f0b2c06b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:19.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b3135a640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b2c0ae0d0 0x7f0b2c06b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:19.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b3135a640 1 -- 172.21.15.64:0/3662823957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c14b880 con 0x7f0b180047c0 2023-11-24T08:05:19.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.234+0000 7f0b31b5b640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b2c0ca9a0 0x7f0b2c06ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:19.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.235+0000 7f0b3135a640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b2c14abf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0b280114e0 tx=0x7f0b280119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:19.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.235+0000 7f0b21ffb640 1 -- 172.21.15.64:0/3662823957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28015870 con 0x7f0b180047c0 2023-11-24T08:05:19.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.235+0000 7f0b21ffb640 1 -- 172.21.15.64:0/3662823957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b280159d0 con 0x7f0b180047c0 2023-11-24T08:05:19.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.235+0000 7f0b21ffb640 1 -- 172.21.15.64:0/3662823957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28015d40 con 0x7f0b180047c0 2023-11-24T08:05:19.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.235+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c14bb60 con 0x7f0b180047c0 2023-11-24T08:05:19.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.235+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c14b130 con 0x7f0b180047c0 2023-11-24T08:05:19.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.236+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2c06fb50 con 0x7f0b180047c0 2023-11-24T08:05:19.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.238+0000 7f0b21ffb640 1 -- 172.21.15.64:0/3662823957 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b28027420 con 0x7f0b180047c0 2023-11-24T08:05:19.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.238+0000 7f0b21ffb640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b1006ae40 0x7f0b1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:19.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.239+0000 7f0b21ffb640 1 -- 172.21.15.64:0/3662823957 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0b28089f10 con 0x7f0b180047c0 2023-11-24T08:05:19.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.239+0000 7f0b30b59640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b1006ae40 0x7f0b1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:19.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.240+0000 7f0b30b59640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b1006ae40 0x7f0b1006d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c033000 tx=0x7f0b1c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.243+0000 7f0b21ffb640 1 -- 172.21.15.64:0/3662823957 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b28054f70 con 0x7f0b180047c0 2023-11-24T08:05:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.475+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0b2c04e920 con 0x7f0b180047c0 2023-11-24T08:05:19.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:19.996+0000 7f0b21ffb640 1 -- 172.21.15.64:0/3662823957 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0b28058590 con 0x7f0b180047c0 2023-11-24T08:05:19.999 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T08:05:20.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.000+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b1006ae40 msgr2=0x7f0b1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.000+0000 7f0b335be640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b1006ae40 0x7f0b1006d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c033000 tx=0x7f0b1c009030 comp rx=0 tx=0).stop 2023-11-24T08:05:20.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.001+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 msgr2=0x7f0b2c14abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.001+0000 7f0b335be640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b2c14abf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0b280114e0 tx=0x7f0b280119a0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.001+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 shutdown_connections 2023-11-24T08:05:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.001+0000 7f0b335be640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b1006ae40 0x7f0b1006d300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.001+0000 7f0b335be640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b2c0ca9a0 0x7f0b2c06ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.001+0000 7f0b335be640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b2c0ae0d0 0x7f0b2c06b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.001+0000 7f0b335be640 1 --2- 172.21.15.64:0/3662823957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b180047c0 0x7f0b2c14abf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.001+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 >> 172.21.15.64:0/3662823957 conn(0x7f0b2c0cdb20 msgr2=0x7f0b2c06ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:20.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.002+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 shutdown_connections 2023-11-24T08:05:20.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.002+0000 7f0b335be640 1 -- 172.21.15.64:0/3662823957 wait complete. 2023-11-24T08:05:20.048 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.234+0000 7fa25f8cf640 1 Processor -- start 2023-11-24T08:05:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.234+0000 7fa25f8cf640 1 -- start start 2023-11-24T08:05:20.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.235+0000 7fa25f8cf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa258161930 0x7fa2580ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.235+0000 7fa25f8cf640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2580ad510 con 0x7fa25806d520 2023-11-24T08:05:20.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.235+0000 7fa25f8cf640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2580acc50 con 0x7fa2580719c0 2023-11-24T08:05:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.235+0000 7fa25f8cf640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa258071e60 con 0x7fa258161930 2023-11-24T08:05:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.236+0000 7fa25de6c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa25806d520 0x7fa25806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32894/0 (socket says 172.21.15.64:32894) 2023-11-24T08:05:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.236+0000 7fa25de6c640 1 -- 172.21.15.64:0/3384449293 learned_addr learned my addr 172.21.15.64:0/3384449293 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.236+0000 7fa25d66b640 1 --2- 172.21.15.64:0/3384449293 >> v2:172.21.15.64:3300/0 conn(0x7fa258161930 0x7fa2580ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.236+0000 7fa25d66b640 1 -- 172.21.15.64:0/3384449293 >> v1:172.21.15.91:6789/0 conn(0x7fa2580719c0 legacy=0x7fa2580ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.236+0000 7fa25d66b640 1 -- 172.21.15.64:0/3384449293 >> v1:172.21.15.64:6789/0 conn(0x7fa25806d520 legacy=0x7fa25806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.237+0000 7fa25d66b640 1 -- 172.21.15.64:0/3384449293 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580acec0 con 0x7fa258161930 2023-11-24T08:05:20.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.238+0000 7fa25d66b640 1 --2- 172.21.15.64:0/3384449293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa258161930 0x7fa2580ae6a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa2540097a0 tx=0x7fa25402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9af9ac689127fc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:20.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.238+0000 7fa24ffff640 1 -- 172.21.15.64:0/3384449293 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254030610 con 0x7fa258161930 2023-11-24T08:05:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.238+0000 7fa24ffff640 1 -- 172.21.15.64:0/3384449293 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254030770 con 0x7fa258161930 2023-11-24T08:05:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.238+0000 7fa24ffff640 1 -- 172.21.15.64:0/3384449293 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254030b00 con 0x7fa258161930 2023-11-24T08:05:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.238+0000 7fa25f8cf640 1 -- 172.21.15.64:0/3384449293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa258161930 msgr2=0x7fa2580ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.238+0000 7fa25f8cf640 1 --2- 172.21.15.64:0/3384449293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa258161930 0x7fa2580ae6a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa2540097a0 tx=0x7fa25402b960 comp rx=0 tx=0).stop 2023-11-24T08:05:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.239+0000 7fa25f8cf640 1 -- 172.21.15.64:0/3384449293 shutdown_connections 2023-11-24T08:05:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.239+0000 7fa25f8cf640 1 --2- 172.21.15.64:0/3384449293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa258161930 0x7fa2580ae6a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.239+0000 7fa25f8cf640 1 -- 172.21.15.64:0/3384449293 >> 172.21.15.64:0/3384449293 conn(0x7fa2580b0d80 msgr2=0x7fa2580b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.239+0000 7fa25f8cf640 1 -- 172.21.15.64:0/3384449293 shutdown_connections 2023-11-24T08:05:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.239+0000 7fa25f8cf640 1 -- 172.21.15.64:0/3384449293 wait complete. 2023-11-24T08:05:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.240+0000 7fa25f8cf640 1 Processor -- start 2023-11-24T08:05:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.241+0000 7fa25f8cf640 1 -- start start 2023-11-24T08:05:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.241+0000 7fa25f8cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25806d520 0x7fa2580cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.241+0000 7fa25f8cf640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2580719c0 0x7fa2580ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.241+0000 7fa25f8cf640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa258161930 0x7fa258163090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25f8cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25815ac90 con 0x7fa25806d520 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25f8cf640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581f76c0 con 0x7fa2580719c0 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25f8cf640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581f7930 con 0x7fa258161930 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25d66b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25806d520 0x7fa2580cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25d66b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25806d520 0x7fa2580cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48344/0 (socket says 172.21.15.64:48344) 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25d66b640 1 -- 172.21.15.64:0/431172108 learned_addr learned my addr 172.21.15.64:0/431172108 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25ce6a640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2580719c0 0x7fa2580ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25d66b640 1 -- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa258161930 msgr2=0x7fa258163090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25d66b640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa258161930 0x7fa258163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25d66b640 1 -- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2580719c0 msgr2=0x7fa2580ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25d66b640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2580719c0 0x7fa2580ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25d66b640 1 -- 172.21.15.64:0/431172108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa254009400 con 0x7fa25806d520 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.242+0000 7fa25ce6a640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2580719c0 0x7fa2580ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.243+0000 7fa25d66b640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25806d520 0x7fa2580cb860 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa254009890 tx=0x7fa2540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.243+0000 7fa24dffb640 1 -- 172.21.15.64:0/431172108 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25403e070 con 0x7fa25806d520 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.243+0000 7fa24dffb640 1 -- 172.21.15.64:0/431172108 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254039a40 con 0x7fa25806d520 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.243+0000 7fa24dffb640 1 -- 172.21.15.64:0/431172108 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254039dd0 con 0x7fa25806d520 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.243+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2581f7c10 con 0x7fa25806d520 2023-11-24T08:05:20.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.243+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2581f81d0 con 0x7fa25806d520 2023-11-24T08:05:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.244+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa258074770 con 0x7fa25806d520 2023-11-24T08:05:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.245+0000 7fa24dffb640 1 -- 172.21.15.64:0/431172108 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa25404b400 con 0x7fa25806d520 2023-11-24T08:05:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.246+0000 7fa24dffb640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa23c06ae40 0x7fa23c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.246+0000 7fa24dffb640 1 -- 172.21.15.64:0/431172108 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2540abbe0 con 0x7fa25806d520 2023-11-24T08:05:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.247+0000 7fa25ce6a640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa23c06ae40 0x7fa23c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.248+0000 7fa25ce6a640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa23c06ae40 0x7fa23c06d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa2581f5110 tx=0x7fa248009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.251+0000 7fa24dffb640 1 -- 172.21.15.64:0/431172108 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa254076bf0 con 0x7fa25806d520 2023-11-24T08:05:20.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.439+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa25806e290 con 0x7fa25806d520 2023-11-24T08:05:20.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.441+0000 7fa24dffb640 1 -- 172.21.15.64:0/431172108 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fa254034090 con 0x7fa25806d520 2023-11-24T08:05:20.443 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:20.443 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:19.477573+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:05:17.535151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:05:18.499872+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:20.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa23c06ae40 msgr2=0x7fa23c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa23c06ae40 0x7fa23c06d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa2581f5110 tx=0x7fa248009040 comp rx=0 tx=0).stop 2023-11-24T08:05:20.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25806d520 msgr2=0x7fa2580cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25806d520 0x7fa2580cb860 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa254009890 tx=0x7fa2540046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 shutdown_connections 2023-11-24T08:05:20.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa23c06ae40 0x7fa23c06d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa258161930 0x7fa258163090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2580719c0 0x7fa2580ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 --2- 172.21.15.64:0/431172108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25806d520 0x7fa2580cb860 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 >> 172.21.15.64:0/431172108 conn(0x7fa2580b0d80 msgr2=0x7fa2580b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 shutdown_connections 2023-11-24T08:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.444+0000 7fa25f8cf640 1 -- 172.21.15.64:0/431172108 wait complete. 2023-11-24T08:05:20.464 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:20.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.683+0000 7fa056be5640 1 Processor -- start 2023-11-24T08:05:20.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.684+0000 7fa056be5640 1 -- start start 2023-11-24T08:05:20.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.685+0000 7fa056be5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa05015b2c0 0x7fa0500cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.685+0000 7fa056be5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0500b46a0 0x7fa05015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.685+0000 7fa056be5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0500d00f0 con 0x7fa05015bbe0 2023-11-24T08:05:20.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.685+0000 7fa056be5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0500d03d0 con 0x7fa05015b2c0 2023-11-24T08:05:20.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.685+0000 7fa056be5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500d06b0 con 0x7fa0500b46a0 2023-11-24T08:05:20.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.685+0000 7fa054981640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa05015b2c0 0x7fa0500cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.685+0000 7fa055182640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa05015bbe0 0x7fa05006ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32898/0 (socket says 172.21.15.64:32898) 2023-11-24T08:05:20.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.685+0000 7fa055182640 1 -- 172.21.15.64:0/1905825565 learned_addr learned my addr 172.21.15.64:0/1905825565 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:20.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.686+0000 7fa04ffff640 1 --2- 172.21.15.64:0/1905825565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0500b46a0 0x7fa05015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.686+0000 7fa054981640 1 -- 172.21.15.64:0/1905825565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0500b46a0 msgr2=0x7fa05015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:20.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.686+0000 7fa04f7fe640 1 -- 172.21.15.64:0/1905825565 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233584133 0 0) 0x7fa0500d00f0 con 0x7fa05015bbe0 2023-11-24T08:05:20.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.686+0000 7fa054981640 1 --2- 172.21.15.64:0/1905825565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0500b46a0 0x7fa05015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.686+0000 7fa054981640 1 -- 172.21.15.64:0/1905825565 >> v1:172.21.15.64:6789/0 conn(0x7fa05015bbe0 legacy=0x7fa05006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.686+0000 7fa054981640 1 -- 172.21.15.64:0/1905825565 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500d0990 con 0x7fa05015b2c0 2023-11-24T08:05:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.686+0000 7fa04ffff640 1 --2- 172.21.15.64:0/1905825565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0500b46a0 0x7fa05015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.687+0000 7fa054981640 1 --2- 172.21.15.64:0/1905825565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05015b2c0 0x7fa0500cfaf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa040009d90 tx=0x7fa04002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabade5719cedf1e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:20.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.687+0000 7fa04f7fe640 1 -- 172.21.15.64:0/1905825565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04002cbd0 con 0x7fa05015b2c0 2023-11-24T08:05:20.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.687+0000 7fa04f7fe640 1 -- 172.21.15.64:0/1905825565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04002cd30 con 0x7fa05015b2c0 2023-11-24T08:05:20.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.687+0000 7fa04f7fe640 1 -- 172.21.15.64:0/1905825565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04002cea0 con 0x7fa05015b2c0 2023-11-24T08:05:20.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.688+0000 7fa056be5640 1 -- 172.21.15.64:0/1905825565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05015b2c0 msgr2=0x7fa0500cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.688+0000 7fa056be5640 1 --2- 172.21.15.64:0/1905825565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05015b2c0 0x7fa0500cfaf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa040009d90 tx=0x7fa04002be20 comp rx=0 tx=0).stop 2023-11-24T08:05:20.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.688+0000 7fa056be5640 1 -- 172.21.15.64:0/1905825565 shutdown_connections 2023-11-24T08:05:20.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.688+0000 7fa056be5640 1 --2- 172.21.15.64:0/1905825565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa0500b46a0 0x7fa05015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.688+0000 7fa056be5640 1 --2- 172.21.15.64:0/1905825565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05015b2c0 0x7fa0500cfaf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.688+0000 7fa056be5640 1 -- 172.21.15.64:0/1905825565 >> 172.21.15.64:0/1905825565 conn(0x7fa0500b2e00 msgr2=0x7fa0500b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:20.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.688+0000 7fa056be5640 1 -- 172.21.15.64:0/1905825565 shutdown_connections 2023-11-24T08:05:20.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.688+0000 7fa056be5640 1 -- 172.21.15.64:0/1905825565 wait complete. 2023-11-24T08:05:20.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.690+0000 7fa056be5640 1 Processor -- start 2023-11-24T08:05:20.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.690+0000 7fa056be5640 1 -- start start 2023-11-24T08:05:20.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.691+0000 7fa056be5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0500b46a0 0x7fa050153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.691+0000 7fa056be5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa05015b2c0 0x7fa0500ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.691+0000 7fa056be5640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa05015bbe0 0x7fa0500d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.691+0000 7fa056be5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500cba30 con 0x7fa0500b46a0 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.691+0000 7fa056be5640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0501f2a80 con 0x7fa05015bbe0 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.691+0000 7fa056be5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500caeb0 con 0x7fa05015b2c0 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.691+0000 7fa054981640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0500b46a0 0x7fa050153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa054981640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0500b46a0 0x7fa050153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48366/0 (socket says 172.21.15.64:48366) 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa054981640 1 -- 172.21.15.64:0/3856440877 learned_addr learned my addr 172.21.15.64:0/3856440877 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa04ffff640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa05015b2c0 0x7fa0500ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa054981640 1 -- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa05015b2c0 msgr2=0x7fa0500ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa054981640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa05015b2c0 0x7fa0500ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa054981640 1 -- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa05015bbe0 msgr2=0x7fa0500d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa055182640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa05015bbe0 0x7fa0500d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa054981640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa05015bbe0 0x7fa0500d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa054981640 1 -- 172.21.15.64:0/3856440877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040009a70 con 0x7fa0500b46a0 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa04ffff640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa05015b2c0 0x7fa0500ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa055182640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa05015bbe0 0x7fa0500d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.692+0000 7fa054981640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0500b46a0 0x7fa050153f50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa040009d90 tx=0x7fa040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:20.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.693+0000 7fa04d7fa640 1 -- 172.21.15.64:0/3856440877 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04002cea0 con 0x7fa0500b46a0 2023-11-24T08:05:20.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.693+0000 7fa04d7fa640 1 -- 172.21.15.64:0/3856440877 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa040040650 con 0x7fa0500b46a0 2023-11-24T08:05:20.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.693+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0500cb190 con 0x7fa0500b46a0 2023-11-24T08:05:20.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.693+0000 7fa04d7fa640 1 -- 172.21.15.64:0/3856440877 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040040a00 con 0x7fa0500b46a0 2023-11-24T08:05:20.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.693+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0500cb750 con 0x7fa0500b46a0 2023-11-24T08:05:20.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.694+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa050074770 con 0x7fa0500b46a0 2023-11-24T08:05:20.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.695+0000 7fa04d7fa640 1 -- 172.21.15.64:0/3856440877 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa040040b60 con 0x7fa0500b46a0 2023-11-24T08:05:20.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.697+0000 7fa04d7fa640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa03006ae40 0x7fa03006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:20.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.697+0000 7fa04d7fa640 1 -- 172.21.15.64:0/3856440877 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0400ac440 con 0x7fa0500b46a0 2023-11-24T08:05:20.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.697+0000 7fa04ffff640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa03006ae40 0x7fa03006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:20.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.698+0000 7fa04ffff640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa03006ae40 0x7fa03006d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa038006fd0 tx=0x7fa038008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:20.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.701+0000 7fa04d7fa640 1 -- 172.21.15.64:0/3856440877 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0400774e0 con 0x7fa0500b46a0 2023-11-24T08:05:20.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.897+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0500b3c00 con 0x7fa0500b46a0 2023-11-24T08:05:20.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.900+0000 7fa04d7fa640 1 -- 172.21.15.64:0/3856440877 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fa04007ab00 con 0x7fa0500b46a0 2023-11-24T08:05:20.902 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:20.903 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:19.477573+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:05:17.535151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:05:18.499872+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:20.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa03006ae40 msgr2=0x7fa03006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa03006ae40 0x7fa03006d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa038006fd0 tx=0x7fa038008040 comp rx=0 tx=0).stop 2023-11-24T08:05:20.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0500b46a0 msgr2=0x7fa050153f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:20.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0500b46a0 0x7fa050153f50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa040009d90 tx=0x7fa040004a40 comp rx=0 tx=0).stop 2023-11-24T08:05:20.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 shutdown_connections 2023-11-24T08:05:20.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa03006ae40 0x7fa03006d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa05015bbe0 0x7fa0500d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa05015b2c0 0x7fa0500ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 --2- 172.21.15.64:0/3856440877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0500b46a0 0x7fa050153f50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:20.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 >> 172.21.15.64:0/3856440877 conn(0x7fa0500b2e00 msgr2=0x7fa0500b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:20.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 shutdown_connections 2023-11-24T08:05:20.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:20.903+0000 7fa056be5640 1 -- 172.21.15.64:0/3856440877 wait complete. 2023-11-24T08:05:20.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:05:21.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.131+0000 7f8090b63640 1 Processor -- start 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.132+0000 7f8090b63640 1 -- start start 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.133+0000 7f8090b63640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f808c15d3b0 0x7f808c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.133+0000 7f8090b63640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f808c0d0fa0 0x7f808c0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.133+0000 7f8090b63640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c0c6bf0 con 0x7f808c0d1bc0 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.133+0000 7f8090b63640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808c0d1430 con 0x7f808c15d3b0 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.133+0000 7f8090b63640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808c0d1fa0 con 0x7f808c0d0fa0 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.133+0000 7f808b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f808c15d3b0 0x7f808c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.133+0000 7f808b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f808c15d3b0 0x7f808c0ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48372/0 (socket says 172.21.15.64:48372) 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.133+0000 7f808b7fe640 1 -- 172.21.15.64:0/1391867859 learned_addr learned my addr 172.21.15.64:0/1391867859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:21.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.134+0000 7f808b7fe640 1 -- 172.21.15.64:0/1391867859 >> v2:172.21.15.91:3300/0 conn(0x7f808c0d0fa0 msgr2=0x7f808c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.134+0000 7f808b7fe640 1 --2- 172.21.15.64:0/1391867859 >> v2:172.21.15.91:3300/0 conn(0x7f808c0d0fa0 0x7f808c0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.134+0000 7f808b7fe640 1 -- 172.21.15.64:0/1391867859 >> v1:172.21.15.91:6789/0 conn(0x7f808c0d1bc0 legacy=0x7f808c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.134+0000 7f808b7fe640 1 -- 172.21.15.64:0/1391867859 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0cedc0 con 0x7f808c15d3b0 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.135+0000 7f808b7fe640 1 --2- 172.21.15.64:0/1391867859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c0ce6c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f808c08c4a0 tx=0x7f807c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681680dd1a7409e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.135+0000 7f808a7fc640 1 -- 172.21.15.64:0/1391867859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c02c9f0 con 0x7f808c15d3b0 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.135+0000 7f808a7fc640 1 -- 172.21.15.64:0/1391867859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c02cb50 con 0x7f808c15d3b0 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.135+0000 7f808a7fc640 1 -- 172.21.15.64:0/1391867859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0343f0 con 0x7f808c15d3b0 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.136+0000 7f8090b63640 1 -- 172.21.15.64:0/1391867859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 msgr2=0x7f808c0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.136+0000 7f8090b63640 1 --2- 172.21.15.64:0/1391867859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c0ce6c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f808c08c4a0 tx=0x7f807c02be50 comp rx=0 tx=0).stop 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.136+0000 7f8090b63640 1 -- 172.21.15.64:0/1391867859 shutdown_connections 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.136+0000 7f8090b63640 1 --2- 172.21.15.64:0/1391867859 >> v2:172.21.15.91:3300/0 conn(0x7f808c0d0fa0 0x7f808c0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:21.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.136+0000 7f8090b63640 1 --2- 172.21.15.64:0/1391867859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c0ce6c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:21.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.136+0000 7f8090b63640 1 -- 172.21.15.64:0/1391867859 >> 172.21.15.64:0/1391867859 conn(0x7f808c0d30a0 msgr2=0x7f808c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:21.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.136+0000 7f8090b63640 1 -- 172.21.15.64:0/1391867859 shutdown_connections 2023-11-24T08:05:21.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.136+0000 7f8090b63640 1 -- 172.21.15.64:0/1391867859 wait complete. 2023-11-24T08:05:21.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.138+0000 7f8090b63640 1 Processor -- start 2023-11-24T08:05:21.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.138+0000 7f8090b63640 1 -- start start 2023-11-24T08:05:21.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f8090b63640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f808c0d0fa0 0x7f808c1e6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:21.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f8090b63640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f808c0d1bc0 0x7f808c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:21.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f8090b63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c1e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:21.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f8090b63640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b61a0 con 0x7f808c15d3b0 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f8090b63640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b6470 con 0x7f808c0d0fa0 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f8090b63640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b6750 con 0x7f808c0d1bc0 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f808bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c1e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f808bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c1e75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48380/0 (socket says 172.21.15.64:48380) 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f808bfff640 1 -- 172.21.15.64:0/3213822789 learned_addr learned my addr 172.21.15.64:0/3213822789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f808b7fe640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f808c0d0fa0 0x7f808c1e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f808bfff640 1 -- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f808c0d1bc0 msgr2=0x7f808c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.139+0000 7f808affd640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f808c0d1bc0 0x7f808c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f808bfff640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f808c0d1bc0 0x7f808c0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f808bfff640 1 -- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f808c0d0fa0 msgr2=0x7f808c1e6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f808bfff640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f808c0d0fa0 0x7f808c1e6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:21.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f808bfff640 1 -- 172.21.15.64:0/3213822789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c009b30 con 0x7f808c15d3b0 2023-11-24T08:05:21.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f808bfff640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c1e75d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f808400b490 tx=0x7f808400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:21.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f8073fff640 1 -- 172.21.15.64:0/3213822789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084011830 con 0x7f808c15d3b0 2023-11-24T08:05:21.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f8073fff640 1 -- 172.21.15.64:0/3213822789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80840119b0 con 0x7f808c15d3b0 2023-11-24T08:05:21.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f8073fff640 1 -- 172.21.15.64:0/3213822789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084011d60 con 0x7f808c15d3b0 2023-11-24T08:05:21.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c0b6a30 con 0x7f808c15d3b0 2023-11-24T08:05:21.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.140+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c0b6ff0 con 0x7f808c15d3b0 2023-11-24T08:05:21.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.142+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808c04e840 con 0x7f808c15d3b0 2023-11-24T08:05:21.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.143+0000 7f8073fff640 1 -- 172.21.15.64:0/3213822789 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f808401a400 con 0x7f808c15d3b0 2023-11-24T08:05:21.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.144+0000 7f8073fff640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f807806ae40 0x7f807806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:21.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.144+0000 7f8073fff640 1 -- 172.21.15.64:0/3213822789 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8084089660 con 0x7f808c15d3b0 2023-11-24T08:05:21.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.145+0000 7f808b7fe640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f807806ae40 0x7f807806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:21.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.146+0000 7f808b7fe640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f807806ae40 0x7f807806d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f807c035000 tx=0x7f807c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:21.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.149+0000 7f8073fff640 1 -- 172.21.15.64:0/3213822789 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80840547a0 con 0x7f808c15d3b0 2023-11-24T08:05:21.238 INFO:tasks.ceph.mds.c.smithi064.stderr:2023-11-24T08:05:21.236+0000 7f6aa7c15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:05:21.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:21.386+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f808c067600 con 0x7f808c15d3b0 2023-11-24T08:05:22.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.024+0000 7f8073fff640 1 -- 172.21.15.64:0/3213822789 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8084057dc0 con 0x7f808c15d3b0 2023-11-24T08:05:22.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f807806ae40 msgr2=0x7f807806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:22.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f807806ae40 0x7f807806d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f807c035000 tx=0x7f807c030000 comp rx=0 tx=0).stop 2023-11-24T08:05:22.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 msgr2=0x7f808c1e75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c1e75d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f808400b490 tx=0x7f808400b950 comp rx=0 tx=0).stop 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 shutdown_connections 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f807806ae40 0x7f807806d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f808c15d3b0 0x7f808c1e75d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f808c0d1bc0 0x7f808c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 --2- 172.21.15.64:0/3213822789 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f808c0d0fa0 0x7f808c1e6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 >> 172.21.15.64:0/3213822789 conn(0x7f808c0d30a0 msgr2=0x7f808c0c7920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 shutdown_connections 2023-11-24T08:05:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.028+0000 7f8090b63640 1 -- 172.21.15.64:0/3213822789 wait complete. 2023-11-24T08:05:22.053 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:05:22.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:05:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.276+0000 7f979d7fb640 1 Processor -- start 2023-11-24T08:05:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.276+0000 7f979d7fb640 1 -- start start 2023-11-24T08:05:22.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.278+0000 7f979d7fb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97980b4140 0x7f9798071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:22.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.278+0000 7f979d7fb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9798070d40 con 0x7f97981439b0 2023-11-24T08:05:22.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.278+0000 7f979d7fb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9798071020 con 0x7f97981435d0 2023-11-24T08:05:22.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.278+0000 7f979d7fb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9798071300 con 0x7f97980b4140 2023-11-24T08:05:22.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.278+0000 7f97977fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97980b4140 0x7f9798071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:22.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.278+0000 7f97977fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97980b4140 0x7f9798071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48384/0 (socket says 172.21.15.64:48384) 2023-11-24T08:05:22.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.278+0000 7f97977fe640 1 -- 172.21.15.64:0/959455047 learned_addr learned my addr 172.21.15.64:0/959455047 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:22.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.279+0000 7f97977fe640 1 -- 172.21.15.64:0/959455047 >> v1:172.21.15.91:6789/0 conn(0x7f97981435d0 legacy=0x7f97980b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:22.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.279+0000 7f97977fe640 1 -- 172.21.15.64:0/959455047 >> v1:172.21.15.64:6789/0 conn(0x7f97981439b0 legacy=0x7f9798070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:22.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.279+0000 7f97977fe640 1 -- 172.21.15.64:0/959455047 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980715e0 con 0x7f97980b4140 2023-11-24T08:05:22.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.280+0000 7f97977fe640 1 --2- 172.21.15.64:0/959455047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97980b4140 0x7f9798071de0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f978c00bb10 tx=0x7f978c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d817e9bab387af34 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:22.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.280+0000 7f9796ffd640 1 -- 172.21.15.64:0/959455047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c02ee90 con 0x7f97980b4140 2023-11-24T08:05:22.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.280+0000 7f9796ffd640 1 -- 172.21.15.64:0/959455047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c036440 con 0x7f97980b4140 2023-11-24T08:05:22.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.280+0000 7f9796ffd640 1 -- 172.21.15.64:0/959455047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c0367d0 con 0x7f97980b4140 2023-11-24T08:05:22.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.281+0000 7f979d7fb640 1 -- 172.21.15.64:0/959455047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97980b4140 msgr2=0x7f9798071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:22.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.281+0000 7f979d7fb640 1 --2- 172.21.15.64:0/959455047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97980b4140 0x7f9798071de0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f978c00bb10 tx=0x7f978c02e0e0 comp rx=0 tx=0).stop 2023-11-24T08:05:22.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.281+0000 7f979d7fb640 1 -- 172.21.15.64:0/959455047 shutdown_connections 2023-11-24T08:05:22.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.281+0000 7f979d7fb640 1 --2- 172.21.15.64:0/959455047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97980b4140 0x7f9798071de0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.281+0000 7f979d7fb640 1 -- 172.21.15.64:0/959455047 >> 172.21.15.64:0/959455047 conn(0x7f97980cec10 msgr2=0x7f97980cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.281+0000 7f979d7fb640 1 -- 172.21.15.64:0/959455047 shutdown_connections 2023-11-24T08:05:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.281+0000 7f979d7fb640 1 -- 172.21.15.64:0/959455047 wait complete. 2023-11-24T08:05:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.283+0000 7f979d7fb640 1 Processor -- start 2023-11-24T08:05:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.283+0000 7f979d7fb640 1 -- start start 2023-11-24T08:05:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979d7fb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97980b4140 0x7f9798159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979d7fb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97981435d0 0x7f979813dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979d7fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97981439b0 0x7f979813a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979d7fb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979813f600 con 0x7f97981439b0 2023-11-24T08:05:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979d7fb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979813e980 con 0x7f97981435d0 2023-11-24T08:05:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979d7fb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979813ec60 con 0x7f97980b4140 2023-11-24T08:05:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979cffa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97981439b0 0x7f979813a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979cffa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97981439b0 0x7f979813a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48394/0 (socket says 172.21.15.64:48394) 2023-11-24T08:05:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.284+0000 7f979cffa640 1 -- 172.21.15.64:0/1320164627 learned_addr learned my addr 172.21.15.64:0/1320164627 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f979cffa640 1 -- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97980b4140 msgr2=0x7f9798159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f979cffa640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97980b4140 0x7f9798159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f979cffa640 1 -- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97981435d0 msgr2=0x7f979813dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f979cffa640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97981435d0 0x7f979813dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f979cffa640 1 -- 172.21.15.64:0/1320164627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c00b770 con 0x7f97981439b0 2023-11-24T08:05:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f979cffa640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97981439b0 0x7f979813a0c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f979000c680 tx=0x7f979000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f9794ff9640 1 -- 172.21.15.64:0/1320164627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790003b70 con 0x7f97981439b0 2023-11-24T08:05:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f9794ff9640 1 -- 172.21.15.64:0/1320164627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790003cd0 con 0x7f97981439b0 2023-11-24T08:05:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979813ef40 con 0x7f97981439b0 2023-11-24T08:05:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.285+0000 7f9794ff9640 1 -- 172.21.15.64:0/1320164627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790005310 con 0x7f97981439b0 2023-11-24T08:05:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.286+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979813e500 con 0x7f97981439b0 2023-11-24T08:05:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.287+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979804e840 con 0x7f97981439b0 2023-11-24T08:05:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.288+0000 7f9794ff9640 1 -- 172.21.15.64:0/1320164627 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9790005470 con 0x7f97981439b0 2023-11-24T08:05:22.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.289+0000 7f9794ff9640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f978406ae40 0x7f978406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:22.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.289+0000 7f9794ff9640 1 -- 172.21.15.64:0/1320164627 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9790089450 con 0x7f97981439b0 2023-11-24T08:05:22.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.289+0000 7f9797fff640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f978406ae40 0x7f978406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.290+0000 7f9797fff640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f978406ae40 0x7f978406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9788003cc0 tx=0x7f97880079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:22.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.293+0000 7f9794ff9640 1 -- 172.21.15.64:0/1320164627 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9790057f10 con 0x7f97981439b0 2023-11-24T08:05:22.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:22.528+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9798067640 con 0x7f97981439b0 2023-11-24T08:05:23.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.028+0000 7f9794ff9640 1 -- 172.21.15.64:0/1320164627 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9790057aa0 con 0x7f97981439b0 2023-11-24T08:05:23.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f978406ae40 msgr2=0x7f978406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f978406ae40 0x7f978406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9788003cc0 tx=0x7f97880079b0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97981439b0 msgr2=0x7f979813a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97981439b0 0x7f979813a0c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f979000c680 tx=0x7f979000cb40 comp rx=0 tx=0).stop 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 shutdown_connections 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f978406ae40 0x7f978406d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97981439b0 0x7f979813a0c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f97981435d0 0x7f979813dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 --2- 172.21.15.64:0/1320164627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f97980b4140 0x7f9798159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 >> 172.21.15.64:0/1320164627 conn(0x7f97980cec10 msgr2=0x7f979815e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.031+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 shutdown_connections 2023-11-24T08:05:23.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.032+0000 7f979d7fb640 1 -- 172.21.15.64:0/1320164627 wait complete. 2023-11-24T08:05:23.054 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:05:23.055 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:23.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.269+0000 7fd33759e640 1 Processor -- start 2023-11-24T08:05:23.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.269+0000 7fd33759e640 1 -- start start 2023-11-24T08:05:23.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.271+0000 7fd33759e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd33813dfd0 0x7fd33815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.271+0000 7fd33759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3380cf050 0x7fd3380cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.271+0000 7fd33759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3380ce7c0 con 0x7fd33806c130 2023-11-24T08:05:23.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.271+0000 7fd33759e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3380cfa10 con 0x7fd33813dfd0 2023-11-24T08:05:23.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.271+0000 7fd33759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd338140900 con 0x7fd3380cf050 2023-11-24T08:05:23.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.271+0000 7fd336d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd33806c130 0x7fd33806c580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32918/0 (socket says 172.21.15.64:32918) 2023-11-24T08:05:23.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.271+0000 7fd336d9d640 1 -- 172.21.15.64:0/1247025127 learned_addr learned my addr 172.21.15.64:0/1247025127 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:23.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.272+0000 7fd335d9b640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3380cf050 0x7fd3380cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.272+0000 7fd33659c640 1 --2- 172.21.15.64:0/1247025127 >> v2:172.21.15.91:3300/0 conn(0x7fd33813dfd0 0x7fd33815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.272+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335582846 0 0) 0x7fd3380ce7c0 con 0x7fd33806c130 2023-11-24T08:05:23.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.272+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd320003520 con 0x7fd33806c130 2023-11-24T08:05:23.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.272+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109523200 0 0) 0x7fd320003520 con 0x7fd33806c130 2023-11-24T08:05:23.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.272+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3380cf050 msgr2=0x7fd3380cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:23.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.272+0000 7fd33559a640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3380cf050 0x7fd3380cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 >> v2:172.21.15.91:3300/0 conn(0x7fd33813dfd0 msgr2=0x7fd33815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:23.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 --2- 172.21.15.64:0/1247025127 >> v2:172.21.15.91:3300/0 conn(0x7fd33813dfd0 0x7fd33815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33815fa70 con 0x7fd33806c130 2023-11-24T08:05:23.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fd3300019f0 con 0x7fd33806c130 2023-11-24T08:05:23.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 >> v1:172.21.15.64:6789/0 conn(0x7fd33806c130 legacy=0x7fd33806c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3200039e0 0x7fd320003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3200047c0 0x7fd320004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd320005180 0x7fd32000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380cfa10 con 0x7fd3200039e0 2023-11-24T08:05:23.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd338140900 con 0x7fd3200047c0 2023-11-24T08:05:23.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32000bec0 con 0x7fd320005180 2023-11-24T08:05:23.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd335d9b640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3380cf050 0x7fd3380cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:05:23.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33659c640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3200039e0 0x7fd320003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd336d9d640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd320005180 0x7fd32000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd335d9b640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3200047c0 0x7fd320004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33659c640 1 -- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd320005180 msgr2=0x7fd32000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:23.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33659c640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd320005180 0x7fd32000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33659c640 1 -- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3200047c0 msgr2=0x7fd320004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:23.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33659c640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3200047c0 0x7fd320004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd33659c640 1 -- 172.21.15.64:0/1247025127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320003520 con 0x7fd3200039e0 2023-11-24T08:05:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd336d9d640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd320005180 0x7fd32000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.273+0000 7fd335d9b640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd3200047c0 0x7fd320004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.275+0000 7fd33659c640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3200039e0 0x7fd320003e00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd328009af0 tx=0x7fd3200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc85cc3b16329c55 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.275+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32802c710 con 0x7fd3200039e0 2023-11-24T08:05:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.275+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32802c870 con 0x7fd3200039e0 2023-11-24T08:05:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.275+0000 7fd33559a640 1 -- 172.21.15.64:0/1247025127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32802cc00 con 0x7fd3200039e0 2023-11-24T08:05:23.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.276+0000 7fd33759e640 1 -- 172.21.15.64:0/1247025127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3200039e0 msgr2=0x7fd320003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.276+0000 7fd33759e640 1 --2- 172.21.15.64:0/1247025127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3200039e0 0x7fd320003e00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd328009af0 tx=0x7fd3200057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.276+0000 7fd336d9d640 1 -- 172.21.15.64:0/1247025127 reap_dead start 2023-11-24T08:05:23.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.276+0000 7fd33759e640 1 -- 172.21.15.64:0/1247025127 shutdown_connections 2023-11-24T08:05:23.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.276+0000 7fd33759e640 1 -- 172.21.15.64:0/1247025127 >> 172.21.15.64:0/1247025127 conn(0x7fd33806a860 msgr2=0x7fd33806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:23.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.276+0000 7fd33759e640 1 -- 172.21.15.64:0/1247025127 shutdown_connections 2023-11-24T08:05:23.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.276+0000 7fd33759e640 1 -- 172.21.15.64:0/1247025127 wait complete. 2023-11-24T08:05:23.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.278+0000 7fd33759e640 1 Processor -- start 2023-11-24T08:05:23.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.278+0000 7fd33759e640 1 -- start start 2023-11-24T08:05:23.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd33759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3200039e0 0x7fd338158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd33759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3380b3740 0x7fd3380b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd33759e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd33815cc90 0x7fd33815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd33759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd338155800 con 0x7fd3380b3740 2023-11-24T08:05:23.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd33759e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33815d650 con 0x7fd33815cc90 2023-11-24T08:05:23.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd33759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33813ae80 con 0x7fd3200039e0 2023-11-24T08:05:23.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd335d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3380b3740 0x7fd3380b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd335d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3380b3740 0x7fd3380b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48414/0 (socket says 172.21.15.64:48414) 2023-11-24T08:05:23.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd335d9b640 1 -- 172.21.15.64:0/3614787092 learned_addr learned my addr 172.21.15.64:0/3614787092 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:23.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd33659c640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3200039e0 0x7fd338158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd335d9b640 1 -- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3200039e0 msgr2=0x7fd338158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd336d9d640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd33815cc90 0x7fd33815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd335d9b640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3200039e0 0x7fd338158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd335d9b640 1 -- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd33815cc90 msgr2=0x7fd33815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd335d9b640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd33815cc90 0x7fd33815d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.279+0000 7fd335d9b640 1 -- 172.21.15.64:0/3614787092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33813aba0 con 0x7fd3380b3740 2023-11-24T08:05:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.280+0000 7fd335d9b640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3380b3740 0x7fd3380b3b80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd32c010900 tx=0x7fd32c010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.280+0000 7fd326ffd640 1 -- 172.21.15.64:0/3614787092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c00ebc0 con 0x7fd3380b3740 2023-11-24T08:05:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.280+0000 7fd326ffd640 1 -- 172.21.15.64:0/3614787092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c00ed20 con 0x7fd3380b3740 2023-11-24T08:05:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.280+0000 7fd326ffd640 1 -- 172.21.15.64:0/3614787092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c01c640 con 0x7fd3380b3740 2023-11-24T08:05:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.280+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3380b2c60 con 0x7fd3380b3740 2023-11-24T08:05:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.280+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3380b3220 con 0x7fd3380b3740 2023-11-24T08:05:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.281+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33804e840 con 0x7fd3380b3740 2023-11-24T08:05:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.282+0000 7fd326ffd640 1 -- 172.21.15.64:0/3614787092 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd32c01c7a0 con 0x7fd3380b3740 2023-11-24T08:05:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.283+0000 7fd326ffd640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd31806ae40 0x7fd31806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.284+0000 7fd326ffd640 1 -- 172.21.15.64:0/3614787092 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd32c089cb0 con 0x7fd3380b3740 2023-11-24T08:05:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.284+0000 7fd33659c640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd31806ae40 0x7fd31806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.285+0000 7fd33659c640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd31806ae40 0x7fd31806d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd328009af0 tx=0x7fd328030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.288+0000 7fd326ffd640 1 -- 172.21.15.64:0/3614787092 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd32c054d70 con 0x7fd3380b3740 2023-11-24T08:05:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.521+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3380ce9a0 con 0x7fd3380b3740 2023-11-24T08:05:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.522+0000 7fd326ffd640 1 -- 172.21.15.64:0/3614787092 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd32c058390 con 0x7fd3380b3740 2023-11-24T08:05:23.524 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:23.525 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1893080737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":1893080737},{"type":"v1","addr":"172.21.15.91:6837","nonce":1893080737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2845212044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2845212044},{"type":"v1","addr":"172.21.15.91:6835","nonce":2845212044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2382740551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2382740551},{"type":"v1","addr":"172.21.15.64:6833","nonce":2382740551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:05:19.478049+0000","modified":"2023-11-24T08:05:22.529755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6835/2394348469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2394348469},{"type":"v1","addr":"172.21.15.64:6835","nonce":2394348469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:05:23.525 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 15 2023-11-24T08:05:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd31806ae40 msgr2=0x7fd31806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd31806ae40 0x7fd31806d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd328009af0 tx=0x7fd328030000 comp rx=0 tx=0).stop 2023-11-24T08:05:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3380b3740 msgr2=0x7fd3380b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3380b3740 0x7fd3380b3b80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd32c010900 tx=0x7fd32c010dc0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 shutdown_connections 2023-11-24T08:05:23.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd31806ae40 0x7fd31806d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd33815cc90 0x7fd33815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3380b3740 0x7fd3380b3b80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 --2- 172.21.15.64:0/3614787092 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd3200039e0 0x7fd338158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 >> 172.21.15.64:0/3614787092 conn(0x7fd33806a860 msgr2=0x7fd33806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 shutdown_connections 2023-11-24T08:05:23.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.525+0000 7fd33759e640 1 -- 172.21.15.64:0/3614787092 wait complete. 2023-11-24T08:05:23.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:23.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.770+0000 7f3a894d3640 1 Processor -- start 2023-11-24T08:05:23.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.771+0000 7f3a894d3640 1 -- start start 2023-11-24T08:05:23.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.772+0000 7f3a894d3640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3a8406d520 0x7f3a8406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.772+0000 7f3a894d3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a840abff0 0x7f3a8415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.772+0000 7f3a894d3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8415c360 con 0x7f3a840ad720 2023-11-24T08:05:23.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.772+0000 7f3a894d3640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a840b41a0 con 0x7f3a840abff0 2023-11-24T08:05:23.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.772+0000 7f3a894d3640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8406de40 con 0x7f3a8406d520 2023-11-24T08:05:23.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.772+0000 7f3a837fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a840abff0 0x7f3a8415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.773+0000 7f3a837fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a840abff0 0x7f3a8415f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48430/0 (socket says 172.21.15.64:48430) 2023-11-24T08:05:23.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.773+0000 7f3a837fe640 1 -- 172.21.15.64:0/695887100 learned_addr learned my addr 172.21.15.64:0/695887100 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:23.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.773+0000 7f3a837fe640 1 -- 172.21.15.64:0/695887100 >> v2:172.21.15.91:3300/0 conn(0x7f3a8406d520 msgr2=0x7f3a8406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:23.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.773+0000 7f3a82ffd640 1 --2- 172.21.15.64:0/695887100 >> v2:172.21.15.91:3300/0 conn(0x7f3a8406d520 0x7f3a8406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.773+0000 7f3a837fe640 1 --2- 172.21.15.64:0/695887100 >> v2:172.21.15.91:3300/0 conn(0x7f3a8406d520 0x7f3a8406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.773+0000 7f3a837fe640 1 -- 172.21.15.64:0/695887100 >> v1:172.21.15.91:6789/0 conn(0x7f3a840ad720 legacy=0x7f3a8415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:23.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.773+0000 7f3a837fe640 1 -- 172.21.15.64:0/695887100 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8415ba30 con 0x7f3a840abff0 2023-11-24T08:05:23.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.773+0000 7f3a82ffd640 1 --2- 172.21.15.64:0/695887100 >> v2:172.21.15.91:3300/0 conn(0x7f3a8406d520 0x7f3a8406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:23.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.774+0000 7f3a837fe640 1 --2- 172.21.15.64:0/695887100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840abff0 0x7f3a8415f3e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3a8406c070 tx=0x7f3a7c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73705bfedd8fd35e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:23.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.774+0000 7f3a827fc640 1 -- 172.21.15.64:0/695887100 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c02ca80 con 0x7f3a840abff0 2023-11-24T08:05:23.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.774+0000 7f3a827fc640 1 -- 172.21.15.64:0/695887100 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c02cbe0 con 0x7f3a840abff0 2023-11-24T08:05:23.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.774+0000 7f3a827fc640 1 -- 172.21.15.64:0/695887100 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0343f0 con 0x7f3a840abff0 2023-11-24T08:05:23.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.775+0000 7f3a894d3640 1 -- 172.21.15.64:0/695887100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840abff0 msgr2=0x7f3a8415f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.775+0000 7f3a894d3640 1 --2- 172.21.15.64:0/695887100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840abff0 0x7f3a8415f3e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3a8406c070 tx=0x7f3a7c02bee0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.775+0000 7f3a894d3640 1 -- 172.21.15.64:0/695887100 shutdown_connections 2023-11-24T08:05:23.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.775+0000 7f3a894d3640 1 --2- 172.21.15.64:0/695887100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840abff0 0x7f3a8415f3e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.775+0000 7f3a894d3640 1 --2- 172.21.15.64:0/695887100 >> v2:172.21.15.91:3300/0 conn(0x7f3a8406d520 0x7f3a8406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.775+0000 7f3a894d3640 1 -- 172.21.15.64:0/695887100 >> 172.21.15.64:0/695887100 conn(0x7f3a840b0d80 msgr2=0x7f3a840b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:23.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.775+0000 7f3a894d3640 1 -- 172.21.15.64:0/695887100 shutdown_connections 2023-11-24T08:05:23.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.775+0000 7f3a894d3640 1 -- 172.21.15.64:0/695887100 wait complete. 2023-11-24T08:05:23.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.777+0000 7f3a894d3640 1 Processor -- start 2023-11-24T08:05:23.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.777+0000 7f3a894d3640 1 -- start start 2023-11-24T08:05:23.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a894d3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3a8406d520 0x7f3a84144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a894d3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a840abff0 0x7f3a84155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a894d3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840ad720 0x7f3a84153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a894d3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84155bc0 con 0x7f3a840ad720 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a894d3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84156d10 con 0x7f3a840abff0 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a894d3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84156ff0 con 0x7f3a8406d520 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840ad720 0x7f3a84153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840ad720 0x7f3a84153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48442/0 (socket says 172.21.15.64:48442) 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.778+0000 7f3a837fe640 1 -- 172.21.15.64:0/3401610059 learned_addr learned my addr 172.21.15.64:0/3401610059 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a7bfff640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a840abff0 0x7f3a84155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a82ffd640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3a8406d520 0x7f3a84144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a837fe640 1 -- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3a8406d520 msgr2=0x7f3a84144410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a837fe640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3a8406d520 0x7f3a84144410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a837fe640 1 -- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a840abff0 msgr2=0x7f3a84155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a837fe640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a840abff0 0x7f3a84155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a837fe640 1 -- 172.21.15.64:0/3401610059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c009b30 con 0x7f3a840ad720 2023-11-24T08:05:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a7bfff640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a840abff0 0x7f3a84155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a837fe640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840ad720 0x7f3a84153240 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c036040 tx=0x7f3a7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:23.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a7b7fe640 1 -- 172.21.15.64:0/3401610059 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c03e450 con 0x7f3a840ad720 2023-11-24T08:05:23.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a7b7fe640 1 -- 172.21.15.64:0/3401610059 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c03e5d0 con 0x7f3a840ad720 2023-11-24T08:05:23.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.779+0000 7f3a7b7fe640 1 -- 172.21.15.64:0/3401610059 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c03e980 con 0x7f3a840ad720 2023-11-24T08:05:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.780+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a841572d0 con 0x7f3a840ad720 2023-11-24T08:05:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.780+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a841557b0 con 0x7f3a840ad720 2023-11-24T08:05:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.781+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a84159280 con 0x7f3a840ad720 2023-11-24T08:05:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.782+0000 7f3a7b7fe640 1 -- 172.21.15.64:0/3401610059 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a7c02cd50 con 0x7f3a840ad720 2023-11-24T08:05:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.783+0000 7f3a7b7fe640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3a6406ae40 0x7f3a6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.783+0000 7f3a7b7fe640 1 -- 172.21.15.64:0/3401610059 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a7c0ac250 con 0x7f3a840ad720 2023-11-24T08:05:23.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.783+0000 7f3a82ffd640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3a6406ae40 0x7f3a6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:23.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.784+0000 7f3a82ffd640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3a6406ae40 0x7f3a6406d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a74008850 tx=0x7f3a74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:23.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.787+0000 7f3a7b7fe640 1 -- 172.21.15.64:0/3401610059 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a84159280 con 0x7f3a840ad720 2023-11-24T08:05:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.982+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a84159280 con 0x7f3a840ad720 2023-11-24T08:05:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.985+0000 7f3a7b7fe640 1 -- 172.21.15.64:0/3401610059 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f3a84159280 con 0x7f3a840ad720 2023-11-24T08:05:23.987 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:23.988 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:21.012478+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:05:17.535151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:05:18.499872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3a6406ae40 msgr2=0x7f3a6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3a6406ae40 0x7f3a6406d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a74008850 tx=0x7f3a74008040 comp rx=0 tx=0).stop 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840ad720 msgr2=0x7f3a84153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840ad720 0x7f3a84153240 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c036040 tx=0x7f3a7c004a40 comp rx=0 tx=0).stop 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 shutdown_connections 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3a6406ae40 0x7f3a6406d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a840ad720 0x7f3a84153240 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3a840abff0 0x7f3a84155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.988+0000 7f3a894d3640 1 --2- 172.21.15.64:0/3401610059 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3a8406d520 0x7f3a84144410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.989+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 >> 172.21.15.64:0/3401610059 conn(0x7f3a840b0d80 msgr2=0x7f3a8415da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.989+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 shutdown_connections 2023-11-24T08:05:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:23.989+0000 7f3a894d3640 1 -- 172.21.15.64:0/3401610059 wait complete. 2023-11-24T08:05:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:05:24.011 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:05:24.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.227+0000 7f8611567640 1 Processor -- start 2023-11-24T08:05:24.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.227+0000 7f8611567640 1 -- start start 2023-11-24T08:05:24.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.229+0000 7f8611567640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f860c0cf760 0x7f860c15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.229+0000 7f8611567640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f860c143950 0x7f860c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.229+0000 7f8611567640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860c071700 con 0x7f860c143570 2023-11-24T08:05:24.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.229+0000 7f8611567640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860c15af30 con 0x7f860c143950 2023-11-24T08:05:24.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.229+0000 7f8611567640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860c15b410 con 0x7f860c0cf760 2023-11-24T08:05:24.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.229+0000 7f8610d66640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f860c143950 0x7f860c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.229+0000 7f8610d66640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f860c143950 0x7f860c15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48458/0 (socket says 172.21.15.64:48458) 2023-11-24T08:05:24.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.229+0000 7f8610d66640 1 -- 172.21.15.64:0/3317562453 learned_addr learned my addr 172.21.15.64:0/3317562453 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:24.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.230+0000 7f8610d66640 1 -- 172.21.15.64:0/3317562453 >> v2:172.21.15.91:3300/0 conn(0x7f860c0cf760 msgr2=0x7f860c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:24.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.230+0000 7f8610d66640 1 --2- 172.21.15.64:0/3317562453 >> v2:172.21.15.91:3300/0 conn(0x7f860c0cf760 0x7f860c15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.230+0000 7f8610d66640 1 -- 172.21.15.64:0/3317562453 >> v1:172.21.15.91:6789/0 conn(0x7f860c143570 legacy=0x7f860c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:24.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.230+0000 7f8610d66640 1 -- 172.21.15.64:0/3317562453 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c15b130 con 0x7f860c143950 2023-11-24T08:05:24.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.231+0000 7f8610d66640 1 --2- 172.21.15.64:0/3317562453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15a9f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f860c0cbe10 tx=0x7f860402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c4addb07d4993 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:24.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.231+0000 7f860affd640 1 -- 172.21.15.64:0/3317562453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860402ec00 con 0x7f860c143950 2023-11-24T08:05:24.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.231+0000 7f860affd640 1 -- 172.21.15.64:0/3317562453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860402ed60 con 0x7f860c143950 2023-11-24T08:05:24.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.231+0000 7f860affd640 1 -- 172.21.15.64:0/3317562453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604036610 con 0x7f860c143950 2023-11-24T08:05:24.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.232+0000 7f8611567640 1 -- 172.21.15.64:0/3317562453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 msgr2=0x7f860c15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:24.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.232+0000 7f8611567640 1 --2- 172.21.15.64:0/3317562453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15a9f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f860c0cbe10 tx=0x7f860402e060 comp rx=0 tx=0).stop 2023-11-24T08:05:24.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.232+0000 7f8611567640 1 -- 172.21.15.64:0/3317562453 shutdown_connections 2023-11-24T08:05:24.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.232+0000 7f8611567640 1 --2- 172.21.15.64:0/3317562453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15a9f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.232+0000 7f8611567640 1 --2- 172.21.15.64:0/3317562453 >> v2:172.21.15.91:3300/0 conn(0x7f860c0cf760 0x7f860c15a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.232+0000 7f8611567640 1 -- 172.21.15.64:0/3317562453 >> 172.21.15.64:0/3317562453 conn(0x7f860c06a860 msgr2=0x7f860c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:24.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.232+0000 7f8611567640 1 -- 172.21.15.64:0/3317562453 shutdown_connections 2023-11-24T08:05:24.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.232+0000 7f8611567640 1 -- 172.21.15.64:0/3317562453 wait complete. 2023-11-24T08:05:24.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.234+0000 7f8611567640 1 Processor -- start 2023-11-24T08:05:24.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.234+0000 7f8611567640 1 -- start start 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.235+0000 7f8611567640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f860c0cf760 0x7f860c14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.235+0000 7f8611567640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f860c143570 0x7f860c14c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.235+0000 7f8611567640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.235+0000 7f8611567640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c14f9a0 con 0x7f860c143950 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.235+0000 7f8611567640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c14ca60 con 0x7f860c143570 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.235+0000 7f8611567640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860c14cd40 con 0x7f860c0cf760 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48466/0 (socket says 172.21.15.64:48466) 2023-11-24T08:05:24.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 -- 172.21.15.64:0/239497313 learned_addr learned my addr 172.21.15.64:0/239497313 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:24.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f860b7fe640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f860c143570 0x7f860c14c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 -- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f860c0cf760 msgr2=0x7f860c14f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:24.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f860c0cf760 0x7f860c14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 -- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f860c143570 msgr2=0x7f860c14c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:24.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f860c143570 0x7f860c14c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 -- 172.21.15.64:0/239497313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604009b30 con 0x7f860c143950 2023-11-24T08:05:24.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.236+0000 7f8610d66640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15bb50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8604038040 tx=0x7f8604004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:24.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.237+0000 7f8608ff9640 1 -- 172.21.15.64:0/239497313 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860402ec00 con 0x7f860c143950 2023-11-24T08:05:24.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.237+0000 7f8608ff9640 1 -- 172.21.15.64:0/239497313 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860403f470 con 0x7f860c143950 2023-11-24T08:05:24.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.237+0000 7f8608ff9640 1 -- 172.21.15.64:0/239497313 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860403f800 con 0x7f860c143950 2023-11-24T08:05:24.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.237+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860c14d020 con 0x7f860c143950 2023-11-24T08:05:24.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.237+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860c0caf00 con 0x7f860c143950 2023-11-24T08:05:24.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.238+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860c14d540 con 0x7f860c143950 2023-11-24T08:05:24.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.239+0000 7f8608ff9640 1 -- 172.21.15.64:0/239497313 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f860403f960 con 0x7f860c143950 2023-11-24T08:05:24.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.240+0000 7f8608ff9640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f806ae40 0x7f85f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.240+0000 7f8608ff9640 1 -- 172.21.15.64:0/239497313 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86040abdf0 con 0x7f860c143950 2023-11-24T08:05:24.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.241+0000 7f860bfff640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f806ae40 0x7f85f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.242+0000 7f860bfff640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f806ae40 0x7f85f806d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f85fc001560 tx=0x7f85fc007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:24.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.244+0000 7f8608ff9640 1 -- 172.21.15.64:0/239497313 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8604076e00 con 0x7f860c143950 2023-11-24T08:05:24.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.437+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f860c15e190 con 0x7f85f806ae40 2023-11-24T08:05:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.466+0000 7f8608ff9640 1 -- 172.21.15.64:0/239497313 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282105 (secure 0 0 0) 0x7f85fc0050d0 con 0x7f85f806ae40 2023-11-24T08:05:24.470 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:05:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f806ae40 msgr2=0x7f85f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f806ae40 0x7f85f806d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f85fc001560 tx=0x7f85fc007990 comp rx=0 tx=0).stop 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 msgr2=0x7f860c15bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15bb50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8604038040 tx=0x7f8604004a40 comp rx=0 tx=0).stop 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 shutdown_connections 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f806ae40 0x7f85f806d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f860c143950 0x7f860c15bb50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f860c143570 0x7f860c14c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 --2- 172.21.15.64:0/239497313 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f860c0cf760 0x7f860c14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 >> 172.21.15.64:0/239497313 conn(0x7f860c06a860 msgr2=0x7f860c0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 shutdown_connections 2023-11-24T08:05:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.473+0000 7f8611567640 1 -- 172.21.15.64:0/239497313 wait complete. 2023-11-24T08:05:24.498 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-11-24T08:05:24.043708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220036,"kb_used_data":3876,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754684,"statfs":{"total":773094113280,"available":772868796416,"internally_reserved":0,"allocated":3969024,"data_stored":2241597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001661"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019366+0000","last_change":"2023-11-24T08:05:19.019366+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019366+0000","last_undegraded":"2023-11-24T08:05:19.019366+0000","last_fullsized":"2023-11-24T08:05:19.019366+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002461+0000","last_change":"2023-11-24T08:05:19.002461+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002461+0000","last_undegraded":"2023-11-24T08:05:19.002461+0000","last_fullsized":"2023-11-24T08:05:19.002461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016949+0000","last_change":"2023-11-24T08:05:19.016949+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016949+0000","last_undegraded":"2023-11-24T08:05:19.016949+0000","last_fullsized":"2023-11-24T08:05:19.016949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017053+0000","last_change":"2023-11-24T08:05:19.017053+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017053+0000","last_undegraded":"2023-11-24T08:05:19.017053+0000","last_fullsized":"2023-11-24T08:05:19.017053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.010185+0000","last_change":"2023-11-24T08:05:19.010185+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.010185+0000","last_undegraded":"2023-11-24T08:05:19.010185+0000","last_fullsized":"2023-11-24T08:05:19.010185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002180+0000","last_change":"2023-11-24T08:05:19.002180+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002180+0000","last_undegraded":"2023-11-24T08:05:19.002180+0000","last_fullsized":"2023-11-24T08:05:19.002180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021735+0000","last_change":"2023-11-24T08:05:19.021735+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021735+0000","last_undegraded":"2023-11-24T08:05:19.021735+0000","last_fullsized":"2023-11-24T08:05:19.021735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.026170+0000","last_change":"2023-11-24T08:05:19.026170+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.026170+0000","last_undegraded":"2023-11-24T08:05:19.026170+0000","last_fullsized":"2023-11-24T08:05:19.026170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007708+0000","last_change":"2023-11-24T08:05:19.007708+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007708+0000","last_undegraded":"2023-11-24T08:05:19.007708+0000","last_fullsized":"2023-11-24T08:05:19.007708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019694+0000","last_change":"2023-11-24T08:05:19.019694+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019694+0000","last_undegraded":"2023-11-24T08:05:19.019694+0000","last_fullsized":"2023-11-24T08:05:19.019694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022034+0000","last_change":"2023-11-24T08:05:19.022034+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022034+0000","last_undegraded":"2023-11-24T08:05:19.022034+0000","last_fullsized":"2023-11-24T08:05:19.022034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.014621+0000","last_change":"2023-11-24T08:05:19.014621+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.014621+0000","last_undegraded":"2023-11-24T08:05:19.014621+0000","last_fullsized":"2023-11-24T08:05:19.014621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009712+0000","last_change":"2023-11-24T08:05:19.009712+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009712+0000","last_undegraded":"2023-11-24T08:05:19.009712+0000","last_fullsized":"2023-11-24T08:05:19.009712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001299+0000","last_change":"2023-11-24T08:05:19.001299+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001299+0000","last_undegraded":"2023-11-24T08:05:19.001299+0000","last_fullsized":"2023-11-24T08:05:19.001299+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.014116+0000","last_change":"2023-11-24T08:05:19.014116+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.014116+0000","last_undegraded":"2023-11-24T08:05:19.014116+0000","last_fullsized":"2023-11-24T08:05:19.014116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021676+0000","last_change":"2023-11-24T08:05:19.021676+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021676+0000","last_undegraded":"2023-11-24T08:05:19.021676+0000","last_fullsized":"2023-11-24T08:05:19.021676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022765+0000","last_change":"2023-11-24T08:05:19.022765+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022765+0000","last_undegraded":"2023-11-24T08:05:19.022765+0000","last_fullsized":"2023-11-24T08:05:19.022765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001872+0000","last_change":"2023-11-24T08:05:19.001872+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001872+0000","last_undegraded":"2023-11-24T08:05:19.001872+0000","last_fullsized":"2023-11-24T08:05:19.001872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003795+0000","last_change":"2023-11-24T08:05:19.003795+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003795+0000","last_undegraded":"2023-11-24T08:05:19.003795+0000","last_fullsized":"2023-11-24T08:05:19.003795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007408+0000","last_change":"2023-11-24T08:05:19.007408+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007408+0000","last_undegraded":"2023-11-24T08:05:19.007408+0000","last_fullsized":"2023-11-24T08:05:19.007408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.010630+0000","last_change":"2023-11-24T08:05:19.010630+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.010630+0000","last_undegraded":"2023-11-24T08:05:19.010630+0000","last_fullsized":"2023-11-24T08:05:19.010630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.006901+0000","last_change":"2023-11-24T08:05:19.006901+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.006901+0000","last_undegraded":"2023-11-24T08:05:19.006901+0000","last_fullsized":"2023-11-24T08:05:19.006901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.011354+0000","last_change":"2023-11-24T08:05:19.011354+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.011354+0000","last_undegraded":"2023-11-24T08:05:19.011354+0000","last_fullsized":"2023-11-24T08:05:19.011354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012941+0000","last_change":"2023-11-24T08:05:19.012941+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012941+0000","last_undegraded":"2023-11-24T08:05:19.012941+0000","last_fullsized":"2023-11-24T08:05:19.012941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018757+0000","last_change":"2023-11-24T08:05:19.018757+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018757+0000","last_undegraded":"2023-11-24T08:05:19.018757+0000","last_fullsized":"2023-11-24T08:05:19.018757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001270+0000","last_change":"2023-11-24T08:05:19.001270+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001270+0000","last_undegraded":"2023-11-24T08:05:19.001270+0000","last_fullsized":"2023-11-24T08:05:19.001270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021345+0000","last_change":"2023-11-24T08:05:19.021345+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021345+0000","last_undegraded":"2023-11-24T08:05:19.021345+0000","last_fullsized":"2023-11-24T08:05:19.021345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.013040+0000","last_change":"2023-11-24T08:05:19.013040+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.013040+0000","last_undegraded":"2023-11-24T08:05:19.013040+0000","last_fullsized":"2023-11-24T08:05:19.013040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009317+0000","last_change":"2023-11-24T08:05:19.009317+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009317+0000","last_undegraded":"2023-11-24T08:05:19.009317+0000","last_fullsized":"2023-11-24T08:05:19.009317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.020254+0000","last_change":"2023-11-24T08:05:19.020254+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.020254+0000","last_undegraded":"2023-11-24T08:05:19.020254+0000","last_fullsized":"2023-11-24T08:05:19.020254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022175+0000","last_change":"2023-11-24T08:05:19.022175+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022175+0000","last_undegraded":"2023-11-24T08:05:19.022175+0000","last_fullsized":"2023-11-24T08:05:19.022175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025740+0000","last_change":"2023-11-24T08:05:19.025740+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025740+0000","last_undegraded":"2023-11-24T08:05:19.025740+0000","last_fullsized":"2023-11-24T08:05:19.025740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016746+0000","last_change":"2023-11-24T08:05:19.016746+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016746+0000","last_undegraded":"2023-11-24T08:05:19.016746+0000","last_fullsized":"2023-11-24T08:05:19.016746+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.038997+0000","last_change":"2023-11-24T08:05:19.034855+0000","last_active":"2023-11-24T08:05:19.038997+0000","last_peered":"2023-11-24T08:05:19.038997+0000","last_clean":"2023-11-24T08:05:19.038997+0000","last_became_active":"2023-11-24T08:05:19.034614+0000","last_became_peered":"2023-11-24T08:05:19.034614+0000","last_unstale":"2023-11-24T08:05:19.038997+0000","last_undegraded":"2023-11-24T08:05:19.038997+0000","last_fullsized":"2023-11-24T08:05:19.038997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017903+0000","last_change":"2023-11-24T08:05:19.017903+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017903+0000","last_undegraded":"2023-11-24T08:05:19.017903+0000","last_fullsized":"2023-11-24T08:05:19.017903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034166+0000","last_change":"2023-11-24T08:05:19.029832+0000","last_active":"2023-11-24T08:05:19.034166+0000","last_peered":"2023-11-24T08:05:19.034166+0000","last_clean":"2023-11-24T08:05:19.034166+0000","last_became_active":"2023-11-24T08:05:19.028467+0000","last_became_peered":"2023-11-24T08:05:19.028467+0000","last_unstale":"2023-11-24T08:05:19.034166+0000","last_undegraded":"2023-11-24T08:05:19.034166+0000","last_fullsized":"2023-11-24T08:05:19.034166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:12.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.004380+0000","last_change":"2023-11-24T08:05:19.004380+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.004380+0000","last_undegraded":"2023-11-24T08:05:19.004380+0000","last_fullsized":"2023-11-24T08:05:19.004380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031446+0000","last_change":"2023-11-24T08:05:19.024698+0000","last_active":"2023-11-24T08:05:19.031446+0000","last_peered":"2023-11-24T08:05:19.031446+0000","last_clean":"2023-11-24T08:05:19.031446+0000","last_became_active":"2023-11-24T08:05:19.024477+0000","last_became_peered":"2023-11-24T08:05:19.024477+0000","last_unstale":"2023-11-24T08:05:19.031446+0000","last_undegraded":"2023-11-24T08:05:19.031446+0000","last_fullsized":"2023-11-24T08:05:19.031446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:50.782618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.020852+0000","last_change":"2023-11-24T08:05:19.020852+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.020852+0000","last_undegraded":"2023-11-24T08:05:19.020852+0000","last_fullsized":"2023-11-24T08:05:19.020852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.028717+0000","last_change":"2023-11-24T08:05:19.024566+0000","last_active":"2023-11-24T08:05:19.028717+0000","last_peered":"2023-11-24T08:05:19.028717+0000","last_clean":"2023-11-24T08:05:19.028717+0000","last_became_active":"2023-11-24T08:05:19.024310+0000","last_became_peered":"2023-11-24T08:05:19.024310+0000","last_unstale":"2023-11-24T08:05:19.028717+0000","last_undegraded":"2023-11-24T08:05:19.028717+0000","last_fullsized":"2023-11-24T08:05:19.028717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:59.756867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019547+0000","last_change":"2023-11-24T08:05:19.019547+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019547+0000","last_undegraded":"2023-11-24T08:05:19.019547+0000","last_fullsized":"2023-11-24T08:05:19.019547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034315+0000","last_change":"2023-11-24T08:05:19.027924+0000","last_active":"2023-11-24T08:05:19.034315+0000","last_peered":"2023-11-24T08:05:19.034315+0000","last_clean":"2023-11-24T08:05:19.034315+0000","last_became_active":"2023-11-24T08:05:19.023079+0000","last_became_peered":"2023-11-24T08:05:19.023079+0000","last_unstale":"2023-11-24T08:05:19.034315+0000","last_undegraded":"2023-11-24T08:05:19.034315+0000","last_fullsized":"2023-11-24T08:05:19.034315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:45.153445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012736+0000","last_change":"2023-11-24T08:05:19.012736+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012736+0000","last_undegraded":"2023-11-24T08:05:19.012736+0000","last_fullsized":"2023-11-24T08:05:19.012736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034070+0000","last_change":"2023-11-24T08:05:19.028892+0000","last_active":"2023-11-24T08:05:19.034070+0000","last_peered":"2023-11-24T08:05:19.034070+0000","last_clean":"2023-11-24T08:05:19.034070+0000","last_became_active":"2023-11-24T08:05:19.027535+0000","last_became_peered":"2023-11-24T08:05:19.027535+0000","last_unstale":"2023-11-24T08:05:19.034070+0000","last_undegraded":"2023-11-24T08:05:19.034070+0000","last_fullsized":"2023-11-24T08:05:19.034070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:45:40.009128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003620+0000","last_change":"2023-11-24T08:05:19.003620+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003620+0000","last_undegraded":"2023-11-24T08:05:19.003620+0000","last_fullsized":"2023-11-24T08:05:19.003620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.019329+0000","last_change":"2023-11-24T08:05:19.015336+0000","last_active":"2023-11-24T08:05:19.019329+0000","last_peered":"2023-11-24T08:05:19.019329+0000","last_clean":"2023-11-24T08:05:19.019329+0000","last_became_active":"2023-11-24T08:05:19.014910+0000","last_became_peered":"2023-11-24T08:05:19.014910+0000","last_unstale":"2023-11-24T08:05:19.019329+0000","last_undegraded":"2023-11-24T08:05:19.019329+0000","last_fullsized":"2023-11-24T08:05:19.019329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:27.754037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017987+0000","last_change":"2023-11-24T08:05:19.017987+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017987+0000","last_undegraded":"2023-11-24T08:05:19.017987+0000","last_fullsized":"2023-11-24T08:05:19.017987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036507+0000","last_change":"2023-11-24T08:05:19.031449+0000","last_active":"2023-11-24T08:05:19.036507+0000","last_peered":"2023-11-24T08:05:19.036507+0000","last_clean":"2023-11-24T08:05:19.036507+0000","last_became_active":"2023-11-24T08:05:19.031114+0000","last_became_peered":"2023-11-24T08:05:19.031114+0000","last_unstale":"2023-11-24T08:05:19.036507+0000","last_undegraded":"2023-11-24T08:05:19.036507+0000","last_fullsized":"2023-11-24T08:05:19.036507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:56.728306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012757+0000","last_change":"2023-11-24T08:05:19.012757+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012757+0000","last_undegraded":"2023-11-24T08:05:19.012757+0000","last_fullsized":"2023-11-24T08:05:19.012757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032407+0000","last_change":"2023-11-24T08:05:19.028843+0000","last_active":"2023-11-24T08:05:19.032407+0000","last_peered":"2023-11-24T08:05:19.032407+0000","last_clean":"2023-11-24T08:05:19.032407+0000","last_became_active":"2023-11-24T08:05:19.027840+0000","last_became_peered":"2023-11-24T08:05:19.027840+0000","last_unstale":"2023-11-24T08:05:19.032407+0000","last_undegraded":"2023-11-24T08:05:19.032407+0000","last_fullsized":"2023-11-24T08:05:19.032407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:12.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003124+0000","last_change":"2023-11-24T08:05:19.003124+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003124+0000","last_undegraded":"2023-11-24T08:05:19.003124+0000","last_fullsized":"2023-11-24T08:05:19.003124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034147+0000","last_change":"2023-11-24T08:05:19.029786+0000","last_active":"2023-11-24T08:05:19.034147+0000","last_peered":"2023-11-24T08:05:19.034147+0000","last_clean":"2023-11-24T08:05:19.034147+0000","last_became_active":"2023-11-24T08:05:19.029469+0000","last_became_peered":"2023-11-24T08:05:19.029469+0000","last_unstale":"2023-11-24T08:05:19.034147+0000","last_undegraded":"2023-11-24T08:05:19.034147+0000","last_fullsized":"2023-11-24T08:05:19.034147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002655+0000","last_change":"2023-11-24T08:05:19.002655+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002655+0000","last_undegraded":"2023-11-24T08:05:19.002655+0000","last_fullsized":"2023-11-24T08:05:19.002655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036658+0000","last_change":"2023-11-24T08:05:19.034418+0000","last_active":"2023-11-24T08:05:19.036658+0000","last_peered":"2023-11-24T08:05:19.036658+0000","last_clean":"2023-11-24T08:05:19.036658+0000","last_became_active":"2023-11-24T08:05:19.033911+0000","last_became_peered":"2023-11-24T08:05:19.033911+0000","last_unstale":"2023-11-24T08:05:19.036658+0000","last_undegraded":"2023-11-24T08:05:19.036658+0000","last_fullsized":"2023-11-24T08:05:19.036658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:50.782618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018899+0000","last_change":"2023-11-24T08:05:19.018899+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018899+0000","last_undegraded":"2023-11-24T08:05:19.018899+0000","last_fullsized":"2023-11-24T08:05:19.018899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031376+0000","last_change":"2023-11-24T08:05:19.023626+0000","last_active":"2023-11-24T08:05:19.031376+0000","last_peered":"2023-11-24T08:05:19.031376+0000","last_clean":"2023-11-24T08:05:19.031376+0000","last_became_active":"2023-11-24T08:05:19.023367+0000","last_became_peered":"2023-11-24T08:05:19.023367+0000","last_unstale":"2023-11-24T08:05:19.031376+0000","last_undegraded":"2023-11-24T08:05:19.031376+0000","last_fullsized":"2023-11-24T08:05:19.031376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:50.145064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017655+0000","last_change":"2023-11-24T08:05:19.017655+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017655+0000","last_undegraded":"2023-11-24T08:05:19.017655+0000","last_fullsized":"2023-11-24T08:05:19.017655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034295+0000","last_change":"2023-11-24T08:05:19.026245+0000","last_active":"2023-11-24T08:05:19.034295+0000","last_peered":"2023-11-24T08:05:19.034295+0000","last_clean":"2023-11-24T08:05:19.034295+0000","last_became_active":"2023-11-24T08:05:19.025943+0000","last_became_peered":"2023-11-24T08:05:19.025943+0000","last_unstale":"2023-11-24T08:05:19.034295+0000","last_undegraded":"2023-11-24T08:05:19.034295+0000","last_fullsized":"2023-11-24T08:05:19.034295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:44.721323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001371+0000","last_change":"2023-11-24T08:05:19.001371+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001371+0000","last_undegraded":"2023-11-24T08:05:19.001371+0000","last_fullsized":"2023-11-24T08:05:19.001371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.033021+0000","last_change":"2023-11-24T08:05:19.027534+0000","last_active":"2023-11-24T08:05:19.033021+0000","last_peered":"2023-11-24T08:05:19.033021+0000","last_clean":"2023-11-24T08:05:19.033021+0000","last_became_active":"2023-11-24T08:05:19.024963+0000","last_became_peered":"2023-11-24T08:05:19.024963+0000","last_unstale":"2023-11-24T08:05:19.033021+0000","last_undegraded":"2023-11-24T08:05:19.033021+0000","last_fullsized":"2023-11-24T08:05:19.033021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:18.282886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025333+0000","last_change":"2023-11-24T08:05:19.025333+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025333+0000","last_undegraded":"2023-11-24T08:05:19.025333+0000","last_fullsized":"2023-11-24T08:05:19.025333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032465+0000","last_change":"2023-11-24T08:05:19.028331+0000","last_active":"2023-11-24T08:05:19.032465+0000","last_peered":"2023-11-24T08:05:19.032465+0000","last_clean":"2023-11-24T08:05:19.032465+0000","last_became_active":"2023-11-24T08:05:19.028037+0000","last_became_peered":"2023-11-24T08:05:19.028037+0000","last_unstale":"2023-11-24T08:05:19.032465+0000","last_undegraded":"2023-11-24T08:05:19.032465+0000","last_fullsized":"2023-11-24T08:05:19.032465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:00.858708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017553+0000","last_change":"2023-11-24T08:05:19.017553+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017553+0000","last_undegraded":"2023-11-24T08:05:19.017553+0000","last_fullsized":"2023-11-24T08:05:19.017553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031349+0000","last_change":"2023-11-24T08:05:19.023564+0000","last_active":"2023-11-24T08:05:19.031349+0000","last_peered":"2023-11-24T08:05:19.031349+0000","last_clean":"2023-11-24T08:05:19.031349+0000","last_became_active":"2023-11-24T08:05:19.022784+0000","last_became_peered":"2023-11-24T08:05:19.022784+0000","last_unstale":"2023-11-24T08:05:19.031349+0000","last_undegraded":"2023-11-24T08:05:19.031349+0000","last_fullsized":"2023-11-24T08:05:19.031349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.008194+0000","last_change":"2023-11-24T08:05:19.008194+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.008194+0000","last_undegraded":"2023-11-24T08:05:19.008194+0000","last_fullsized":"2023-11-24T08:05:19.008194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034866+0000","last_change":"2023-11-24T08:05:19.030608+0000","last_active":"2023-11-24T08:05:19.034866+0000","last_peered":"2023-11-24T08:05:19.034866+0000","last_clean":"2023-11-24T08:05:19.034866+0000","last_became_active":"2023-11-24T08:05:19.030359+0000","last_became_peered":"2023-11-24T08:05:19.030359+0000","last_unstale":"2023-11-24T08:05:19.034866+0000","last_undegraded":"2023-11-24T08:05:19.034866+0000","last_fullsized":"2023-11-24T08:05:19.034866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.026741+0000","last_change":"2023-11-24T08:05:19.026741+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.026741+0000","last_undegraded":"2023-11-24T08:05:19.026741+0000","last_fullsized":"2023-11-24T08:05:19.026741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034369+0000","last_change":"2023-11-24T08:05:19.027262+0000","last_active":"2023-11-24T08:05:19.034369+0000","last_peered":"2023-11-24T08:05:19.034369+0000","last_clean":"2023-11-24T08:05:19.034369+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:19.034369+0000","last_undegraded":"2023-11-24T08:05:19.034369+0000","last_fullsized":"2023-11-24T08:05:19.034369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:39.801164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016569+0000","last_change":"2023-11-24T08:05:19.016569+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016569+0000","last_undegraded":"2023-11-24T08:05:19.016569+0000","last_fullsized":"2023-11-24T08:05:19.016569+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034100+0000","last_change":"2023-11-24T08:05:19.025861+0000","last_active":"2023-11-24T08:05:19.034100+0000","last_peered":"2023-11-24T08:05:19.034100+0000","last_clean":"2023-11-24T08:05:19.034100+0000","last_became_active":"2023-11-24T08:05:19.024918+0000","last_became_peered":"2023-11-24T08:05:19.024918+0000","last_unstale":"2023-11-24T08:05:19.034100+0000","last_undegraded":"2023-11-24T08:05:19.034100+0000","last_fullsized":"2023-11-24T08:05:19.034100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:59.275982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.008986+0000","last_change":"2023-11-24T08:05:19.008986+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.008986+0000","last_undegraded":"2023-11-24T08:05:19.008986+0000","last_fullsized":"2023-11-24T08:05:19.008986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.030949+0000","last_change":"2023-11-24T08:05:19.017985+0000","last_active":"2023-11-24T08:05:19.030949+0000","last_peered":"2023-11-24T08:05:19.030949+0000","last_clean":"2023-11-24T08:05:19.030949+0000","last_became_active":"2023-11-24T08:05:19.017444+0000","last_became_peered":"2023-11-24T08:05:19.017444+0000","last_unstale":"2023-11-24T08:05:19.030949+0000","last_undegraded":"2023-11-24T08:05:19.030949+0000","last_fullsized":"2023-11-24T08:05:19.030949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:45:30.962562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007396+0000","last_change":"2023-11-24T08:05:19.007396+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007396+0000","last_undegraded":"2023-11-24T08:05:19.007396+0000","last_fullsized":"2023-11-24T08:05:19.007396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031706+0000","last_change":"2023-11-24T08:05:19.026334+0000","last_active":"2023-11-24T08:05:19.031706+0000","last_peered":"2023-11-24T08:05:19.031706+0000","last_clean":"2023-11-24T08:05:19.031706+0000","last_became_active":"2023-11-24T08:05:19.025952+0000","last_became_peered":"2023-11-24T08:05:19.025952+0000","last_unstale":"2023-11-24T08:05:19.031706+0000","last_undegraded":"2023-11-24T08:05:19.031706+0000","last_fullsized":"2023-11-24T08:05:19.031706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009085+0000","last_change":"2023-11-24T08:05:19.009085+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009085+0000","last_undegraded":"2023-11-24T08:05:19.009085+0000","last_fullsized":"2023-11-24T08:05:19.009085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036500+0000","last_change":"2023-11-24T08:05:19.034595+0000","last_active":"2023-11-24T08:05:19.036500+0000","last_peered":"2023-11-24T08:05:19.036500+0000","last_clean":"2023-11-24T08:05:19.036500+0000","last_became_active":"2023-11-24T08:05:19.033790+0000","last_became_peered":"2023-11-24T08:05:19.033790+0000","last_unstale":"2023-11-24T08:05:19.036500+0000","last_undegraded":"2023-11-24T08:05:19.036500+0000","last_fullsized":"2023-11-24T08:05:19.036500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007348+0000","last_change":"2023-11-24T08:05:19.007348+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007348+0000","last_undegraded":"2023-11-24T08:05:19.007348+0000","last_fullsized":"2023-11-24T08:05:19.007348+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034034+0000","last_change":"2023-11-24T08:05:19.028570+0000","last_active":"2023-11-24T08:05:19.034034+0000","last_peered":"2023-11-24T08:05:19.034034+0000","last_clean":"2023-11-24T08:05:19.034034+0000","last_became_active":"2023-11-24T08:05:19.028257+0000","last_became_peered":"2023-11-24T08:05:19.028257+0000","last_unstale":"2023-11-24T08:05:19.034034+0000","last_undegraded":"2023-11-24T08:05:19.034034+0000","last_fullsized":"2023-11-24T08:05:19.034034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:59.756867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022389+0000","last_change":"2023-11-24T08:05:19.022389+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022389+0000","last_undegraded":"2023-11-24T08:05:19.022389+0000","last_fullsized":"2023-11-24T08:05:19.022389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036304+0000","last_change":"2023-11-24T08:05:19.032519+0000","last_active":"2023-11-24T08:05:19.036304+0000","last_peered":"2023-11-24T08:05:19.036304+0000","last_clean":"2023-11-24T08:05:19.036304+0000","last_became_active":"2023-11-24T08:05:19.032159+0000","last_became_peered":"2023-11-24T08:05:19.032159+0000","last_unstale":"2023-11-24T08:05:19.036304+0000","last_undegraded":"2023-11-24T08:05:19.036304+0000","last_fullsized":"2023-11-24T08:05:19.036304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:27.754037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007343+0000","last_change":"2023-11-24T08:05:19.007343+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007343+0000","last_undegraded":"2023-11-24T08:05:19.007343+0000","last_fullsized":"2023-11-24T08:05:19.007343+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.039674+0000","last_change":"2023-11-24T08:05:19.034899+0000","last_active":"2023-11-24T08:05:19.039674+0000","last_peered":"2023-11-24T08:05:19.039674+0000","last_clean":"2023-11-24T08:05:19.039674+0000","last_became_active":"2023-11-24T08:05:19.034569+0000","last_became_peered":"2023-11-24T08:05:19.034569+0000","last_unstale":"2023-11-24T08:05:19.039674+0000","last_undegraded":"2023-11-24T08:05:19.039674+0000","last_fullsized":"2023-11-24T08:05:19.039674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018629+0000","last_change":"2023-11-24T08:05:19.018629+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018629+0000","last_undegraded":"2023-11-24T08:05:19.018629+0000","last_fullsized":"2023-11-24T08:05:19.018629+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031764+0000","last_change":"2023-11-24T08:05:19.027343+0000","last_active":"2023-11-24T08:05:19.031764+0000","last_peered":"2023-11-24T08:05:19.031764+0000","last_clean":"2023-11-24T08:05:19.031764+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:19.031764+0000","last_undegraded":"2023-11-24T08:05:19.031764+0000","last_fullsized":"2023-11-24T08:05:19.031764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:50.145064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.011239+0000","last_change":"2023-11-24T08:05:19.011239+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.011239+0000","last_undegraded":"2023-11-24T08:05:19.011239+0000","last_fullsized":"2023-11-24T08:05:19.011239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032952+0000","last_change":"2023-11-24T08:05:19.025711+0000","last_active":"2023-11-24T08:05:19.032952+0000","last_peered":"2023-11-24T08:05:19.032952+0000","last_clean":"2023-11-24T08:05:19.032952+0000","last_became_active":"2023-11-24T08:05:19.025421+0000","last_became_peered":"2023-11-24T08:05:19.025421+0000","last_unstale":"2023-11-24T08:05:19.032952+0000","last_undegraded":"2023-11-24T08:05:19.032952+0000","last_fullsized":"2023-11-24T08:05:19.032952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019282+0000","last_change":"2023-11-24T08:05:19.019282+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019282+0000","last_undegraded":"2023-11-24T08:05:19.019282+0000","last_fullsized":"2023-11-24T08:05:19.019282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034137+0000","last_change":"2023-11-24T08:05:19.029520+0000","last_active":"2023-11-24T08:05:19.034137+0000","last_peered":"2023-11-24T08:05:19.034137+0000","last_clean":"2023-11-24T08:05:19.034137+0000","last_became_active":"2023-11-24T08:05:19.029227+0000","last_became_peered":"2023-11-24T08:05:19.029227+0000","last_unstale":"2023-11-24T08:05:19.034137+0000","last_undegraded":"2023-11-24T08:05:19.034137+0000","last_fullsized":"2023-11-24T08:05:19.034137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017498+0000","last_change":"2023-11-24T08:05:19.017498+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017498+0000","last_undegraded":"2023-11-24T08:05:19.017498+0000","last_fullsized":"2023-11-24T08:05:19.017498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034959+0000","last_change":"2023-11-24T08:05:19.031328+0000","last_active":"2023-11-24T08:05:19.034959+0000","last_peered":"2023-11-24T08:05:19.034959+0000","last_clean":"2023-11-24T08:05:19.034959+0000","last_became_active":"2023-11-24T08:05:19.031046+0000","last_became_peered":"2023-11-24T08:05:19.031046+0000","last_unstale":"2023-11-24T08:05:19.034959+0000","last_undegraded":"2023-11-24T08:05:19.034959+0000","last_fullsized":"2023-11-24T08:05:19.034959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:04.782054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001684+0000","last_change":"2023-11-24T08:05:19.001684+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001684+0000","last_undegraded":"2023-11-24T08:05:19.001684+0000","last_fullsized":"2023-11-24T08:05:19.001684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036519+0000","last_change":"2023-11-24T08:05:19.030600+0000","last_active":"2023-11-24T08:05:19.036519+0000","last_peered":"2023-11-24T08:05:19.036519+0000","last_clean":"2023-11-24T08:05:19.036519+0000","last_became_active":"2023-11-24T08:05:19.028703+0000","last_became_peered":"2023-11-24T08:05:19.028703+0000","last_unstale":"2023-11-24T08:05:19.036519+0000","last_undegraded":"2023-11-24T08:05:19.036519+0000","last_fullsized":"2023-11-24T08:05:19.036519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:40.635942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009362+0000","last_change":"2023-11-24T08:05:19.009362+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009362+0000","last_undegraded":"2023-11-24T08:05:19.009362+0000","last_fullsized":"2023-11-24T08:05:19.009362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036848+0000","last_change":"2023-11-24T08:05:19.033681+0000","last_active":"2023-11-24T08:05:19.036848+0000","last_peered":"2023-11-24T08:05:19.036848+0000","last_clean":"2023-11-24T08:05:19.036848+0000","last_became_active":"2023-11-24T08:05:19.033420+0000","last_became_peered":"2023-11-24T08:05:19.033420+0000","last_unstale":"2023-11-24T08:05:19.036848+0000","last_undegraded":"2023-11-24T08:05:19.036848+0000","last_fullsized":"2023-11-24T08:05:19.036848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025637+0000","last_change":"2023-11-24T08:05:19.025637+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025637+0000","last_undegraded":"2023-11-24T08:05:19.025637+0000","last_fullsized":"2023-11-24T08:05:19.025637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.041907+0000","last_change":"2023-11-24T08:05:19.036326+0000","last_active":"2023-11-24T08:05:19.041907+0000","last_peered":"2023-11-24T08:05:19.041907+0000","last_clean":"2023-11-24T08:05:19.041907+0000","last_became_active":"2023-11-24T08:05:19.035962+0000","last_became_peered":"2023-11-24T08:05:19.035962+0000","last_unstale":"2023-11-24T08:05:19.041907+0000","last_undegraded":"2023-11-24T08:05:19.041907+0000","last_fullsized":"2023-11-24T08:05:19.041907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":174,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-24T08:05:17.997541+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:05:17.997541+0000","last_peered":"2023-11-24T08:05:17.997541+0000","last_clean":"2023-11-24T08:05:17.997541+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:05:17.997541+0000","last_undegraded":"2023-11-24T08:05:17.997541+0000","last_fullsized":"2023-11-24T08:05:17.997541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.876}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":1.3240000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":1.1200000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":4.0179999999999998,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.369}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.3759999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.284}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.554},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.484}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.823,"15min":1.823},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":6.8129999999999997,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.147},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.7509999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.879,"15min":1.879},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":9.8569999999999993,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":9.9450000000000003,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.667}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":1.601},{"interface":"front","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.5209999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.751},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.22}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.151},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.0569999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.419}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":1.325}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.379}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":1.496},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.3600000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":1.5580000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.1659999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.089}]},{"osd":2,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0409999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.21},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3779999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.262},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.278}]},{"osd":6,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.3440000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.5780000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.441}]},{"osd":2,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.3500000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.125},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.1579999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.528}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.034}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.9969999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.161},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.88800000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.84199999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.3149999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":3.161,"5min":3.161,"15min":3.161},"last":1.401}]},{"osd":5,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.365}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.218},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.5860000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.323},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.524}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":0.94199999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.92100000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.3340000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3049999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.49},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.5329999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.677},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.4179999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.609,"5min":1.609,"15min":1.609},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.7330000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.6120000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":722084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.321}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.2549999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.6320000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":1.659}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.704},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":2.2389999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":2.0590000000000002},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.845}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":2.1819999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.9630000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":2.1600000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":3.371,"5min":3.371,"15min":3.371},"last":2.2610000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:05:24.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:05:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.768+0000 7fdfd8ed1640 1 Processor -- start 2023-11-24T08:05:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.768+0000 7fdfd8ed1640 1 -- start start 2023-11-24T08:05:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.770+0000 7fdfd8ed1640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdfd413dfd0 0x7fdfd415be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.770+0000 7fdfd8ed1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdfd40cf050 0x7fdfd40cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.770+0000 7fdfd8ed1640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd40ce7c0 con 0x7fdfd406c130 2023-11-24T08:05:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.770+0000 7fdfd8ed1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd40cfa40 con 0x7fdfd40cf050 2023-11-24T08:05:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.770+0000 7fdfd8ed1640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd4140900 con 0x7fdfd413dfd0 2023-11-24T08:05:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.770+0000 7fdfcbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdfd40cf050 0x7fdfd40cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.770+0000 7fdfcbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdfd40cf050 0x7fdfd40cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48468/0 (socket says 172.21.15.64:48468) 2023-11-24T08:05:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.770+0000 7fdfcbfff640 1 -- 172.21.15.64:0/2829186562 learned_addr learned my addr 172.21.15.64:0/2829186562 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.771+0000 7fdfcbfff640 1 -- 172.21.15.64:0/2829186562 >> v2:172.21.15.91:3300/0 conn(0x7fdfd413dfd0 msgr2=0x7fdfd415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.771+0000 7fdfd37fe640 1 --2- 172.21.15.64:0/2829186562 >> v2:172.21.15.91:3300/0 conn(0x7fdfd413dfd0 0x7fdfd415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.771+0000 7fdfcbfff640 1 --2- 172.21.15.64:0/2829186562 >> v2:172.21.15.91:3300/0 conn(0x7fdfd413dfd0 0x7fdfd415be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.771+0000 7fdfcbfff640 1 -- 172.21.15.64:0/2829186562 >> v1:172.21.15.91:6789/0 conn(0x7fdfd406c130 legacy=0x7fdfd406c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.771+0000 7fdfcbfff640 1 -- 172.21.15.64:0/2829186562 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd415fa70 con 0x7fdfd40cf050 2023-11-24T08:05:24.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.772+0000 7fdfcbfff640 1 --2- 172.21.15.64:0/2829186562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd40cf500 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40cfc10 tx=0x7fdfbc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3a0f04d47a307 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:24.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.772+0000 7fdfd2ffd640 1 -- 172.21.15.64:0/2829186562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc031870 con 0x7fdfd40cf050 2023-11-24T08:05:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd2ffd640 1 -- 172.21.15.64:0/2829186562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc0319d0 con 0x7fdfd40cf050 2023-11-24T08:05:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd2ffd640 1 -- 172.21.15.64:0/2829186562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc031d60 con 0x7fdfd40cf050 2023-11-24T08:05:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/2829186562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 msgr2=0x7fdfd40cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/2829186562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd40cf500 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40cfc10 tx=0x7fdfbc02b910 comp rx=0 tx=0).stop 2023-11-24T08:05:24.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/2829186562 shutdown_connections 2023-11-24T08:05:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/2829186562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd40cf500 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/2829186562 >> v2:172.21.15.91:3300/0 conn(0x7fdfd413dfd0 0x7fdfd415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/2829186562 >> 172.21.15.64:0/2829186562 conn(0x7fdfd406a860 msgr2=0x7fdfd406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/2829186562 shutdown_connections 2023-11-24T08:05:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.773+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/2829186562 wait complete. 2023-11-24T08:05:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.775+0000 7fdfd8ed1640 1 Processor -- start 2023-11-24T08:05:24.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.775+0000 7fdfd8ed1640 1 -- start start 2023-11-24T08:05:24.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.776+0000 7fdfd8ed1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfd406c130 0x7fdfd4070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.776+0000 7fdfd8ed1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd4159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.776+0000 7fdfd8ed1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfd413dfd0 0x7fdfd4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.776+0000 7fdfd8ed1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41554d0 con 0x7fdfd40cf050 2023-11-24T08:05:24.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.776+0000 7fdfd8ed1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4158e60 con 0x7fdfd406c130 2023-11-24T08:05:24.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.776+0000 7fdfd8ed1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4070650 con 0x7fdfd413dfd0 2023-11-24T08:05:24.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd4159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd4159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48472/0 (socket says 172.21.15.64:48472) 2023-11-24T08:05:24.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 -- 172.21.15.64:0/348045056 learned_addr learned my addr 172.21.15.64:0/348045056 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:24.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfd37fe640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfd406c130 0x7fdfd4070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfd3fff640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfd413dfd0 0x7fdfd4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 -- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfd413dfd0 msgr2=0x7fdfd4154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:24.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfd413dfd0 0x7fdfd4154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 -- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfd406c130 msgr2=0x7fdfd4070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:24.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfd406c130 0x7fdfd4070110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:24.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 -- 172.21.15.64:0/348045056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc0094d0 con 0x7fdfd40cf050 2023-11-24T08:05:24.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.777+0000 7fdfcbfff640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd4159b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:24.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.778+0000 7fdfd0ff9640 1 -- 172.21.15.64:0/348045056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc031840 con 0x7fdfd40cf050 2023-11-24T08:05:24.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.778+0000 7fdfd0ff9640 1 -- 172.21.15.64:0/348045056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc0443f0 con 0x7fdfd40cf050 2023-11-24T08:05:24.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.778+0000 7fdfd0ff9640 1 -- 172.21.15.64:0/348045056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0447a0 con 0x7fdfd40cf050 2023-11-24T08:05:24.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.778+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd4071720 con 0x7fdfd40cf050 2023-11-24T08:05:24.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.778+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd4159390 con 0x7fdfd40cf050 2023-11-24T08:05:24.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.779+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd404e840 con 0x7fdfd40cf050 2023-11-24T08:05:24.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.780+0000 7fdfd0ff9640 1 -- 172.21.15.64:0/348045056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfbc04d450 con 0x7fdfd40cf050 2023-11-24T08:05:24.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.781+0000 7fdfd0ff9640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdfb406ae40 0x7fdfb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:24.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.781+0000 7fdfd0ff9640 1 -- 172.21.15.64:0/348045056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfbc02d040 con 0x7fdfd40cf050 2023-11-24T08:05:24.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.782+0000 7fdfd37fe640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdfb406ae40 0x7fdfb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:24.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.783+0000 7fdfd37fe640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdfb406ae40 0x7fdfb406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4006fd0 tx=0x7fdfc4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:24.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.785+0000 7fdfd0ff9640 1 -- 172.21.15.64:0/348045056 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdfbc076e80 con 0x7fdfd40cf050 2023-11-24T08:05:24.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.973+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfd415f420 con 0x7fdfb406ae40 2023-11-24T08:05:24.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:24.995+0000 7fdfd0ff9640 1 -- 172.21.15.64:0/348045056 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282105 (secure 0 0 0) 0x7fdfc400c050 con 0x7fdfb406ae40 2023-11-24T08:05:24.998 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:05:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdfb406ae40 msgr2=0x7fdfb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdfb406ae40 0x7fdfb406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4006fd0 tx=0x7fdfc4008040 comp rx=0 tx=0).stop 2023-11-24T08:05:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 msgr2=0x7fdfd4159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd4159b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 shutdown_connections 2023-11-24T08:05:25.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdfb406ae40 0x7fdfb406d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdfd413dfd0 0x7fdfd4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd40cf050 0x7fdfd4159b90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 --2- 172.21.15.64:0/348045056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdfd406c130 0x7fdfd4070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.001+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 >> 172.21.15.64:0/348045056 conn(0x7fdfd406a860 msgr2=0x7fdfd415de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:25.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.002+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 shutdown_connections 2023-11-24T08:05:25.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.002+0000 7fdfd8ed1640 1 -- 172.21.15.64:0/348045056 wait complete. 2023-11-24T08:05:25.021 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-11-24T08:05:24.043708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220036,"kb_used_data":3876,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754684,"statfs":{"total":773094113280,"available":772868796416,"internally_reserved":0,"allocated":3969024,"data_stored":2241597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001661"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019366+0000","last_change":"2023-11-24T08:05:19.019366+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019366+0000","last_undegraded":"2023-11-24T08:05:19.019366+0000","last_fullsized":"2023-11-24T08:05:19.019366+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002461+0000","last_change":"2023-11-24T08:05:19.002461+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002461+0000","last_undegraded":"2023-11-24T08:05:19.002461+0000","last_fullsized":"2023-11-24T08:05:19.002461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016949+0000","last_change":"2023-11-24T08:05:19.016949+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016949+0000","last_undegraded":"2023-11-24T08:05:19.016949+0000","last_fullsized":"2023-11-24T08:05:19.016949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017053+0000","last_change":"2023-11-24T08:05:19.017053+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017053+0000","last_undegraded":"2023-11-24T08:05:19.017053+0000","last_fullsized":"2023-11-24T08:05:19.017053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.010185+0000","last_change":"2023-11-24T08:05:19.010185+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.010185+0000","last_undegraded":"2023-11-24T08:05:19.010185+0000","last_fullsized":"2023-11-24T08:05:19.010185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002180+0000","last_change":"2023-11-24T08:05:19.002180+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002180+0000","last_undegraded":"2023-11-24T08:05:19.002180+0000","last_fullsized":"2023-11-24T08:05:19.002180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021735+0000","last_change":"2023-11-24T08:05:19.021735+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021735+0000","last_undegraded":"2023-11-24T08:05:19.021735+0000","last_fullsized":"2023-11-24T08:05:19.021735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.026170+0000","last_change":"2023-11-24T08:05:19.026170+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.026170+0000","last_undegraded":"2023-11-24T08:05:19.026170+0000","last_fullsized":"2023-11-24T08:05:19.026170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007708+0000","last_change":"2023-11-24T08:05:19.007708+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007708+0000","last_undegraded":"2023-11-24T08:05:19.007708+0000","last_fullsized":"2023-11-24T08:05:19.007708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019694+0000","last_change":"2023-11-24T08:05:19.019694+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019694+0000","last_undegraded":"2023-11-24T08:05:19.019694+0000","last_fullsized":"2023-11-24T08:05:19.019694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022034+0000","last_change":"2023-11-24T08:05:19.022034+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022034+0000","last_undegraded":"2023-11-24T08:05:19.022034+0000","last_fullsized":"2023-11-24T08:05:19.022034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.014621+0000","last_change":"2023-11-24T08:05:19.014621+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.014621+0000","last_undegraded":"2023-11-24T08:05:19.014621+0000","last_fullsized":"2023-11-24T08:05:19.014621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009712+0000","last_change":"2023-11-24T08:05:19.009712+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009712+0000","last_undegraded":"2023-11-24T08:05:19.009712+0000","last_fullsized":"2023-11-24T08:05:19.009712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001299+0000","last_change":"2023-11-24T08:05:19.001299+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001299+0000","last_undegraded":"2023-11-24T08:05:19.001299+0000","last_fullsized":"2023-11-24T08:05:19.001299+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.014116+0000","last_change":"2023-11-24T08:05:19.014116+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.014116+0000","last_undegraded":"2023-11-24T08:05:19.014116+0000","last_fullsized":"2023-11-24T08:05:19.014116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021676+0000","last_change":"2023-11-24T08:05:19.021676+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021676+0000","last_undegraded":"2023-11-24T08:05:19.021676+0000","last_fullsized":"2023-11-24T08:05:19.021676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022765+0000","last_change":"2023-11-24T08:05:19.022765+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022765+0000","last_undegraded":"2023-11-24T08:05:19.022765+0000","last_fullsized":"2023-11-24T08:05:19.022765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001872+0000","last_change":"2023-11-24T08:05:19.001872+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001872+0000","last_undegraded":"2023-11-24T08:05:19.001872+0000","last_fullsized":"2023-11-24T08:05:19.001872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003795+0000","last_change":"2023-11-24T08:05:19.003795+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003795+0000","last_undegraded":"2023-11-24T08:05:19.003795+0000","last_fullsized":"2023-11-24T08:05:19.003795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007408+0000","last_change":"2023-11-24T08:05:19.007408+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007408+0000","last_undegraded":"2023-11-24T08:05:19.007408+0000","last_fullsized":"2023-11-24T08:05:19.007408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.010630+0000","last_change":"2023-11-24T08:05:19.010630+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.010630+0000","last_undegraded":"2023-11-24T08:05:19.010630+0000","last_fullsized":"2023-11-24T08:05:19.010630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.006901+0000","last_change":"2023-11-24T08:05:19.006901+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.006901+0000","last_undegraded":"2023-11-24T08:05:19.006901+0000","last_fullsized":"2023-11-24T08:05:19.006901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.011354+0000","last_change":"2023-11-24T08:05:19.011354+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.011354+0000","last_undegraded":"2023-11-24T08:05:19.011354+0000","last_fullsized":"2023-11-24T08:05:19.011354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012941+0000","last_change":"2023-11-24T08:05:19.012941+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012941+0000","last_undegraded":"2023-11-24T08:05:19.012941+0000","last_fullsized":"2023-11-24T08:05:19.012941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018757+0000","last_change":"2023-11-24T08:05:19.018757+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018757+0000","last_undegraded":"2023-11-24T08:05:19.018757+0000","last_fullsized":"2023-11-24T08:05:19.018757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001270+0000","last_change":"2023-11-24T08:05:19.001270+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001270+0000","last_undegraded":"2023-11-24T08:05:19.001270+0000","last_fullsized":"2023-11-24T08:05:19.001270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021345+0000","last_change":"2023-11-24T08:05:19.021345+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021345+0000","last_undegraded":"2023-11-24T08:05:19.021345+0000","last_fullsized":"2023-11-24T08:05:19.021345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.013040+0000","last_change":"2023-11-24T08:05:19.013040+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.013040+0000","last_undegraded":"2023-11-24T08:05:19.013040+0000","last_fullsized":"2023-11-24T08:05:19.013040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009317+0000","last_change":"2023-11-24T08:05:19.009317+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009317+0000","last_undegraded":"2023-11-24T08:05:19.009317+0000","last_fullsized":"2023-11-24T08:05:19.009317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.020254+0000","last_change":"2023-11-24T08:05:19.020254+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.020254+0000","last_undegraded":"2023-11-24T08:05:19.020254+0000","last_fullsized":"2023-11-24T08:05:19.020254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022175+0000","last_change":"2023-11-24T08:05:19.022175+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022175+0000","last_undegraded":"2023-11-24T08:05:19.022175+0000","last_fullsized":"2023-11-24T08:05:19.022175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025740+0000","last_change":"2023-11-24T08:05:19.025740+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025740+0000","last_undegraded":"2023-11-24T08:05:19.025740+0000","last_fullsized":"2023-11-24T08:05:19.025740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016746+0000","last_change":"2023-11-24T08:05:19.016746+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016746+0000","last_undegraded":"2023-11-24T08:05:19.016746+0000","last_fullsized":"2023-11-24T08:05:19.016746+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.038997+0000","last_change":"2023-11-24T08:05:19.034855+0000","last_active":"2023-11-24T08:05:19.038997+0000","last_peered":"2023-11-24T08:05:19.038997+0000","last_clean":"2023-11-24T08:05:19.038997+0000","last_became_active":"2023-11-24T08:05:19.034614+0000","last_became_peered":"2023-11-24T08:05:19.034614+0000","last_unstale":"2023-11-24T08:05:19.038997+0000","last_undegraded":"2023-11-24T08:05:19.038997+0000","last_fullsized":"2023-11-24T08:05:19.038997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017903+0000","last_change":"2023-11-24T08:05:19.017903+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017903+0000","last_undegraded":"2023-11-24T08:05:19.017903+0000","last_fullsized":"2023-11-24T08:05:19.017903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034166+0000","last_change":"2023-11-24T08:05:19.029832+0000","last_active":"2023-11-24T08:05:19.034166+0000","last_peered":"2023-11-24T08:05:19.034166+0000","last_clean":"2023-11-24T08:05:19.034166+0000","last_became_active":"2023-11-24T08:05:19.028467+0000","last_became_peered":"2023-11-24T08:05:19.028467+0000","last_unstale":"2023-11-24T08:05:19.034166+0000","last_undegraded":"2023-11-24T08:05:19.034166+0000","last_fullsized":"2023-11-24T08:05:19.034166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:12.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.004380+0000","last_change":"2023-11-24T08:05:19.004380+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.004380+0000","last_undegraded":"2023-11-24T08:05:19.004380+0000","last_fullsized":"2023-11-24T08:05:19.004380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031446+0000","last_change":"2023-11-24T08:05:19.024698+0000","last_active":"2023-11-24T08:05:19.031446+0000","last_peered":"2023-11-24T08:05:19.031446+0000","last_clean":"2023-11-24T08:05:19.031446+0000","last_became_active":"2023-11-24T08:05:19.024477+0000","last_became_peered":"2023-11-24T08:05:19.024477+0000","last_unstale":"2023-11-24T08:05:19.031446+0000","last_undegraded":"2023-11-24T08:05:19.031446+0000","last_fullsized":"2023-11-24T08:05:19.031446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:50.782618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.020852+0000","last_change":"2023-11-24T08:05:19.020852+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.020852+0000","last_undegraded":"2023-11-24T08:05:19.020852+0000","last_fullsized":"2023-11-24T08:05:19.020852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.028717+0000","last_change":"2023-11-24T08:05:19.024566+0000","last_active":"2023-11-24T08:05:19.028717+0000","last_peered":"2023-11-24T08:05:19.028717+0000","last_clean":"2023-11-24T08:05:19.028717+0000","last_became_active":"2023-11-24T08:05:19.024310+0000","last_became_peered":"2023-11-24T08:05:19.024310+0000","last_unstale":"2023-11-24T08:05:19.028717+0000","last_undegraded":"2023-11-24T08:05:19.028717+0000","last_fullsized":"2023-11-24T08:05:19.028717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:59.756867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019547+0000","last_change":"2023-11-24T08:05:19.019547+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019547+0000","last_undegraded":"2023-11-24T08:05:19.019547+0000","last_fullsized":"2023-11-24T08:05:19.019547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034315+0000","last_change":"2023-11-24T08:05:19.027924+0000","last_active":"2023-11-24T08:05:19.034315+0000","last_peered":"2023-11-24T08:05:19.034315+0000","last_clean":"2023-11-24T08:05:19.034315+0000","last_became_active":"2023-11-24T08:05:19.023079+0000","last_became_peered":"2023-11-24T08:05:19.023079+0000","last_unstale":"2023-11-24T08:05:19.034315+0000","last_undegraded":"2023-11-24T08:05:19.034315+0000","last_fullsized":"2023-11-24T08:05:19.034315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:45.153445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012736+0000","last_change":"2023-11-24T08:05:19.012736+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012736+0000","last_undegraded":"2023-11-24T08:05:19.012736+0000","last_fullsized":"2023-11-24T08:05:19.012736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034070+0000","last_change":"2023-11-24T08:05:19.028892+0000","last_active":"2023-11-24T08:05:19.034070+0000","last_peered":"2023-11-24T08:05:19.034070+0000","last_clean":"2023-11-24T08:05:19.034070+0000","last_became_active":"2023-11-24T08:05:19.027535+0000","last_became_peered":"2023-11-24T08:05:19.027535+0000","last_unstale":"2023-11-24T08:05:19.034070+0000","last_undegraded":"2023-11-24T08:05:19.034070+0000","last_fullsized":"2023-11-24T08:05:19.034070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:45:40.009128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003620+0000","last_change":"2023-11-24T08:05:19.003620+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003620+0000","last_undegraded":"2023-11-24T08:05:19.003620+0000","last_fullsized":"2023-11-24T08:05:19.003620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.019329+0000","last_change":"2023-11-24T08:05:19.015336+0000","last_active":"2023-11-24T08:05:19.019329+0000","last_peered":"2023-11-24T08:05:19.019329+0000","last_clean":"2023-11-24T08:05:19.019329+0000","last_became_active":"2023-11-24T08:05:19.014910+0000","last_became_peered":"2023-11-24T08:05:19.014910+0000","last_unstale":"2023-11-24T08:05:19.019329+0000","last_undegraded":"2023-11-24T08:05:19.019329+0000","last_fullsized":"2023-11-24T08:05:19.019329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:27.754037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017987+0000","last_change":"2023-11-24T08:05:19.017987+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017987+0000","last_undegraded":"2023-11-24T08:05:19.017987+0000","last_fullsized":"2023-11-24T08:05:19.017987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036507+0000","last_change":"2023-11-24T08:05:19.031449+0000","last_active":"2023-11-24T08:05:19.036507+0000","last_peered":"2023-11-24T08:05:19.036507+0000","last_clean":"2023-11-24T08:05:19.036507+0000","last_became_active":"2023-11-24T08:05:19.031114+0000","last_became_peered":"2023-11-24T08:05:19.031114+0000","last_unstale":"2023-11-24T08:05:19.036507+0000","last_undegraded":"2023-11-24T08:05:19.036507+0000","last_fullsized":"2023-11-24T08:05:19.036507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:56.728306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012757+0000","last_change":"2023-11-24T08:05:19.012757+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012757+0000","last_undegraded":"2023-11-24T08:05:19.012757+0000","last_fullsized":"2023-11-24T08:05:19.012757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032407+0000","last_change":"2023-11-24T08:05:19.028843+0000","last_active":"2023-11-24T08:05:19.032407+0000","last_peered":"2023-11-24T08:05:19.032407+0000","last_clean":"2023-11-24T08:05:19.032407+0000","last_became_active":"2023-11-24T08:05:19.027840+0000","last_became_peered":"2023-11-24T08:05:19.027840+0000","last_unstale":"2023-11-24T08:05:19.032407+0000","last_undegraded":"2023-11-24T08:05:19.032407+0000","last_fullsized":"2023-11-24T08:05:19.032407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:12.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003124+0000","last_change":"2023-11-24T08:05:19.003124+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003124+0000","last_undegraded":"2023-11-24T08:05:19.003124+0000","last_fullsized":"2023-11-24T08:05:19.003124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034147+0000","last_change":"2023-11-24T08:05:19.029786+0000","last_active":"2023-11-24T08:05:19.034147+0000","last_peered":"2023-11-24T08:05:19.034147+0000","last_clean":"2023-11-24T08:05:19.034147+0000","last_became_active":"2023-11-24T08:05:19.029469+0000","last_became_peered":"2023-11-24T08:05:19.029469+0000","last_unstale":"2023-11-24T08:05:19.034147+0000","last_undegraded":"2023-11-24T08:05:19.034147+0000","last_fullsized":"2023-11-24T08:05:19.034147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002655+0000","last_change":"2023-11-24T08:05:19.002655+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002655+0000","last_undegraded":"2023-11-24T08:05:19.002655+0000","last_fullsized":"2023-11-24T08:05:19.002655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036658+0000","last_change":"2023-11-24T08:05:19.034418+0000","last_active":"2023-11-24T08:05:19.036658+0000","last_peered":"2023-11-24T08:05:19.036658+0000","last_clean":"2023-11-24T08:05:19.036658+0000","last_became_active":"2023-11-24T08:05:19.033911+0000","last_became_peered":"2023-11-24T08:05:19.033911+0000","last_unstale":"2023-11-24T08:05:19.036658+0000","last_undegraded":"2023-11-24T08:05:19.036658+0000","last_fullsized":"2023-11-24T08:05:19.036658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:50.782618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018899+0000","last_change":"2023-11-24T08:05:19.018899+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018899+0000","last_undegraded":"2023-11-24T08:05:19.018899+0000","last_fullsized":"2023-11-24T08:05:19.018899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031376+0000","last_change":"2023-11-24T08:05:19.023626+0000","last_active":"2023-11-24T08:05:19.031376+0000","last_peered":"2023-11-24T08:05:19.031376+0000","last_clean":"2023-11-24T08:05:19.031376+0000","last_became_active":"2023-11-24T08:05:19.023367+0000","last_became_peered":"2023-11-24T08:05:19.023367+0000","last_unstale":"2023-11-24T08:05:19.031376+0000","last_undegraded":"2023-11-24T08:05:19.031376+0000","last_fullsized":"2023-11-24T08:05:19.031376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:50.145064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017655+0000","last_change":"2023-11-24T08:05:19.017655+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017655+0000","last_undegraded":"2023-11-24T08:05:19.017655+0000","last_fullsized":"2023-11-24T08:05:19.017655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034295+0000","last_change":"2023-11-24T08:05:19.026245+0000","last_active":"2023-11-24T08:05:19.034295+0000","last_peered":"2023-11-24T08:05:19.034295+0000","last_clean":"2023-11-24T08:05:19.034295+0000","last_became_active":"2023-11-24T08:05:19.025943+0000","last_became_peered":"2023-11-24T08:05:19.025943+0000","last_unstale":"2023-11-24T08:05:19.034295+0000","last_undegraded":"2023-11-24T08:05:19.034295+0000","last_fullsized":"2023-11-24T08:05:19.034295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:44.721323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001371+0000","last_change":"2023-11-24T08:05:19.001371+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001371+0000","last_undegraded":"2023-11-24T08:05:19.001371+0000","last_fullsized":"2023-11-24T08:05:19.001371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.033021+0000","last_change":"2023-11-24T08:05:19.027534+0000","last_active":"2023-11-24T08:05:19.033021+0000","last_peered":"2023-11-24T08:05:19.033021+0000","last_clean":"2023-11-24T08:05:19.033021+0000","last_became_active":"2023-11-24T08:05:19.024963+0000","last_became_peered":"2023-11-24T08:05:19.024963+0000","last_unstale":"2023-11-24T08:05:19.033021+0000","last_undegraded":"2023-11-24T08:05:19.033021+0000","last_fullsized":"2023-11-24T08:05:19.033021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:18.282886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025333+0000","last_change":"2023-11-24T08:05:19.025333+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025333+0000","last_undegraded":"2023-11-24T08:05:19.025333+0000","last_fullsized":"2023-11-24T08:05:19.025333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032465+0000","last_change":"2023-11-24T08:05:19.028331+0000","last_active":"2023-11-24T08:05:19.032465+0000","last_peered":"2023-11-24T08:05:19.032465+0000","last_clean":"2023-11-24T08:05:19.032465+0000","last_became_active":"2023-11-24T08:05:19.028037+0000","last_became_peered":"2023-11-24T08:05:19.028037+0000","last_unstale":"2023-11-24T08:05:19.032465+0000","last_undegraded":"2023-11-24T08:05:19.032465+0000","last_fullsized":"2023-11-24T08:05:19.032465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:00.858708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017553+0000","last_change":"2023-11-24T08:05:19.017553+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017553+0000","last_undegraded":"2023-11-24T08:05:19.017553+0000","last_fullsized":"2023-11-24T08:05:19.017553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031349+0000","last_change":"2023-11-24T08:05:19.023564+0000","last_active":"2023-11-24T08:05:19.031349+0000","last_peered":"2023-11-24T08:05:19.031349+0000","last_clean":"2023-11-24T08:05:19.031349+0000","last_became_active":"2023-11-24T08:05:19.022784+0000","last_became_peered":"2023-11-24T08:05:19.022784+0000","last_unstale":"2023-11-24T08:05:19.031349+0000","last_undegraded":"2023-11-24T08:05:19.031349+0000","last_fullsized":"2023-11-24T08:05:19.031349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.008194+0000","last_change":"2023-11-24T08:05:19.008194+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.008194+0000","last_undegraded":"2023-11-24T08:05:19.008194+0000","last_fullsized":"2023-11-24T08:05:19.008194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034866+0000","last_change":"2023-11-24T08:05:19.030608+0000","last_active":"2023-11-24T08:05:19.034866+0000","last_peered":"2023-11-24T08:05:19.034866+0000","last_clean":"2023-11-24T08:05:19.034866+0000","last_became_active":"2023-11-24T08:05:19.030359+0000","last_became_peered":"2023-11-24T08:05:19.030359+0000","last_unstale":"2023-11-24T08:05:19.034866+0000","last_undegraded":"2023-11-24T08:05:19.034866+0000","last_fullsized":"2023-11-24T08:05:19.034866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.026741+0000","last_change":"2023-11-24T08:05:19.026741+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.026741+0000","last_undegraded":"2023-11-24T08:05:19.026741+0000","last_fullsized":"2023-11-24T08:05:19.026741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034369+0000","last_change":"2023-11-24T08:05:19.027262+0000","last_active":"2023-11-24T08:05:19.034369+0000","last_peered":"2023-11-24T08:05:19.034369+0000","last_clean":"2023-11-24T08:05:19.034369+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:19.034369+0000","last_undegraded":"2023-11-24T08:05:19.034369+0000","last_fullsized":"2023-11-24T08:05:19.034369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:39.801164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016569+0000","last_change":"2023-11-24T08:05:19.016569+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016569+0000","last_undegraded":"2023-11-24T08:05:19.016569+0000","last_fullsized":"2023-11-24T08:05:19.016569+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034100+0000","last_change":"2023-11-24T08:05:19.025861+0000","last_active":"2023-11-24T08:05:19.034100+0000","last_peered":"2023-11-24T08:05:19.034100+0000","last_clean":"2023-11-24T08:05:19.034100+0000","last_became_active":"2023-11-24T08:05:19.024918+0000","last_became_peered":"2023-11-24T08:05:19.024918+0000","last_unstale":"2023-11-24T08:05:19.034100+0000","last_undegraded":"2023-11-24T08:05:19.034100+0000","last_fullsized":"2023-11-24T08:05:19.034100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:59.275982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.008986+0000","last_change":"2023-11-24T08:05:19.008986+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.008986+0000","last_undegraded":"2023-11-24T08:05:19.008986+0000","last_fullsized":"2023-11-24T08:05:19.008986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.030949+0000","last_change":"2023-11-24T08:05:19.017985+0000","last_active":"2023-11-24T08:05:19.030949+0000","last_peered":"2023-11-24T08:05:19.030949+0000","last_clean":"2023-11-24T08:05:19.030949+0000","last_became_active":"2023-11-24T08:05:19.017444+0000","last_became_peered":"2023-11-24T08:05:19.017444+0000","last_unstale":"2023-11-24T08:05:19.030949+0000","last_undegraded":"2023-11-24T08:05:19.030949+0000","last_fullsized":"2023-11-24T08:05:19.030949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:45:30.962562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007396+0000","last_change":"2023-11-24T08:05:19.007396+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007396+0000","last_undegraded":"2023-11-24T08:05:19.007396+0000","last_fullsized":"2023-11-24T08:05:19.007396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031706+0000","last_change":"2023-11-24T08:05:19.026334+0000","last_active":"2023-11-24T08:05:19.031706+0000","last_peered":"2023-11-24T08:05:19.031706+0000","last_clean":"2023-11-24T08:05:19.031706+0000","last_became_active":"2023-11-24T08:05:19.025952+0000","last_became_peered":"2023-11-24T08:05:19.025952+0000","last_unstale":"2023-11-24T08:05:19.031706+0000","last_undegraded":"2023-11-24T08:05:19.031706+0000","last_fullsized":"2023-11-24T08:05:19.031706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009085+0000","last_change":"2023-11-24T08:05:19.009085+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009085+0000","last_undegraded":"2023-11-24T08:05:19.009085+0000","last_fullsized":"2023-11-24T08:05:19.009085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036500+0000","last_change":"2023-11-24T08:05:19.034595+0000","last_active":"2023-11-24T08:05:19.036500+0000","last_peered":"2023-11-24T08:05:19.036500+0000","last_clean":"2023-11-24T08:05:19.036500+0000","last_became_active":"2023-11-24T08:05:19.033790+0000","last_became_peered":"2023-11-24T08:05:19.033790+0000","last_unstale":"2023-11-24T08:05:19.036500+0000","last_undegraded":"2023-11-24T08:05:19.036500+0000","last_fullsized":"2023-11-24T08:05:19.036500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007348+0000","last_change":"2023-11-24T08:05:19.007348+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007348+0000","last_undegraded":"2023-11-24T08:05:19.007348+0000","last_fullsized":"2023-11-24T08:05:19.007348+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034034+0000","last_change":"2023-11-24T08:05:19.028570+0000","last_active":"2023-11-24T08:05:19.034034+0000","last_peered":"2023-11-24T08:05:19.034034+0000","last_clean":"2023-11-24T08:05:19.034034+0000","last_became_active":"2023-11-24T08:05:19.028257+0000","last_became_peered":"2023-11-24T08:05:19.028257+0000","last_unstale":"2023-11-24T08:05:19.034034+0000","last_undegraded":"2023-11-24T08:05:19.034034+0000","last_fullsized":"2023-11-24T08:05:19.034034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:59.756867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022389+0000","last_change":"2023-11-24T08:05:19.022389+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022389+0000","last_undegraded":"2023-11-24T08:05:19.022389+0000","last_fullsized":"2023-11-24T08:05:19.022389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036304+0000","last_change":"2023-11-24T08:05:19.032519+0000","last_active":"2023-11-24T08:05:19.036304+0000","last_peered":"2023-11-24T08:05:19.036304+0000","last_clean":"2023-11-24T08:05:19.036304+0000","last_became_active":"2023-11-24T08:05:19.032159+0000","last_became_peered":"2023-11-24T08:05:19.032159+0000","last_unstale":"2023-11-24T08:05:19.036304+0000","last_undegraded":"2023-11-24T08:05:19.036304+0000","last_fullsized":"2023-11-24T08:05:19.036304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:27.754037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007343+0000","last_change":"2023-11-24T08:05:19.007343+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007343+0000","last_undegraded":"2023-11-24T08:05:19.007343+0000","last_fullsized":"2023-11-24T08:05:19.007343+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.039674+0000","last_change":"2023-11-24T08:05:19.034899+0000","last_active":"2023-11-24T08:05:19.039674+0000","last_peered":"2023-11-24T08:05:19.039674+0000","last_clean":"2023-11-24T08:05:19.039674+0000","last_became_active":"2023-11-24T08:05:19.034569+0000","last_became_peered":"2023-11-24T08:05:19.034569+0000","last_unstale":"2023-11-24T08:05:19.039674+0000","last_undegraded":"2023-11-24T08:05:19.039674+0000","last_fullsized":"2023-11-24T08:05:19.039674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018629+0000","last_change":"2023-11-24T08:05:19.018629+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018629+0000","last_undegraded":"2023-11-24T08:05:19.018629+0000","last_fullsized":"2023-11-24T08:05:19.018629+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031764+0000","last_change":"2023-11-24T08:05:19.027343+0000","last_active":"2023-11-24T08:05:19.031764+0000","last_peered":"2023-11-24T08:05:19.031764+0000","last_clean":"2023-11-24T08:05:19.031764+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:19.031764+0000","last_undegraded":"2023-11-24T08:05:19.031764+0000","last_fullsized":"2023-11-24T08:05:19.031764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:50.145064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.011239+0000","last_change":"2023-11-24T08:05:19.011239+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.011239+0000","last_undegraded":"2023-11-24T08:05:19.011239+0000","last_fullsized":"2023-11-24T08:05:19.011239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032952+0000","last_change":"2023-11-24T08:05:19.025711+0000","last_active":"2023-11-24T08:05:19.032952+0000","last_peered":"2023-11-24T08:05:19.032952+0000","last_clean":"2023-11-24T08:05:19.032952+0000","last_became_active":"2023-11-24T08:05:19.025421+0000","last_became_peered":"2023-11-24T08:05:19.025421+0000","last_unstale":"2023-11-24T08:05:19.032952+0000","last_undegraded":"2023-11-24T08:05:19.032952+0000","last_fullsized":"2023-11-24T08:05:19.032952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019282+0000","last_change":"2023-11-24T08:05:19.019282+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019282+0000","last_undegraded":"2023-11-24T08:05:19.019282+0000","last_fullsized":"2023-11-24T08:05:19.019282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034137+0000","last_change":"2023-11-24T08:05:19.029520+0000","last_active":"2023-11-24T08:05:19.034137+0000","last_peered":"2023-11-24T08:05:19.034137+0000","last_clean":"2023-11-24T08:05:19.034137+0000","last_became_active":"2023-11-24T08:05:19.029227+0000","last_became_peered":"2023-11-24T08:05:19.029227+0000","last_unstale":"2023-11-24T08:05:19.034137+0000","last_undegraded":"2023-11-24T08:05:19.034137+0000","last_fullsized":"2023-11-24T08:05:19.034137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017498+0000","last_change":"2023-11-24T08:05:19.017498+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017498+0000","last_undegraded":"2023-11-24T08:05:19.017498+0000","last_fullsized":"2023-11-24T08:05:19.017498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034959+0000","last_change":"2023-11-24T08:05:19.031328+0000","last_active":"2023-11-24T08:05:19.034959+0000","last_peered":"2023-11-24T08:05:19.034959+0000","last_clean":"2023-11-24T08:05:19.034959+0000","last_became_active":"2023-11-24T08:05:19.031046+0000","last_became_peered":"2023-11-24T08:05:19.031046+0000","last_unstale":"2023-11-24T08:05:19.034959+0000","last_undegraded":"2023-11-24T08:05:19.034959+0000","last_fullsized":"2023-11-24T08:05:19.034959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:04.782054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001684+0000","last_change":"2023-11-24T08:05:19.001684+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001684+0000","last_undegraded":"2023-11-24T08:05:19.001684+0000","last_fullsized":"2023-11-24T08:05:19.001684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036519+0000","last_change":"2023-11-24T08:05:19.030600+0000","last_active":"2023-11-24T08:05:19.036519+0000","last_peered":"2023-11-24T08:05:19.036519+0000","last_clean":"2023-11-24T08:05:19.036519+0000","last_became_active":"2023-11-24T08:05:19.028703+0000","last_became_peered":"2023-11-24T08:05:19.028703+0000","last_unstale":"2023-11-24T08:05:19.036519+0000","last_undegraded":"2023-11-24T08:05:19.036519+0000","last_fullsized":"2023-11-24T08:05:19.036519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:40.635942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009362+0000","last_change":"2023-11-24T08:05:19.009362+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009362+0000","last_undegraded":"2023-11-24T08:05:19.009362+0000","last_fullsized":"2023-11-24T08:05:19.009362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036848+0000","last_change":"2023-11-24T08:05:19.033681+0000","last_active":"2023-11-24T08:05:19.036848+0000","last_peered":"2023-11-24T08:05:19.036848+0000","last_clean":"2023-11-24T08:05:19.036848+0000","last_became_active":"2023-11-24T08:05:19.033420+0000","last_became_peered":"2023-11-24T08:05:19.033420+0000","last_unstale":"2023-11-24T08:05:19.036848+0000","last_undegraded":"2023-11-24T08:05:19.036848+0000","last_fullsized":"2023-11-24T08:05:19.036848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025637+0000","last_change":"2023-11-24T08:05:19.025637+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025637+0000","last_undegraded":"2023-11-24T08:05:19.025637+0000","last_fullsized":"2023-11-24T08:05:19.025637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.041907+0000","last_change":"2023-11-24T08:05:19.036326+0000","last_active":"2023-11-24T08:05:19.041907+0000","last_peered":"2023-11-24T08:05:19.041907+0000","last_clean":"2023-11-24T08:05:19.041907+0000","last_became_active":"2023-11-24T08:05:19.035962+0000","last_became_peered":"2023-11-24T08:05:19.035962+0000","last_unstale":"2023-11-24T08:05:19.041907+0000","last_undegraded":"2023-11-24T08:05:19.041907+0000","last_fullsized":"2023-11-24T08:05:19.041907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":174,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-24T08:05:17.997541+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:05:17.997541+0000","last_peered":"2023-11-24T08:05:17.997541+0000","last_clean":"2023-11-24T08:05:17.997541+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:05:17.997541+0000","last_undegraded":"2023-11-24T08:05:17.997541+0000","last_fullsized":"2023-11-24T08:05:17.997541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.876}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":1.3240000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":1.1200000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":4.0179999999999998,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.369}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.3759999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.284}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.554},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.484}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.823,"15min":1.823},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":6.8129999999999997,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.147},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.7509999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.879,"15min":1.879},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":9.8569999999999993,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":9.9450000000000003,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.667}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":1.601},{"interface":"front","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.5209999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.751},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.22}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.151},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.0569999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.419}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":1.325}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.379}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":1.496},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.3600000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":1.5580000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.1659999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.089}]},{"osd":2,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0409999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.21},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3779999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.262},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.278}]},{"osd":6,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.3440000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.5780000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.441}]},{"osd":2,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.3500000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.125},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.1579999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.528}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.034}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.9969999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.161},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.88800000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.84199999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.3149999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":3.161,"5min":3.161,"15min":3.161},"last":1.401}]},{"osd":5,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.365}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.218},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.5860000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.323},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.524}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":0.94199999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.92100000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.3340000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3049999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.49},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.5329999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.677},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.4179999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.609,"5min":1.609,"15min":1.609},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.7330000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.6120000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":722084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.321}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.2549999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.6320000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":1.659}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.704},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":2.2389999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":2.0590000000000002},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.845}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":2.1819999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.9630000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":2.1600000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":3.371,"5min":3.371,"15min":3.371},"last":2.2610000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:05:25.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:05:25.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.299+0000 7f7e2597c640 1 Processor -- start 2023-11-24T08:05:25.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.299+0000 7f7e2597c640 1 -- start start 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e2597c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7e2006d520 0x7f7e2006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e2597c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7e200ad720 0x7f7e2015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e2597c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200abff0 0x7f7e2015f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e2597c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e200b41a0 con 0x7f7e200ad720 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e2597c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2015c360 con 0x7f7e2006d520 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e2597c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2006de40 con 0x7f7e200abff0 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e17fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7e200ad720 0x7f7e2015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e17fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7e200ad720 0x7f7e2015ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48482/0 (socket says 172.21.15.64:48482) 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e17fff640 1 -- 172.21.15.64:0/3162703352 learned_addr learned my addr 172.21.15.64:0/3162703352 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e1effd640 1 --2- 172.21.15.64:0/3162703352 >> v2:172.21.15.91:3300/0 conn(0x7f7e2006d520 0x7f7e2006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.301+0000 7f7e1f7fe640 1 --2- 172.21.15.64:0/3162703352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200abff0 0x7f7e2015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.302+0000 7f7e17fff640 1 -- 172.21.15.64:0/3162703352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200abff0 msgr2=0x7f7e2015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.302+0000 7f7e17fff640 1 --2- 172.21.15.64:0/3162703352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200abff0 0x7f7e2015f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.302+0000 7f7e17fff640 1 -- 172.21.15.64:0/3162703352 >> v2:172.21.15.91:3300/0 conn(0x7f7e2006d520 msgr2=0x7f7e2006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.302+0000 7f7e17fff640 1 --2- 172.21.15.64:0/3162703352 >> v2:172.21.15.91:3300/0 conn(0x7f7e2006d520 0x7f7e2006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.302+0000 7f7e17fff640 1 -- 172.21.15.64:0/3162703352 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2015ba30 con 0x7f7e200ad720 2023-11-24T08:05:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.303+0000 7f7e17fff640 1 --2- 172.21.15.64:0/3162703352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200ad720 0x7f7e2015ecd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7e2006c070 tx=0x7f7e0802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d210810f2ad3634c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.303+0000 7f7e1e7fc640 1 -- 172.21.15.64:0/3162703352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08031860 con 0x7f7e200ad720 2023-11-24T08:05:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.303+0000 7f7e1e7fc640 1 -- 172.21.15.64:0/3162703352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e080319c0 con 0x7f7e200ad720 2023-11-24T08:05:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.303+0000 7f7e1e7fc640 1 -- 172.21.15.64:0/3162703352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08031d50 con 0x7f7e200ad720 2023-11-24T08:05:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 -- 172.21.15.64:0/3162703352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200ad720 msgr2=0x7f7e2015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 --2- 172.21.15.64:0/3162703352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200ad720 0x7f7e2015ecd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7e2006c070 tx=0x7f7e0802b900 comp rx=0 tx=0).stop 2023-11-24T08:05:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 -- 172.21.15.64:0/3162703352 shutdown_connections 2023-11-24T08:05:25.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 --2- 172.21.15.64:0/3162703352 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200abff0 0x7f7e2015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 --2- 172.21.15.64:0/3162703352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200ad720 0x7f7e2015ecd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 --2- 172.21.15.64:0/3162703352 >> v2:172.21.15.91:3300/0 conn(0x7f7e2006d520 0x7f7e2006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 -- 172.21.15.64:0/3162703352 >> 172.21.15.64:0/3162703352 conn(0x7f7e200b0d80 msgr2=0x7f7e200b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:25.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 -- 172.21.15.64:0/3162703352 shutdown_connections 2023-11-24T08:05:25.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.304+0000 7f7e2597c640 1 -- 172.21.15.64:0/3162703352 wait complete. 2023-11-24T08:05:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.306+0000 7f7e2597c640 1 Processor -- start 2023-11-24T08:05:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.307+0000 7f7e2597c640 1 -- start start 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e2597c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e2006d520 0x7f7e20155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e2597c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200abff0 0x7f7e200ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e2597c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200ad720 0x7f7e2015a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e2597c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20155bb0 con 0x7f7e200abff0 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e2597c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200cadc0 con 0x7f7e2006d520 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e2597c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20071290 con 0x7f7e200ad720 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200abff0 0x7f7e200ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200abff0 0x7f7e200ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48498/0 (socket says 172.21.15.64:48498) 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.308+0000 7f7e17fff640 1 -- 172.21.15.64:0/2160442278 learned_addr learned my addr 172.21.15.64:0/2160442278 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e1effd640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e2006d520 0x7f7e20155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e1f7fe640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200ad720 0x7f7e2015a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e17fff640 1 -- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200ad720 msgr2=0x7f7e2015a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e17fff640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200ad720 0x7f7e2015a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e17fff640 1 -- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e2006d520 msgr2=0x7f7e20155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e17fff640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e2006d520 0x7f7e20155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e17fff640 1 -- 172.21.15.64:0/2160442278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080094d0 con 0x7f7e200abff0 2023-11-24T08:05:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e1f7fe640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200ad720 0x7f7e2015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e17fff640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200abff0 0x7f7e200ca880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7e0802b940 tx=0x7f7e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:25.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e1cff9640 1 -- 172.21.15.64:0/2160442278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0803ad30 con 0x7f7e200abff0 2023-11-24T08:05:25.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.309+0000 7f7e1cff9640 1 -- 172.21.15.64:0/2160442278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e080443f0 con 0x7f7e200abff0 2023-11-24T08:05:25.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.310+0000 7f7e1cff9640 1 -- 172.21.15.64:0/2160442278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080447a0 con 0x7f7e200abff0 2023-11-24T08:05:25.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.310+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e200c9da0 con 0x7f7e200abff0 2023-11-24T08:05:25.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.310+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e200ca360 con 0x7f7e200abff0 2023-11-24T08:05:25.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.311+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e200ac840 con 0x7f7e200abff0 2023-11-24T08:05:25.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.312+0000 7f7e1cff9640 1 -- 172.21.15.64:0/2160442278 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e0804d450 con 0x7f7e200abff0 2023-11-24T08:05:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.313+0000 7f7e1cff9640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e0006ae40 0x7f7e0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.313+0000 7f7e1cff9640 1 -- 172.21.15.64:0/2160442278 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e080abc60 con 0x7f7e200abff0 2023-11-24T08:05:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.313+0000 7f7e1effd640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e0006ae40 0x7f7e0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.315+0000 7f7e1effd640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e0006ae40 0x7f7e0006d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e10006fd0 tx=0x7f7e10008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:25.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.317+0000 7f7e1cff9640 1 -- 172.21.15.64:0/2160442278 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e08076c70 con 0x7f7e200abff0 2023-11-24T08:05:25.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.581+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e2015b6e0 con 0x7f7e0006ae40 2023-11-24T08:05:25.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.604+0000 7f7e1cff9640 1 -- 172.21.15.64:0/2160442278 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282105 (secure 0 0 0) 0x7f7e1000f830 con 0x7f7e0006ae40 2023-11-24T08:05:25.608 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:05:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e0006ae40 msgr2=0x7f7e0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e0006ae40 0x7f7e0006d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e10006fd0 tx=0x7f7e10008040 comp rx=0 tx=0).stop 2023-11-24T08:05:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200abff0 msgr2=0x7f7e200ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200abff0 0x7f7e200ca880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7e0802b940 tx=0x7f7e080046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 shutdown_connections 2023-11-24T08:05:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e0006ae40 0x7f7e0006d300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e200ad720 0x7f7e2015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e200abff0 0x7f7e200ca880 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 --2- 172.21.15.64:0/2160442278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e2006d520 0x7f7e20155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 >> 172.21.15.64:0/2160442278 conn(0x7f7e200b0d80 msgr2=0x7f7e200af650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 shutdown_connections 2023-11-24T08:05:25.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.610+0000 7f7e2597c640 1 -- 172.21.15.64:0/2160442278 wait complete. 2023-11-24T08:05:25.632 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-11-24T08:05:24.043708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220036,"kb_used_data":3876,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754684,"statfs":{"total":773094113280,"available":772868796416,"internally_reserved":0,"allocated":3969024,"data_stored":2241597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001661"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019366+0000","last_change":"2023-11-24T08:05:19.019366+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019366+0000","last_undegraded":"2023-11-24T08:05:19.019366+0000","last_fullsized":"2023-11-24T08:05:19.019366+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002461+0000","last_change":"2023-11-24T08:05:19.002461+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002461+0000","last_undegraded":"2023-11-24T08:05:19.002461+0000","last_fullsized":"2023-11-24T08:05:19.002461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016949+0000","last_change":"2023-11-24T08:05:19.016949+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016949+0000","last_undegraded":"2023-11-24T08:05:19.016949+0000","last_fullsized":"2023-11-24T08:05:19.016949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017053+0000","last_change":"2023-11-24T08:05:19.017053+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017053+0000","last_undegraded":"2023-11-24T08:05:19.017053+0000","last_fullsized":"2023-11-24T08:05:19.017053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.010185+0000","last_change":"2023-11-24T08:05:19.010185+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.010185+0000","last_undegraded":"2023-11-24T08:05:19.010185+0000","last_fullsized":"2023-11-24T08:05:19.010185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002180+0000","last_change":"2023-11-24T08:05:19.002180+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002180+0000","last_undegraded":"2023-11-24T08:05:19.002180+0000","last_fullsized":"2023-11-24T08:05:19.002180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021735+0000","last_change":"2023-11-24T08:05:19.021735+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021735+0000","last_undegraded":"2023-11-24T08:05:19.021735+0000","last_fullsized":"2023-11-24T08:05:19.021735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.026170+0000","last_change":"2023-11-24T08:05:19.026170+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.026170+0000","last_undegraded":"2023-11-24T08:05:19.026170+0000","last_fullsized":"2023-11-24T08:05:19.026170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007708+0000","last_change":"2023-11-24T08:05:19.007708+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007708+0000","last_undegraded":"2023-11-24T08:05:19.007708+0000","last_fullsized":"2023-11-24T08:05:19.007708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019694+0000","last_change":"2023-11-24T08:05:19.019694+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019694+0000","last_undegraded":"2023-11-24T08:05:19.019694+0000","last_fullsized":"2023-11-24T08:05:19.019694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022034+0000","last_change":"2023-11-24T08:05:19.022034+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022034+0000","last_undegraded":"2023-11-24T08:05:19.022034+0000","last_fullsized":"2023-11-24T08:05:19.022034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.014621+0000","last_change":"2023-11-24T08:05:19.014621+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.014621+0000","last_undegraded":"2023-11-24T08:05:19.014621+0000","last_fullsized":"2023-11-24T08:05:19.014621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009712+0000","last_change":"2023-11-24T08:05:19.009712+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009712+0000","last_undegraded":"2023-11-24T08:05:19.009712+0000","last_fullsized":"2023-11-24T08:05:19.009712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001299+0000","last_change":"2023-11-24T08:05:19.001299+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001299+0000","last_undegraded":"2023-11-24T08:05:19.001299+0000","last_fullsized":"2023-11-24T08:05:19.001299+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.014116+0000","last_change":"2023-11-24T08:05:19.014116+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.014116+0000","last_undegraded":"2023-11-24T08:05:19.014116+0000","last_fullsized":"2023-11-24T08:05:19.014116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021676+0000","last_change":"2023-11-24T08:05:19.021676+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021676+0000","last_undegraded":"2023-11-24T08:05:19.021676+0000","last_fullsized":"2023-11-24T08:05:19.021676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022765+0000","last_change":"2023-11-24T08:05:19.022765+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022765+0000","last_undegraded":"2023-11-24T08:05:19.022765+0000","last_fullsized":"2023-11-24T08:05:19.022765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001872+0000","last_change":"2023-11-24T08:05:19.001872+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001872+0000","last_undegraded":"2023-11-24T08:05:19.001872+0000","last_fullsized":"2023-11-24T08:05:19.001872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003795+0000","last_change":"2023-11-24T08:05:19.003795+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003795+0000","last_undegraded":"2023-11-24T08:05:19.003795+0000","last_fullsized":"2023-11-24T08:05:19.003795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007408+0000","last_change":"2023-11-24T08:05:19.007408+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007408+0000","last_undegraded":"2023-11-24T08:05:19.007408+0000","last_fullsized":"2023-11-24T08:05:19.007408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.010630+0000","last_change":"2023-11-24T08:05:19.010630+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.010630+0000","last_undegraded":"2023-11-24T08:05:19.010630+0000","last_fullsized":"2023-11-24T08:05:19.010630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.006901+0000","last_change":"2023-11-24T08:05:19.006901+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.006901+0000","last_undegraded":"2023-11-24T08:05:19.006901+0000","last_fullsized":"2023-11-24T08:05:19.006901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.011354+0000","last_change":"2023-11-24T08:05:19.011354+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.011354+0000","last_undegraded":"2023-11-24T08:05:19.011354+0000","last_fullsized":"2023-11-24T08:05:19.011354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012941+0000","last_change":"2023-11-24T08:05:19.012941+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012941+0000","last_undegraded":"2023-11-24T08:05:19.012941+0000","last_fullsized":"2023-11-24T08:05:19.012941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018757+0000","last_change":"2023-11-24T08:05:19.018757+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018757+0000","last_undegraded":"2023-11-24T08:05:19.018757+0000","last_fullsized":"2023-11-24T08:05:19.018757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001270+0000","last_change":"2023-11-24T08:05:19.001270+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001270+0000","last_undegraded":"2023-11-24T08:05:19.001270+0000","last_fullsized":"2023-11-24T08:05:19.001270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.021345+0000","last_change":"2023-11-24T08:05:19.021345+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.021345+0000","last_undegraded":"2023-11-24T08:05:19.021345+0000","last_fullsized":"2023-11-24T08:05:19.021345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.013040+0000","last_change":"2023-11-24T08:05:19.013040+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.013040+0000","last_undegraded":"2023-11-24T08:05:19.013040+0000","last_fullsized":"2023-11-24T08:05:19.013040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009317+0000","last_change":"2023-11-24T08:05:19.009317+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009317+0000","last_undegraded":"2023-11-24T08:05:19.009317+0000","last_fullsized":"2023-11-24T08:05:19.009317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.020254+0000","last_change":"2023-11-24T08:05:19.020254+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.020254+0000","last_undegraded":"2023-11-24T08:05:19.020254+0000","last_fullsized":"2023-11-24T08:05:19.020254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022175+0000","last_change":"2023-11-24T08:05:19.022175+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022175+0000","last_undegraded":"2023-11-24T08:05:19.022175+0000","last_fullsized":"2023-11-24T08:05:19.022175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025740+0000","last_change":"2023-11-24T08:05:19.025740+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025740+0000","last_undegraded":"2023-11-24T08:05:19.025740+0000","last_fullsized":"2023-11-24T08:05:19.025740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016746+0000","last_change":"2023-11-24T08:05:19.016746+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016746+0000","last_undegraded":"2023-11-24T08:05:19.016746+0000","last_fullsized":"2023-11-24T08:05:19.016746+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.038997+0000","last_change":"2023-11-24T08:05:19.034855+0000","last_active":"2023-11-24T08:05:19.038997+0000","last_peered":"2023-11-24T08:05:19.038997+0000","last_clean":"2023-11-24T08:05:19.038997+0000","last_became_active":"2023-11-24T08:05:19.034614+0000","last_became_peered":"2023-11-24T08:05:19.034614+0000","last_unstale":"2023-11-24T08:05:19.038997+0000","last_undegraded":"2023-11-24T08:05:19.038997+0000","last_fullsized":"2023-11-24T08:05:19.038997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017903+0000","last_change":"2023-11-24T08:05:19.017903+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017903+0000","last_undegraded":"2023-11-24T08:05:19.017903+0000","last_fullsized":"2023-11-24T08:05:19.017903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034166+0000","last_change":"2023-11-24T08:05:19.029832+0000","last_active":"2023-11-24T08:05:19.034166+0000","last_peered":"2023-11-24T08:05:19.034166+0000","last_clean":"2023-11-24T08:05:19.034166+0000","last_became_active":"2023-11-24T08:05:19.028467+0000","last_became_peered":"2023-11-24T08:05:19.028467+0000","last_unstale":"2023-11-24T08:05:19.034166+0000","last_undegraded":"2023-11-24T08:05:19.034166+0000","last_fullsized":"2023-11-24T08:05:19.034166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:12.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.004380+0000","last_change":"2023-11-24T08:05:19.004380+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.004380+0000","last_undegraded":"2023-11-24T08:05:19.004380+0000","last_fullsized":"2023-11-24T08:05:19.004380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031446+0000","last_change":"2023-11-24T08:05:19.024698+0000","last_active":"2023-11-24T08:05:19.031446+0000","last_peered":"2023-11-24T08:05:19.031446+0000","last_clean":"2023-11-24T08:05:19.031446+0000","last_became_active":"2023-11-24T08:05:19.024477+0000","last_became_peered":"2023-11-24T08:05:19.024477+0000","last_unstale":"2023-11-24T08:05:19.031446+0000","last_undegraded":"2023-11-24T08:05:19.031446+0000","last_fullsized":"2023-11-24T08:05:19.031446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:50.782618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.020852+0000","last_change":"2023-11-24T08:05:19.020852+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.020852+0000","last_undegraded":"2023-11-24T08:05:19.020852+0000","last_fullsized":"2023-11-24T08:05:19.020852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.028717+0000","last_change":"2023-11-24T08:05:19.024566+0000","last_active":"2023-11-24T08:05:19.028717+0000","last_peered":"2023-11-24T08:05:19.028717+0000","last_clean":"2023-11-24T08:05:19.028717+0000","last_became_active":"2023-11-24T08:05:19.024310+0000","last_became_peered":"2023-11-24T08:05:19.024310+0000","last_unstale":"2023-11-24T08:05:19.028717+0000","last_undegraded":"2023-11-24T08:05:19.028717+0000","last_fullsized":"2023-11-24T08:05:19.028717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:59.756867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019547+0000","last_change":"2023-11-24T08:05:19.019547+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019547+0000","last_undegraded":"2023-11-24T08:05:19.019547+0000","last_fullsized":"2023-11-24T08:05:19.019547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034315+0000","last_change":"2023-11-24T08:05:19.027924+0000","last_active":"2023-11-24T08:05:19.034315+0000","last_peered":"2023-11-24T08:05:19.034315+0000","last_clean":"2023-11-24T08:05:19.034315+0000","last_became_active":"2023-11-24T08:05:19.023079+0000","last_became_peered":"2023-11-24T08:05:19.023079+0000","last_unstale":"2023-11-24T08:05:19.034315+0000","last_undegraded":"2023-11-24T08:05:19.034315+0000","last_fullsized":"2023-11-24T08:05:19.034315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:45.153445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012736+0000","last_change":"2023-11-24T08:05:19.012736+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012736+0000","last_undegraded":"2023-11-24T08:05:19.012736+0000","last_fullsized":"2023-11-24T08:05:19.012736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034070+0000","last_change":"2023-11-24T08:05:19.028892+0000","last_active":"2023-11-24T08:05:19.034070+0000","last_peered":"2023-11-24T08:05:19.034070+0000","last_clean":"2023-11-24T08:05:19.034070+0000","last_became_active":"2023-11-24T08:05:19.027535+0000","last_became_peered":"2023-11-24T08:05:19.027535+0000","last_unstale":"2023-11-24T08:05:19.034070+0000","last_undegraded":"2023-11-24T08:05:19.034070+0000","last_fullsized":"2023-11-24T08:05:19.034070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:45:40.009128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003620+0000","last_change":"2023-11-24T08:05:19.003620+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003620+0000","last_undegraded":"2023-11-24T08:05:19.003620+0000","last_fullsized":"2023-11-24T08:05:19.003620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.019329+0000","last_change":"2023-11-24T08:05:19.015336+0000","last_active":"2023-11-24T08:05:19.019329+0000","last_peered":"2023-11-24T08:05:19.019329+0000","last_clean":"2023-11-24T08:05:19.019329+0000","last_became_active":"2023-11-24T08:05:19.014910+0000","last_became_peered":"2023-11-24T08:05:19.014910+0000","last_unstale":"2023-11-24T08:05:19.019329+0000","last_undegraded":"2023-11-24T08:05:19.019329+0000","last_fullsized":"2023-11-24T08:05:19.019329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:27.754037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017987+0000","last_change":"2023-11-24T08:05:19.017987+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017987+0000","last_undegraded":"2023-11-24T08:05:19.017987+0000","last_fullsized":"2023-11-24T08:05:19.017987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036507+0000","last_change":"2023-11-24T08:05:19.031449+0000","last_active":"2023-11-24T08:05:19.036507+0000","last_peered":"2023-11-24T08:05:19.036507+0000","last_clean":"2023-11-24T08:05:19.036507+0000","last_became_active":"2023-11-24T08:05:19.031114+0000","last_became_peered":"2023-11-24T08:05:19.031114+0000","last_unstale":"2023-11-24T08:05:19.036507+0000","last_undegraded":"2023-11-24T08:05:19.036507+0000","last_fullsized":"2023-11-24T08:05:19.036507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:56.728306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.012757+0000","last_change":"2023-11-24T08:05:19.012757+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.012757+0000","last_undegraded":"2023-11-24T08:05:19.012757+0000","last_fullsized":"2023-11-24T08:05:19.012757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032407+0000","last_change":"2023-11-24T08:05:19.028843+0000","last_active":"2023-11-24T08:05:19.032407+0000","last_peered":"2023-11-24T08:05:19.032407+0000","last_clean":"2023-11-24T08:05:19.032407+0000","last_became_active":"2023-11-24T08:05:19.027840+0000","last_became_peered":"2023-11-24T08:05:19.027840+0000","last_unstale":"2023-11-24T08:05:19.032407+0000","last_undegraded":"2023-11-24T08:05:19.032407+0000","last_fullsized":"2023-11-24T08:05:19.032407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:12.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.003124+0000","last_change":"2023-11-24T08:05:19.003124+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.003124+0000","last_undegraded":"2023-11-24T08:05:19.003124+0000","last_fullsized":"2023-11-24T08:05:19.003124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034147+0000","last_change":"2023-11-24T08:05:19.029786+0000","last_active":"2023-11-24T08:05:19.034147+0000","last_peered":"2023-11-24T08:05:19.034147+0000","last_clean":"2023-11-24T08:05:19.034147+0000","last_became_active":"2023-11-24T08:05:19.029469+0000","last_became_peered":"2023-11-24T08:05:19.029469+0000","last_unstale":"2023-11-24T08:05:19.034147+0000","last_undegraded":"2023-11-24T08:05:19.034147+0000","last_fullsized":"2023-11-24T08:05:19.034147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.002655+0000","last_change":"2023-11-24T08:05:19.002655+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.002655+0000","last_undegraded":"2023-11-24T08:05:19.002655+0000","last_fullsized":"2023-11-24T08:05:19.002655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036658+0000","last_change":"2023-11-24T08:05:19.034418+0000","last_active":"2023-11-24T08:05:19.036658+0000","last_peered":"2023-11-24T08:05:19.036658+0000","last_clean":"2023-11-24T08:05:19.036658+0000","last_became_active":"2023-11-24T08:05:19.033911+0000","last_became_peered":"2023-11-24T08:05:19.033911+0000","last_unstale":"2023-11-24T08:05:19.036658+0000","last_undegraded":"2023-11-24T08:05:19.036658+0000","last_fullsized":"2023-11-24T08:05:19.036658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:50.782618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018899+0000","last_change":"2023-11-24T08:05:19.018899+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018899+0000","last_undegraded":"2023-11-24T08:05:19.018899+0000","last_fullsized":"2023-11-24T08:05:19.018899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031376+0000","last_change":"2023-11-24T08:05:19.023626+0000","last_active":"2023-11-24T08:05:19.031376+0000","last_peered":"2023-11-24T08:05:19.031376+0000","last_clean":"2023-11-24T08:05:19.031376+0000","last_became_active":"2023-11-24T08:05:19.023367+0000","last_became_peered":"2023-11-24T08:05:19.023367+0000","last_unstale":"2023-11-24T08:05:19.031376+0000","last_undegraded":"2023-11-24T08:05:19.031376+0000","last_fullsized":"2023-11-24T08:05:19.031376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:50.145064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017655+0000","last_change":"2023-11-24T08:05:19.017655+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017655+0000","last_undegraded":"2023-11-24T08:05:19.017655+0000","last_fullsized":"2023-11-24T08:05:19.017655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034295+0000","last_change":"2023-11-24T08:05:19.026245+0000","last_active":"2023-11-24T08:05:19.034295+0000","last_peered":"2023-11-24T08:05:19.034295+0000","last_clean":"2023-11-24T08:05:19.034295+0000","last_became_active":"2023-11-24T08:05:19.025943+0000","last_became_peered":"2023-11-24T08:05:19.025943+0000","last_unstale":"2023-11-24T08:05:19.034295+0000","last_undegraded":"2023-11-24T08:05:19.034295+0000","last_fullsized":"2023-11-24T08:05:19.034295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:44.721323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001371+0000","last_change":"2023-11-24T08:05:19.001371+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001371+0000","last_undegraded":"2023-11-24T08:05:19.001371+0000","last_fullsized":"2023-11-24T08:05:19.001371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.033021+0000","last_change":"2023-11-24T08:05:19.027534+0000","last_active":"2023-11-24T08:05:19.033021+0000","last_peered":"2023-11-24T08:05:19.033021+0000","last_clean":"2023-11-24T08:05:19.033021+0000","last_became_active":"2023-11-24T08:05:19.024963+0000","last_became_peered":"2023-11-24T08:05:19.024963+0000","last_unstale":"2023-11-24T08:05:19.033021+0000","last_undegraded":"2023-11-24T08:05:19.033021+0000","last_fullsized":"2023-11-24T08:05:19.033021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:18.282886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025333+0000","last_change":"2023-11-24T08:05:19.025333+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025333+0000","last_undegraded":"2023-11-24T08:05:19.025333+0000","last_fullsized":"2023-11-24T08:05:19.025333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032465+0000","last_change":"2023-11-24T08:05:19.028331+0000","last_active":"2023-11-24T08:05:19.032465+0000","last_peered":"2023-11-24T08:05:19.032465+0000","last_clean":"2023-11-24T08:05:19.032465+0000","last_became_active":"2023-11-24T08:05:19.028037+0000","last_became_peered":"2023-11-24T08:05:19.028037+0000","last_unstale":"2023-11-24T08:05:19.032465+0000","last_undegraded":"2023-11-24T08:05:19.032465+0000","last_fullsized":"2023-11-24T08:05:19.032465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:00.858708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017553+0000","last_change":"2023-11-24T08:05:19.017553+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017553+0000","last_undegraded":"2023-11-24T08:05:19.017553+0000","last_fullsized":"2023-11-24T08:05:19.017553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031349+0000","last_change":"2023-11-24T08:05:19.023564+0000","last_active":"2023-11-24T08:05:19.031349+0000","last_peered":"2023-11-24T08:05:19.031349+0000","last_clean":"2023-11-24T08:05:19.031349+0000","last_became_active":"2023-11-24T08:05:19.022784+0000","last_became_peered":"2023-11-24T08:05:19.022784+0000","last_unstale":"2023-11-24T08:05:19.031349+0000","last_undegraded":"2023-11-24T08:05:19.031349+0000","last_fullsized":"2023-11-24T08:05:19.031349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.008194+0000","last_change":"2023-11-24T08:05:19.008194+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.008194+0000","last_undegraded":"2023-11-24T08:05:19.008194+0000","last_fullsized":"2023-11-24T08:05:19.008194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034866+0000","last_change":"2023-11-24T08:05:19.030608+0000","last_active":"2023-11-24T08:05:19.034866+0000","last_peered":"2023-11-24T08:05:19.034866+0000","last_clean":"2023-11-24T08:05:19.034866+0000","last_became_active":"2023-11-24T08:05:19.030359+0000","last_became_peered":"2023-11-24T08:05:19.030359+0000","last_unstale":"2023-11-24T08:05:19.034866+0000","last_undegraded":"2023-11-24T08:05:19.034866+0000","last_fullsized":"2023-11-24T08:05:19.034866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.026741+0000","last_change":"2023-11-24T08:05:19.026741+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.026741+0000","last_undegraded":"2023-11-24T08:05:19.026741+0000","last_fullsized":"2023-11-24T08:05:19.026741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034369+0000","last_change":"2023-11-24T08:05:19.027262+0000","last_active":"2023-11-24T08:05:19.034369+0000","last_peered":"2023-11-24T08:05:19.034369+0000","last_clean":"2023-11-24T08:05:19.034369+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:19.034369+0000","last_undegraded":"2023-11-24T08:05:19.034369+0000","last_fullsized":"2023-11-24T08:05:19.034369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:39.801164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.016569+0000","last_change":"2023-11-24T08:05:19.016569+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.016569+0000","last_undegraded":"2023-11-24T08:05:19.016569+0000","last_fullsized":"2023-11-24T08:05:19.016569+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034100+0000","last_change":"2023-11-24T08:05:19.025861+0000","last_active":"2023-11-24T08:05:19.034100+0000","last_peered":"2023-11-24T08:05:19.034100+0000","last_clean":"2023-11-24T08:05:19.034100+0000","last_became_active":"2023-11-24T08:05:19.024918+0000","last_became_peered":"2023-11-24T08:05:19.024918+0000","last_unstale":"2023-11-24T08:05:19.034100+0000","last_undegraded":"2023-11-24T08:05:19.034100+0000","last_fullsized":"2023-11-24T08:05:19.034100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:59.275982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.008986+0000","last_change":"2023-11-24T08:05:19.008986+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.008986+0000","last_undegraded":"2023-11-24T08:05:19.008986+0000","last_fullsized":"2023-11-24T08:05:19.008986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.030949+0000","last_change":"2023-11-24T08:05:19.017985+0000","last_active":"2023-11-24T08:05:19.030949+0000","last_peered":"2023-11-24T08:05:19.030949+0000","last_clean":"2023-11-24T08:05:19.030949+0000","last_became_active":"2023-11-24T08:05:19.017444+0000","last_became_peered":"2023-11-24T08:05:19.017444+0000","last_unstale":"2023-11-24T08:05:19.030949+0000","last_undegraded":"2023-11-24T08:05:19.030949+0000","last_fullsized":"2023-11-24T08:05:19.030949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:45:30.962562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007396+0000","last_change":"2023-11-24T08:05:19.007396+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007396+0000","last_undegraded":"2023-11-24T08:05:19.007396+0000","last_fullsized":"2023-11-24T08:05:19.007396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031706+0000","last_change":"2023-11-24T08:05:19.026334+0000","last_active":"2023-11-24T08:05:19.031706+0000","last_peered":"2023-11-24T08:05:19.031706+0000","last_clean":"2023-11-24T08:05:19.031706+0000","last_became_active":"2023-11-24T08:05:19.025952+0000","last_became_peered":"2023-11-24T08:05:19.025952+0000","last_unstale":"2023-11-24T08:05:19.031706+0000","last_undegraded":"2023-11-24T08:05:19.031706+0000","last_fullsized":"2023-11-24T08:05:19.031706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009085+0000","last_change":"2023-11-24T08:05:19.009085+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009085+0000","last_undegraded":"2023-11-24T08:05:19.009085+0000","last_fullsized":"2023-11-24T08:05:19.009085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036500+0000","last_change":"2023-11-24T08:05:19.034595+0000","last_active":"2023-11-24T08:05:19.036500+0000","last_peered":"2023-11-24T08:05:19.036500+0000","last_clean":"2023-11-24T08:05:19.036500+0000","last_became_active":"2023-11-24T08:05:19.033790+0000","last_became_peered":"2023-11-24T08:05:19.033790+0000","last_unstale":"2023-11-24T08:05:19.036500+0000","last_undegraded":"2023-11-24T08:05:19.036500+0000","last_fullsized":"2023-11-24T08:05:19.036500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007348+0000","last_change":"2023-11-24T08:05:19.007348+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007348+0000","last_undegraded":"2023-11-24T08:05:19.007348+0000","last_fullsized":"2023-11-24T08:05:19.007348+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034034+0000","last_change":"2023-11-24T08:05:19.028570+0000","last_active":"2023-11-24T08:05:19.034034+0000","last_peered":"2023-11-24T08:05:19.034034+0000","last_clean":"2023-11-24T08:05:19.034034+0000","last_became_active":"2023-11-24T08:05:19.028257+0000","last_became_peered":"2023-11-24T08:05:19.028257+0000","last_unstale":"2023-11-24T08:05:19.034034+0000","last_undegraded":"2023-11-24T08:05:19.034034+0000","last_fullsized":"2023-11-24T08:05:19.034034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:59.756867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.022389+0000","last_change":"2023-11-24T08:05:19.022389+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.022389+0000","last_undegraded":"2023-11-24T08:05:19.022389+0000","last_fullsized":"2023-11-24T08:05:19.022389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036304+0000","last_change":"2023-11-24T08:05:19.032519+0000","last_active":"2023-11-24T08:05:19.036304+0000","last_peered":"2023-11-24T08:05:19.036304+0000","last_clean":"2023-11-24T08:05:19.036304+0000","last_became_active":"2023-11-24T08:05:19.032159+0000","last_became_peered":"2023-11-24T08:05:19.032159+0000","last_unstale":"2023-11-24T08:05:19.036304+0000","last_undegraded":"2023-11-24T08:05:19.036304+0000","last_fullsized":"2023-11-24T08:05:19.036304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:27.754037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.007343+0000","last_change":"2023-11-24T08:05:19.007343+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.007343+0000","last_undegraded":"2023-11-24T08:05:19.007343+0000","last_fullsized":"2023-11-24T08:05:19.007343+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.039674+0000","last_change":"2023-11-24T08:05:19.034899+0000","last_active":"2023-11-24T08:05:19.039674+0000","last_peered":"2023-11-24T08:05:19.039674+0000","last_clean":"2023-11-24T08:05:19.039674+0000","last_became_active":"2023-11-24T08:05:19.034569+0000","last_became_peered":"2023-11-24T08:05:19.034569+0000","last_unstale":"2023-11-24T08:05:19.039674+0000","last_undegraded":"2023-11-24T08:05:19.039674+0000","last_fullsized":"2023-11-24T08:05:19.039674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.018629+0000","last_change":"2023-11-24T08:05:19.018629+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.018629+0000","last_undegraded":"2023-11-24T08:05:19.018629+0000","last_fullsized":"2023-11-24T08:05:19.018629+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.031764+0000","last_change":"2023-11-24T08:05:19.027343+0000","last_active":"2023-11-24T08:05:19.031764+0000","last_peered":"2023-11-24T08:05:19.031764+0000","last_clean":"2023-11-24T08:05:19.031764+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:19.031764+0000","last_undegraded":"2023-11-24T08:05:19.031764+0000","last_fullsized":"2023-11-24T08:05:19.031764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:50.145064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.011239+0000","last_change":"2023-11-24T08:05:19.011239+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.011239+0000","last_undegraded":"2023-11-24T08:05:19.011239+0000","last_fullsized":"2023-11-24T08:05:19.011239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.032952+0000","last_change":"2023-11-24T08:05:19.025711+0000","last_active":"2023-11-24T08:05:19.032952+0000","last_peered":"2023-11-24T08:05:19.032952+0000","last_clean":"2023-11-24T08:05:19.032952+0000","last_became_active":"2023-11-24T08:05:19.025421+0000","last_became_peered":"2023-11-24T08:05:19.025421+0000","last_unstale":"2023-11-24T08:05:19.032952+0000","last_undegraded":"2023-11-24T08:05:19.032952+0000","last_fullsized":"2023-11-24T08:05:19.032952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.019282+0000","last_change":"2023-11-24T08:05:19.019282+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.019282+0000","last_undegraded":"2023-11-24T08:05:19.019282+0000","last_fullsized":"2023-11-24T08:05:19.019282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034137+0000","last_change":"2023-11-24T08:05:19.029520+0000","last_active":"2023-11-24T08:05:19.034137+0000","last_peered":"2023-11-24T08:05:19.034137+0000","last_clean":"2023-11-24T08:05:19.034137+0000","last_became_active":"2023-11-24T08:05:19.029227+0000","last_became_peered":"2023-11-24T08:05:19.029227+0000","last_unstale":"2023-11-24T08:05:19.034137+0000","last_undegraded":"2023-11-24T08:05:19.034137+0000","last_fullsized":"2023-11-24T08:05:19.034137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.017498+0000","last_change":"2023-11-24T08:05:19.017498+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.017498+0000","last_undegraded":"2023-11-24T08:05:19.017498+0000","last_fullsized":"2023-11-24T08:05:19.017498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.034959+0000","last_change":"2023-11-24T08:05:19.031328+0000","last_active":"2023-11-24T08:05:19.034959+0000","last_peered":"2023-11-24T08:05:19.034959+0000","last_clean":"2023-11-24T08:05:19.034959+0000","last_became_active":"2023-11-24T08:05:19.031046+0000","last_became_peered":"2023-11-24T08:05:19.031046+0000","last_unstale":"2023-11-24T08:05:19.034959+0000","last_undegraded":"2023-11-24T08:05:19.034959+0000","last_fullsized":"2023-11-24T08:05:19.034959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:04.782054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.001684+0000","last_change":"2023-11-24T08:05:19.001684+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.001684+0000","last_undegraded":"2023-11-24T08:05:19.001684+0000","last_fullsized":"2023-11-24T08:05:19.001684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036519+0000","last_change":"2023-11-24T08:05:19.030600+0000","last_active":"2023-11-24T08:05:19.036519+0000","last_peered":"2023-11-24T08:05:19.036519+0000","last_clean":"2023-11-24T08:05:19.036519+0000","last_became_active":"2023-11-24T08:05:19.028703+0000","last_became_peered":"2023-11-24T08:05:19.028703+0000","last_unstale":"2023-11-24T08:05:19.036519+0000","last_undegraded":"2023-11-24T08:05:19.036519+0000","last_fullsized":"2023-11-24T08:05:19.036519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:40.635942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.009362+0000","last_change":"2023-11-24T08:05:19.009362+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.009362+0000","last_undegraded":"2023-11-24T08:05:19.009362+0000","last_fullsized":"2023-11-24T08:05:19.009362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.036848+0000","last_change":"2023-11-24T08:05:19.033681+0000","last_active":"2023-11-24T08:05:19.036848+0000","last_peered":"2023-11-24T08:05:19.036848+0000","last_clean":"2023-11-24T08:05:19.036848+0000","last_became_active":"2023-11-24T08:05:19.033420+0000","last_became_peered":"2023-11-24T08:05:19.033420+0000","last_unstale":"2023-11-24T08:05:19.036848+0000","last_undegraded":"2023-11-24T08:05:19.036848+0000","last_fullsized":"2023-11-24T08:05:19.036848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-11-24T08:05:19.025637+0000","last_change":"2023-11-24T08:05:19.025637+0000","last_active":"2023-11-24T08:05:18.980078+0000","last_peered":"2023-11-24T08:05:18.980078+0000","last_clean":"2023-11-24T08:05:18.980078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:05:19.025637+0000","last_undegraded":"2023-11-24T08:05:19.025637+0000","last_fullsized":"2023-11-24T08:05:19.025637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T08:05:19.041907+0000","last_change":"2023-11-24T08:05:19.036326+0000","last_active":"2023-11-24T08:05:19.041907+0000","last_peered":"2023-11-24T08:05:19.041907+0000","last_clean":"2023-11-24T08:05:19.041907+0000","last_became_active":"2023-11-24T08:05:19.035962+0000","last_became_peered":"2023-11-24T08:05:19.035962+0000","last_unstale":"2023-11-24T08:05:19.041907+0000","last_undegraded":"2023-11-24T08:05:19.041907+0000","last_fullsized":"2023-11-24T08:05:19.041907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:26.199922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":174,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-24T08:05:17.997541+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:05:17.997541+0000","last_peered":"2023-11-24T08:05:17.997541+0000","last_clean":"2023-11-24T08:05:17.997541+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:05:17.997541+0000","last_undegraded":"2023-11-24T08:05:17.997541+0000","last_fullsized":"2023-11-24T08:05:17.997541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.876}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":1.3240000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":1.1200000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":4.0179999999999998,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.369}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.3759999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.284}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.554},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.484}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.823,"15min":1.823},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":6.8129999999999997,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.147},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.7509999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.879,"15min":1.879},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":9.8569999999999993,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":9.9450000000000003,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.667}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":1.601},{"interface":"front","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.5209999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.751},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.22}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.151},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.0569999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.419}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":1.325}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.379}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":1.496},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.3600000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":1.5580000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.1659999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.089}]},{"osd":2,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0409999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.21},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3779999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.262},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.278}]},{"osd":6,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.3440000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.5780000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.441}]},{"osd":2,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.3500000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.125},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.1579999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.528}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.034}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.9969999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.161},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.88800000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.84199999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.3149999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":3.161,"5min":3.161,"15min":3.161},"last":1.401}]},{"osd":5,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.365}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.218},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.5860000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.323},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.524}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":0.94199999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.92100000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.3340000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3049999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.49},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.5329999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.677},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.4179999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.609,"5min":1.609,"15min":1.609},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.7330000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.6120000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":722084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.321}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.2549999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.6320000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":1.659}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.704},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":2.2389999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":2.0590000000000002},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.845}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":2.1819999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.9630000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":2.1600000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":3.371,"5min":3.371,"15min":3.371},"last":2.2610000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:05:25.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-11-24T08:05:25.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.019366+0000', 'last_change': '2023-11-24T08:05:19.019366+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.019366+0000', 'last_undegraded': '2023-11-24T08:05:19.019366+0000', 'last_fullsized': '2023-11-24T08:05:19.019366+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:05:25.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-11-24T08:05:25.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.002461+0000', 'last_change': '2023-11-24T08:05:19.002461+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.002461+0000', 'last_undegraded': '2023-11-24T08:05:19.002461+0000', 'last_fullsized': '2023-11-24T08:05:19.002461+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-11-24T08:05:25.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.016949+0000', 'last_change': '2023-11-24T08:05:19.016949+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.016949+0000', 'last_undegraded': '2023-11-24T08:05:19.016949+0000', 'last_fullsized': '2023-11-24T08:05:19.016949+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-11-24T08:05:25.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.017053+0000', 'last_change': '2023-11-24T08:05:19.017053+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.017053+0000', 'last_undegraded': '2023-11-24T08:05:19.017053+0000', 'last_fullsized': '2023-11-24T08:05:19.017053+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.010185+0000', 'last_change': '2023-11-24T08:05:19.010185+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.010185+0000', 'last_undegraded': '2023-11-24T08:05:19.010185+0000', 'last_fullsized': '2023-11-24T08:05:19.010185+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.002180+0000', 'last_change': '2023-11-24T08:05:19.002180+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.002180+0000', 'last_undegraded': '2023-11-24T08:05:19.002180+0000', 'last_fullsized': '2023-11-24T08:05:19.002180+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.021735+0000', 'last_change': '2023-11-24T08:05:19.021735+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.021735+0000', 'last_undegraded': '2023-11-24T08:05:19.021735+0000', 'last_fullsized': '2023-11-24T08:05:19.021735+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.026170+0000', 'last_change': '2023-11-24T08:05:19.026170+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.026170+0000', 'last_undegraded': '2023-11-24T08:05:19.026170+0000', 'last_fullsized': '2023-11-24T08:05:19.026170+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-11-24T08:05:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.007708+0000', 'last_change': '2023-11-24T08:05:19.007708+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.007708+0000', 'last_undegraded': '2023-11-24T08:05:19.007708+0000', 'last_fullsized': '2023-11-24T08:05:19.007708+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.019694+0000', 'last_change': '2023-11-24T08:05:19.019694+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.019694+0000', 'last_undegraded': '2023-11-24T08:05:19.019694+0000', 'last_fullsized': '2023-11-24T08:05:19.019694+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.022034+0000', 'last_change': '2023-11-24T08:05:19.022034+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.022034+0000', 'last_undegraded': '2023-11-24T08:05:19.022034+0000', 'last_fullsized': '2023-11-24T08:05:19.022034+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.014621+0000', 'last_change': '2023-11-24T08:05:19.014621+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.014621+0000', 'last_undegraded': '2023-11-24T08:05:19.014621+0000', 'last_fullsized': '2023-11-24T08:05:19.014621+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.009712+0000', 'last_change': '2023-11-24T08:05:19.009712+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.009712+0000', 'last_undegraded': '2023-11-24T08:05:19.009712+0000', 'last_fullsized': '2023-11-24T08:05:19.009712+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-11-24T08:05:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.001299+0000', 'last_change': '2023-11-24T08:05:19.001299+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.001299+0000', 'last_undegraded': '2023-11-24T08:05:19.001299+0000', 'last_fullsized': '2023-11-24T08:05:19.001299+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.014116+0000', 'last_change': '2023-11-24T08:05:19.014116+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.014116+0000', 'last_undegraded': '2023-11-24T08:05:19.014116+0000', 'last_fullsized': '2023-11-24T08:05:19.014116+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.021676+0000', 'last_change': '2023-11-24T08:05:19.021676+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.021676+0000', 'last_undegraded': '2023-11-24T08:05:19.021676+0000', 'last_fullsized': '2023-11-24T08:05:19.021676+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.022765+0000', 'last_change': '2023-11-24T08:05:19.022765+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.022765+0000', 'last_undegraded': '2023-11-24T08:05:19.022765+0000', 'last_fullsized': '2023-11-24T08:05:19.022765+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.001872+0000', 'last_change': '2023-11-24T08:05:19.001872+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.001872+0000', 'last_undegraded': '2023-11-24T08:05:19.001872+0000', 'last_fullsized': '2023-11-24T08:05:19.001872+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-11-24T08:05:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.003795+0000', 'last_change': '2023-11-24T08:05:19.003795+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.003795+0000', 'last_undegraded': '2023-11-24T08:05:19.003795+0000', 'last_fullsized': '2023-11-24T08:05:19.003795+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.007408+0000', 'last_change': '2023-11-24T08:05:19.007408+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.007408+0000', 'last_undegraded': '2023-11-24T08:05:19.007408+0000', 'last_fullsized': '2023-11-24T08:05:19.007408+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.010630+0000', 'last_change': '2023-11-24T08:05:19.010630+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.010630+0000', 'last_undegraded': '2023-11-24T08:05:19.010630+0000', 'last_fullsized': '2023-11-24T08:05:19.010630+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.006901+0000', 'last_change': '2023-11-24T08:05:19.006901+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.006901+0000', 'last_undegraded': '2023-11-24T08:05:19.006901+0000', 'last_fullsized': '2023-11-24T08:05:19.006901+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.011354+0000', 'last_change': '2023-11-24T08:05:19.011354+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.011354+0000', 'last_undegraded': '2023-11-24T08:05:19.011354+0000', 'last_fullsized': '2023-11-24T08:05:19.011354+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:05:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.012941+0000', 'last_change': '2023-11-24T08:05:19.012941+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.012941+0000', 'last_undegraded': '2023-11-24T08:05:19.012941+0000', 'last_fullsized': '2023-11-24T08:05:19.012941+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.018757+0000', 'last_change': '2023-11-24T08:05:19.018757+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.018757+0000', 'last_undegraded': '2023-11-24T08:05:19.018757+0000', 'last_fullsized': '2023-11-24T08:05:19.018757+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.001270+0000', 'last_change': '2023-11-24T08:05:19.001270+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.001270+0000', 'last_undegraded': '2023-11-24T08:05:19.001270+0000', 'last_fullsized': '2023-11-24T08:05:19.001270+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.021345+0000', 'last_change': '2023-11-24T08:05:19.021345+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.021345+0000', 'last_undegraded': '2023-11-24T08:05:19.021345+0000', 'last_fullsized': '2023-11-24T08:05:19.021345+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.013040+0000', 'last_change': '2023-11-24T08:05:19.013040+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.013040+0000', 'last_undegraded': '2023-11-24T08:05:19.013040+0000', 'last_fullsized': '2023-11-24T08:05:19.013040+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.009317+0000', 'last_change': '2023-11-24T08:05:19.009317+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.009317+0000', 'last_undegraded': '2023-11-24T08:05:19.009317+0000', 'last_fullsized': '2023-11-24T08:05:19.009317+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.020254+0000', 'last_change': '2023-11-24T08:05:19.020254+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.020254+0000', 'last_undegraded': '2023-11-24T08:05:19.020254+0000', 'last_fullsized': '2023-11-24T08:05:19.020254+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.022175+0000', 'last_change': '2023-11-24T08:05:19.022175+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.022175+0000', 'last_undegraded': '2023-11-24T08:05:19.022175+0000', 'last_fullsized': '2023-11-24T08:05:19.022175+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.025740+0000', 'last_change': '2023-11-24T08:05:19.025740+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.025740+0000', 'last_undegraded': '2023-11-24T08:05:19.025740+0000', 'last_fullsized': '2023-11-24T08:05:19.025740+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.016746+0000', 'last_change': '2023-11-24T08:05:19.016746+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.016746+0000', 'last_undegraded': '2023-11-24T08:05:19.016746+0000', 'last_fullsized': '2023-11-24T08:05:19.016746+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:05:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.017903+0000', 'last_change': '2023-11-24T08:05:19.017903+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.017903+0000', 'last_undegraded': '2023-11-24T08:05:19.017903+0000', 'last_fullsized': '2023-11-24T08:05:19.017903+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.004380+0000', 'last_change': '2023-11-24T08:05:19.004380+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.004380+0000', 'last_undegraded': '2023-11-24T08:05:19.004380+0000', 'last_fullsized': '2023-11-24T08:05:19.004380+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.020852+0000', 'last_change': '2023-11-24T08:05:19.020852+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.020852+0000', 'last_undegraded': '2023-11-24T08:05:19.020852+0000', 'last_fullsized': '2023-11-24T08:05:19.020852+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.019547+0000', 'last_change': '2023-11-24T08:05:19.019547+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.019547+0000', 'last_undegraded': '2023-11-24T08:05:19.019547+0000', 'last_fullsized': '2023-11-24T08:05:19.019547+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.012736+0000', 'last_change': '2023-11-24T08:05:19.012736+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.012736+0000', 'last_undegraded': '2023-11-24T08:05:19.012736+0000', 'last_fullsized': '2023-11-24T08:05:19.012736+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.003620+0000', 'last_change': '2023-11-24T08:05:19.003620+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.003620+0000', 'last_undegraded': '2023-11-24T08:05:19.003620+0000', 'last_fullsized': '2023-11-24T08:05:19.003620+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.017987+0000', 'last_change': '2023-11-24T08:05:19.017987+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.017987+0000', 'last_undegraded': '2023-11-24T08:05:19.017987+0000', 'last_fullsized': '2023-11-24T08:05:19.017987+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.012757+0000', 'last_change': '2023-11-24T08:05:19.012757+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.012757+0000', 'last_undegraded': '2023-11-24T08:05:19.012757+0000', 'last_fullsized': '2023-11-24T08:05:19.012757+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.003124+0000', 'last_change': '2023-11-24T08:05:19.003124+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.003124+0000', 'last_undegraded': '2023-11-24T08:05:19.003124+0000', 'last_fullsized': '2023-11-24T08:05:19.003124+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.002655+0000', 'last_change': '2023-11-24T08:05:19.002655+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.002655+0000', 'last_undegraded': '2023-11-24T08:05:19.002655+0000', 'last_fullsized': '2023-11-24T08:05:19.002655+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.018899+0000', 'last_change': '2023-11-24T08:05:19.018899+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.018899+0000', 'last_undegraded': '2023-11-24T08:05:19.018899+0000', 'last_fullsized': '2023-11-24T08:05:19.018899+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.017655+0000', 'last_change': '2023-11-24T08:05:19.017655+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.017655+0000', 'last_undegraded': '2023-11-24T08:05:19.017655+0000', 'last_fullsized': '2023-11-24T08:05:19.017655+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.001371+0000', 'last_change': '2023-11-24T08:05:19.001371+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.001371+0000', 'last_undegraded': '2023-11-24T08:05:19.001371+0000', 'last_fullsized': '2023-11-24T08:05:19.001371+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.025333+0000', 'last_change': '2023-11-24T08:05:19.025333+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.025333+0000', 'last_undegraded': '2023-11-24T08:05:19.025333+0000', 'last_fullsized': '2023-11-24T08:05:19.025333+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.017553+0000', 'last_change': '2023-11-24T08:05:19.017553+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.017553+0000', 'last_undegraded': '2023-11-24T08:05:19.017553+0000', 'last_fullsized': '2023-11-24T08:05:19.017553+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:05:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-11-24T08:05:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.008194+0000', 'last_change': '2023-11-24T08:05:19.008194+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.008194+0000', 'last_undegraded': '2023-11-24T08:05:19.008194+0000', 'last_fullsized': '2023-11-24T08:05:19.008194+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:05:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-11-24T08:05:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.026741+0000', 'last_change': '2023-11-24T08:05:19.026741+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.026741+0000', 'last_undegraded': '2023-11-24T08:05:19.026741+0000', 'last_fullsized': '2023-11-24T08:05:19.026741+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-11-24T08:05:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.016569+0000', 'last_change': '2023-11-24T08:05:19.016569+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.016569+0000', 'last_undegraded': '2023-11-24T08:05:19.016569+0000', 'last_fullsized': '2023-11-24T08:05:19.016569+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:05:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-11-24T08:05:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.008986+0000', 'last_change': '2023-11-24T08:05:19.008986+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.008986+0000', 'last_undegraded': '2023-11-24T08:05:19.008986+0000', 'last_fullsized': '2023-11-24T08:05:19.008986+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.007396+0000', 'last_change': '2023-11-24T08:05:19.007396+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.007396+0000', 'last_undegraded': '2023-11-24T08:05:19.007396+0000', 'last_fullsized': '2023-11-24T08:05:19.007396+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.009085+0000', 'last_change': '2023-11-24T08:05:19.009085+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.009085+0000', 'last_undegraded': '2023-11-24T08:05:19.009085+0000', 'last_fullsized': '2023-11-24T08:05:19.009085+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.007348+0000', 'last_change': '2023-11-24T08:05:19.007348+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.007348+0000', 'last_undegraded': '2023-11-24T08:05:19.007348+0000', 'last_fullsized': '2023-11-24T08:05:19.007348+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.022389+0000', 'last_change': '2023-11-24T08:05:19.022389+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.022389+0000', 'last_undegraded': '2023-11-24T08:05:19.022389+0000', 'last_fullsized': '2023-11-24T08:05:19.022389+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-11-24T08:05:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.007343+0000', 'last_change': '2023-11-24T08:05:19.007343+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.007343+0000', 'last_undegraded': '2023-11-24T08:05:19.007343+0000', 'last_fullsized': '2023-11-24T08:05:19.007343+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.018629+0000', 'last_change': '2023-11-24T08:05:19.018629+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.018629+0000', 'last_undegraded': '2023-11-24T08:05:19.018629+0000', 'last_fullsized': '2023-11-24T08:05:19.018629+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.011239+0000', 'last_change': '2023-11-24T08:05:19.011239+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.011239+0000', 'last_undegraded': '2023-11-24T08:05:19.011239+0000', 'last_fullsized': '2023-11-24T08:05:19.011239+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.019282+0000', 'last_change': '2023-11-24T08:05:19.019282+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.019282+0000', 'last_undegraded': '2023-11-24T08:05:19.019282+0000', 'last_fullsized': '2023-11-24T08:05:19.019282+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.017498+0000', 'last_change': '2023-11-24T08:05:19.017498+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.017498+0000', 'last_undegraded': '2023-11-24T08:05:19.017498+0000', 'last_fullsized': '2023-11-24T08:05:19.017498+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-11-24T08:05:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.001684+0000', 'last_change': '2023-11-24T08:05:19.001684+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.001684+0000', 'last_undegraded': '2023-11-24T08:05:19.001684+0000', 'last_fullsized': '2023-11-24T08:05:19.001684+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:05:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-11-24T08:05:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.009362+0000', 'last_change': '2023-11-24T08:05:19.009362+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.009362+0000', 'last_undegraded': '2023-11-24T08:05:19.009362+0000', 'last_fullsized': '2023-11-24T08:05:19.009362+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:05:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-11-24T08:05:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:05:19.025637+0000', 'last_change': '2023-11-24T08:05:19.025637+0000', 'last_active': '2023-11-24T08:05:18.980078+0000', 'last_peered': '2023-11-24T08:05:18.980078+0000', 'last_clean': '2023-11-24T08:05:18.980078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:05:19.025637+0000', 'last_undegraded': '2023-11-24T08:05:19.025637+0000', 'last_fullsized': '2023-11-24T08:05:19.025637+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'last_clean_scrub_stamp': '2023-11-24T08:05:18.980078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:05:25.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T08:05:25.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.898+0000 7faca396f640 1 Processor -- start 2023-11-24T08:05:25.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.898+0000 7faca396f640 1 -- start start 2023-11-24T08:05:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.899+0000 7faca396f640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fac9c06dc30 0x7fac9c0acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.899+0000 7faca396f640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c071cd0 con 0x7fac9c160970 2023-11-24T08:05:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.899+0000 7faca396f640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c0b2120 con 0x7fac9c070780 2023-11-24T08:05:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.899+0000 7faca396f640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9c0b2400 con 0x7fac9c06dc30 2023-11-24T08:05:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.900+0000 7faca170b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fac9c160970 0x7fac9c0b2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32932/0 (socket says 172.21.15.64:32932) 2023-11-24T08:05:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.900+0000 7faca170b640 1 -- 172.21.15.64:0/273050675 learned_addr learned my addr 172.21.15.64:0/273050675 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.900+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564696418 0 0) 0x7fac9c071cd0 con 0x7fac9c160970 2023-11-24T08:05:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.900+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac88003520 con 0x7fac9c160970 2023-11-24T08:05:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.900+0000 7faca1f0c640 1 --2- 172.21.15.64:0/273050675 >> v2:172.21.15.91:3300/0 conn(0x7fac9c06dc30 0x7fac9c0acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3436968840 0 0) 0x7fac88003520 con 0x7fac9c160970 2023-11-24T08:05:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 >> v2:172.21.15.91:3300/0 conn(0x7fac9c06dc30 msgr2=0x7fac9c0acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 --2- 172.21.15.64:0/273050675 >> v2:172.21.15.91:3300/0 conn(0x7fac9c06dc30 0x7fac9c0acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 >> v1:172.21.15.91:6789/0 conn(0x7fac9c070780 legacy=0x7fac9c06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c0b26e0 con 0x7fac9c160970 2023-11-24T08:05:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fac98001670 con 0x7fac9c160970 2023-11-24T08:05:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 >> v1:172.21.15.64:6789/0 conn(0x7fac9c160970 legacy=0x7fac9c0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac88003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac880047c0 0x7fac88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac88005250 0x7fac8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0b2400 con 0x7fac880039c0 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8800c1a0 con 0x7fac88005250 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8800c360 con 0x7fac880047c0 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7faca170b640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac88003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7faca1f0c640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac88005250 0x7fac8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7faca170b640 1 -- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac880047c0 msgr2=0x7fac88004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7faca0f0a640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac880047c0 0x7fac88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7faca170b640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac880047c0 0x7fac88004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.901+0000 7faca170b640 1 -- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac88005250 msgr2=0x7fac8800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.902+0000 7faca170b640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac88005250 0x7fac8800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.902+0000 7faca170b640 1 -- 172.21.15.64:0/273050675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88003520 con 0x7fac880039c0 2023-11-24T08:05:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.902+0000 7faca1f0c640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac88005250 0x7fac8800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.902+0000 7faca1f0c640 1 -- 172.21.15.64:0/273050675 reap_dead start 2023-11-24T08:05:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.902+0000 7faca0f0a640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac880047c0 0x7fac88004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.903+0000 7faca170b640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac88003dc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c0e0 tx=0x7fac880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4d03f8ad4da9c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.903+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98005ad0 con 0x7fac880039c0 2023-11-24T08:05:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.903+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98005c30 con 0x7fac880039c0 2023-11-24T08:05:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.903+0000 7fac93fff640 1 -- 172.21.15.64:0/273050675 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9802c810 con 0x7fac880039c0 2023-11-24T08:05:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.903+0000 7faca396f640 1 -- 172.21.15.64:0/273050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 msgr2=0x7fac88003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.903+0000 7faca396f640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac88003dc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c0e0 tx=0x7fac880057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.904+0000 7faca396f640 1 -- 172.21.15.64:0/273050675 shutdown_connections 2023-11-24T08:05:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.904+0000 7faca396f640 1 --2- 172.21.15.64:0/273050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac88003dc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.904+0000 7faca396f640 1 -- 172.21.15.64:0/273050675 >> 172.21.15.64:0/273050675 conn(0x7fac9c0b0d80 msgr2=0x7fac9c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.904+0000 7faca396f640 1 -- 172.21.15.64:0/273050675 shutdown_connections 2023-11-24T08:05:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.904+0000 7faca396f640 1 -- 172.21.15.64:0/273050675 wait complete. 2023-11-24T08:05:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.905+0000 7faca396f640 1 Processor -- start 2023-11-24T08:05:25.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.906+0000 7faca396f640 1 -- start start 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.906+0000 7faca396f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac9c1f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca396f640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9c15b5d0 0x7fac9c15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca396f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac9c1520e0 0x7fac9c152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca396f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1538b0 con 0x7fac880039c0 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca396f640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c14ac20 con 0x7fac9c15b5d0 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca396f640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1547b0 con 0x7fac9c1520e0 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca170b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac9c1f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca170b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac9c1f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48512/0 (socket says 172.21.15.64:48512) 2023-11-24T08:05:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca170b640 1 -- 172.21.15.64:0/718412178 learned_addr learned my addr 172.21.15.64:0/718412178 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca0f0a640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9c15b5d0 0x7fac9c15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca1f0c640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac9c1520e0 0x7fac9c152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca170b640 1 -- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac9c1520e0 msgr2=0x7fac9c152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca170b640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac9c1520e0 0x7fac9c152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca170b640 1 -- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9c15b5d0 msgr2=0x7fac9c15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca170b640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9c15b5d0 0x7fac9c15ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca170b640 1 -- 172.21.15.64:0/718412178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c1f2df0 con 0x7fac880039c0 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.907+0000 7faca0f0a640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9c15b5d0 0x7fac9c15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.908+0000 7faca170b640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac9c1f28b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c0e0 tx=0x7fac9802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.908+0000 7fac91ffb640 1 -- 172.21.15.64:0/718412178 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9803c040 con 0x7fac880039c0 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.908+0000 7fac91ffb640 1 -- 172.21.15.64:0/718412178 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98007270 con 0x7fac880039c0 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.908+0000 7fac91ffb640 1 -- 172.21.15.64:0/718412178 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98007600 con 0x7fac880039c0 2023-11-24T08:05:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.908+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c152aa0 con 0x7fac880039c0 2023-11-24T08:05:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.908+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c152d30 con 0x7fac880039c0 2023-11-24T08:05:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.909+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c15bf90 con 0x7fac880039c0 2023-11-24T08:05:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.910+0000 7fac91ffb640 1 -- 172.21.15.64:0/718412178 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac98032070 con 0x7fac880039c0 2023-11-24T08:05:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.911+0000 7fac91ffb640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac8006ae40 0x7fac8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.911+0000 7fac91ffb640 1 -- 172.21.15.64:0/718412178 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac980aaa80 con 0x7fac880039c0 2023-11-24T08:05:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.912+0000 7faca0f0a640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac8006ae40 0x7fac8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.913+0000 7faca0f0a640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac8006ae40 0x7fac8006d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac8c007fd0 tx=0x7fac8c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:25.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:25.916+0000 7fac91ffb640 1 -- 172.21.15.64:0/718412178 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac98075cf0 con 0x7fac880039c0 2023-11-24T08:05:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.177+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fac9c074770 con 0x7fac880039c0 2023-11-24T08:05:26.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.178+0000 7fac91ffb640 1 -- 172.21.15.64:0/718412178 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1033 (secure 0 0 0) 0x7fac98046020 con 0x7fac880039c0 2023-11-24T08:05:26.180 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:26.180 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":115,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1700813023,"num_in_osds":8,"osd_in_since":1700813014,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225316864,"bytes_avail":772868796416,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5220}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-11-24T08:05:02.037147+0000","services":{}},"progress_events":{}} 2023-11-24T08:05:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac8006ae40 msgr2=0x7fac8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac8006ae40 0x7fac8006d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac8c007fd0 tx=0x7fac8c009040 comp rx=0 tx=0).stop 2023-11-24T08:05:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 msgr2=0x7fac9c1f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac9c1f28b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c0e0 tx=0x7fac9802e070 comp rx=0 tx=0).stop 2023-11-24T08:05:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 shutdown_connections 2023-11-24T08:05:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fac8006ae40 0x7fac8006d300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fac9c1520e0 0x7fac9c152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fac9c15b5d0 0x7fac9c15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 --2- 172.21.15.64:0/718412178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac880039c0 0x7fac9c1f28b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 >> 172.21.15.64:0/718412178 conn(0x7fac9c0b0d80 msgr2=0x7fac9c0add70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 shutdown_connections 2023-11-24T08:05:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.181+0000 7faca396f640 1 -- 172.21.15.64:0/718412178 wait complete. 2023-11-24T08:05:26.202 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T08:05:26.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:05:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.419+0000 7ff5c501c640 1 Processor -- start 2023-11-24T08:05:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.420+0000 7ff5c501c640 1 -- start start 2023-11-24T08:05:26.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.421+0000 7ff5c501c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5c015a100 0x7ff5c00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:26.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.422+0000 7ff5c501c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c013da30 con 0x7ff5c00abff0 2023-11-24T08:05:26.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.422+0000 7ff5c501c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c006e340 con 0x7ff5c00ae7e0 2023-11-24T08:05:26.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.422+0000 7ff5c501c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00ac840 con 0x7ff5c015a100 2023-11-24T08:05:26.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.422+0000 7ff5b659c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff5c00abff0 0x7ff5c006d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32942/0 (socket says 172.21.15.64:32942) 2023-11-24T08:05:26.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.422+0000 7ff5b659c640 1 -- 172.21.15.64:0/631026060 learned_addr learned my addr 172.21.15.64:0/631026060 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:26.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.422+0000 7ff5bed9d640 1 --2- 172.21.15.64:0/631026060 >> v2:172.21.15.64:3300/0 conn(0x7ff5c015a100 0x7ff5c00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:26.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.422+0000 7ff5be59c640 1 -- 172.21.15.64:0/631026060 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334978435 0 0) 0x7ff5c013da30 con 0x7ff5c00abff0 2023-11-24T08:05:26.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.422+0000 7ff5be59c640 1 -- 172.21.15.64:0/631026060 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5ac003520 con 0x7ff5c00abff0 2023-11-24T08:05:26.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.423+0000 7ff5bed9d640 1 -- 172.21.15.64:0/631026060 >> v1:172.21.15.91:6789/0 conn(0x7ff5c00ae7e0 legacy=0x7ff5c006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.423+0000 7ff5bed9d640 1 -- 172.21.15.64:0/631026060 >> v1:172.21.15.64:6789/0 conn(0x7ff5c00abff0 legacy=0x7ff5c006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.423+0000 7ff5bed9d640 1 -- 172.21.15.64:0/631026060 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00ac560 con 0x7ff5c015a100 2023-11-24T08:05:26.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.424+0000 7ff5bed9d640 1 --2- 172.21.15.64:0/631026060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c015a100 0x7ff5c00b3290 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00097a0 tx=0x7ff5b002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec1e4e8dc323736 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:26.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.424+0000 7ff5be59c640 1 -- 172.21.15.64:0/631026060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0030610 con 0x7ff5c015a100 2023-11-24T08:05:26.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.424+0000 7ff5be59c640 1 -- 172.21.15.64:0/631026060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b0030770 con 0x7ff5c015a100 2023-11-24T08:05:26.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.424+0000 7ff5be59c640 1 -- 172.21.15.64:0/631026060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0030b00 con 0x7ff5c015a100 2023-11-24T08:05:26.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.424+0000 7ff5c501c640 1 -- 172.21.15.64:0/631026060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c015a100 msgr2=0x7ff5c00b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.424+0000 7ff5c501c640 1 --2- 172.21.15.64:0/631026060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c015a100 0x7ff5c00b3290 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00097a0 tx=0x7ff5b002b960 comp rx=0 tx=0).stop 2023-11-24T08:05:26.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.425+0000 7ff5c501c640 1 -- 172.21.15.64:0/631026060 shutdown_connections 2023-11-24T08:05:26.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.425+0000 7ff5c501c640 1 --2- 172.21.15.64:0/631026060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c015a100 0x7ff5c00b3290 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.425+0000 7ff5c501c640 1 -- 172.21.15.64:0/631026060 >> 172.21.15.64:0/631026060 conn(0x7ff5c00b0d80 msgr2=0x7ff5c00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:26.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.425+0000 7ff5c501c640 1 -- 172.21.15.64:0/631026060 shutdown_connections 2023-11-24T08:05:26.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.425+0000 7ff5c501c640 1 -- 172.21.15.64:0/631026060 wait complete. 2023-11-24T08:05:26.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.426+0000 7ff5c501c640 1 Processor -- start 2023-11-24T08:05:26.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.426+0000 7ff5c501c640 1 -- start start 2023-11-24T08:05:26.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.427+0000 7ff5c501c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00abff0 0x7ff5c014df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:26.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.427+0000 7ff5c501c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5c00ae7e0 0x7ff5c00af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:26.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.427+0000 7ff5c501c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5c015a100 0x7ff5c00cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:26.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.427+0000 7ff5c501c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01f5230 con 0x7ff5c00abff0 2023-11-24T08:05:26.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.427+0000 7ff5c501c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01f5510 con 0x7ff5c015a100 2023-11-24T08:05:26.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.427+0000 7ff5c501c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01f25a0 con 0x7ff5c00ae7e0 2023-11-24T08:05:26.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00abff0 0x7ff5c014df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:26.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00abff0 0x7ff5c014df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48528/0 (socket says 172.21.15.64:48528) 2023-11-24T08:05:26.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 -- 172.21.15.64:0/4012684287 learned_addr learned my addr 172.21.15.64:0/4012684287 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:26.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bf59e640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5c015a100 0x7ff5c00cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:26.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 -- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5c00ae7e0 msgr2=0x7ff5c00af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5c00ae7e0 0x7ff5c00af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 -- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5c015a100 msgr2=0x7ff5c00cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5c015a100 0x7ff5c00cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 -- 172.21.15.64:0/4012684287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0009400 con 0x7ff5c00abff0 2023-11-24T08:05:26.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bf59e640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5c015a100 0x7ff5c00cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:26.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.428+0000 7ff5bed9d640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00abff0 0x7ff5c014df40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff5b002bf70 tx=0x7ff5b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:26.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.429+0000 7ff5b7fff640 1 -- 172.21.15.64:0/4012684287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b003e040 con 0x7ff5c00abff0 2023-11-24T08:05:26.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.429+0000 7ff5b7fff640 1 -- 172.21.15.64:0/4012684287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b0039cb0 con 0x7ff5c00abff0 2023-11-24T08:05:26.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.429+0000 7ff5b7fff640 1 -- 172.21.15.64:0/4012684287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0044630 con 0x7ff5c00abff0 2023-11-24T08:05:26.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.429+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c01f2880 con 0x7ff5c00abff0 2023-11-24T08:05:26.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.429+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c00b23a0 con 0x7ff5c00abff0 2023-11-24T08:05:26.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.430+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c01f2a90 con 0x7ff5c00abff0 2023-11-24T08:05:26.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.431+0000 7ff5b7fff640 1 -- 172.21.15.64:0/4012684287 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5b0043390 con 0x7ff5c00abff0 2023-11-24T08:05:26.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.432+0000 7ff5b7fff640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff5a006ae40 0x7ff5a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:26.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.432+0000 7ff5b7fff640 1 -- 172.21.15.64:0/4012684287 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5b00ab790 con 0x7ff5c00abff0 2023-11-24T08:05:26.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.433+0000 7ff5b659c640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff5a006ae40 0x7ff5a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:26.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.434+0000 7ff5b659c640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff5a006ae40 0x7ff5a006d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00b2a60 tx=0x7ff5a8007930 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:26.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.436+0000 7ff5b7fff640 1 -- 172.21.15.64:0/4012684287 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5b0076830 con 0x7ff5c00abff0 2023-11-24T08:05:26.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.629+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5c013fe40 con 0x7ff5a006ae40 2023-11-24T08:05:26.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.653+0000 7ff5b7fff640 1 -- 172.21.15.64:0/4012684287 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287327 (secure 0 0 0) 0x7ff5c013fe40 con 0x7ff5a006ae40 2023-11-24T08:05:26.657 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:26.660 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff5a006ae40 msgr2=0x7ff5a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff5a006ae40 0x7ff5a006d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00b2a60 tx=0x7ff5a8007930 comp rx=0 tx=0).stop 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00abff0 msgr2=0x7ff5c014df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00abff0 0x7ff5c014df40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff5b002bf70 tx=0x7ff5b0004730 comp rx=0 tx=0).stop 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 shutdown_connections 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff5a006ae40 0x7ff5a006d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff5c015a100 0x7ff5c00cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff5c00ae7e0 0x7ff5c00af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 --2- 172.21.15.64:0/4012684287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00abff0 0x7ff5c014df40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:26.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 >> 172.21.15.64:0/4012684287 conn(0x7ff5c00b0d80 msgr2=0x7ff5c00b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:26.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 shutdown_connections 2023-11-24T08:05:26.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:26.660+0000 7ff5c501c640 1 -- 172.21.15.64:0/4012684287 wait complete. 2023-11-24T08:05:26.683 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-11-24T08:05:26.044544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220388,"kb_used_data":4228,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754332,"statfs":{"total":773094113280,"available":772868435968,"internally_reserved":0,"allocated":4329472,"data_stored":2401153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.048332"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029877+0000","last_change":"2023-11-24T08:05:20.048285+0000","last_active":"2023-11-24T08:05:21.029877+0000","last_peered":"2023-11-24T08:05:21.029877+0000","last_clean":"2023-11-24T08:05:21.029877+0000","last_became_active":"2023-11-24T08:05:20.040125+0000","last_became_peered":"2023-11-24T08:05:20.040125+0000","last_unstale":"2023-11-24T08:05:21.029877+0000","last_undegraded":"2023-11-24T08:05:21.029877+0000","last_fullsized":"2023-11-24T08:05:21.029877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:03.008639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488420+0000","last_change":"2023-11-24T08:05:20.014890+0000","last_active":"2023-11-24T08:05:22.488420+0000","last_peered":"2023-11-24T08:05:22.488420+0000","last_clean":"2023-11-24T08:05:22.488420+0000","last_became_active":"2023-11-24T08:05:20.014395+0000","last_became_peered":"2023-11-24T08:05:20.014395+0000","last_unstale":"2023-11-24T08:05:22.488420+0000","last_undegraded":"2023-11-24T08:05:22.488420+0000","last_fullsized":"2023-11-24T08:05:22.488420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:48.665206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212382+0000","last_change":"2023-11-24T08:05:20.044829+0000","last_active":"2023-11-24T08:05:21.212382+0000","last_peered":"2023-11-24T08:05:21.212382+0000","last_clean":"2023-11-24T08:05:21.212382+0000","last_became_active":"2023-11-24T08:05:20.042415+0000","last_became_peered":"2023-11-24T08:05:20.042415+0000","last_unstale":"2023-11-24T08:05:21.212382+0000","last_undegraded":"2023-11-24T08:05:21.212382+0000","last_fullsized":"2023-11-24T08:05:21.212382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:23:08.742886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025421+0000","last_change":"2023-11-24T08:05:20.043475+0000","last_active":"2023-11-24T08:05:21.025421+0000","last_peered":"2023-11-24T08:05:21.025421+0000","last_clean":"2023-11-24T08:05:21.025421+0000","last_became_active":"2023-11-24T08:05:20.043194+0000","last_became_peered":"2023-11-24T08:05:20.043194+0000","last_unstale":"2023-11-24T08:05:21.025421+0000","last_undegraded":"2023-11-24T08:05:21.025421+0000","last_fullsized":"2023-11-24T08:05:21.025421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:31.661335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387046+0000","last_change":"2023-11-24T08:05:20.024517+0000","last_active":"2023-11-24T08:05:22.387046+0000","last_peered":"2023-11-24T08:05:22.387046+0000","last_clean":"2023-11-24T08:05:22.387046+0000","last_became_active":"2023-11-24T08:05:20.024257+0000","last_became_peered":"2023-11-24T08:05:20.024257+0000","last_unstale":"2023-11-24T08:05:22.387046+0000","last_undegraded":"2023-11-24T08:05:22.387046+0000","last_fullsized":"2023-11-24T08:05:22.387046+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:19.298610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.487388+0000","last_change":"2023-11-24T08:05:20.034242+0000","last_active":"2023-11-24T08:05:22.487388+0000","last_peered":"2023-11-24T08:05:22.487388+0000","last_clean":"2023-11-24T08:05:22.487388+0000","last_became_active":"2023-11-24T08:05:20.033879+0000","last_became_peered":"2023-11-24T08:05:20.033879+0000","last_unstale":"2023-11-24T08:05:22.487388+0000","last_undegraded":"2023-11-24T08:05:22.487388+0000","last_fullsized":"2023-11-24T08:05:22.487388+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:12.630573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029125+0000","last_change":"2023-11-24T08:05:20.045362+0000","last_active":"2023-11-24T08:05:21.029125+0000","last_peered":"2023-11-24T08:05:21.029125+0000","last_clean":"2023-11-24T08:05:21.029125+0000","last_became_active":"2023-11-24T08:05:20.045018+0000","last_became_peered":"2023-11-24T08:05:20.045018+0000","last_unstale":"2023-11-24T08:05:21.029125+0000","last_undegraded":"2023-11-24T08:05:21.029125+0000","last_fullsized":"2023-11-24T08:05:21.029125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:58.999070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.028580+0000","last_change":"2023-11-24T08:05:20.034522+0000","last_active":"2023-11-24T08:05:21.028580+0000","last_peered":"2023-11-24T08:05:21.028580+0000","last_clean":"2023-11-24T08:05:21.028580+0000","last_became_active":"2023-11-24T08:05:20.034211+0000","last_became_peered":"2023-11-24T08:05:20.034211+0000","last_unstale":"2023-11-24T08:05:21.028580+0000","last_undegraded":"2023-11-24T08:05:21.028580+0000","last_fullsized":"2023-11-24T08:05:21.028580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:29.978935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.490564+0000","last_change":"2023-11-24T08:05:20.047938+0000","last_active":"2023-11-24T08:05:22.490564+0000","last_peered":"2023-11-24T08:05:22.490564+0000","last_clean":"2023-11-24T08:05:22.490564+0000","last_became_active":"2023-11-24T08:05:20.047676+0000","last_became_peered":"2023-11-24T08:05:20.047676+0000","last_unstale":"2023-11-24T08:05:22.490564+0000","last_undegraded":"2023-11-24T08:05:22.490564+0000","last_fullsized":"2023-11-24T08:05:22.490564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:31.661335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.026026+0000","last_change":"2023-11-24T08:05:20.042230+0000","last_active":"2023-11-24T08:05:21.026026+0000","last_peered":"2023-11-24T08:05:21.026026+0000","last_clean":"2023-11-24T08:05:21.026026+0000","last_became_active":"2023-11-24T08:05:20.041889+0000","last_became_peered":"2023-11-24T08:05:20.041889+0000","last_unstale":"2023-11-24T08:05:21.026026+0000","last_undegraded":"2023-11-24T08:05:21.026026+0000","last_fullsized":"2023-11-24T08:05:21.026026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:11.244829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030788+0000","last_change":"2023-11-24T08:05:20.044547+0000","last_active":"2023-11-24T08:05:21.030788+0000","last_peered":"2023-11-24T08:05:21.030788+0000","last_clean":"2023-11-24T08:05:21.030788+0000","last_became_active":"2023-11-24T08:05:20.044177+0000","last_became_peered":"2023-11-24T08:05:20.044177+0000","last_unstale":"2023-11-24T08:05:21.030788+0000","last_undegraded":"2023-11-24T08:05:21.030788+0000","last_fullsized":"2023-11-24T08:05:21.030788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:59.317581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212287+0000","last_change":"2023-11-24T08:05:20.038371+0000","last_active":"2023-11-24T08:05:21.212287+0000","last_peered":"2023-11-24T08:05:21.212287+0000","last_clean":"2023-11-24T08:05:21.212287+0000","last_became_active":"2023-11-24T08:05:20.037745+0000","last_became_peered":"2023-11-24T08:05:20.037745+0000","last_unstale":"2023-11-24T08:05:21.212287+0000","last_undegraded":"2023-11-24T08:05:21.212287+0000","last_fullsized":"2023-11-24T08:05:21.212287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:58.850425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387237+0000","last_change":"2023-11-24T08:05:20.038931+0000","last_active":"2023-11-24T08:05:22.387237+0000","last_peered":"2023-11-24T08:05:22.387237+0000","last_clean":"2023-11-24T08:05:22.387237+0000","last_became_active":"2023-11-24T08:05:20.037478+0000","last_became_peered":"2023-11-24T08:05:20.037478+0000","last_unstale":"2023-11-24T08:05:22.387237+0000","last_undegraded":"2023-11-24T08:05:22.387237+0000","last_fullsized":"2023-11-24T08:05:22.387237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:27.215645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.487524+0000","last_change":"2023-11-24T08:05:20.013966+0000","last_active":"2023-11-24T08:05:22.487524+0000","last_peered":"2023-11-24T08:05:22.487524+0000","last_clean":"2023-11-24T08:05:22.487524+0000","last_became_active":"2023-11-24T08:05:20.013534+0000","last_became_peered":"2023-11-24T08:05:20.013534+0000","last_unstale":"2023-11-24T08:05:22.487524+0000","last_undegraded":"2023-11-24T08:05:22.487524+0000","last_fullsized":"2023-11-24T08:05:22.487524+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:58.999070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.211739+0000","last_change":"2023-11-24T08:05:20.033505+0000","last_active":"2023-11-24T08:05:21.211739+0000","last_peered":"2023-11-24T08:05:21.211739+0000","last_clean":"2023-11-24T08:05:21.211739+0000","last_became_active":"2023-11-24T08:05:20.033237+0000","last_became_peered":"2023-11-24T08:05:20.033237+0000","last_unstale":"2023-11-24T08:05:21.211739+0000","last_undegraded":"2023-11-24T08:05:21.211739+0000","last_fullsized":"2023-11-24T08:05:21.211739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:31.661335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029415+0000","last_change":"2023-11-24T08:05:20.039494+0000","last_active":"2023-11-24T08:05:21.029415+0000","last_peered":"2023-11-24T08:05:21.029415+0000","last_clean":"2023-11-24T08:05:21.029415+0000","last_became_active":"2023-11-24T08:05:20.039230+0000","last_became_peered":"2023-11-24T08:05:20.039230+0000","last_unstale":"2023-11-24T08:05:21.029415+0000","last_undegraded":"2023-11-24T08:05:21.029415+0000","last_fullsized":"2023-11-24T08:05:21.029415+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:40.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.031287+0000","last_change":"2023-11-24T08:05:20.052638+0000","last_active":"2023-11-24T08:05:21.031287+0000","last_peered":"2023-11-24T08:05:21.031287+0000","last_clean":"2023-11-24T08:05:21.031287+0000","last_became_active":"2023-11-24T08:05:20.050897+0000","last_became_peered":"2023-11-24T08:05:20.050897+0000","last_unstale":"2023-11-24T08:05:21.031287+0000","last_undegraded":"2023-11-24T08:05:21.031287+0000","last_fullsized":"2023-11-24T08:05:21.031287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:48.665206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488439+0000","last_change":"2023-11-24T08:05:20.045198+0000","last_active":"2023-11-24T08:05:22.488439+0000","last_peered":"2023-11-24T08:05:22.488439+0000","last_clean":"2023-11-24T08:05:22.488439+0000","last_became_active":"2023-11-24T08:05:20.044903+0000","last_became_peered":"2023-11-24T08:05:20.044903+0000","last_unstale":"2023-11-24T08:05:22.488439+0000","last_undegraded":"2023-11-24T08:05:22.488439+0000","last_fullsized":"2023-11-24T08:05:22.488439+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:15:34.932650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.487744+0000","last_change":"2023-11-24T08:05:20.046999+0000","last_active":"2023-11-24T08:05:22.487744+0000","last_peered":"2023-11-24T08:05:22.487744+0000","last_clean":"2023-11-24T08:05:22.487744+0000","last_became_active":"2023-11-24T08:05:20.046723+0000","last_became_peered":"2023-11-24T08:05:20.046723+0000","last_unstale":"2023-11-24T08:05:22.487744+0000","last_undegraded":"2023-11-24T08:05:22.487744+0000","last_fullsized":"2023-11-24T08:05:22.487744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:40.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387981+0000","last_change":"2023-11-24T08:05:20.033528+0000","last_active":"2023-11-24T08:05:22.387981+0000","last_peered":"2023-11-24T08:05:22.387981+0000","last_clean":"2023-11-24T08:05:22.387981+0000","last_became_active":"2023-11-24T08:05:20.033275+0000","last_became_peered":"2023-11-24T08:05:20.033275+0000","last_unstale":"2023-11-24T08:05:22.387981+0000","last_undegraded":"2023-11-24T08:05:22.387981+0000","last_fullsized":"2023-11-24T08:05:22.387981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:51.798341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489820+0000","last_change":"2023-11-24T08:05:20.046798+0000","last_active":"2023-11-24T08:05:22.489820+0000","last_peered":"2023-11-24T08:05:22.489820+0000","last_clean":"2023-11-24T08:05:22.489820+0000","last_became_active":"2023-11-24T08:05:20.046474+0000","last_became_peered":"2023-11-24T08:05:20.046474+0000","last_unstale":"2023-11-24T08:05:22.489820+0000","last_undegraded":"2023-11-24T08:05:22.489820+0000","last_fullsized":"2023-11-24T08:05:22.489820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:40.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489618+0000","last_change":"2023-11-24T08:05:20.041572+0000","last_active":"2023-11-24T08:05:22.489618+0000","last_peered":"2023-11-24T08:05:22.489618+0000","last_clean":"2023-11-24T08:05:22.489618+0000","last_became_active":"2023-11-24T08:05:20.041291+0000","last_became_peered":"2023-11-24T08:05:20.041291+0000","last_unstale":"2023-11-24T08:05:22.489618+0000","last_undegraded":"2023-11-24T08:05:22.489618+0000","last_fullsized":"2023-11-24T08:05:22.489618+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:26.095937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069200+0000","last_change":"2023-11-24T08:05:20.032505+0000","last_active":"2023-11-24T08:05:22.069200+0000","last_peered":"2023-11-24T08:05:22.069200+0000","last_clean":"2023-11-24T08:05:22.069200+0000","last_became_active":"2023-11-24T08:05:20.031170+0000","last_became_peered":"2023-11-24T08:05:20.031170+0000","last_unstale":"2023-11-24T08:05:22.069200+0000","last_undegraded":"2023-11-24T08:05:22.069200+0000","last_fullsized":"2023-11-24T08:05:22.069200+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:26.095937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.211707+0000","last_change":"2023-11-24T08:05:20.032978+0000","last_active":"2023-11-24T08:05:21.211707+0000","last_peered":"2023-11-24T08:05:21.211707+0000","last_clean":"2023-11-24T08:05:21.211707+0000","last_became_active":"2023-11-24T08:05:20.032546+0000","last_became_peered":"2023-11-24T08:05:20.032546+0000","last_unstale":"2023-11-24T08:05:21.211707+0000","last_undegraded":"2023-11-24T08:05:21.211707+0000","last_fullsized":"2023-11-24T08:05:21.211707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:00:29.933202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025801+0000","last_change":"2023-11-24T08:05:20.038780+0000","last_active":"2023-11-24T08:05:21.025801+0000","last_peered":"2023-11-24T08:05:21.025801+0000","last_clean":"2023-11-24T08:05:21.025801+0000","last_became_active":"2023-11-24T08:05:20.038486+0000","last_became_peered":"2023-11-24T08:05:20.038486+0000","last_unstale":"2023-11-24T08:05:21.025801+0000","last_undegraded":"2023-11-24T08:05:21.025801+0000","last_fullsized":"2023-11-24T08:05:21.025801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:58.850425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488792+0000","last_change":"2023-11-24T08:05:20.034907+0000","last_active":"2023-11-24T08:05:22.488792+0000","last_peered":"2023-11-24T08:05:22.488792+0000","last_clean":"2023-11-24T08:05:22.488792+0000","last_became_active":"2023-11-24T08:05:20.034618+0000","last_became_peered":"2023-11-24T08:05:20.034618+0000","last_unstale":"2023-11-24T08:05:22.488792+0000","last_undegraded":"2023-11-24T08:05:22.488792+0000","last_fullsized":"2023-11-24T08:05:22.488792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:22:20.845895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025604+0000","last_change":"2023-11-24T08:05:20.049538+0000","last_active":"2023-11-24T08:05:21.025604+0000","last_peered":"2023-11-24T08:05:21.025604+0000","last_clean":"2023-11-24T08:05:21.025604+0000","last_became_active":"2023-11-24T08:05:20.049259+0000","last_became_peered":"2023-11-24T08:05:20.049259+0000","last_unstale":"2023-11-24T08:05:21.025604+0000","last_undegraded":"2023-11-24T08:05:21.025604+0000","last_fullsized":"2023-11-24T08:05:21.025604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:00:29.933202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212434+0000","last_change":"2023-11-24T08:05:20.040763+0000","last_active":"2023-11-24T08:05:21.212434+0000","last_peered":"2023-11-24T08:05:21.212434+0000","last_clean":"2023-11-24T08:05:21.212434+0000","last_became_active":"2023-11-24T08:05:20.040526+0000","last_became_peered":"2023-11-24T08:05:20.040526+0000","last_unstale":"2023-11-24T08:05:21.212434+0000","last_undegraded":"2023-11-24T08:05:21.212434+0000","last_fullsized":"2023-11-24T08:05:21.212434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:12.630573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489848+0000","last_change":"2023-11-24T08:05:20.047203+0000","last_active":"2023-11-24T08:05:22.489848+0000","last_peered":"2023-11-24T08:05:22.489848+0000","last_clean":"2023-11-24T08:05:22.489848+0000","last_became_active":"2023-11-24T08:05:20.046942+0000","last_became_peered":"2023-11-24T08:05:20.046942+0000","last_unstale":"2023-11-24T08:05:22.489848+0000","last_undegraded":"2023-11-24T08:05:22.489848+0000","last_fullsized":"2023-11-24T08:05:22.489848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:09.824526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029102+0000","last_change":"2023-11-24T08:05:20.048590+0000","last_active":"2023-11-24T08:05:21.029102+0000","last_peered":"2023-11-24T08:05:21.029102+0000","last_clean":"2023-11-24T08:05:21.029102+0000","last_became_active":"2023-11-24T08:05:20.048300+0000","last_became_peered":"2023-11-24T08:05:20.048300+0000","last_unstale":"2023-11-24T08:05:21.029102+0000","last_undegraded":"2023-11-24T08:05:21.029102+0000","last_fullsized":"2023-11-24T08:05:21.029102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:12.630573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030245+0000","last_change":"2023-11-24T08:05:20.027198+0000","last_active":"2023-11-24T08:05:21.030245+0000","last_peered":"2023-11-24T08:05:21.030245+0000","last_clean":"2023-11-24T08:05:21.030245+0000","last_became_active":"2023-11-24T08:05:20.026798+0000","last_became_peered":"2023-11-24T08:05:20.026798+0000","last_unstale":"2023-11-24T08:05:21.030245+0000","last_undegraded":"2023-11-24T08:05:21.030245+0000","last_fullsized":"2023-11-24T08:05:21.030245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:26.095937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.028603+0000","last_change":"2023-11-24T08:05:20.049045+0000","last_active":"2023-11-24T08:05:21.028603+0000","last_peered":"2023-11-24T08:05:21.028603+0000","last_clean":"2023-11-24T08:05:21.028603+0000","last_became_active":"2023-11-24T08:05:20.048771+0000","last_became_peered":"2023-11-24T08:05:20.048771+0000","last_unstale":"2023-11-24T08:05:21.028603+0000","last_undegraded":"2023-11-24T08:05:21.028603+0000","last_fullsized":"2023-11-24T08:05:21.028603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:58.850425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069578+0000","last_change":"2023-11-24T08:05:20.043487+0000","last_active":"2023-11-24T08:05:22.069578+0000","last_peered":"2023-11-24T08:05:22.069578+0000","last_clean":"2023-11-24T08:05:22.069578+0000","last_became_active":"2023-11-24T08:05:20.041729+0000","last_became_peered":"2023-11-24T08:05:20.041729+0000","last_unstale":"2023-11-24T08:05:22.069578+0000","last_undegraded":"2023-11-24T08:05:22.069578+0000","last_fullsized":"2023-11-24T08:05:22.069578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:07.888449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.048870+0000","last_change":"2023-11-24T08:05:19.034855+0000","last_active":"2023-11-24T08:05:20.048870+0000","last_peered":"2023-11-24T08:05:20.048870+0000","last_clean":"2023-11-24T08:05:20.048870+0000","last_became_active":"2023-11-24T08:05:19.034614+0000","last_became_peered":"2023-11-24T08:05:19.034614+0000","last_unstale":"2023-11-24T08:05:20.048870+0000","last_undegraded":"2023-11-24T08:05:20.048870+0000","last_fullsized":"2023-11-24T08:05:20.048870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:09.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030127+0000","last_change":"2023-11-24T08:05:20.035745+0000","last_active":"2023-11-24T08:05:21.030127+0000","last_peered":"2023-11-24T08:05:21.030127+0000","last_clean":"2023-11-24T08:05:21.030127+0000","last_became_active":"2023-11-24T08:05:20.035379+0000","last_became_peered":"2023-11-24T08:05:20.035379+0000","last_unstale":"2023-11-24T08:05:21.030127+0000","last_undegraded":"2023-11-24T08:05:21.030127+0000","last_fullsized":"2023-11-24T08:05:21.030127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:48.077409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.034128+0000","last_change":"2023-11-24T08:05:19.029832+0000","last_active":"2023-11-24T08:05:20.034128+0000","last_peered":"2023-11-24T08:05:20.034128+0000","last_clean":"2023-11-24T08:05:20.034128+0000","last_became_active":"2023-11-24T08:05:19.028467+0000","last_became_peered":"2023-11-24T08:05:19.028467+0000","last_unstale":"2023-11-24T08:05:20.034128+0000","last_undegraded":"2023-11-24T08:05:20.034128+0000","last_fullsized":"2023-11-24T08:05:20.034128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:49.838563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.490190+0000","last_change":"2023-11-24T08:05:20.040574+0000","last_active":"2023-11-24T08:05:22.490190+0000","last_peered":"2023-11-24T08:05:22.490190+0000","last_clean":"2023-11-24T08:05:22.490190+0000","last_became_active":"2023-11-24T08:05:20.033808+0000","last_became_peered":"2023-11-24T08:05:20.033808+0000","last_unstale":"2023-11-24T08:05:22.490190+0000","last_undegraded":"2023-11-24T08:05:22.490190+0000","last_fullsized":"2023-11-24T08:05:22.490190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:07.888449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.044957+0000","last_change":"2023-11-24T08:05:19.024698+0000","last_active":"2023-11-24T08:05:20.044957+0000","last_peered":"2023-11-24T08:05:20.044957+0000","last_clean":"2023-11-24T08:05:20.044957+0000","last_became_active":"2023-11-24T08:05:19.024477+0000","last_became_peered":"2023-11-24T08:05:19.024477+0000","last_unstale":"2023-11-24T08:05:20.044957+0000","last_undegraded":"2023-11-24T08:05:20.044957+0000","last_fullsized":"2023-11-24T08:05:20.044957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:10.229106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025723+0000","last_change":"2023-11-24T08:05:20.044222+0000","last_active":"2023-11-24T08:05:21.025723+0000","last_peered":"2023-11-24T08:05:21.025723+0000","last_clean":"2023-11-24T08:05:21.025723+0000","last_became_active":"2023-11-24T08:05:20.043968+0000","last_became_peered":"2023-11-24T08:05:20.043968+0000","last_unstale":"2023-11-24T08:05:21.025723+0000","last_undegraded":"2023-11-24T08:05:21.025723+0000","last_fullsized":"2023-11-24T08:05:21.025723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:29.978935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.038698+0000","last_change":"2023-11-24T08:05:19.024566+0000","last_active":"2023-11-24T08:05:20.038698+0000","last_peered":"2023-11-24T08:05:20.038698+0000","last_clean":"2023-11-24T08:05:20.038698+0000","last_became_active":"2023-11-24T08:05:19.024310+0000","last_became_peered":"2023-11-24T08:05:19.024310+0000","last_unstale":"2023-11-24T08:05:20.038698+0000","last_undegraded":"2023-11-24T08:05:20.038698+0000","last_fullsized":"2023-11-24T08:05:20.038698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:32:05.718595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030588+0000","last_change":"2023-11-24T08:05:20.052189+0000","last_active":"2023-11-24T08:05:21.030588+0000","last_peered":"2023-11-24T08:05:21.030588+0000","last_clean":"2023-11-24T08:05:21.030588+0000","last_became_active":"2023-11-24T08:05:20.051875+0000","last_became_peered":"2023-11-24T08:05:20.051875+0000","last_unstale":"2023-11-24T08:05:21.030588+0000","last_undegraded":"2023-11-24T08:05:21.030588+0000","last_fullsized":"2023-11-24T08:05:21.030588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:59.448273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.052012+0000","last_change":"2023-11-24T08:05:19.027924+0000","last_active":"2023-11-24T08:05:20.052012+0000","last_peered":"2023-11-24T08:05:20.052012+0000","last_clean":"2023-11-24T08:05:20.052012+0000","last_became_active":"2023-11-24T08:05:19.023079+0000","last_became_peered":"2023-11-24T08:05:19.023079+0000","last_unstale":"2023-11-24T08:05:20.052012+0000","last_undegraded":"2023-11-24T08:05:20.052012+0000","last_fullsized":"2023-11-24T08:05:20.052012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212127+0000","last_change":"2023-11-24T08:05:20.040082+0000","last_active":"2023-11-24T08:05:21.212127+0000","last_peered":"2023-11-24T08:05:21.212127+0000","last_clean":"2023-11-24T08:05:21.212127+0000","last_became_active":"2023-11-24T08:05:20.039676+0000","last_became_peered":"2023-11-24T08:05:20.039676+0000","last_unstale":"2023-11-24T08:05:21.212127+0000","last_undegraded":"2023-11-24T08:05:21.212127+0000","last_fullsized":"2023-11-24T08:05:21.212127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:11.244829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.041125+0000","last_change":"2023-11-24T08:05:19.028892+0000","last_active":"2023-11-24T08:05:20.041125+0000","last_peered":"2023-11-24T08:05:20.041125+0000","last_clean":"2023-11-24T08:05:20.041125+0000","last_became_active":"2023-11-24T08:05:19.027535+0000","last_became_peered":"2023-11-24T08:05:19.027535+0000","last_unstale":"2023-11-24T08:05:20.041125+0000","last_undegraded":"2023-11-24T08:05:20.041125+0000","last_fullsized":"2023-11-24T08:05:20.041125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:49.838563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489406+0000","last_change":"2023-11-24T08:05:20.043805+0000","last_active":"2023-11-24T08:05:22.489406+0000","last_peered":"2023-11-24T08:05:22.489406+0000","last_clean":"2023-11-24T08:05:22.489406+0000","last_became_active":"2023-11-24T08:05:20.043507+0000","last_became_peered":"2023-11-24T08:05:20.043507+0000","last_unstale":"2023-11-24T08:05:22.489406+0000","last_undegraded":"2023-11-24T08:05:22.489406+0000","last_fullsized":"2023-11-24T08:05:22.489406+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:56.826339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.007232+0000","last_change":"2023-11-24T08:05:19.015336+0000","last_active":"2023-11-24T08:05:20.007232+0000","last_peered":"2023-11-24T08:05:20.007232+0000","last_clean":"2023-11-24T08:05:20.007232+0000","last_became_active":"2023-11-24T08:05:19.014910+0000","last_became_peered":"2023-11-24T08:05:19.014910+0000","last_unstale":"2023-11-24T08:05:20.007232+0000","last_undegraded":"2023-11-24T08:05:20.007232+0000","last_fullsized":"2023-11-24T08:05:20.007232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:30.645612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.024970+0000","last_change":"2023-11-24T08:05:20.043160+0000","last_active":"2023-11-24T08:05:21.024970+0000","last_peered":"2023-11-24T08:05:21.024970+0000","last_clean":"2023-11-24T08:05:21.024970+0000","last_became_active":"2023-11-24T08:05:20.042825+0000","last_became_peered":"2023-11-24T08:05:20.042825+0000","last_unstale":"2023-11-24T08:05:21.024970+0000","last_undegraded":"2023-11-24T08:05:21.024970+0000","last_fullsized":"2023-11-24T08:05:21.024970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:09.824526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.023648+0000","last_change":"2023-11-24T08:05:19.031449+0000","last_active":"2023-11-24T08:05:20.023648+0000","last_peered":"2023-11-24T08:05:20.023648+0000","last_clean":"2023-11-24T08:05:20.023648+0000","last_became_active":"2023-11-24T08:05:19.031114+0000","last_became_peered":"2023-11-24T08:05:19.031114+0000","last_unstale":"2023-11-24T08:05:20.023648+0000","last_undegraded":"2023-11-24T08:05:20.023648+0000","last_fullsized":"2023-11-24T08:05:20.023648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:09.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212507+0000","last_change":"2023-11-24T08:05:20.046213+0000","last_active":"2023-11-24T08:05:21.212507+0000","last_peered":"2023-11-24T08:05:21.212507+0000","last_clean":"2023-11-24T08:05:21.212507+0000","last_became_active":"2023-11-24T08:05:20.045904+0000","last_became_peered":"2023-11-24T08:05:20.045904+0000","last_unstale":"2023-11-24T08:05:21.212507+0000","last_undegraded":"2023-11-24T08:05:21.212507+0000","last_fullsized":"2023-11-24T08:05:21.212507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:58.999070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.011355+0000","last_change":"2023-11-24T08:05:19.028843+0000","last_active":"2023-11-24T08:05:20.011355+0000","last_peered":"2023-11-24T08:05:20.011355+0000","last_clean":"2023-11-24T08:05:20.011355+0000","last_became_active":"2023-11-24T08:05:19.027840+0000","last_became_peered":"2023-11-24T08:05:19.027840+0000","last_unstale":"2023-11-24T08:05:20.011355+0000","last_undegraded":"2023-11-24T08:05:20.011355+0000","last_fullsized":"2023-11-24T08:05:20.011355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:57.834702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488125+0000","last_change":"2023-11-24T08:05:20.031652+0000","last_active":"2023-11-24T08:05:22.488125+0000","last_peered":"2023-11-24T08:05:22.488125+0000","last_clean":"2023-11-24T08:05:22.488125+0000","last_became_active":"2023-11-24T08:05:20.031337+0000","last_became_peered":"2023-11-24T08:05:20.031337+0000","last_unstale":"2023-11-24T08:05:22.488125+0000","last_undegraded":"2023-11-24T08:05:22.488125+0000","last_fullsized":"2023-11-24T08:05:22.488125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:59.317581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.006389+0000","last_change":"2023-11-24T08:05:19.029786+0000","last_active":"2023-11-24T08:05:20.006389+0000","last_peered":"2023-11-24T08:05:20.006389+0000","last_clean":"2023-11-24T08:05:20.006389+0000","last_became_active":"2023-11-24T08:05:19.029469+0000","last_became_peered":"2023-11-24T08:05:19.029469+0000","last_unstale":"2023-11-24T08:05:20.006389+0000","last_undegraded":"2023-11-24T08:05:20.006389+0000","last_fullsized":"2023-11-24T08:05:20.006389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:47.061685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488566+0000","last_change":"2023-11-24T08:05:20.030806+0000","last_active":"2023-11-24T08:05:22.488566+0000","last_peered":"2023-11-24T08:05:22.488566+0000","last_clean":"2023-11-24T08:05:22.488566+0000","last_became_active":"2023-11-24T08:05:20.030520+0000","last_became_peered":"2023-11-24T08:05:20.030520+0000","last_unstale":"2023-11-24T08:05:22.488566+0000","last_undegraded":"2023-11-24T08:05:22.488566+0000","last_fullsized":"2023-11-24T08:05:22.488566+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:36:44.264543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.039244+0000","last_change":"2023-11-24T08:05:19.034418+0000","last_active":"2023-11-24T08:05:20.039244+0000","last_peered":"2023-11-24T08:05:20.039244+0000","last_clean":"2023-11-24T08:05:20.039244+0000","last_became_active":"2023-11-24T08:05:19.033911+0000","last_became_peered":"2023-11-24T08:05:19.033911+0000","last_unstale":"2023-11-24T08:05:20.039244+0000","last_undegraded":"2023-11-24T08:05:20.039244+0000","last_fullsized":"2023-11-24T08:05:20.039244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:25.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212332+0000","last_change":"2023-11-24T08:05:20.049925+0000","last_active":"2023-11-24T08:05:21.212332+0000","last_peered":"2023-11-24T08:05:21.212332+0000","last_clean":"2023-11-24T08:05:21.212332+0000","last_became_active":"2023-11-24T08:05:20.049651+0000","last_became_peered":"2023-11-24T08:05:20.049651+0000","last_unstale":"2023-11-24T08:05:21.212332+0000","last_undegraded":"2023-11-24T08:05:21.212332+0000","last_fullsized":"2023-11-24T08:05:21.212332+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:29.978935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.057654+0000","last_change":"2023-11-24T08:05:19.023626+0000","last_active":"2023-11-24T08:05:20.057654+0000","last_peered":"2023-11-24T08:05:20.057654+0000","last_clean":"2023-11-24T08:05:20.057654+0000","last_became_active":"2023-11-24T08:05:19.023367+0000","last_became_peered":"2023-11-24T08:05:19.023367+0000","last_unstale":"2023-11-24T08:05:20.057654+0000","last_undegraded":"2023-11-24T08:05:20.057654+0000","last_fullsized":"2023-11-24T08:05:20.057654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:28.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025095+0000","last_change":"2023-11-24T08:05:20.042063+0000","last_active":"2023-11-24T08:05:21.025095+0000","last_peered":"2023-11-24T08:05:21.025095+0000","last_clean":"2023-11-24T08:05:21.025095+0000","last_became_active":"2023-11-24T08:05:20.041782+0000","last_became_peered":"2023-11-24T08:05:20.041782+0000","last_unstale":"2023-11-24T08:05:21.025095+0000","last_undegraded":"2023-11-24T08:05:21.025095+0000","last_fullsized":"2023-11-24T08:05:21.025095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:07.888449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.052804+0000","last_change":"2023-11-24T08:05:19.026245+0000","last_active":"2023-11-24T08:05:20.052804+0000","last_peered":"2023-11-24T08:05:20.052804+0000","last_clean":"2023-11-24T08:05:20.052804+0000","last_became_active":"2023-11-24T08:05:19.025943+0000","last_became_peered":"2023-11-24T08:05:19.025943+0000","last_unstale":"2023-11-24T08:05:20.052804+0000","last_undegraded":"2023-11-24T08:05:20.052804+0000","last_fullsized":"2023-11-24T08:05:20.052804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488064+0000","last_change":"2023-11-24T08:05:20.014733+0000","last_active":"2023-11-24T08:05:22.488064+0000","last_peered":"2023-11-24T08:05:22.488064+0000","last_clean":"2023-11-24T08:05:22.488064+0000","last_became_active":"2023-11-24T08:05:20.013816+0000","last_became_peered":"2023-11-24T08:05:20.013816+0000","last_unstale":"2023-11-24T08:05:22.488064+0000","last_undegraded":"2023-11-24T08:05:22.488064+0000","last_fullsized":"2023-11-24T08:05:22.488064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:59.448273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.025669+0000","last_change":"2023-11-24T08:05:19.027534+0000","last_active":"2023-11-24T08:05:20.025669+0000","last_peered":"2023-11-24T08:05:20.025669+0000","last_clean":"2023-11-24T08:05:20.025669+0000","last_became_active":"2023-11-24T08:05:19.024963+0000","last_became_peered":"2023-11-24T08:05:19.024963+0000","last_unstale":"2023-11-24T08:05:20.025669+0000","last_undegraded":"2023-11-24T08:05:20.025669+0000","last_fullsized":"2023-11-24T08:05:20.025669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:49:41.186235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029005+0000","last_change":"2023-11-24T08:05:20.051911+0000","last_active":"2023-11-24T08:05:21.029005+0000","last_peered":"2023-11-24T08:05:21.029005+0000","last_clean":"2023-11-24T08:05:21.029005+0000","last_became_active":"2023-11-24T08:05:20.051638+0000","last_became_peered":"2023-11-24T08:05:20.051638+0000","last_unstale":"2023-11-24T08:05:21.029005+0000","last_undegraded":"2023-11-24T08:05:21.029005+0000","last_fullsized":"2023-11-24T08:05:21.029005+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:11.244829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.003756+0000","last_change":"2023-11-24T08:05:19.028331+0000","last_active":"2023-11-24T08:05:20.003756+0000","last_peered":"2023-11-24T08:05:20.003756+0000","last_clean":"2023-11-24T08:05:20.003756+0000","last_became_active":"2023-11-24T08:05:19.028037+0000","last_became_peered":"2023-11-24T08:05:19.028037+0000","last_unstale":"2023-11-24T08:05:20.003756+0000","last_undegraded":"2023-11-24T08:05:20.003756+0000","last_fullsized":"2023-11-24T08:05:20.003756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:01.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030205+0000","last_change":"2023-11-24T08:05:20.043970+0000","last_active":"2023-11-24T08:05:21.030205+0000","last_peered":"2023-11-24T08:05:21.030205+0000","last_clean":"2023-11-24T08:05:21.030205+0000","last_became_active":"2023-11-24T08:05:20.043637+0000","last_became_peered":"2023-11-24T08:05:20.043637+0000","last_unstale":"2023-11-24T08:05:21.030205+0000","last_undegraded":"2023-11-24T08:05:21.030205+0000","last_fullsized":"2023-11-24T08:05:21.030205+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:56.826339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.048620+0000","last_change":"2023-11-24T08:05:19.023564+0000","last_active":"2023-11-24T08:05:20.048620+0000","last_peered":"2023-11-24T08:05:20.048620+0000","last_clean":"2023-11-24T08:05:20.048620+0000","last_became_active":"2023-11-24T08:05:19.022784+0000","last_became_peered":"2023-11-24T08:05:19.022784+0000","last_unstale":"2023-11-24T08:05:20.048620+0000","last_undegraded":"2023-11-24T08:05:20.048620+0000","last_fullsized":"2023-11-24T08:05:20.048620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:55.810615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.490399+0000","last_change":"2023-11-24T08:05:20.043461+0000","last_active":"2023-11-24T08:05:22.490399+0000","last_peered":"2023-11-24T08:05:22.490399+0000","last_clean":"2023-11-24T08:05:22.490399+0000","last_became_active":"2023-11-24T08:05:20.043158+0000","last_became_peered":"2023-11-24T08:05:20.043158+0000","last_unstale":"2023-11-24T08:05:22.490399+0000","last_undegraded":"2023-11-24T08:05:22.490399+0000","last_fullsized":"2023-11-24T08:05:22.490399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:03.221967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.049798+0000","last_change":"2023-11-24T08:05:19.030608+0000","last_active":"2023-11-24T08:05:20.049798+0000","last_peered":"2023-11-24T08:05:20.049798+0000","last_clean":"2023-11-24T08:05:20.049798+0000","last_became_active":"2023-11-24T08:05:19.030359+0000","last_became_peered":"2023-11-24T08:05:19.030359+0000","last_unstale":"2023-11-24T08:05:20.049798+0000","last_undegraded":"2023-11-24T08:05:20.049798+0000","last_fullsized":"2023-11-24T08:05:20.049798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:55.810615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212825+0000","last_change":"2023-11-24T08:05:20.053318+0000","last_active":"2023-11-24T08:05:21.212825+0000","last_peered":"2023-11-24T08:05:21.212825+0000","last_clean":"2023-11-24T08:05:21.212825+0000","last_became_active":"2023-11-24T08:05:20.053051+0000","last_became_peered":"2023-11-24T08:05:20.053051+0000","last_unstale":"2023-11-24T08:05:21.212825+0000","last_undegraded":"2023-11-24T08:05:21.212825+0000","last_fullsized":"2023-11-24T08:05:21.212825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:40.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.043159+0000","last_change":"2023-11-24T08:05:19.027262+0000","last_active":"2023-11-24T08:05:20.043159+0000","last_peered":"2023-11-24T08:05:20.043159+0000","last_clean":"2023-11-24T08:05:20.043159+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:20.043159+0000","last_undegraded":"2023-11-24T08:05:20.043159+0000","last_fullsized":"2023-11-24T08:05:20.043159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:01.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025356+0000","last_change":"2023-11-24T08:05:20.049308+0000","last_active":"2023-11-24T08:05:21.025356+0000","last_peered":"2023-11-24T08:05:21.025356+0000","last_clean":"2023-11-24T08:05:21.025356+0000","last_became_active":"2023-11-24T08:05:20.048759+0000","last_became_peered":"2023-11-24T08:05:20.048759+0000","last_unstale":"2023-11-24T08:05:21.025356+0000","last_undegraded":"2023-11-24T08:05:21.025356+0000","last_fullsized":"2023-11-24T08:05:21.025356+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:03.221967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.050110+0000","last_change":"2023-11-24T08:05:19.025861+0000","last_active":"2023-11-24T08:05:20.050110+0000","last_peered":"2023-11-24T08:05:20.050110+0000","last_clean":"2023-11-24T08:05:20.050110+0000","last_became_active":"2023-11-24T08:05:19.024918+0000","last_became_peered":"2023-11-24T08:05:19.024918+0000","last_unstale":"2023-11-24T08:05:20.050110+0000","last_undegraded":"2023-11-24T08:05:20.050110+0000","last_fullsized":"2023-11-24T08:05:20.050110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:12.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387473+0000","last_change":"2023-11-24T08:05:20.038066+0000","last_active":"2023-11-24T08:05:22.387473+0000","last_peered":"2023-11-24T08:05:22.387473+0000","last_clean":"2023-11-24T08:05:22.387473+0000","last_became_active":"2023-11-24T08:05:20.037794+0000","last_became_peered":"2023-11-24T08:05:20.037794+0000","last_unstale":"2023-11-24T08:05:22.387473+0000","last_undegraded":"2023-11-24T08:05:22.387473+0000","last_fullsized":"2023-11-24T08:05:22.387473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:51.160788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.016802+0000","last_change":"2023-11-24T08:05:19.017985+0000","last_active":"2023-11-24T08:05:20.016802+0000","last_peered":"2023-11-24T08:05:20.016802+0000","last_clean":"2023-11-24T08:05:20.016802+0000","last_became_active":"2023-11-24T08:05:19.017444+0000","last_became_peered":"2023-11-24T08:05:19.017444+0000","last_unstale":"2023-11-24T08:05:20.016802+0000","last_undegraded":"2023-11-24T08:05:20.016802+0000","last_fullsized":"2023-11-24T08:05:20.016802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:04.782054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387581+0000","last_change":"2023-11-24T08:05:20.044447+0000","last_active":"2023-11-24T08:05:22.387581+0000","last_peered":"2023-11-24T08:05:22.387581+0000","last_clean":"2023-11-24T08:05:22.387581+0000","last_became_active":"2023-11-24T08:05:20.044152+0000","last_became_peered":"2023-11-24T08:05:20.044152+0000","last_unstale":"2023-11-24T08:05:22.387581+0000","last_undegraded":"2023-11-24T08:05:22.387581+0000","last_fullsized":"2023-11-24T08:05:22.387581+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:28.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.038408+0000","last_change":"2023-11-24T08:05:19.026334+0000","last_active":"2023-11-24T08:05:20.038408+0000","last_peered":"2023-11-24T08:05:20.038408+0000","last_clean":"2023-11-24T08:05:20.038408+0000","last_became_active":"2023-11-24T08:05:19.025952+0000","last_became_peered":"2023-11-24T08:05:19.025952+0000","last_unstale":"2023-11-24T08:05:20.038408+0000","last_undegraded":"2023-11-24T08:05:20.038408+0000","last_fullsized":"2023-11-24T08:05:20.038408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:00.858708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387380+0000","last_change":"2023-11-24T08:05:20.020123+0000","last_active":"2023-11-24T08:05:22.387380+0000","last_peered":"2023-11-24T08:05:22.387380+0000","last_clean":"2023-11-24T08:05:22.387380+0000","last_became_active":"2023-11-24T08:05:20.019725+0000","last_became_peered":"2023-11-24T08:05:20.019725+0000","last_unstale":"2023-11-24T08:05:22.387380+0000","last_undegraded":"2023-11-24T08:05:22.387380+0000","last_fullsized":"2023-11-24T08:05:22.387380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:54.464456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.016170+0000","last_change":"2023-11-24T08:05:19.034595+0000","last_active":"2023-11-24T08:05:20.016170+0000","last_peered":"2023-11-24T08:05:20.016170+0000","last_clean":"2023-11-24T08:05:20.016170+0000","last_became_active":"2023-11-24T08:05:19.033790+0000","last_became_peered":"2023-11-24T08:05:19.033790+0000","last_unstale":"2023-11-24T08:05:20.016170+0000","last_undegraded":"2023-11-24T08:05:20.016170+0000","last_fullsized":"2023-11-24T08:05:20.016170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:22.374300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387359+0000","last_change":"2023-11-24T08:05:20.022542+0000","last_active":"2023-11-24T08:05:22.387359+0000","last_peered":"2023-11-24T08:05:22.387359+0000","last_clean":"2023-11-24T08:05:22.387359+0000","last_became_active":"2023-11-24T08:05:20.022244+0000","last_became_peered":"2023-11-24T08:05:20.022244+0000","last_unstale":"2023-11-24T08:05:22.387359+0000","last_undegraded":"2023-11-24T08:05:22.387359+0000","last_fullsized":"2023-11-24T08:05:22.387359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:03.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.022210+0000","last_change":"2023-11-24T08:05:19.028570+0000","last_active":"2023-11-24T08:05:20.022210+0000","last_peered":"2023-11-24T08:05:20.022210+0000","last_clean":"2023-11-24T08:05:20.022210+0000","last_became_active":"2023-11-24T08:05:19.028257+0000","last_became_peered":"2023-11-24T08:05:19.028257+0000","last_unstale":"2023-11-24T08:05:20.022210+0000","last_undegraded":"2023-11-24T08:05:20.022210+0000","last_fullsized":"2023-11-24T08:05:20.022210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:50.782618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.028983+0000","last_change":"2023-11-24T08:05:20.044676+0000","last_active":"2023-11-24T08:05:21.028983+0000","last_peered":"2023-11-24T08:05:21.028983+0000","last_clean":"2023-11-24T08:05:21.028983+0000","last_became_active":"2023-11-24T08:05:20.044335+0000","last_became_peered":"2023-11-24T08:05:20.044335+0000","last_unstale":"2023-11-24T08:05:21.028983+0000","last_undegraded":"2023-11-24T08:05:21.028983+0000","last_fullsized":"2023-11-24T08:05:21.028983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:23:08.742886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.018624+0000","last_change":"2023-11-24T08:05:19.032519+0000","last_active":"2023-11-24T08:05:20.018624+0000","last_peered":"2023-11-24T08:05:20.018624+0000","last_clean":"2023-11-24T08:05:20.018624+0000","last_became_active":"2023-11-24T08:05:19.032159+0000","last_became_peered":"2023-11-24T08:05:19.032159+0000","last_unstale":"2023-11-24T08:05:20.018624+0000","last_undegraded":"2023-11-24T08:05:20.018624+0000","last_fullsized":"2023-11-24T08:05:20.018624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:22.374300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387497+0000","last_change":"2023-11-24T08:05:20.040541+0000","last_active":"2023-11-24T08:05:22.387497+0000","last_peered":"2023-11-24T08:05:22.387497+0000","last_clean":"2023-11-24T08:05:22.387497+0000","last_became_active":"2023-11-24T08:05:20.040010+0000","last_became_peered":"2023-11-24T08:05:20.040010+0000","last_unstale":"2023-11-24T08:05:22.387497+0000","last_undegraded":"2023-11-24T08:05:22.387497+0000","last_fullsized":"2023-11-24T08:05:22.387497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:13.377447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.045270+0000","last_change":"2023-11-24T08:05:19.034899+0000","last_active":"2023-11-24T08:05:20.045270+0000","last_peered":"2023-11-24T08:05:20.045270+0000","last_clean":"2023-11-24T08:05:20.045270+0000","last_became_active":"2023-11-24T08:05:19.034569+0000","last_became_peered":"2023-11-24T08:05:19.034569+0000","last_unstale":"2023-11-24T08:05:20.045270+0000","last_undegraded":"2023-11-24T08:05:20.045270+0000","last_fullsized":"2023-11-24T08:05:20.045270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:25.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212862+0000","last_change":"2023-11-24T08:05:20.046655+0000","last_active":"2023-11-24T08:05:21.212862+0000","last_peered":"2023-11-24T08:05:21.212862+0000","last_clean":"2023-11-24T08:05:21.212862+0000","last_became_active":"2023-11-24T08:05:20.046208+0000","last_became_peered":"2023-11-24T08:05:20.046208+0000","last_unstale":"2023-11-24T08:05:21.212862+0000","last_undegraded":"2023-11-24T08:05:21.212862+0000","last_fullsized":"2023-11-24T08:05:21.212862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:59.448273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.006250+0000","last_change":"2023-11-24T08:05:19.027343+0000","last_active":"2023-11-24T08:05:20.006250+0000","last_peered":"2023-11-24T08:05:20.006250+0000","last_clean":"2023-11-24T08:05:20.006250+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:20.006250+0000","last_undegraded":"2023-11-24T08:05:20.006250+0000","last_fullsized":"2023-11-24T08:05:20.006250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:01.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069399+0000","last_change":"2023-11-24T08:05:20.042900+0000","last_active":"2023-11-24T08:05:22.069399+0000","last_peered":"2023-11-24T08:05:22.069399+0000","last_clean":"2023-11-24T08:05:22.069399+0000","last_became_active":"2023-11-24T08:05:20.042551+0000","last_became_peered":"2023-11-24T08:05:20.042551+0000","last_unstale":"2023-11-24T08:05:22.069399+0000","last_undegraded":"2023-11-24T08:05:22.069399+0000","last_fullsized":"2023-11-24T08:05:22.069399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:40.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.026545+0000","last_change":"2023-11-24T08:05:19.025711+0000","last_active":"2023-11-24T08:05:20.026545+0000","last_peered":"2023-11-24T08:05:20.026545+0000","last_clean":"2023-11-24T08:05:20.026545+0000","last_became_active":"2023-11-24T08:05:19.025421+0000","last_became_peered":"2023-11-24T08:05:19.025421+0000","last_unstale":"2023-11-24T08:05:20.026545+0000","last_undegraded":"2023-11-24T08:05:20.026545+0000","last_fullsized":"2023-11-24T08:05:20.026545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:22.374300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030708+0000","last_change":"2023-11-24T08:05:20.026311+0000","last_active":"2023-11-24T08:05:21.030708+0000","last_peered":"2023-11-24T08:05:21.030708+0000","last_clean":"2023-11-24T08:05:21.030708+0000","last_became_active":"2023-11-24T08:05:20.025991+0000","last_became_peered":"2023-11-24T08:05:20.025991+0000","last_unstale":"2023-11-24T08:05:21.030708+0000","last_undegraded":"2023-11-24T08:05:21.030708+0000","last_fullsized":"2023-11-24T08:05:21.030708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:40.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.005925+0000","last_change":"2023-11-24T08:05:19.029520+0000","last_active":"2023-11-24T08:05:20.005925+0000","last_peered":"2023-11-24T08:05:20.005925+0000","last_clean":"2023-11-24T08:05:20.005925+0000","last_became_active":"2023-11-24T08:05:19.029227+0000","last_became_peered":"2023-11-24T08:05:19.029227+0000","last_unstale":"2023-11-24T08:05:20.005925+0000","last_undegraded":"2023-11-24T08:05:20.005925+0000","last_fullsized":"2023-11-24T08:05:20.005925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:22.374300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069510+0000","last_change":"2023-11-24T08:05:20.028443+0000","last_active":"2023-11-24T08:05:22.069510+0000","last_peered":"2023-11-24T08:05:22.069510+0000","last_clean":"2023-11-24T08:05:22.069510+0000","last_became_active":"2023-11-24T08:05:20.028163+0000","last_became_peered":"2023-11-24T08:05:20.028163+0000","last_unstale":"2023-11-24T08:05:22.069510+0000","last_undegraded":"2023-11-24T08:05:22.069510+0000","last_fullsized":"2023-11-24T08:05:22.069510+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:09.824526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.026227+0000","last_change":"2023-11-24T08:05:19.031328+0000","last_active":"2023-11-24T08:05:20.026227+0000","last_peered":"2023-11-24T08:05:20.026227+0000","last_clean":"2023-11-24T08:05:20.026227+0000","last_became_active":"2023-11-24T08:05:19.031046+0000","last_became_peered":"2023-11-24T08:05:19.031046+0000","last_unstale":"2023-11-24T08:05:20.026227+0000","last_undegraded":"2023-11-24T08:05:20.026227+0000","last_fullsized":"2023-11-24T08:05:20.026227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:09.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.487031+0000","last_change":"2023-11-24T08:05:20.038413+0000","last_active":"2023-11-24T08:05:22.487031+0000","last_peered":"2023-11-24T08:05:22.487031+0000","last_clean":"2023-11-24T08:05:22.487031+0000","last_became_active":"2023-11-24T08:05:20.034335+0000","last_became_peered":"2023-11-24T08:05:20.034335+0000","last_unstale":"2023-11-24T08:05:22.487031+0000","last_undegraded":"2023-11-24T08:05:22.487031+0000","last_fullsized":"2023-11-24T08:05:22.487031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:23:08.742886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.022377+0000","last_change":"2023-11-24T08:05:19.030600+0000","last_active":"2023-11-24T08:05:20.022377+0000","last_peered":"2023-11-24T08:05:20.022377+0000","last_clean":"2023-11-24T08:05:20.022377+0000","last_became_active":"2023-11-24T08:05:19.028703+0000","last_became_peered":"2023-11-24T08:05:19.028703+0000","last_unstale":"2023-11-24T08:05:20.022377+0000","last_undegraded":"2023-11-24T08:05:20.022377+0000","last_fullsized":"2023-11-24T08:05:20.022377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:00.858708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489277+0000","last_change":"2023-11-24T08:05:20.039291+0000","last_active":"2023-11-24T08:05:22.489277+0000","last_peered":"2023-11-24T08:05:22.489277+0000","last_clean":"2023-11-24T08:05:22.489277+0000","last_became_active":"2023-11-24T08:05:20.038950+0000","last_became_peered":"2023-11-24T08:05:20.038950+0000","last_unstale":"2023-11-24T08:05:22.489277+0000","last_undegraded":"2023-11-24T08:05:22.489277+0000","last_fullsized":"2023-11-24T08:05:22.489277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:48.077409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.003758+0000","last_change":"2023-11-24T08:05:19.033681+0000","last_active":"2023-11-24T08:05:20.003758+0000","last_peered":"2023-11-24T08:05:20.003758+0000","last_clean":"2023-11-24T08:05:20.003758+0000","last_became_active":"2023-11-24T08:05:19.033420+0000","last_became_peered":"2023-11-24T08:05:19.033420+0000","last_unstale":"2023-11-24T08:05:20.003758+0000","last_undegraded":"2023-11-24T08:05:20.003758+0000","last_fullsized":"2023-11-24T08:05:20.003758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:47.061685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.031354+0000","last_change":"2023-11-24T08:05:20.035316+0000","last_active":"2023-11-24T08:05:21.031354+0000","last_peered":"2023-11-24T08:05:21.031354+0000","last_clean":"2023-11-24T08:05:21.031354+0000","last_became_active":"2023-11-24T08:05:20.035044+0000","last_became_peered":"2023-11-24T08:05:20.035044+0000","last_unstale":"2023-11-24T08:05:21.031354+0000","last_undegraded":"2023-11-24T08:05:21.031354+0000","last_fullsized":"2023-11-24T08:05:21.031354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:15:34.932650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.015217+0000","last_change":"2023-11-24T08:05:19.036326+0000","last_active":"2023-11-24T08:05:20.015217+0000","last_peered":"2023-11-24T08:05:20.015217+0000","last_clean":"2023-11-24T08:05:20.015217+0000","last_became_active":"2023-11-24T08:05:19.035962+0000","last_became_peered":"2023-11-24T08:05:19.035962+0000","last_unstale":"2023-11-24T08:05:20.015217+0000","last_undegraded":"2023-11-24T08:05:20.015217+0000","last_fullsized":"2023-11-24T08:05:20.015217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:49:41.186235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.027171+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:05:20.027171+0000","last_peered":"2023-11-24T08:05:20.027171+0000","last_clean":"2023-11-24T08:05:20.027171+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:05:20.027171+0000","last_undegraded":"2023-11-24T08:05:20.027171+0000","last_fullsized":"2023-11-24T08:05:20.027171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":742535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.995}]},{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.29}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.8879999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.464}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.099},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":1.391}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":1.216}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":4.0179999999999998,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.1799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":152693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.2270000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.411}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.5},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.5860000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.823,"15min":1.823},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":6.8129999999999997,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.639},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.895}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.879,"15min":1.879},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":9.8569999999999993,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.762},{"interface":"front","average":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":9.9450000000000003,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.8400000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.6819999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":152749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.4810000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.98299999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.5820000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.143},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.91400000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.262},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":1.3089999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.351}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.6919999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":1.548}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":152181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.073},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.224}]},{"osd":1,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.2030000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.026},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.3460000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.171}]},{"osd":5,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.389},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.153}]},{"osd":6,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.282},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.2589999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.127},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.446}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":153711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.73599999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":1.6359999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.163},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.2669999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.452}]},{"osd":5,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.393}]},{"osd":6,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.496},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.4330000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.357},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":2.0430000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":152147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.131},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.272}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.9969999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.208}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.552}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.3149999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":3.161,"5min":3.161,"15min":3.161},"last":1.502}]},{"osd":5,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.7649999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.367}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.825},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.4419999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.9199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":153711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.216},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.097}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3939999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.165}]},{"osd":6,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.4590000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.609,"5min":1.609,"15min":1.609},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.7330000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.429}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":741426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.4390000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":0.85499999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.6320000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":1.0449999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.5629999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.419}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":1.742},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":3.371,"5min":3.371,"15min":3.371},"last":1.3720000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:05:29.688 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:05:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.924+0000 7fc08ffff640 1 Processor -- start 2023-11-24T08:05:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.925+0000 7fc08ffff640 1 -- start start 2023-11-24T08:05:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.926+0000 7fc08ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc09015d5b0 0x7fc0900d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.926+0000 7fc08ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc0900d14a0 0x7fc0900d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 0x7fc0900cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc08ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0900cf0c0 con 0x7fc09015d5b0 2023-11-24T08:05:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc08ffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0900d1e40 con 0x7fc0900d14a0 2023-11-24T08:05:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc08ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900b01b0 con 0x7fc0900cf7a0 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc08effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc09015d5b0 0x7fc0900d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc08effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc09015d5b0 0x7fc0900d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47254/0 (socket says 172.21.15.64:47254) 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc08effd640 1 -- 172.21.15.64:0/1408209304 learned_addr learned my addr 172.21.15.64:0/1408209304 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc087fff640 1 --2- 172.21.15.64:0/1408209304 >> v2:172.21.15.91:3300/0 conn(0x7fc0900d14a0 0x7fc0900d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.927+0000 7fc08f7fe640 1 --2- 172.21.15.64:0/1408209304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 0x7fc0900cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.928+0000 7fc08effd640 1 -- 172.21.15.64:0/1408209304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 msgr2=0x7fc0900cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.928+0000 7fc08effd640 1 --2- 172.21.15.64:0/1408209304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 0x7fc0900cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.928+0000 7fc08effd640 1 -- 172.21.15.64:0/1408209304 >> v2:172.21.15.91:3300/0 conn(0x7fc0900d14a0 msgr2=0x7fc0900d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.928+0000 7fc08effd640 1 --2- 172.21.15.64:0/1408209304 >> v2:172.21.15.91:3300/0 conn(0x7fc0900d14a0 0x7fc0900d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.928+0000 7fc08effd640 1 -- 172.21.15.64:0/1408209304 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09015d990 con 0x7fc09015d5b0 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.929+0000 7fc08effd640 1 --2- 172.21.15.64:0/1408209304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc09015d5b0 0x7fc0900d3990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc090161000 tx=0x7fc08002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d62ff64474fb75a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.929+0000 7fc08e7fc640 1 -- 172.21.15.64:0/1408209304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080031640 con 0x7fc09015d5b0 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.929+0000 7fc08e7fc640 1 -- 172.21.15.64:0/1408209304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0800317a0 con 0x7fc09015d5b0 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.929+0000 7fc08e7fc640 1 -- 172.21.15.64:0/1408209304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080031b30 con 0x7fc09015d5b0 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 -- 172.21.15.64:0/1408209304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc09015d5b0 msgr2=0x7fc0900d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 --2- 172.21.15.64:0/1408209304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc09015d5b0 0x7fc0900d3990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc090161000 tx=0x7fc08002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 -- 172.21.15.64:0/1408209304 shutdown_connections 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 --2- 172.21.15.64:0/1408209304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 0x7fc0900cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 --2- 172.21.15.64:0/1408209304 >> v2:172.21.15.91:3300/0 conn(0x7fc0900d14a0 0x7fc0900d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 --2- 172.21.15.64:0/1408209304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc09015d5b0 0x7fc0900d3990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 -- 172.21.15.64:0/1408209304 >> 172.21.15.64:0/1408209304 conn(0x7fc09009c3b0 msgr2=0x7fc0900d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 -- 172.21.15.64:0/1408209304 shutdown_connections 2023-11-24T08:05:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.930+0000 7fc08ffff640 1 -- 172.21.15.64:0/1408209304 wait complete. 2023-11-24T08:05:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.932+0000 7fc08ffff640 1 Processor -- start 2023-11-24T08:05:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.932+0000 7fc08ffff640 1 -- start start 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 0x7fc09014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0900d14a0 0x7fc0900cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc09015d5b0 0x7fc0900cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc08ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900ca140 con 0x7fc0900d14a0 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc08ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900c7b40 con 0x7fc09015d5b0 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc08ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900c7e20 con 0x7fc0900cf7a0 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc087fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0900d14a0 0x7fc0900cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc087fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0900d14a0 0x7fc0900cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47258/0 (socket says 172.21.15.64:47258) 2023-11-24T08:05:29.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc087fff640 1 -- 172.21.15.64:0/2242175460 learned_addr learned my addr 172.21.15.64:0/2242175460 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc08effd640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 0x7fc09014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.934+0000 7fc08f7fe640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc09015d5b0 0x7fc0900cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc087fff640 1 -- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 msgr2=0x7fc09014f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc087fff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 0x7fc09014f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc087fff640 1 -- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc09015d5b0 msgr2=0x7fc0900cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc087fff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc09015d5b0 0x7fc0900cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc087fff640 1 -- 172.21.15.64:0/2242175460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0800094d0 con 0x7fc0900d14a0 2023-11-24T08:05:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc087fff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0900d14a0 0x7fc0900cc3c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d5f0 tx=0x7fc07800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc0877fe640 1 -- 172.21.15.64:0/2242175460 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780125f0 con 0x7fc0900d14a0 2023-11-24T08:05:29.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc0877fe640 1 -- 172.21.15.64:0/2242175460 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc078012750 con 0x7fc0900d14a0 2023-11-24T08:05:29.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0900c8100 con 0x7fc0900d14a0 2023-11-24T08:05:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc0877fe640 1 -- 172.21.15.64:0/2242175460 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078012b00 con 0x7fc0900d14a0 2023-11-24T08:05:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.935+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09014f980 con 0x7fc0900d14a0 2023-11-24T08:05:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.937+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09004e840 con 0x7fc0900d14a0 2023-11-24T08:05:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.938+0000 7fc0877fe640 1 -- 172.21.15.64:0/2242175460 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc078012c60 con 0x7fc0900d14a0 2023-11-24T08:05:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.939+0000 7fc0877fe640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc07006ae40 0x7fc07006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.940+0000 7fc0877fe640 1 -- 172.21.15.64:0/2242175460 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc078089ac0 con 0x7fc0900d14a0 2023-11-24T08:05:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.940+0000 7fc08effd640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc07006ae40 0x7fc07006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.941+0000 7fc08effd640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc07006ae40 0x7fc07006d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc080036000 tx=0x7fc08003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:29.944+0000 7fc0877fe640 1 -- 172.21.15.64:0/2242175460 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc078054ad0 con 0x7fc0900d14a0 2023-11-24T08:05:30.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.138+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0900665c0 con 0x7fc07006ae40 2023-11-24T08:05:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.161+0000 7fc0877fe640 1 -- 172.21.15.64:0/2242175460 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287204 (secure 0 0 0) 0x7fc0900665c0 con 0x7fc07006ae40 2023-11-24T08:05:30.167 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:30.169 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:05:30.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.167+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc07006ae40 msgr2=0x7fc07006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:30.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.167+0000 7fc08ffff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc07006ae40 0x7fc07006d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc080036000 tx=0x7fc08003c040 comp rx=0 tx=0).stop 2023-11-24T08:05:30.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.167+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0900d14a0 msgr2=0x7fc0900cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:30.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.167+0000 7fc08ffff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0900d14a0 0x7fc0900cc3c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d5f0 tx=0x7fc07800dab0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.168+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 shutdown_connections 2023-11-24T08:05:30.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.168+0000 7fc08ffff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc07006ae40 0x7fc07006d300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.168+0000 7fc08ffff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc09015d5b0 0x7fc0900cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.168+0000 7fc08ffff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0900d14a0 0x7fc0900cc3c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.168+0000 7fc08ffff640 1 --2- 172.21.15.64:0/2242175460 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc0900cf7a0 0x7fc09014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.168+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 >> 172.21.15.64:0/2242175460 conn(0x7fc09009c3b0 msgr2=0x7fc09015dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:30.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.168+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 shutdown_connections 2023-11-24T08:05:30.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.168+0000 7fc08ffff640 1 -- 172.21.15.64:0/2242175460 wait complete. 2023-11-24T08:05:30.192 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-24T08:05:30.046440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220388,"kb_used_data":4228,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754332,"statfs":{"total":773094113280,"available":772868435968,"internally_reserved":0,"allocated":4329472,"data_stored":2401153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003825"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029877+0000","last_change":"2023-11-24T08:05:20.048285+0000","last_active":"2023-11-24T08:05:21.029877+0000","last_peered":"2023-11-24T08:05:21.029877+0000","last_clean":"2023-11-24T08:05:21.029877+0000","last_became_active":"2023-11-24T08:05:20.040125+0000","last_became_peered":"2023-11-24T08:05:20.040125+0000","last_unstale":"2023-11-24T08:05:21.029877+0000","last_undegraded":"2023-11-24T08:05:21.029877+0000","last_fullsized":"2023-11-24T08:05:21.029877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:03.008639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488420+0000","last_change":"2023-11-24T08:05:20.014890+0000","last_active":"2023-11-24T08:05:22.488420+0000","last_peered":"2023-11-24T08:05:22.488420+0000","last_clean":"2023-11-24T08:05:22.488420+0000","last_became_active":"2023-11-24T08:05:20.014395+0000","last_became_peered":"2023-11-24T08:05:20.014395+0000","last_unstale":"2023-11-24T08:05:22.488420+0000","last_undegraded":"2023-11-24T08:05:22.488420+0000","last_fullsized":"2023-11-24T08:05:22.488420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:48.665206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212382+0000","last_change":"2023-11-24T08:05:20.044829+0000","last_active":"2023-11-24T08:05:21.212382+0000","last_peered":"2023-11-24T08:05:21.212382+0000","last_clean":"2023-11-24T08:05:21.212382+0000","last_became_active":"2023-11-24T08:05:20.042415+0000","last_became_peered":"2023-11-24T08:05:20.042415+0000","last_unstale":"2023-11-24T08:05:21.212382+0000","last_undegraded":"2023-11-24T08:05:21.212382+0000","last_fullsized":"2023-11-24T08:05:21.212382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:23:08.742886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025421+0000","last_change":"2023-11-24T08:05:20.043475+0000","last_active":"2023-11-24T08:05:21.025421+0000","last_peered":"2023-11-24T08:05:21.025421+0000","last_clean":"2023-11-24T08:05:21.025421+0000","last_became_active":"2023-11-24T08:05:20.043194+0000","last_became_peered":"2023-11-24T08:05:20.043194+0000","last_unstale":"2023-11-24T08:05:21.025421+0000","last_undegraded":"2023-11-24T08:05:21.025421+0000","last_fullsized":"2023-11-24T08:05:21.025421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:31.661335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387046+0000","last_change":"2023-11-24T08:05:20.024517+0000","last_active":"2023-11-24T08:05:22.387046+0000","last_peered":"2023-11-24T08:05:22.387046+0000","last_clean":"2023-11-24T08:05:22.387046+0000","last_became_active":"2023-11-24T08:05:20.024257+0000","last_became_peered":"2023-11-24T08:05:20.024257+0000","last_unstale":"2023-11-24T08:05:22.387046+0000","last_undegraded":"2023-11-24T08:05:22.387046+0000","last_fullsized":"2023-11-24T08:05:22.387046+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:19.298610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.487388+0000","last_change":"2023-11-24T08:05:20.034242+0000","last_active":"2023-11-24T08:05:22.487388+0000","last_peered":"2023-11-24T08:05:22.487388+0000","last_clean":"2023-11-24T08:05:22.487388+0000","last_became_active":"2023-11-24T08:05:20.033879+0000","last_became_peered":"2023-11-24T08:05:20.033879+0000","last_unstale":"2023-11-24T08:05:22.487388+0000","last_undegraded":"2023-11-24T08:05:22.487388+0000","last_fullsized":"2023-11-24T08:05:22.487388+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:12.630573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029125+0000","last_change":"2023-11-24T08:05:20.045362+0000","last_active":"2023-11-24T08:05:21.029125+0000","last_peered":"2023-11-24T08:05:21.029125+0000","last_clean":"2023-11-24T08:05:21.029125+0000","last_became_active":"2023-11-24T08:05:20.045018+0000","last_became_peered":"2023-11-24T08:05:20.045018+0000","last_unstale":"2023-11-24T08:05:21.029125+0000","last_undegraded":"2023-11-24T08:05:21.029125+0000","last_fullsized":"2023-11-24T08:05:21.029125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:58.999070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.028580+0000","last_change":"2023-11-24T08:05:20.034522+0000","last_active":"2023-11-24T08:05:21.028580+0000","last_peered":"2023-11-24T08:05:21.028580+0000","last_clean":"2023-11-24T08:05:21.028580+0000","last_became_active":"2023-11-24T08:05:20.034211+0000","last_became_peered":"2023-11-24T08:05:20.034211+0000","last_unstale":"2023-11-24T08:05:21.028580+0000","last_undegraded":"2023-11-24T08:05:21.028580+0000","last_fullsized":"2023-11-24T08:05:21.028580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:29.978935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.490564+0000","last_change":"2023-11-24T08:05:20.047938+0000","last_active":"2023-11-24T08:05:22.490564+0000","last_peered":"2023-11-24T08:05:22.490564+0000","last_clean":"2023-11-24T08:05:22.490564+0000","last_became_active":"2023-11-24T08:05:20.047676+0000","last_became_peered":"2023-11-24T08:05:20.047676+0000","last_unstale":"2023-11-24T08:05:22.490564+0000","last_undegraded":"2023-11-24T08:05:22.490564+0000","last_fullsized":"2023-11-24T08:05:22.490564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:31.661335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.026026+0000","last_change":"2023-11-24T08:05:20.042230+0000","last_active":"2023-11-24T08:05:21.026026+0000","last_peered":"2023-11-24T08:05:21.026026+0000","last_clean":"2023-11-24T08:05:21.026026+0000","last_became_active":"2023-11-24T08:05:20.041889+0000","last_became_peered":"2023-11-24T08:05:20.041889+0000","last_unstale":"2023-11-24T08:05:21.026026+0000","last_undegraded":"2023-11-24T08:05:21.026026+0000","last_fullsized":"2023-11-24T08:05:21.026026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:11.244829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030788+0000","last_change":"2023-11-24T08:05:20.044547+0000","last_active":"2023-11-24T08:05:21.030788+0000","last_peered":"2023-11-24T08:05:21.030788+0000","last_clean":"2023-11-24T08:05:21.030788+0000","last_became_active":"2023-11-24T08:05:20.044177+0000","last_became_peered":"2023-11-24T08:05:20.044177+0000","last_unstale":"2023-11-24T08:05:21.030788+0000","last_undegraded":"2023-11-24T08:05:21.030788+0000","last_fullsized":"2023-11-24T08:05:21.030788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:59.317581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212287+0000","last_change":"2023-11-24T08:05:20.038371+0000","last_active":"2023-11-24T08:05:21.212287+0000","last_peered":"2023-11-24T08:05:21.212287+0000","last_clean":"2023-11-24T08:05:21.212287+0000","last_became_active":"2023-11-24T08:05:20.037745+0000","last_became_peered":"2023-11-24T08:05:20.037745+0000","last_unstale":"2023-11-24T08:05:21.212287+0000","last_undegraded":"2023-11-24T08:05:21.212287+0000","last_fullsized":"2023-11-24T08:05:21.212287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:58.850425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387237+0000","last_change":"2023-11-24T08:05:20.038931+0000","last_active":"2023-11-24T08:05:22.387237+0000","last_peered":"2023-11-24T08:05:22.387237+0000","last_clean":"2023-11-24T08:05:22.387237+0000","last_became_active":"2023-11-24T08:05:20.037478+0000","last_became_peered":"2023-11-24T08:05:20.037478+0000","last_unstale":"2023-11-24T08:05:22.387237+0000","last_undegraded":"2023-11-24T08:05:22.387237+0000","last_fullsized":"2023-11-24T08:05:22.387237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:27.215645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.487524+0000","last_change":"2023-11-24T08:05:20.013966+0000","last_active":"2023-11-24T08:05:22.487524+0000","last_peered":"2023-11-24T08:05:22.487524+0000","last_clean":"2023-11-24T08:05:22.487524+0000","last_became_active":"2023-11-24T08:05:20.013534+0000","last_became_peered":"2023-11-24T08:05:20.013534+0000","last_unstale":"2023-11-24T08:05:22.487524+0000","last_undegraded":"2023-11-24T08:05:22.487524+0000","last_fullsized":"2023-11-24T08:05:22.487524+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:58.999070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.211739+0000","last_change":"2023-11-24T08:05:20.033505+0000","last_active":"2023-11-24T08:05:21.211739+0000","last_peered":"2023-11-24T08:05:21.211739+0000","last_clean":"2023-11-24T08:05:21.211739+0000","last_became_active":"2023-11-24T08:05:20.033237+0000","last_became_peered":"2023-11-24T08:05:20.033237+0000","last_unstale":"2023-11-24T08:05:21.211739+0000","last_undegraded":"2023-11-24T08:05:21.211739+0000","last_fullsized":"2023-11-24T08:05:21.211739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:31.661335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029415+0000","last_change":"2023-11-24T08:05:20.039494+0000","last_active":"2023-11-24T08:05:21.029415+0000","last_peered":"2023-11-24T08:05:21.029415+0000","last_clean":"2023-11-24T08:05:21.029415+0000","last_became_active":"2023-11-24T08:05:20.039230+0000","last_became_peered":"2023-11-24T08:05:20.039230+0000","last_unstale":"2023-11-24T08:05:21.029415+0000","last_undegraded":"2023-11-24T08:05:21.029415+0000","last_fullsized":"2023-11-24T08:05:21.029415+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:40.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.031287+0000","last_change":"2023-11-24T08:05:20.052638+0000","last_active":"2023-11-24T08:05:21.031287+0000","last_peered":"2023-11-24T08:05:21.031287+0000","last_clean":"2023-11-24T08:05:21.031287+0000","last_became_active":"2023-11-24T08:05:20.050897+0000","last_became_peered":"2023-11-24T08:05:20.050897+0000","last_unstale":"2023-11-24T08:05:21.031287+0000","last_undegraded":"2023-11-24T08:05:21.031287+0000","last_fullsized":"2023-11-24T08:05:21.031287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:48.665206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488439+0000","last_change":"2023-11-24T08:05:20.045198+0000","last_active":"2023-11-24T08:05:22.488439+0000","last_peered":"2023-11-24T08:05:22.488439+0000","last_clean":"2023-11-24T08:05:22.488439+0000","last_became_active":"2023-11-24T08:05:20.044903+0000","last_became_peered":"2023-11-24T08:05:20.044903+0000","last_unstale":"2023-11-24T08:05:22.488439+0000","last_undegraded":"2023-11-24T08:05:22.488439+0000","last_fullsized":"2023-11-24T08:05:22.488439+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:15:34.932650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.487744+0000","last_change":"2023-11-24T08:05:20.046999+0000","last_active":"2023-11-24T08:05:22.487744+0000","last_peered":"2023-11-24T08:05:22.487744+0000","last_clean":"2023-11-24T08:05:22.487744+0000","last_became_active":"2023-11-24T08:05:20.046723+0000","last_became_peered":"2023-11-24T08:05:20.046723+0000","last_unstale":"2023-11-24T08:05:22.487744+0000","last_undegraded":"2023-11-24T08:05:22.487744+0000","last_fullsized":"2023-11-24T08:05:22.487744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:40.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387981+0000","last_change":"2023-11-24T08:05:20.033528+0000","last_active":"2023-11-24T08:05:22.387981+0000","last_peered":"2023-11-24T08:05:22.387981+0000","last_clean":"2023-11-24T08:05:22.387981+0000","last_became_active":"2023-11-24T08:05:20.033275+0000","last_became_peered":"2023-11-24T08:05:20.033275+0000","last_unstale":"2023-11-24T08:05:22.387981+0000","last_undegraded":"2023-11-24T08:05:22.387981+0000","last_fullsized":"2023-11-24T08:05:22.387981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:51.798341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489820+0000","last_change":"2023-11-24T08:05:20.046798+0000","last_active":"2023-11-24T08:05:22.489820+0000","last_peered":"2023-11-24T08:05:22.489820+0000","last_clean":"2023-11-24T08:05:22.489820+0000","last_became_active":"2023-11-24T08:05:20.046474+0000","last_became_peered":"2023-11-24T08:05:20.046474+0000","last_unstale":"2023-11-24T08:05:22.489820+0000","last_undegraded":"2023-11-24T08:05:22.489820+0000","last_fullsized":"2023-11-24T08:05:22.489820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:40.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489618+0000","last_change":"2023-11-24T08:05:20.041572+0000","last_active":"2023-11-24T08:05:22.489618+0000","last_peered":"2023-11-24T08:05:22.489618+0000","last_clean":"2023-11-24T08:05:22.489618+0000","last_became_active":"2023-11-24T08:05:20.041291+0000","last_became_peered":"2023-11-24T08:05:20.041291+0000","last_unstale":"2023-11-24T08:05:22.489618+0000","last_undegraded":"2023-11-24T08:05:22.489618+0000","last_fullsized":"2023-11-24T08:05:22.489618+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:26.095937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069200+0000","last_change":"2023-11-24T08:05:20.032505+0000","last_active":"2023-11-24T08:05:22.069200+0000","last_peered":"2023-11-24T08:05:22.069200+0000","last_clean":"2023-11-24T08:05:22.069200+0000","last_became_active":"2023-11-24T08:05:20.031170+0000","last_became_peered":"2023-11-24T08:05:20.031170+0000","last_unstale":"2023-11-24T08:05:22.069200+0000","last_undegraded":"2023-11-24T08:05:22.069200+0000","last_fullsized":"2023-11-24T08:05:22.069200+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:26.095937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.211707+0000","last_change":"2023-11-24T08:05:20.032978+0000","last_active":"2023-11-24T08:05:21.211707+0000","last_peered":"2023-11-24T08:05:21.211707+0000","last_clean":"2023-11-24T08:05:21.211707+0000","last_became_active":"2023-11-24T08:05:20.032546+0000","last_became_peered":"2023-11-24T08:05:20.032546+0000","last_unstale":"2023-11-24T08:05:21.211707+0000","last_undegraded":"2023-11-24T08:05:21.211707+0000","last_fullsized":"2023-11-24T08:05:21.211707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:00:29.933202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025801+0000","last_change":"2023-11-24T08:05:20.038780+0000","last_active":"2023-11-24T08:05:21.025801+0000","last_peered":"2023-11-24T08:05:21.025801+0000","last_clean":"2023-11-24T08:05:21.025801+0000","last_became_active":"2023-11-24T08:05:20.038486+0000","last_became_peered":"2023-11-24T08:05:20.038486+0000","last_unstale":"2023-11-24T08:05:21.025801+0000","last_undegraded":"2023-11-24T08:05:21.025801+0000","last_fullsized":"2023-11-24T08:05:21.025801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:58.850425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488792+0000","last_change":"2023-11-24T08:05:20.034907+0000","last_active":"2023-11-24T08:05:22.488792+0000","last_peered":"2023-11-24T08:05:22.488792+0000","last_clean":"2023-11-24T08:05:22.488792+0000","last_became_active":"2023-11-24T08:05:20.034618+0000","last_became_peered":"2023-11-24T08:05:20.034618+0000","last_unstale":"2023-11-24T08:05:22.488792+0000","last_undegraded":"2023-11-24T08:05:22.488792+0000","last_fullsized":"2023-11-24T08:05:22.488792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:22:20.845895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025604+0000","last_change":"2023-11-24T08:05:20.049538+0000","last_active":"2023-11-24T08:05:21.025604+0000","last_peered":"2023-11-24T08:05:21.025604+0000","last_clean":"2023-11-24T08:05:21.025604+0000","last_became_active":"2023-11-24T08:05:20.049259+0000","last_became_peered":"2023-11-24T08:05:20.049259+0000","last_unstale":"2023-11-24T08:05:21.025604+0000","last_undegraded":"2023-11-24T08:05:21.025604+0000","last_fullsized":"2023-11-24T08:05:21.025604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:00:29.933202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212434+0000","last_change":"2023-11-24T08:05:20.040763+0000","last_active":"2023-11-24T08:05:21.212434+0000","last_peered":"2023-11-24T08:05:21.212434+0000","last_clean":"2023-11-24T08:05:21.212434+0000","last_became_active":"2023-11-24T08:05:20.040526+0000","last_became_peered":"2023-11-24T08:05:20.040526+0000","last_unstale":"2023-11-24T08:05:21.212434+0000","last_undegraded":"2023-11-24T08:05:21.212434+0000","last_fullsized":"2023-11-24T08:05:21.212434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:12.630573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489848+0000","last_change":"2023-11-24T08:05:20.047203+0000","last_active":"2023-11-24T08:05:22.489848+0000","last_peered":"2023-11-24T08:05:22.489848+0000","last_clean":"2023-11-24T08:05:22.489848+0000","last_became_active":"2023-11-24T08:05:20.046942+0000","last_became_peered":"2023-11-24T08:05:20.046942+0000","last_unstale":"2023-11-24T08:05:22.489848+0000","last_undegraded":"2023-11-24T08:05:22.489848+0000","last_fullsized":"2023-11-24T08:05:22.489848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:09.824526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029102+0000","last_change":"2023-11-24T08:05:20.048590+0000","last_active":"2023-11-24T08:05:21.029102+0000","last_peered":"2023-11-24T08:05:21.029102+0000","last_clean":"2023-11-24T08:05:21.029102+0000","last_became_active":"2023-11-24T08:05:20.048300+0000","last_became_peered":"2023-11-24T08:05:20.048300+0000","last_unstale":"2023-11-24T08:05:21.029102+0000","last_undegraded":"2023-11-24T08:05:21.029102+0000","last_fullsized":"2023-11-24T08:05:21.029102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:12.630573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030245+0000","last_change":"2023-11-24T08:05:20.027198+0000","last_active":"2023-11-24T08:05:21.030245+0000","last_peered":"2023-11-24T08:05:21.030245+0000","last_clean":"2023-11-24T08:05:21.030245+0000","last_became_active":"2023-11-24T08:05:20.026798+0000","last_became_peered":"2023-11-24T08:05:20.026798+0000","last_unstale":"2023-11-24T08:05:21.030245+0000","last_undegraded":"2023-11-24T08:05:21.030245+0000","last_fullsized":"2023-11-24T08:05:21.030245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:26.095937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.028603+0000","last_change":"2023-11-24T08:05:20.049045+0000","last_active":"2023-11-24T08:05:21.028603+0000","last_peered":"2023-11-24T08:05:21.028603+0000","last_clean":"2023-11-24T08:05:21.028603+0000","last_became_active":"2023-11-24T08:05:20.048771+0000","last_became_peered":"2023-11-24T08:05:20.048771+0000","last_unstale":"2023-11-24T08:05:21.028603+0000","last_undegraded":"2023-11-24T08:05:21.028603+0000","last_fullsized":"2023-11-24T08:05:21.028603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:58.850425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069578+0000","last_change":"2023-11-24T08:05:20.043487+0000","last_active":"2023-11-24T08:05:22.069578+0000","last_peered":"2023-11-24T08:05:22.069578+0000","last_clean":"2023-11-24T08:05:22.069578+0000","last_became_active":"2023-11-24T08:05:20.041729+0000","last_became_peered":"2023-11-24T08:05:20.041729+0000","last_unstale":"2023-11-24T08:05:22.069578+0000","last_undegraded":"2023-11-24T08:05:22.069578+0000","last_fullsized":"2023-11-24T08:05:22.069578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:07.888449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.048870+0000","last_change":"2023-11-24T08:05:19.034855+0000","last_active":"2023-11-24T08:05:20.048870+0000","last_peered":"2023-11-24T08:05:20.048870+0000","last_clean":"2023-11-24T08:05:20.048870+0000","last_became_active":"2023-11-24T08:05:19.034614+0000","last_became_peered":"2023-11-24T08:05:19.034614+0000","last_unstale":"2023-11-24T08:05:20.048870+0000","last_undegraded":"2023-11-24T08:05:20.048870+0000","last_fullsized":"2023-11-24T08:05:20.048870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:09.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030127+0000","last_change":"2023-11-24T08:05:20.035745+0000","last_active":"2023-11-24T08:05:21.030127+0000","last_peered":"2023-11-24T08:05:21.030127+0000","last_clean":"2023-11-24T08:05:21.030127+0000","last_became_active":"2023-11-24T08:05:20.035379+0000","last_became_peered":"2023-11-24T08:05:20.035379+0000","last_unstale":"2023-11-24T08:05:21.030127+0000","last_undegraded":"2023-11-24T08:05:21.030127+0000","last_fullsized":"2023-11-24T08:05:21.030127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:48.077409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.034128+0000","last_change":"2023-11-24T08:05:19.029832+0000","last_active":"2023-11-24T08:05:20.034128+0000","last_peered":"2023-11-24T08:05:20.034128+0000","last_clean":"2023-11-24T08:05:20.034128+0000","last_became_active":"2023-11-24T08:05:19.028467+0000","last_became_peered":"2023-11-24T08:05:19.028467+0000","last_unstale":"2023-11-24T08:05:20.034128+0000","last_undegraded":"2023-11-24T08:05:20.034128+0000","last_fullsized":"2023-11-24T08:05:20.034128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:49.838563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.490190+0000","last_change":"2023-11-24T08:05:20.040574+0000","last_active":"2023-11-24T08:05:22.490190+0000","last_peered":"2023-11-24T08:05:22.490190+0000","last_clean":"2023-11-24T08:05:22.490190+0000","last_became_active":"2023-11-24T08:05:20.033808+0000","last_became_peered":"2023-11-24T08:05:20.033808+0000","last_unstale":"2023-11-24T08:05:22.490190+0000","last_undegraded":"2023-11-24T08:05:22.490190+0000","last_fullsized":"2023-11-24T08:05:22.490190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:07.888449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.044957+0000","last_change":"2023-11-24T08:05:19.024698+0000","last_active":"2023-11-24T08:05:20.044957+0000","last_peered":"2023-11-24T08:05:20.044957+0000","last_clean":"2023-11-24T08:05:20.044957+0000","last_became_active":"2023-11-24T08:05:19.024477+0000","last_became_peered":"2023-11-24T08:05:19.024477+0000","last_unstale":"2023-11-24T08:05:20.044957+0000","last_undegraded":"2023-11-24T08:05:20.044957+0000","last_fullsized":"2023-11-24T08:05:20.044957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:10.229106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025723+0000","last_change":"2023-11-24T08:05:20.044222+0000","last_active":"2023-11-24T08:05:21.025723+0000","last_peered":"2023-11-24T08:05:21.025723+0000","last_clean":"2023-11-24T08:05:21.025723+0000","last_became_active":"2023-11-24T08:05:20.043968+0000","last_became_peered":"2023-11-24T08:05:20.043968+0000","last_unstale":"2023-11-24T08:05:21.025723+0000","last_undegraded":"2023-11-24T08:05:21.025723+0000","last_fullsized":"2023-11-24T08:05:21.025723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:29.978935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.038698+0000","last_change":"2023-11-24T08:05:19.024566+0000","last_active":"2023-11-24T08:05:20.038698+0000","last_peered":"2023-11-24T08:05:20.038698+0000","last_clean":"2023-11-24T08:05:20.038698+0000","last_became_active":"2023-11-24T08:05:19.024310+0000","last_became_peered":"2023-11-24T08:05:19.024310+0000","last_unstale":"2023-11-24T08:05:20.038698+0000","last_undegraded":"2023-11-24T08:05:20.038698+0000","last_fullsized":"2023-11-24T08:05:20.038698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:32:05.718595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030588+0000","last_change":"2023-11-24T08:05:20.052189+0000","last_active":"2023-11-24T08:05:21.030588+0000","last_peered":"2023-11-24T08:05:21.030588+0000","last_clean":"2023-11-24T08:05:21.030588+0000","last_became_active":"2023-11-24T08:05:20.051875+0000","last_became_peered":"2023-11-24T08:05:20.051875+0000","last_unstale":"2023-11-24T08:05:21.030588+0000","last_undegraded":"2023-11-24T08:05:21.030588+0000","last_fullsized":"2023-11-24T08:05:21.030588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:59.448273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.052012+0000","last_change":"2023-11-24T08:05:19.027924+0000","last_active":"2023-11-24T08:05:20.052012+0000","last_peered":"2023-11-24T08:05:20.052012+0000","last_clean":"2023-11-24T08:05:20.052012+0000","last_became_active":"2023-11-24T08:05:19.023079+0000","last_became_peered":"2023-11-24T08:05:19.023079+0000","last_unstale":"2023-11-24T08:05:20.052012+0000","last_undegraded":"2023-11-24T08:05:20.052012+0000","last_fullsized":"2023-11-24T08:05:20.052012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:53.448732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212127+0000","last_change":"2023-11-24T08:05:20.040082+0000","last_active":"2023-11-24T08:05:21.212127+0000","last_peered":"2023-11-24T08:05:21.212127+0000","last_clean":"2023-11-24T08:05:21.212127+0000","last_became_active":"2023-11-24T08:05:20.039676+0000","last_became_peered":"2023-11-24T08:05:20.039676+0000","last_unstale":"2023-11-24T08:05:21.212127+0000","last_undegraded":"2023-11-24T08:05:21.212127+0000","last_fullsized":"2023-11-24T08:05:21.212127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:11.244829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.041125+0000","last_change":"2023-11-24T08:05:19.028892+0000","last_active":"2023-11-24T08:05:20.041125+0000","last_peered":"2023-11-24T08:05:20.041125+0000","last_clean":"2023-11-24T08:05:20.041125+0000","last_became_active":"2023-11-24T08:05:19.027535+0000","last_became_peered":"2023-11-24T08:05:19.027535+0000","last_unstale":"2023-11-24T08:05:20.041125+0000","last_undegraded":"2023-11-24T08:05:20.041125+0000","last_fullsized":"2023-11-24T08:05:20.041125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:49.838563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489406+0000","last_change":"2023-11-24T08:05:20.043805+0000","last_active":"2023-11-24T08:05:22.489406+0000","last_peered":"2023-11-24T08:05:22.489406+0000","last_clean":"2023-11-24T08:05:22.489406+0000","last_became_active":"2023-11-24T08:05:20.043507+0000","last_became_peered":"2023-11-24T08:05:20.043507+0000","last_unstale":"2023-11-24T08:05:22.489406+0000","last_undegraded":"2023-11-24T08:05:22.489406+0000","last_fullsized":"2023-11-24T08:05:22.489406+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:56.826339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.007232+0000","last_change":"2023-11-24T08:05:19.015336+0000","last_active":"2023-11-24T08:05:20.007232+0000","last_peered":"2023-11-24T08:05:20.007232+0000","last_clean":"2023-11-24T08:05:20.007232+0000","last_became_active":"2023-11-24T08:05:19.014910+0000","last_became_peered":"2023-11-24T08:05:19.014910+0000","last_unstale":"2023-11-24T08:05:20.007232+0000","last_undegraded":"2023-11-24T08:05:20.007232+0000","last_fullsized":"2023-11-24T08:05:20.007232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:30.645612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.024970+0000","last_change":"2023-11-24T08:05:20.043160+0000","last_active":"2023-11-24T08:05:21.024970+0000","last_peered":"2023-11-24T08:05:21.024970+0000","last_clean":"2023-11-24T08:05:21.024970+0000","last_became_active":"2023-11-24T08:05:20.042825+0000","last_became_peered":"2023-11-24T08:05:20.042825+0000","last_unstale":"2023-11-24T08:05:21.024970+0000","last_undegraded":"2023-11-24T08:05:21.024970+0000","last_fullsized":"2023-11-24T08:05:21.024970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:09.824526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.023648+0000","last_change":"2023-11-24T08:05:19.031449+0000","last_active":"2023-11-24T08:05:20.023648+0000","last_peered":"2023-11-24T08:05:20.023648+0000","last_clean":"2023-11-24T08:05:20.023648+0000","last_became_active":"2023-11-24T08:05:19.031114+0000","last_became_peered":"2023-11-24T08:05:19.031114+0000","last_unstale":"2023-11-24T08:05:20.023648+0000","last_undegraded":"2023-11-24T08:05:20.023648+0000","last_fullsized":"2023-11-24T08:05:20.023648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:09.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212507+0000","last_change":"2023-11-24T08:05:20.046213+0000","last_active":"2023-11-24T08:05:21.212507+0000","last_peered":"2023-11-24T08:05:21.212507+0000","last_clean":"2023-11-24T08:05:21.212507+0000","last_became_active":"2023-11-24T08:05:20.045904+0000","last_became_peered":"2023-11-24T08:05:20.045904+0000","last_unstale":"2023-11-24T08:05:21.212507+0000","last_undegraded":"2023-11-24T08:05:21.212507+0000","last_fullsized":"2023-11-24T08:05:21.212507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:58.999070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.011355+0000","last_change":"2023-11-24T08:05:19.028843+0000","last_active":"2023-11-24T08:05:20.011355+0000","last_peered":"2023-11-24T08:05:20.011355+0000","last_clean":"2023-11-24T08:05:20.011355+0000","last_became_active":"2023-11-24T08:05:19.027840+0000","last_became_peered":"2023-11-24T08:05:19.027840+0000","last_unstale":"2023-11-24T08:05:20.011355+0000","last_undegraded":"2023-11-24T08:05:20.011355+0000","last_fullsized":"2023-11-24T08:05:20.011355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:57.834702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488125+0000","last_change":"2023-11-24T08:05:20.031652+0000","last_active":"2023-11-24T08:05:22.488125+0000","last_peered":"2023-11-24T08:05:22.488125+0000","last_clean":"2023-11-24T08:05:22.488125+0000","last_became_active":"2023-11-24T08:05:20.031337+0000","last_became_peered":"2023-11-24T08:05:20.031337+0000","last_unstale":"2023-11-24T08:05:22.488125+0000","last_undegraded":"2023-11-24T08:05:22.488125+0000","last_fullsized":"2023-11-24T08:05:22.488125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:59.317581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.006389+0000","last_change":"2023-11-24T08:05:19.029786+0000","last_active":"2023-11-24T08:05:20.006389+0000","last_peered":"2023-11-24T08:05:20.006389+0000","last_clean":"2023-11-24T08:05:20.006389+0000","last_became_active":"2023-11-24T08:05:19.029469+0000","last_became_peered":"2023-11-24T08:05:19.029469+0000","last_unstale":"2023-11-24T08:05:20.006389+0000","last_undegraded":"2023-11-24T08:05:20.006389+0000","last_fullsized":"2023-11-24T08:05:20.006389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:47.061685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488566+0000","last_change":"2023-11-24T08:05:20.030806+0000","last_active":"2023-11-24T08:05:22.488566+0000","last_peered":"2023-11-24T08:05:22.488566+0000","last_clean":"2023-11-24T08:05:22.488566+0000","last_became_active":"2023-11-24T08:05:20.030520+0000","last_became_peered":"2023-11-24T08:05:20.030520+0000","last_unstale":"2023-11-24T08:05:22.488566+0000","last_undegraded":"2023-11-24T08:05:22.488566+0000","last_fullsized":"2023-11-24T08:05:22.488566+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:36:44.264543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.039244+0000","last_change":"2023-11-24T08:05:19.034418+0000","last_active":"2023-11-24T08:05:20.039244+0000","last_peered":"2023-11-24T08:05:20.039244+0000","last_clean":"2023-11-24T08:05:20.039244+0000","last_became_active":"2023-11-24T08:05:19.033911+0000","last_became_peered":"2023-11-24T08:05:19.033911+0000","last_unstale":"2023-11-24T08:05:20.039244+0000","last_undegraded":"2023-11-24T08:05:20.039244+0000","last_fullsized":"2023-11-24T08:05:20.039244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:25.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212332+0000","last_change":"2023-11-24T08:05:20.049925+0000","last_active":"2023-11-24T08:05:21.212332+0000","last_peered":"2023-11-24T08:05:21.212332+0000","last_clean":"2023-11-24T08:05:21.212332+0000","last_became_active":"2023-11-24T08:05:20.049651+0000","last_became_peered":"2023-11-24T08:05:20.049651+0000","last_unstale":"2023-11-24T08:05:21.212332+0000","last_undegraded":"2023-11-24T08:05:21.212332+0000","last_fullsized":"2023-11-24T08:05:21.212332+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:29.978935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.057654+0000","last_change":"2023-11-24T08:05:19.023626+0000","last_active":"2023-11-24T08:05:20.057654+0000","last_peered":"2023-11-24T08:05:20.057654+0000","last_clean":"2023-11-24T08:05:20.057654+0000","last_became_active":"2023-11-24T08:05:19.023367+0000","last_became_peered":"2023-11-24T08:05:19.023367+0000","last_unstale":"2023-11-24T08:05:20.057654+0000","last_undegraded":"2023-11-24T08:05:20.057654+0000","last_fullsized":"2023-11-24T08:05:20.057654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:28.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025095+0000","last_change":"2023-11-24T08:05:20.042063+0000","last_active":"2023-11-24T08:05:21.025095+0000","last_peered":"2023-11-24T08:05:21.025095+0000","last_clean":"2023-11-24T08:05:21.025095+0000","last_became_active":"2023-11-24T08:05:20.041782+0000","last_became_peered":"2023-11-24T08:05:20.041782+0000","last_unstale":"2023-11-24T08:05:21.025095+0000","last_undegraded":"2023-11-24T08:05:21.025095+0000","last_fullsized":"2023-11-24T08:05:21.025095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:07.888449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.052804+0000","last_change":"2023-11-24T08:05:19.026245+0000","last_active":"2023-11-24T08:05:20.052804+0000","last_peered":"2023-11-24T08:05:20.052804+0000","last_clean":"2023-11-24T08:05:20.052804+0000","last_became_active":"2023-11-24T08:05:19.025943+0000","last_became_peered":"2023-11-24T08:05:19.025943+0000","last_unstale":"2023-11-24T08:05:20.052804+0000","last_undegraded":"2023-11-24T08:05:20.052804+0000","last_fullsized":"2023-11-24T08:05:20.052804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:02.911316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.488064+0000","last_change":"2023-11-24T08:05:20.014733+0000","last_active":"2023-11-24T08:05:22.488064+0000","last_peered":"2023-11-24T08:05:22.488064+0000","last_clean":"2023-11-24T08:05:22.488064+0000","last_became_active":"2023-11-24T08:05:20.013816+0000","last_became_peered":"2023-11-24T08:05:20.013816+0000","last_unstale":"2023-11-24T08:05:22.488064+0000","last_undegraded":"2023-11-24T08:05:22.488064+0000","last_fullsized":"2023-11-24T08:05:22.488064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:59.448273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.025669+0000","last_change":"2023-11-24T08:05:19.027534+0000","last_active":"2023-11-24T08:05:20.025669+0000","last_peered":"2023-11-24T08:05:20.025669+0000","last_clean":"2023-11-24T08:05:20.025669+0000","last_became_active":"2023-11-24T08:05:19.024963+0000","last_became_peered":"2023-11-24T08:05:19.024963+0000","last_unstale":"2023-11-24T08:05:20.025669+0000","last_undegraded":"2023-11-24T08:05:20.025669+0000","last_fullsized":"2023-11-24T08:05:20.025669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:49:41.186235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.029005+0000","last_change":"2023-11-24T08:05:20.051911+0000","last_active":"2023-11-24T08:05:21.029005+0000","last_peered":"2023-11-24T08:05:21.029005+0000","last_clean":"2023-11-24T08:05:21.029005+0000","last_became_active":"2023-11-24T08:05:20.051638+0000","last_became_peered":"2023-11-24T08:05:20.051638+0000","last_unstale":"2023-11-24T08:05:21.029005+0000","last_undegraded":"2023-11-24T08:05:21.029005+0000","last_fullsized":"2023-11-24T08:05:21.029005+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:11.244829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.003756+0000","last_change":"2023-11-24T08:05:19.028331+0000","last_active":"2023-11-24T08:05:20.003756+0000","last_peered":"2023-11-24T08:05:20.003756+0000","last_clean":"2023-11-24T08:05:20.003756+0000","last_became_active":"2023-11-24T08:05:19.028037+0000","last_became_peered":"2023-11-24T08:05:19.028037+0000","last_unstale":"2023-11-24T08:05:20.003756+0000","last_undegraded":"2023-11-24T08:05:20.003756+0000","last_fullsized":"2023-11-24T08:05:20.003756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:01.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030205+0000","last_change":"2023-11-24T08:05:20.043970+0000","last_active":"2023-11-24T08:05:21.030205+0000","last_peered":"2023-11-24T08:05:21.030205+0000","last_clean":"2023-11-24T08:05:21.030205+0000","last_became_active":"2023-11-24T08:05:20.043637+0000","last_became_peered":"2023-11-24T08:05:20.043637+0000","last_unstale":"2023-11-24T08:05:21.030205+0000","last_undegraded":"2023-11-24T08:05:21.030205+0000","last_fullsized":"2023-11-24T08:05:21.030205+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:56.826339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.048620+0000","last_change":"2023-11-24T08:05:19.023564+0000","last_active":"2023-11-24T08:05:20.048620+0000","last_peered":"2023-11-24T08:05:20.048620+0000","last_clean":"2023-11-24T08:05:20.048620+0000","last_became_active":"2023-11-24T08:05:19.022784+0000","last_became_peered":"2023-11-24T08:05:19.022784+0000","last_unstale":"2023-11-24T08:05:20.048620+0000","last_undegraded":"2023-11-24T08:05:20.048620+0000","last_fullsized":"2023-11-24T08:05:20.048620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:55.810615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.490399+0000","last_change":"2023-11-24T08:05:20.043461+0000","last_active":"2023-11-24T08:05:22.490399+0000","last_peered":"2023-11-24T08:05:22.490399+0000","last_clean":"2023-11-24T08:05:22.490399+0000","last_became_active":"2023-11-24T08:05:20.043158+0000","last_became_peered":"2023-11-24T08:05:20.043158+0000","last_unstale":"2023-11-24T08:05:22.490399+0000","last_undegraded":"2023-11-24T08:05:22.490399+0000","last_fullsized":"2023-11-24T08:05:22.490399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:03.221967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.049798+0000","last_change":"2023-11-24T08:05:19.030608+0000","last_active":"2023-11-24T08:05:20.049798+0000","last_peered":"2023-11-24T08:05:20.049798+0000","last_clean":"2023-11-24T08:05:20.049798+0000","last_became_active":"2023-11-24T08:05:19.030359+0000","last_became_peered":"2023-11-24T08:05:19.030359+0000","last_unstale":"2023-11-24T08:05:20.049798+0000","last_undegraded":"2023-11-24T08:05:20.049798+0000","last_fullsized":"2023-11-24T08:05:20.049798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:55.810615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212825+0000","last_change":"2023-11-24T08:05:20.053318+0000","last_active":"2023-11-24T08:05:21.212825+0000","last_peered":"2023-11-24T08:05:21.212825+0000","last_clean":"2023-11-24T08:05:21.212825+0000","last_became_active":"2023-11-24T08:05:20.053051+0000","last_became_peered":"2023-11-24T08:05:20.053051+0000","last_unstale":"2023-11-24T08:05:21.212825+0000","last_undegraded":"2023-11-24T08:05:21.212825+0000","last_fullsized":"2023-11-24T08:05:21.212825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:40.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.043159+0000","last_change":"2023-11-24T08:05:19.027262+0000","last_active":"2023-11-24T08:05:20.043159+0000","last_peered":"2023-11-24T08:05:20.043159+0000","last_clean":"2023-11-24T08:05:20.043159+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:20.043159+0000","last_undegraded":"2023-11-24T08:05:20.043159+0000","last_fullsized":"2023-11-24T08:05:20.043159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:01.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.025356+0000","last_change":"2023-11-24T08:05:20.049308+0000","last_active":"2023-11-24T08:05:21.025356+0000","last_peered":"2023-11-24T08:05:21.025356+0000","last_clean":"2023-11-24T08:05:21.025356+0000","last_became_active":"2023-11-24T08:05:20.048759+0000","last_became_peered":"2023-11-24T08:05:20.048759+0000","last_unstale":"2023-11-24T08:05:21.025356+0000","last_undegraded":"2023-11-24T08:05:21.025356+0000","last_fullsized":"2023-11-24T08:05:21.025356+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:03.221967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.050110+0000","last_change":"2023-11-24T08:05:19.025861+0000","last_active":"2023-11-24T08:05:20.050110+0000","last_peered":"2023-11-24T08:05:20.050110+0000","last_clean":"2023-11-24T08:05:20.050110+0000","last_became_active":"2023-11-24T08:05:19.024918+0000","last_became_peered":"2023-11-24T08:05:19.024918+0000","last_unstale":"2023-11-24T08:05:20.050110+0000","last_undegraded":"2023-11-24T08:05:20.050110+0000","last_fullsized":"2023-11-24T08:05:20.050110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:12.361723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387473+0000","last_change":"2023-11-24T08:05:20.038066+0000","last_active":"2023-11-24T08:05:22.387473+0000","last_peered":"2023-11-24T08:05:22.387473+0000","last_clean":"2023-11-24T08:05:22.387473+0000","last_became_active":"2023-11-24T08:05:20.037794+0000","last_became_peered":"2023-11-24T08:05:20.037794+0000","last_unstale":"2023-11-24T08:05:22.387473+0000","last_undegraded":"2023-11-24T08:05:22.387473+0000","last_fullsized":"2023-11-24T08:05:22.387473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:51.160788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.016802+0000","last_change":"2023-11-24T08:05:19.017985+0000","last_active":"2023-11-24T08:05:20.016802+0000","last_peered":"2023-11-24T08:05:20.016802+0000","last_clean":"2023-11-24T08:05:20.016802+0000","last_became_active":"2023-11-24T08:05:19.017444+0000","last_became_peered":"2023-11-24T08:05:19.017444+0000","last_unstale":"2023-11-24T08:05:20.016802+0000","last_undegraded":"2023-11-24T08:05:20.016802+0000","last_fullsized":"2023-11-24T08:05:20.016802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:04.782054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387581+0000","last_change":"2023-11-24T08:05:20.044447+0000","last_active":"2023-11-24T08:05:22.387581+0000","last_peered":"2023-11-24T08:05:22.387581+0000","last_clean":"2023-11-24T08:05:22.387581+0000","last_became_active":"2023-11-24T08:05:20.044152+0000","last_became_peered":"2023-11-24T08:05:20.044152+0000","last_unstale":"2023-11-24T08:05:22.387581+0000","last_undegraded":"2023-11-24T08:05:22.387581+0000","last_fullsized":"2023-11-24T08:05:22.387581+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:28.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.038408+0000","last_change":"2023-11-24T08:05:19.026334+0000","last_active":"2023-11-24T08:05:20.038408+0000","last_peered":"2023-11-24T08:05:20.038408+0000","last_clean":"2023-11-24T08:05:20.038408+0000","last_became_active":"2023-11-24T08:05:19.025952+0000","last_became_peered":"2023-11-24T08:05:19.025952+0000","last_unstale":"2023-11-24T08:05:20.038408+0000","last_undegraded":"2023-11-24T08:05:20.038408+0000","last_fullsized":"2023-11-24T08:05:20.038408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:00.858708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387380+0000","last_change":"2023-11-24T08:05:20.020123+0000","last_active":"2023-11-24T08:05:22.387380+0000","last_peered":"2023-11-24T08:05:22.387380+0000","last_clean":"2023-11-24T08:05:22.387380+0000","last_became_active":"2023-11-24T08:05:20.019725+0000","last_became_peered":"2023-11-24T08:05:20.019725+0000","last_unstale":"2023-11-24T08:05:22.387380+0000","last_undegraded":"2023-11-24T08:05:22.387380+0000","last_fullsized":"2023-11-24T08:05:22.387380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:54.464456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.016170+0000","last_change":"2023-11-24T08:05:19.034595+0000","last_active":"2023-11-24T08:05:20.016170+0000","last_peered":"2023-11-24T08:05:20.016170+0000","last_clean":"2023-11-24T08:05:20.016170+0000","last_became_active":"2023-11-24T08:05:19.033790+0000","last_became_peered":"2023-11-24T08:05:19.033790+0000","last_unstale":"2023-11-24T08:05:20.016170+0000","last_undegraded":"2023-11-24T08:05:20.016170+0000","last_fullsized":"2023-11-24T08:05:20.016170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:22.374300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387359+0000","last_change":"2023-11-24T08:05:20.022542+0000","last_active":"2023-11-24T08:05:22.387359+0000","last_peered":"2023-11-24T08:05:22.387359+0000","last_clean":"2023-11-24T08:05:22.387359+0000","last_became_active":"2023-11-24T08:05:20.022244+0000","last_became_peered":"2023-11-24T08:05:20.022244+0000","last_unstale":"2023-11-24T08:05:22.387359+0000","last_undegraded":"2023-11-24T08:05:22.387359+0000","last_fullsized":"2023-11-24T08:05:22.387359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:03.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.022210+0000","last_change":"2023-11-24T08:05:19.028570+0000","last_active":"2023-11-24T08:05:20.022210+0000","last_peered":"2023-11-24T08:05:20.022210+0000","last_clean":"2023-11-24T08:05:20.022210+0000","last_became_active":"2023-11-24T08:05:19.028257+0000","last_became_peered":"2023-11-24T08:05:19.028257+0000","last_unstale":"2023-11-24T08:05:20.022210+0000","last_undegraded":"2023-11-24T08:05:20.022210+0000","last_fullsized":"2023-11-24T08:05:20.022210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:50.782618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.028983+0000","last_change":"2023-11-24T08:05:20.044676+0000","last_active":"2023-11-24T08:05:21.028983+0000","last_peered":"2023-11-24T08:05:21.028983+0000","last_clean":"2023-11-24T08:05:21.028983+0000","last_became_active":"2023-11-24T08:05:20.044335+0000","last_became_peered":"2023-11-24T08:05:20.044335+0000","last_unstale":"2023-11-24T08:05:21.028983+0000","last_undegraded":"2023-11-24T08:05:21.028983+0000","last_fullsized":"2023-11-24T08:05:21.028983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:23:08.742886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.018624+0000","last_change":"2023-11-24T08:05:19.032519+0000","last_active":"2023-11-24T08:05:20.018624+0000","last_peered":"2023-11-24T08:05:20.018624+0000","last_clean":"2023-11-24T08:05:20.018624+0000","last_became_active":"2023-11-24T08:05:19.032159+0000","last_became_peered":"2023-11-24T08:05:19.032159+0000","last_unstale":"2023-11-24T08:05:20.018624+0000","last_undegraded":"2023-11-24T08:05:20.018624+0000","last_fullsized":"2023-11-24T08:05:20.018624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:22.374300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.387497+0000","last_change":"2023-11-24T08:05:20.040541+0000","last_active":"2023-11-24T08:05:22.387497+0000","last_peered":"2023-11-24T08:05:22.387497+0000","last_clean":"2023-11-24T08:05:22.387497+0000","last_became_active":"2023-11-24T08:05:20.040010+0000","last_became_peered":"2023-11-24T08:05:20.040010+0000","last_unstale":"2023-11-24T08:05:22.387497+0000","last_undegraded":"2023-11-24T08:05:22.387497+0000","last_fullsized":"2023-11-24T08:05:22.387497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:13.377447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.045270+0000","last_change":"2023-11-24T08:05:19.034899+0000","last_active":"2023-11-24T08:05:20.045270+0000","last_peered":"2023-11-24T08:05:20.045270+0000","last_clean":"2023-11-24T08:05:20.045270+0000","last_became_active":"2023-11-24T08:05:19.034569+0000","last_became_peered":"2023-11-24T08:05:19.034569+0000","last_unstale":"2023-11-24T08:05:20.045270+0000","last_undegraded":"2023-11-24T08:05:20.045270+0000","last_fullsized":"2023-11-24T08:05:20.045270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:25.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.212862+0000","last_change":"2023-11-24T08:05:20.046655+0000","last_active":"2023-11-24T08:05:21.212862+0000","last_peered":"2023-11-24T08:05:21.212862+0000","last_clean":"2023-11-24T08:05:21.212862+0000","last_became_active":"2023-11-24T08:05:20.046208+0000","last_became_peered":"2023-11-24T08:05:20.046208+0000","last_unstale":"2023-11-24T08:05:21.212862+0000","last_undegraded":"2023-11-24T08:05:21.212862+0000","last_fullsized":"2023-11-24T08:05:21.212862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:59.448273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069706+0000","last_change":"2023-11-24T08:05:19.027343+0000","last_active":"2023-11-24T08:05:22.069706+0000","last_peered":"2023-11-24T08:05:22.069706+0000","last_clean":"2023-11-24T08:05:22.069706+0000","last_became_active":"2023-11-24T08:05:19.027023+0000","last_became_peered":"2023-11-24T08:05:19.027023+0000","last_unstale":"2023-11-24T08:05:22.069706+0000","last_undegraded":"2023-11-24T08:05:22.069706+0000","last_fullsized":"2023-11-24T08:05:22.069706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:01.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069399+0000","last_change":"2023-11-24T08:05:20.042900+0000","last_active":"2023-11-24T08:05:22.069399+0000","last_peered":"2023-11-24T08:05:22.069399+0000","last_clean":"2023-11-24T08:05:22.069399+0000","last_became_active":"2023-11-24T08:05:20.042551+0000","last_became_peered":"2023-11-24T08:05:20.042551+0000","last_unstale":"2023-11-24T08:05:22.069399+0000","last_undegraded":"2023-11-24T08:05:22.069399+0000","last_fullsized":"2023-11-24T08:05:22.069399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:40.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.026545+0000","last_change":"2023-11-24T08:05:19.025711+0000","last_active":"2023-11-24T08:05:20.026545+0000","last_peered":"2023-11-24T08:05:20.026545+0000","last_clean":"2023-11-24T08:05:20.026545+0000","last_became_active":"2023-11-24T08:05:19.025421+0000","last_became_peered":"2023-11-24T08:05:19.025421+0000","last_unstale":"2023-11-24T08:05:20.026545+0000","last_undegraded":"2023-11-24T08:05:20.026545+0000","last_fullsized":"2023-11-24T08:05:20.026545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:22.374300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.030708+0000","last_change":"2023-11-24T08:05:20.026311+0000","last_active":"2023-11-24T08:05:21.030708+0000","last_peered":"2023-11-24T08:05:21.030708+0000","last_clean":"2023-11-24T08:05:21.030708+0000","last_became_active":"2023-11-24T08:05:20.025991+0000","last_became_peered":"2023-11-24T08:05:20.025991+0000","last_unstale":"2023-11-24T08:05:21.030708+0000","last_undegraded":"2023-11-24T08:05:21.030708+0000","last_fullsized":"2023-11-24T08:05:21.030708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:40.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.005925+0000","last_change":"2023-11-24T08:05:19.029520+0000","last_active":"2023-11-24T08:05:20.005925+0000","last_peered":"2023-11-24T08:05:20.005925+0000","last_clean":"2023-11-24T08:05:20.005925+0000","last_became_active":"2023-11-24T08:05:19.029227+0000","last_became_peered":"2023-11-24T08:05:19.029227+0000","last_unstale":"2023-11-24T08:05:20.005925+0000","last_undegraded":"2023-11-24T08:05:20.005925+0000","last_fullsized":"2023-11-24T08:05:20.005925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:22.374300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.069510+0000","last_change":"2023-11-24T08:05:20.028443+0000","last_active":"2023-11-24T08:05:22.069510+0000","last_peered":"2023-11-24T08:05:22.069510+0000","last_clean":"2023-11-24T08:05:22.069510+0000","last_became_active":"2023-11-24T08:05:20.028163+0000","last_became_peered":"2023-11-24T08:05:20.028163+0000","last_unstale":"2023-11-24T08:05:22.069510+0000","last_undegraded":"2023-11-24T08:05:22.069510+0000","last_fullsized":"2023-11-24T08:05:22.069510+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:09.824526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.026227+0000","last_change":"2023-11-24T08:05:19.031328+0000","last_active":"2023-11-24T08:05:20.026227+0000","last_peered":"2023-11-24T08:05:20.026227+0000","last_clean":"2023-11-24T08:05:20.026227+0000","last_became_active":"2023-11-24T08:05:19.031046+0000","last_became_peered":"2023-11-24T08:05:19.031046+0000","last_unstale":"2023-11-24T08:05:20.026227+0000","last_undegraded":"2023-11-24T08:05:20.026227+0000","last_fullsized":"2023-11-24T08:05:20.026227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:09.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.487031+0000","last_change":"2023-11-24T08:05:20.038413+0000","last_active":"2023-11-24T08:05:22.487031+0000","last_peered":"2023-11-24T08:05:22.487031+0000","last_clean":"2023-11-24T08:05:22.487031+0000","last_became_active":"2023-11-24T08:05:20.034335+0000","last_became_peered":"2023-11-24T08:05:20.034335+0000","last_unstale":"2023-11-24T08:05:22.487031+0000","last_undegraded":"2023-11-24T08:05:22.487031+0000","last_fullsized":"2023-11-24T08:05:22.487031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:23:08.742886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.022377+0000","last_change":"2023-11-24T08:05:19.030600+0000","last_active":"2023-11-24T08:05:20.022377+0000","last_peered":"2023-11-24T08:05:20.022377+0000","last_clean":"2023-11-24T08:05:20.022377+0000","last_became_active":"2023-11-24T08:05:19.028703+0000","last_became_peered":"2023-11-24T08:05:19.028703+0000","last_unstale":"2023-11-24T08:05:20.022377+0000","last_undegraded":"2023-11-24T08:05:20.022377+0000","last_fullsized":"2023-11-24T08:05:20.022377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:00.858708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:22.489277+0000","last_change":"2023-11-24T08:05:20.039291+0000","last_active":"2023-11-24T08:05:22.489277+0000","last_peered":"2023-11-24T08:05:22.489277+0000","last_clean":"2023-11-24T08:05:22.489277+0000","last_became_active":"2023-11-24T08:05:20.038950+0000","last_became_peered":"2023-11-24T08:05:20.038950+0000","last_unstale":"2023-11-24T08:05:22.489277+0000","last_undegraded":"2023-11-24T08:05:22.489277+0000","last_fullsized":"2023-11-24T08:05:22.489277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:48.077409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.003758+0000","last_change":"2023-11-24T08:05:19.033681+0000","last_active":"2023-11-24T08:05:20.003758+0000","last_peered":"2023-11-24T08:05:20.003758+0000","last_clean":"2023-11-24T08:05:20.003758+0000","last_became_active":"2023-11-24T08:05:19.033420+0000","last_became_peered":"2023-11-24T08:05:19.033420+0000","last_unstale":"2023-11-24T08:05:20.003758+0000","last_undegraded":"2023-11-24T08:05:20.003758+0000","last_fullsized":"2023-11-24T08:05:20.003758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:47.061685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T08:05:21.031354+0000","last_change":"2023-11-24T08:05:20.035316+0000","last_active":"2023-11-24T08:05:21.031354+0000","last_peered":"2023-11-24T08:05:21.031354+0000","last_clean":"2023-11-24T08:05:21.031354+0000","last_became_active":"2023-11-24T08:05:20.035044+0000","last_became_peered":"2023-11-24T08:05:20.035044+0000","last_unstale":"2023-11-24T08:05:21.031354+0000","last_undegraded":"2023-11-24T08:05:21.031354+0000","last_fullsized":"2023-11-24T08:05:21.031354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:18.980078+0000","last_clean_scrub_stamp":"2023-11-24T08:05:18.980078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:15:34.932650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.015217+0000","last_change":"2023-11-24T08:05:19.036326+0000","last_active":"2023-11-24T08:05:20.015217+0000","last_peered":"2023-11-24T08:05:20.015217+0000","last_clean":"2023-11-24T08:05:20.015217+0000","last_became_active":"2023-11-24T08:05:19.035962+0000","last_became_peered":"2023-11-24T08:05:19.035962+0000","last_unstale":"2023-11-24T08:05:20.015217+0000","last_undegraded":"2023-11-24T08:05:20.015217+0000","last_fullsized":"2023-11-24T08:05:20.015217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:05:17.964355+0000","last_clean_scrub_stamp":"2023-11-24T08:05:17.964355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:49:41.186235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T08:05:20.027171+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:05:20.027171+0000","last_peered":"2023-11-24T08:05:20.027171+0000","last_clean":"2023-11-24T08:05:20.027171+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:05:20.027171+0000","last_undegraded":"2023-11-24T08:05:20.027171+0000","last_fullsized":"2023-11-24T08:05:20.027171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":742535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.3839999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.639}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3089999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":1.1559999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":1.272}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":4.0179999999999998,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.508},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.468}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":152693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.377}]},{"osd":1,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.0649999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.2689999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.478}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.823,"15min":1.823},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":6.8129999999999997,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.234},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.879,"15min":1.879},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":9.8569999999999993,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.028},{"interface":"front","average":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":9.9450000000000003,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.502}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":1.296},{"interface":"front","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.331}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":152749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.181}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.095}]},{"osd":2,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.468}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":1.321}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.0269999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":1.7769999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":152181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.2110000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.5289999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.448},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.484}]},{"osd":5,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.337},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.627}]},{"osd":6,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.556},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.7110000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.732},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.677}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":153711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":1.085},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.2709999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.444}]},{"osd":2,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.145},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.5900000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2250000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.349},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":2.2410000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":2.1600000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.528}]},{"osd":7,"last update":"Fri Nov 24 08:04:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.401},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.4910000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":152147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.046},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.91700000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.9969999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.298},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.85299999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.1759999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.3149999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":3.161,"5min":3.161,"15min":3.161},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.3939999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.627},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.3400000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.778},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.6859999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":153711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.105}]},{"osd":2,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.296}]},{"osd":3,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.468},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.98299999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.5760000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.3480000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.165}]},{"osd":7,"last update":"Fri Nov 24 08:04:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.609,"5min":1.609,"15min":1.609},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.7330000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.367}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":741426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.85699999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.286},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.6320000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":1.3959999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.256},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.5109999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":1.089},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.355}]},{"osd":6,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":1.137},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.4630000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:04:47 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":1.585},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":3.371,"5min":3.371,"15min":3.371},"last":1.4359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:05:30.197 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:05:30.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:05:30.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.473+0000 7fe1f29ef640 1 Processor -- start 2023-11-24T08:05:30.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.473+0000 7fe1f29ef640 1 -- start start 2023-11-24T08:05:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.475+0000 7fe1f29ef640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe1ec13cf40 0x7fe1ec06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.475+0000 7fe1f29ef640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 0x7fe1ec13f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.475+0000 7fe1f29ef640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1ec13fc40 0x7fe1ec15daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.475+0000 7fe1f29ef640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ec06dd90 con 0x7fe1ec13fc40 2023-11-24T08:05:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.475+0000 7fe1f29ef640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ec140430 con 0x7fe1ec13cf40 2023-11-24T08:05:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.475+0000 7fe1f29ef640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec071220 con 0x7fe1ec06d910 2023-11-24T08:05:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.475+0000 7fe1f21ee640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1ec13fc40 0x7fe1ec15daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f21ee640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1ec13fc40 0x7fe1ec15daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47270/0 (socket says 172.21.15.64:47270) 2023-11-24T08:05:30.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f21ee640 1 -- 172.21.15.64:0/1624940259 learned_addr learned my addr 172.21.15.64:0/1624940259 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:30.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f11ec640 1 --2- 172.21.15.64:0/1624940259 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 0x7fe1ec13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:30.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f19ed640 1 --2- 172.21.15.64:0/1624940259 >> v2:172.21.15.91:3300/0 conn(0x7fe1ec13cf40 0x7fe1ec06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:30.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f21ee640 1 -- 172.21.15.64:0/1624940259 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 msgr2=0x7fe1ec13f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f21ee640 1 --2- 172.21.15.64:0/1624940259 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 0x7fe1ec13f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f21ee640 1 -- 172.21.15.64:0/1624940259 >> v2:172.21.15.91:3300/0 conn(0x7fe1ec13cf40 msgr2=0x7fe1ec06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f21ee640 1 --2- 172.21.15.64:0/1624940259 >> v2:172.21.15.91:3300/0 conn(0x7fe1ec13cf40 0x7fe1ec06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.476+0000 7fe1f21ee640 1 -- 172.21.15.64:0/1624940259 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec071500 con 0x7fe1ec13fc40 2023-11-24T08:05:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.477+0000 7fe1f21ee640 1 --2- 172.21.15.64:0/1624940259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13fc40 0x7fe1ec15daa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0b2190 tx=0x7fe1e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1378132612b9731 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.478+0000 7fe1f09eb640 1 -- 172.21.15.64:0/1624940259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e402ca70 con 0x7fe1ec13fc40 2023-11-24T08:05:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.478+0000 7fe1f09eb640 1 -- 172.21.15.64:0/1624940259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e402cbd0 con 0x7fe1ec13fc40 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.478+0000 7fe1f09eb640 1 -- 172.21.15.64:0/1624940259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40343f0 con 0x7fe1ec13fc40 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.478+0000 7fe1f29ef640 1 -- 172.21.15.64:0/1624940259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13fc40 msgr2=0x7fe1ec15daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.478+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/1624940259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13fc40 0x7fe1ec15daa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0b2190 tx=0x7fe1e402bed0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.479+0000 7fe1f29ef640 1 -- 172.21.15.64:0/1624940259 shutdown_connections 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.479+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/1624940259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13fc40 0x7fe1ec15daa0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.479+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/1624940259 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 0x7fe1ec13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.479+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/1624940259 >> v2:172.21.15.91:3300/0 conn(0x7fe1ec13cf40 0x7fe1ec06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.479+0000 7fe1f29ef640 1 -- 172.21.15.64:0/1624940259 >> 172.21.15.64:0/1624940259 conn(0x7fe1ec0b0aa0 msgr2=0x7fe1ec0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.479+0000 7fe1f29ef640 1 -- 172.21.15.64:0/1624940259 shutdown_connections 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.479+0000 7fe1f29ef640 1 -- 172.21.15.64:0/1624940259 wait complete. 2023-11-24T08:05:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.480+0000 7fe1f29ef640 1 Processor -- start 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.481+0000 7fe1f29ef640 1 -- start start 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.481+0000 7fe1f29ef640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 0x7fe1ec157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f29ef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13cf40 0x7fe1ec15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f29ef640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1ec13fc40 0x7fe1ec141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f29ef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec0af9a0 con 0x7fe1ec13cf40 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f29ef640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec1406a0 con 0x7fe1ec13fc40 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f29ef640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec140980 con 0x7fe1ec06d910 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f11ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13cf40 0x7fe1ec15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f11ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13cf40 0x7fe1ec15c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47280/0 (socket says 172.21.15.64:47280) 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f11ec640 1 -- 172.21.15.64:0/3613269607 learned_addr learned my addr 172.21.15.64:0/3613269607 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f19ed640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 0x7fe1ec157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f21ee640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1ec13fc40 0x7fe1ec141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f11ec640 1 -- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 msgr2=0x7fe1ec157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f11ec640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 0x7fe1ec157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f11ec640 1 -- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1ec13fc40 msgr2=0x7fe1ec141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f11ec640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1ec13fc40 0x7fe1ec141460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.482+0000 7fe1f11ec640 1 -- 172.21.15.64:0/3613269607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e4009b30 con 0x7fe1ec13cf40 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.483+0000 7fe1f11ec640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13cf40 0x7fe1ec15c150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00d800 tx=0x7fe1dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.483+0000 7fe1e27fc640 1 -- 172.21.15.64:0/3613269607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0096a0 con 0x7fe1ec13cf40 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.483+0000 7fe1e27fc640 1 -- 172.21.15.64:0/3613269607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc009800 con 0x7fe1ec13cf40 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.483+0000 7fe1e27fc640 1 -- 172.21.15.64:0/3613269607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc009b90 con 0x7fe1ec13cf40 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.483+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec140c60 con 0x7fe1ec13cf40 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.483+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec157d30 con 0x7fe1ec13cf40 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.484+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec04e840 con 0x7fe1ec13cf40 2023-11-24T08:05:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.485+0000 7fe1e27fc640 1 -- 172.21.15.64:0/3613269607 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe1dc009cf0 con 0x7fe1ec13cf40 2023-11-24T08:05:30.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.486+0000 7fe1e27fc640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe1d006ae40 0x7fe1d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:30.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.486+0000 7fe1e27fc640 1 -- 172.21.15.64:0/3613269607 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1dc089a60 con 0x7fe1ec13cf40 2023-11-24T08:05:30.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.487+0000 7fe1f19ed640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe1d006ae40 0x7fe1d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:30.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.488+0000 7fe1f19ed640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe1d006ae40 0x7fe1d006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:30.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.490+0000 7fe1e27fc640 1 -- 172.21.15.64:0/3613269607 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1dc054b00 con 0x7fe1ec13cf40 2023-11-24T08:05:30.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.763+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe1ec1439d0 con 0x7fe1ec13cf40 2023-11-24T08:05:30.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.769+0000 7fe1e27fc640 1 -- 172.21.15.64:0/3613269607 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe1dc058120 con 0x7fe1ec13cf40 2023-11-24T08:05:30.771 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-11-24T08:05:30.771 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA== 2023-11-24T08:05:30.771 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow" 2023-11-24T08:05:30.771 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow r" 2023-11-24T08:05:30.771 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:05:30.771 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe1d006ae40 msgr2=0x7fe1d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe1d006ae40 0x7fe1d006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8008040 comp rx=0 tx=0).stop 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13cf40 msgr2=0x7fe1ec15c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13cf40 0x7fe1ec15c150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00d800 tx=0x7fe1dc00dcc0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 shutdown_connections 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe1d006ae40 0x7fe1d006d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe1ec13fc40 0x7fe1ec141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1ec13cf40 0x7fe1ec15c150 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 --2- 172.21.15.64:0/3613269607 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe1ec06d910 0x7fe1ec157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:30.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 >> 172.21.15.64:0/3613269607 conn(0x7fe1ec0b0aa0 msgr2=0x7fe1ec159050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:30.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 shutdown_connections 2023-11-24T08:05:30.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:30.772+0000 7fe1f29ef640 1 -- 172.21.15.64:0/3613269607 wait complete. 2023-11-24T08:05:30.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:31.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.016+0000 7f77f3fff640 1 Processor -- start 2023-11-24T08:05:31.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.016+0000 7f77f3fff640 1 -- start start 2023-11-24T08:05:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.018+0000 7f77f3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f77f4157770 0x7f77f406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.018+0000 7f77f3fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f77f415df80 0x7f77f4070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.018+0000 7f77f3fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f4161670 con 0x7f77f406d080 2023-11-24T08:05:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.018+0000 7f77f3fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f415e410 con 0x7f77f4157770 2023-11-24T08:05:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.018+0000 7f77f3fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f406d500 con 0x7f77f415df80 2023-11-24T08:05:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.018+0000 7f77f27fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f77f406d080 0x7f77f415d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54268/0 (socket says 172.21.15.64:54268) 2023-11-24T08:05:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.018+0000 7f77f27fc640 1 -- 172.21.15.64:0/2504958523 learned_addr learned my addr 172.21.15.64:0/2504958523 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.018+0000 7f77f2ffd640 1 --2- 172.21.15.64:0/2504958523 >> v2:172.21.15.64:3300/0 conn(0x7f77f4157770 0x7f77f406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.019+0000 7f77f37fe640 1 --2- 172.21.15.64:0/2504958523 >> v2:172.21.15.91:3300/0 conn(0x7f77f415df80 0x7f77f4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.019+0000 7f77f2ffd640 1 -- 172.21.15.64:0/2504958523 >> v2:172.21.15.91:3300/0 conn(0x7f77f415df80 msgr2=0x7f77f4070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.019+0000 7f77f2ffd640 1 --2- 172.21.15.64:0/2504958523 >> v2:172.21.15.91:3300/0 conn(0x7f77f415df80 0x7f77f4070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.019+0000 7f77f1ffb640 1 -- 172.21.15.64:0/2504958523 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624069718 0 0) 0x7f77f4161670 con 0x7f77f406d080 2023-11-24T08:05:31.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.019+0000 7f77f2ffd640 1 -- 172.21.15.64:0/2504958523 >> v1:172.21.15.64:6789/0 conn(0x7f77f406d080 legacy=0x7f77f415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.019+0000 7f77f2ffd640 1 -- 172.21.15.64:0/2504958523 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f415e610 con 0x7f77f4157770 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.020+0000 7f77f2ffd640 1 --2- 172.21.15.64:0/2504958523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f4157770 0x7f77f406cb40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f77f406ca90 tx=0x7f77e402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e566d8b370a23bd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.020+0000 7f77f1ffb640 1 -- 172.21.15.64:0/2504958523 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e402cc70 con 0x7f77f4157770 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.020+0000 7f77f1ffb640 1 -- 172.21.15.64:0/2504958523 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e402cdd0 con 0x7f77f4157770 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.020+0000 7f77f1ffb640 1 -- 172.21.15.64:0/2504958523 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4034660 con 0x7f77f4157770 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.021+0000 7f77f3fff640 1 -- 172.21.15.64:0/2504958523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f4157770 msgr2=0x7f77f406cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.021+0000 7f77f3fff640 1 --2- 172.21.15.64:0/2504958523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f4157770 0x7f77f406cb40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f77f406ca90 tx=0x7f77e402bec0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.021+0000 7f77f3fff640 1 -- 172.21.15.64:0/2504958523 shutdown_connections 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.021+0000 7f77f3fff640 1 --2- 172.21.15.64:0/2504958523 >> v2:172.21.15.91:3300/0 conn(0x7f77f415df80 0x7f77f4070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.021+0000 7f77f3fff640 1 --2- 172.21.15.64:0/2504958523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f4157770 0x7f77f406cb40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.021+0000 7f77f3fff640 1 -- 172.21.15.64:0/2504958523 >> 172.21.15.64:0/2504958523 conn(0x7f77f406a860 msgr2=0x7f77f406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:31.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.021+0000 7f77f3fff640 1 -- 172.21.15.64:0/2504958523 shutdown_connections 2023-11-24T08:05:31.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.021+0000 7f77f3fff640 1 -- 172.21.15.64:0/2504958523 wait complete. 2023-11-24T08:05:31.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.023+0000 7f77f3fff640 1 Processor -- start 2023-11-24T08:05:31.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.023+0000 7f77f3fff640 1 -- start start 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f406d080 0x7f77f415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f4157770 0x7f77f4142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f415df80 0x7f77f41446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f3fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4142380 con 0x7f77f406d080 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f3fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4143100 con 0x7f77f415df80 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f3fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41433e0 con 0x7f77f4157770 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f406d080 0x7f77f415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f406d080 0x7f77f415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47296/0 (socket says 172.21.15.64:47296) 2023-11-24T08:05:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f2ffd640 1 -- 172.21.15.64:0/3317757024 learned_addr learned my addr 172.21.15.64:0/3317757024 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f2ffd640 1 -- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f4157770 msgr2=0x7f77f4142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f2ffd640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f4157770 0x7f77f4142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.024+0000 7f77f2ffd640 1 -- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f415df80 msgr2=0x7f77f41446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.025+0000 7f77f2ffd640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f415df80 0x7f77f41446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.025+0000 7f77f2ffd640 1 -- 172.21.15.64:0/3317757024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4009a70 con 0x7f77f406d080 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.025+0000 7f77f2ffd640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f406d080 0x7f77f415cc30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f77e4037040 tx=0x7f77e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.025+0000 7f77db7fe640 1 -- 172.21.15.64:0/3317757024 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4040650 con 0x7f77f406d080 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.025+0000 7f77db7fe640 1 -- 172.21.15.64:0/3317757024 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e40407b0 con 0x7f77f406d080 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.025+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f41436c0 con 0x7f77f406d080 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.025+0000 7f77db7fe640 1 -- 172.21.15.64:0/3317757024 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4040b60 con 0x7f77f406d080 2023-11-24T08:05:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.025+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f4143c80 con 0x7f77f406d080 2023-11-24T08:05:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.027+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f406e240 con 0x7f77f406d080 2023-11-24T08:05:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.028+0000 7f77db7fe640 1 -- 172.21.15.64:0/3317757024 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77e4040cc0 con 0x7f77f406d080 2023-11-24T08:05:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.028+0000 7f77db7fe640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77e006ae40 0x7f77e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.029+0000 7f77db7fe640 1 -- 172.21.15.64:0/3317757024 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77e40ac430 con 0x7f77f406d080 2023-11-24T08:05:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.029+0000 7f77f27fc640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77e006ae40 0x7f77e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.030+0000 7f77f27fc640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77e006ae40 0x7f77e006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f77e8007fd0 tx=0x7f77e8007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.033+0000 7f77db7fe640 1 -- 172.21.15.64:0/3317757024 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77e40774e0 con 0x7f77f406d080 2023-11-24T08:05:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.265+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77f404e840 con 0x7f77f406d080 2023-11-24T08:05:31.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.267+0000 7f77db7fe640 1 -- 172.21.15.64:0/3317757024 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f77e407ab00 con 0x7f77f406d080 2023-11-24T08:05:31.269 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:31.269 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1893080737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":1893080737},{"type":"v1","addr":"172.21.15.91:6837","nonce":1893080737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2845212044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2845212044},{"type":"v1","addr":"172.21.15.91:6835","nonce":2845212044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2382740551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2382740551},{"type":"v1","addr":"172.21.15.64:6833","nonce":2382740551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:05:19.478049+0000","modified":"2023-11-24T08:05:22.529755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6835/2394348469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2394348469},{"type":"v1","addr":"172.21.15.64:6835","nonce":2394348469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:05:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 15 2023-11-24T08:05:31.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.269+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77e006ae40 msgr2=0x7f77e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.269+0000 7f77f3fff640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77e006ae40 0x7f77e006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f77e8007fd0 tx=0x7f77e8007960 comp rx=0 tx=0).stop 2023-11-24T08:05:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.269+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f406d080 msgr2=0x7f77f415cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.269+0000 7f77f3fff640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f406d080 0x7f77f415cc30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f77e4037040 tx=0x7f77e4004a40 comp rx=0 tx=0).stop 2023-11-24T08:05:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.269+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 shutdown_connections 2023-11-24T08:05:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.269+0000 7f77f3fff640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77e006ae40 0x7f77e006d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.270+0000 7f77f3fff640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f415df80 0x7f77f41446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.270+0000 7f77f3fff640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f4157770 0x7f77f4142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.270+0000 7f77f3fff640 1 --2- 172.21.15.64:0/3317757024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f406d080 0x7f77f415cc30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.270+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 >> 172.21.15.64:0/3317757024 conn(0x7f77f406a860 msgr2=0x7f77f4155420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:31.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.270+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 shutdown_connections 2023-11-24T08:05:31.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.270+0000 7f77f3fff640 1 -- 172.21.15.64:0/3317757024 wait complete. 2023-11-24T08:05:31.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:05:19.478049+0000', 'modified': '2023-11-24T08:05:22.529755+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5220}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5220': {'gid': 5220, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.64:6835/2394348469', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 2394348469}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 2394348469}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:05:31.696 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:05:31.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T08:05:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.929+0000 7fca9ffff640 1 Processor -- start 2023-11-24T08:05:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.930+0000 7fca9ffff640 1 -- start start 2023-11-24T08:05:31.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.932+0000 7fca9ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcaa00b2550 0x7fcaa0070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.932+0000 7fca9ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaa00b2930 0x7fcaa0160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.932+0000 7fca9ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa0070a90 con 0x7fcaa00b2d70 2023-11-24T08:05:31.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.932+0000 7fca9ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa0070cf0 con 0x7fcaa00b2930 2023-11-24T08:05:31.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.932+0000 7fca9ffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa0070fd0 con 0x7fcaa00b2550 2023-11-24T08:05:31.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.932+0000 7fca97fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaa00b2930 0x7fcaa0160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.932+0000 7fca9f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcaa00b2d70 0x7fcaa015da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54270/0 (socket says 172.21.15.64:54270) 2023-11-24T08:05:31.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.932+0000 7fca9f7fe640 1 -- 172.21.15.64:0/3316078650 learned_addr learned my addr 172.21.15.64:0/3316078650 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:31.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.933+0000 7fca9effd640 1 --2- 172.21.15.64:0/3316078650 >> v2:172.21.15.91:3300/0 conn(0x7fcaa00b2550 0x7fcaa0070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.933+0000 7fca9e7fc640 1 -- 172.21.15.64:0/3316078650 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995567142 0 0) 0x7fcaa0070a90 con 0x7fcaa00b2d70 2023-11-24T08:05:31.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.933+0000 7fca97fff640 1 -- 172.21.15.64:0/3316078650 >> v2:172.21.15.91:3300/0 conn(0x7fcaa00b2550 msgr2=0x7fcaa0070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.933+0000 7fca97fff640 1 --2- 172.21.15.64:0/3316078650 >> v2:172.21.15.91:3300/0 conn(0x7fcaa00b2550 0x7fcaa0070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.933+0000 7fca97fff640 1 -- 172.21.15.64:0/3316078650 >> v1:172.21.15.64:6789/0 conn(0x7fcaa00b2d70 legacy=0x7fcaa015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.933+0000 7fca97fff640 1 -- 172.21.15.64:0/3316078650 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa00712b0 con 0x7fcaa00b2930 2023-11-24T08:05:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.934+0000 7fca97fff640 1 --2- 172.21.15.64:0/3316078650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2930 0x7fcaa0160ff0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcaa006fdc0 tx=0x7fca8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bb7ce0b71d804 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.934+0000 7fca9e7fc640 1 -- 172.21.15.64:0/3316078650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88031830 con 0x7fcaa00b2930 2023-11-24T08:05:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.934+0000 7fca9e7fc640 1 -- 172.21.15.64:0/3316078650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca88031990 con 0x7fcaa00b2930 2023-11-24T08:05:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.934+0000 7fca9e7fc640 1 -- 172.21.15.64:0/3316078650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88031d20 con 0x7fcaa00b2930 2023-11-24T08:05:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.935+0000 7fca9ffff640 1 -- 172.21.15.64:0/3316078650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2930 msgr2=0x7fcaa0160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.935+0000 7fca9ffff640 1 --2- 172.21.15.64:0/3316078650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2930 0x7fcaa0160ff0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcaa006fdc0 tx=0x7fca8802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.935+0000 7fca9ffff640 1 -- 172.21.15.64:0/3316078650 shutdown_connections 2023-11-24T08:05:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.935+0000 7fca9ffff640 1 --2- 172.21.15.64:0/3316078650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2930 0x7fcaa0160ff0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.935+0000 7fca9ffff640 1 --2- 172.21.15.64:0/3316078650 >> v2:172.21.15.91:3300/0 conn(0x7fcaa00b2550 0x7fcaa0070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.935+0000 7fca9ffff640 1 -- 172.21.15.64:0/3316078650 >> 172.21.15.64:0/3316078650 conn(0x7fcaa00b0840 msgr2=0x7fcaa00b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:31.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.935+0000 7fca9ffff640 1 -- 172.21.15.64:0/3316078650 shutdown_connections 2023-11-24T08:05:31.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.935+0000 7fca9ffff640 1 -- 172.21.15.64:0/3316078650 wait complete. 2023-11-24T08:05:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.937+0000 7fca9ffff640 1 Processor -- start 2023-11-24T08:05:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.937+0000 7fca9ffff640 1 -- start start 2023-11-24T08:05:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.938+0000 7fca9ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaa00b2550 0x7fcaa015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.938+0000 7fca9ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaa00b2930 0x7fcaa014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.938+0000 7fca9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2d70 0x7fcaa015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.938+0000 7fca9ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa014cd10 con 0x7fcaa00b2d70 2023-11-24T08:05:31.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.938+0000 7fca9ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0153fd0 con 0x7fcaa00b2550 2023-11-24T08:05:31.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.938+0000 7fca9ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa015b060 con 0x7fcaa00b2930 2023-11-24T08:05:31.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.938+0000 7fca9f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2d70 0x7fcaa015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2d70 0x7fcaa015ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47318/0 (socket says 172.21.15.64:47318) 2023-11-24T08:05:31.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9f7fe640 1 -- 172.21.15.64:0/1433736685 learned_addr learned my addr 172.21.15.64:0/1433736685 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9effd640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaa00b2550 0x7fcaa015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca97fff640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaa00b2930 0x7fcaa014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9f7fe640 1 -- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaa00b2930 msgr2=0x7fcaa014c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9f7fe640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaa00b2930 0x7fcaa014c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9f7fe640 1 -- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaa00b2550 msgr2=0x7fcaa015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9f7fe640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaa00b2550 0x7fcaa015dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9f7fe640 1 -- 172.21.15.64:0/1433736685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88009400 con 0x7fcaa00b2d70 2023-11-24T08:05:31.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.939+0000 7fca9f7fe640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2d70 0x7fcaa015ab20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fca9800cac0 tx=0x7fca9800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.940+0000 7fca977fe640 1 -- 172.21.15.64:0/1433736685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800ac30 con 0x7fcaa00b2d70 2023-11-24T08:05:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.940+0000 7fca977fe640 1 -- 172.21.15.64:0/1433736685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9800ad90 con 0x7fcaa00b2d70 2023-11-24T08:05:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.940+0000 7fca977fe640 1 -- 172.21.15.64:0/1433736685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98012620 con 0x7fcaa00b2d70 2023-11-24T08:05:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.940+0000 7fca9ffff640 1 -- 172.21.15.64:0/1433736685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa014d0c0 con 0x7fcaa00b2d70 2023-11-24T08:05:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.940+0000 7fca9ffff640 1 -- 172.21.15.64:0/1433736685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa015e5f0 con 0x7fcaa00b2d70 2023-11-24T08:05:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.943+0000 7fca977fe640 1 -- 172.21.15.64:0/1433736685 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca98007070 con 0x7fcaa00b2d70 2023-11-24T08:05:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.944+0000 7fca977fe640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fca8006ae70 0x7fca8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.944+0000 7fca977fe640 1 -- 172.21.15.64:0/1433736685 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca98089610 con 0x7fcaa00b2d70 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.944+0000 7fcaa801f740 1 Processor -- start 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.944+0000 7fcaa801f740 1 -- start start 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.945+0000 7fca9effd640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fca8006ae70 0x7fca8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.945+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 0x556f61276450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.945+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61270d70 0x556f61275270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.945+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f612757b0 0x556f612b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.945+0000 7fcaa801f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f611d4880 con 0x556f612757b0 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.945+0000 7fcaa801f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f611e3ae0 con 0x556f61276070 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.945+0000 7fcaa801f740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f611e11d0 con 0x556f61270d70 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.945+0000 7fca9f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f612757b0 0x556f612b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f612757b0 0x556f612b9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47332/0 (socket says 172.21.15.64:47332) 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9f7fe640 1 -- 172.21.15.64:0/2552995662 learned_addr learned my addr 172.21.15.64:0/2552995662 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca97fff640 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61270d70 0x556f61275270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9effd640 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 0x556f61276450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9effd640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fca8006ae70 0x7fca8006d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fca90006fd0 tx=0x7fca90008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9f7fe640 1 -- 172.21.15.64:0/2552995662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61270d70 msgr2=0x556f61275270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9f7fe640 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61270d70 0x556f61275270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9f7fe640 1 -- 172.21.15.64:0/2552995662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 msgr2=0x556f61276450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9f7fe640 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 0x556f61276450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9f7fe640 1 -- 172.21.15.64:0/2552995662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f611a4860 con 0x556f612757b0 2023-11-24T08:05:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca9f7fe640 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f612757b0 0x556f612b9730 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fca980547e0 tx=0x7fca98057d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e13d8deb9bdc87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.946+0000 7fca94ff9640 1 -- 172.21.15.64:0/2552995662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9806f020 con 0x556f612757b0 2023-11-24T08:05:31.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fca94ff9640 1 -- 172.21.15.64:0/2552995662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca980605e0 con 0x556f612757b0 2023-11-24T08:05:31.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fca94ff9640 1 -- 172.21.15.64:0/2552995662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9805c120 con 0x556f612757b0 2023-11-24T08:05:31.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 -- 172.21.15.64:0/2552995662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f612757b0 msgr2=0x556f612b9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f612757b0 0x556f612b9730 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fca980547e0 tx=0x7fca98057d70 comp rx=0 tx=0).stop 2023-11-24T08:05:31.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 -- 172.21.15.64:0/2552995662 shutdown_connections 2023-11-24T08:05:31.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f612757b0 0x556f612b9730 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61270d70 0x556f61275270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2552995662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 0x556f61276450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 -- 172.21.15.64:0/2552995662 >> 172.21.15.64:0/2552995662 conn(0x556f61237c70 msgr2=0x556f612bbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 -- 172.21.15.64:0/2552995662 shutdown_connections 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.947+0000 7fcaa801f740 1 -- 172.21.15.64:0/2552995662 wait complete. 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.948+0000 7fcaa801f740 1 -- start start 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.948+0000 7fcaa801f740 1 Processor -- start 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.949+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.949+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f612757b0 0x556f612beb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.949+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 0x556f6129b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.949+0000 7fcaa801f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f611dcac0 con 0x556f61270d70 2023-11-24T08:05:31.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.949+0000 7fcaa801f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f611792f0 con 0x556f61276070 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.949+0000 7fcaa801f740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f61196c70 con 0x556f612757b0 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.949+0000 7fcaa801f740 10 client.0 _ll_register_callbacks cb 0x556f6119b3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612942c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47346/0 (socket says 172.21.15.64:47346) 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 -- 172.21.15.64:0/2607904616 learned_addr learned my addr 172.21.15.64:0/2607904616 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca97fff640 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f612757b0 0x556f612beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 -- 172.21.15.64:0/2607904616 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f612757b0 msgr2=0x556f612beb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f612757b0 0x556f612beb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 -- 172.21.15.64:0/2607904616 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 msgr2=0x556f6129b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 0x556f6129b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 -- 172.21.15.64:0/2607904616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa015e5f0 con 0x556f61270d70 2023-11-24T08:05:31.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca97fff640 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f612757b0 0x556f612beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:31.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca9effd640 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612942c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca900146f0 tx=0x7fca90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f017a83681ee4baa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.950+0000 7fca7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:05:31.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fca7dffb640 1 -- 172.21.15.64:0/2607904616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90015960 con 0x556f61270d70 2023-11-24T08:05:31.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 1 -- 172.21.15.64:0/2607904616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f611d5aa0 con 0x556f61270d70 2023-11-24T08:05:31.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fca7dffb640 1 -- 172.21.15.64:0/2607904616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca90015ac0 con 0x556f61270d70 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fca7dffb640 1 -- 172.21.15.64:0/2607904616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca90015c80 con 0x556f61270d70 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 10 client.5424 fetch_fsmap learned FSMap version 15 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 1 -- 172.21.15.64:0/2607904616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x556f611db140 con 0x556f61270d70 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fca7dffb640 1 -- 172.21.15.64:0/2607904616 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f611d5aa0 con 0x556f61270d70 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fca7dffb640 1 -- 172.21.15.64:0/2607904616 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca90028060 con 0x556f61270d70 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 10 client.5424 fetch_fsmap finished waiting for FSMap version 15 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 10 client.5424 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 20 client.5424 populate_metadata read hostname 'smithi064' 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] conn(0x556f61298a50 0x556f612d8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 4 client.5424 mds_command: new command op to 5220 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.951+0000 7fcaa801f740 1 -- 172.21.15.64:0/2607904616 --> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f611d57c0 con 0x556f61298a50 2023-11-24T08:05:31.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.952+0000 7fca97fff640 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] conn(0x556f61298a50 0x556f612d8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.952+0000 7fca97fff640 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] conn(0x556f61298a50 0x556f612d8d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac82f7de62b53ff6 server_cookie=eaa2296cce1641ac in_seq=0 out_seq=0 2023-11-24T08:05:31.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.952+0000 7fca7dffb640 10 client.5424 ms_handle_connect on v2:172.21.15.64:6834/2394348469 2023-11-24T08:05:31.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.953+0000 7fca7dffb640 1 -- 172.21.15.64:0/2607904616 <== mds.0 v2:172.21.15.64:6834/2394348469 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f611d57c0 con 0x556f61298a50 2023-11-24T08:05:31.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.953+0000 7fca7dffb640 10 client.5424 handle_command_reply: tid=0 2023-11-24T08:05:31.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.953+0000 7fcaa801f740 1 client.5424 shutdown 2023-11-24T08:05:31.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.953+0000 7fcaa801f740 1 -- 172.21.15.64:0/2607904616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 msgr2=0x556f612942c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.953+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612942c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca900146f0 tx=0x7fca90004730 comp rx=0 tx=0).stop 2023-11-24T08:05:31.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 1 -- 172.21.15.64:0/2607904616 shutdown_connections 2023-11-24T08:05:31.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] conn(0x556f61298a50 0x556f612d8d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f61276070 0x556f6129b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f612757b0 0x556f612beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 1 --2- 172.21.15.64:0/2607904616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612942c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 1 -- 172.21.15.64:0/2607904616 >> 172.21.15.64:0/2607904616 conn(0x556f61237c70 msgr2=0x556f61276e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:31.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fca7dffb640 0 client.5424 ms_handle_reset on v2:172.21.15.64:6834/2394348469 2023-11-24T08:05:31.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 1 -- 172.21.15.64:0/2607904616 shutdown_connections 2023-11-24T08:05:31.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 1 -- 172.21.15.64:0/2607904616 wait complete. 2023-11-24T08:05:31.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.954+0000 7fcaa801f740 20 client.5424 trim_cache size 0 max 16384 2023-11-24T08:05:31.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.971+0000 7fcaa801f740 1 Processor -- start 2023-11-24T08:05:31.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.971+0000 7fcaa801f740 1 -- start start 2023-11-24T08:05:31.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.972+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612dd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.972+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f612757b0 0x556f6126fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.972+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61276070 0x556f61270310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.972+0000 7fcaa801f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f60f3ff20 con 0x556f61270d70 2023-11-24T08:05:31.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.972+0000 7fcaa801f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f612a2de0 con 0x556f612757b0 2023-11-24T08:05:31.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.972+0000 7fcaa801f740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f612c36e0 con 0x556f61276070 2023-11-24T08:05:31.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612dd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47354/0 (socket says 172.21.15.64:47354) 2023-11-24T08:05:31.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 -- 172.21.15.64:0/1153003846 learned_addr learned my addr 172.21.15.64:0/1153003846 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:31.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 -- 172.21.15.64:0/1153003846 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61276070 msgr2=0x556f61270310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 --2- 172.21.15.64:0/1153003846 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61276070 0x556f61270310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 -- 172.21.15.64:0/1153003846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f612757b0 msgr2=0x556f6126fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 --2- 172.21.15.64:0/1153003846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f612757b0 0x556f6126fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 -- 172.21.15.64:0/1153003846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f61178b80 con 0x556f61270d70 2023-11-24T08:05:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.973+0000 7fca9effd640 1 --2- 172.21.15.64:0/1153003846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612dd590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fca90015fd0 tx=0x7fca9001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50c899c000e061e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fca957fa640 1 -- 172.21.15.64:0/1153003846 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90015760 con 0x556f61270d70 2023-11-24T08:05:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fca957fa640 1 -- 172.21.15.64:0/1153003846 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca900158e0 con 0x556f61270d70 2023-11-24T08:05:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fca957fa640 1 -- 172.21.15.64:0/1153003846 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90021630 con 0x556f61270d70 2023-11-24T08:05:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fcaa801f740 1 -- 172.21.15.64:0/1153003846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 msgr2=0x556f612dd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fcaa801f740 1 --2- 172.21.15.64:0/1153003846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612dd590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fca90015fd0 tx=0x7fca9001b000 comp rx=0 tx=0).stop 2023-11-24T08:05:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fcaa801f740 1 -- 172.21.15.64:0/1153003846 shutdown_connections 2023-11-24T08:05:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fcaa801f740 1 --2- 172.21.15.64:0/1153003846 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61276070 0x556f61270310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fcaa801f740 1 --2- 172.21.15.64:0/1153003846 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f612757b0 0x556f6126fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fcaa801f740 1 --2- 172.21.15.64:0/1153003846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f612dd590 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.974+0000 7fcaa801f740 1 -- 172.21.15.64:0/1153003846 >> 172.21.15.64:0/1153003846 conn(0x556f61237c70 msgr2=0x556f612994e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.975+0000 7fcaa801f740 1 -- 172.21.15.64:0/1153003846 shutdown_connections 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.975+0000 7fcaa801f740 1 -- 172.21.15.64:0/1153003846 wait complete. 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.975+0000 7fcaa801f740 1 -- start start 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.975+0000 7fcaa801f740 1 Processor -- start 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.976+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f6129c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f612757b0 0x556f6129cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fcaa801f740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61276070 0x556f61292930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fcaa801f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f612a2900 con 0x556f61270d70 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fcaa801f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61299890 con 0x556f612757b0 2023-11-24T08:05:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fcaa801f740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f612a0930 con 0x556f61276070 2023-11-24T08:05:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fcaa801f740 10 client.0 _ll_register_callbacks cb 0x556f61277570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:05:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f6129c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f6129c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47358/0 (socket says 172.21.15.64:47358) 2023-11-24T08:05:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 -- 172.21.15.64:0/509563371 learned_addr learned my addr 172.21.15.64:0/509563371 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 -- 172.21.15.64:0/509563371 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61276070 msgr2=0x556f61292930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61276070 0x556f61292930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 -- 172.21.15.64:0/509563371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f612757b0 msgr2=0x556f6129cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f612757b0 0x556f6129cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 -- 172.21.15.64:0/509563371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f611db140 con 0x556f61270d70 2023-11-24T08:05:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca9effd640 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f6129c720 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fca9000f5b0 tx=0x7fca90016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903290968e9d2e75 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.977+0000 7fca7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:05:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fca7dffb640 1 -- 172.21.15.64:0/509563371 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90021630 con 0x556f61270d70 2023-11-24T08:05:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fca7dffb640 1 -- 172.21.15.64:0/509563371 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca90028960 con 0x556f61270d70 2023-11-24T08:05:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 1 -- 172.21.15.64:0/509563371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca980605e0 con 0x556f61270d70 2023-11-24T08:05:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fca7dffb640 1 -- 172.21.15.64:0/509563371 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca90028b20 con 0x556f61270d70 2023-11-24T08:05:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 10 client.5430 fetch_fsmap learned FSMap version 15 2023-11-24T08:05:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 1 -- 172.21.15.64:0/509563371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x556f6119e170 con 0x556f61270d70 2023-11-24T08:05:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fca7dffb640 1 -- 172.21.15.64:0/509563371 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca980605e0 con 0x556f61270d70 2023-11-24T08:05:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fca7dffb640 1 -- 172.21.15.64:0/509563371 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca90028020 con 0x556f61270d70 2023-11-24T08:05:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 10 client.5430 fetch_fsmap finished waiting for FSMap version 15 2023-11-24T08:05:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 10 client.5430 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:05:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 20 client.5430 populate_metadata read hostname 'smithi064' 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] conn(0x556f612bdf80 0x556f61292e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 4 client.5430 mds_command: new command op to 5220 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.978+0000 7fcaa801f740 1 -- 172.21.15.64:0/509563371 --> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556f60edd6c0 con 0x556f612bdf80 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.979+0000 7fca97fff640 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] conn(0x556f612bdf80 0x556f61292e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.979+0000 7fca7dffb640 10 client.5430 ms_handle_connect on v2:172.21.15.64:6834/2394348469 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.979+0000 7fca97fff640 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] conn(0x556f612bdf80 0x556f61292e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed3931b2d6ff9b9 server_cookie=29160966a72ed892 in_seq=0 out_seq=0 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fca7dffb640 1 -- 172.21.15.64:0/509563371 <== mds.0 v2:172.21.15.64:6834/2394348469 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x556f60edd6c0 con 0x556f612bdf80 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fca7dffb640 10 client.5430 handle_command_reply: tid=0 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 client.5430 shutdown 2023-11-24T08:05:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 -- 172.21.15.64:0/509563371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 msgr2=0x556f6129c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f6129c720 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fca9000f5b0 tx=0x7fca90016000 comp rx=0 tx=0).stop 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 -- 172.21.15.64:0/509563371 shutdown_connections 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.64:6834/2394348469,v1:172.21.15.64:6835/2394348469] conn(0x556f612bdf80 0x556f61292e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x556f61276070 0x556f61292930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fca7dffb640 0 client.5430 ms_handle_reset on v2:172.21.15.64:6834/2394348469 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x556f612757b0 0x556f6129cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 --2- 172.21.15.64:0/509563371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f61270d70 0x556f6129c720 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 -- 172.21.15.64:0/509563371 >> 172.21.15.64:0/509563371 conn(0x556f61237c70 msgr2=0x556f612c3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 -- 172.21.15.64:0/509563371 shutdown_connections 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 1 -- 172.21.15.64:0/509563371 wait complete. 2023-11-24T08:05:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.980+0000 7fcaa801f740 20 client.5430 trim_cache size 0 max 16384 2023-11-24T08:05:31.989 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:05:31.989 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918", 2023-11-24T08:05:31.989 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-11-24T08:05:31.989 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5220, 2023-11-24T08:05:31.989 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-11-24T08:05:31.990 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-11-24T08:05:31.990 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-11-24T08:05:31.990 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 11.981211125, 2023-11-24T08:05:31.990 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 15, 2023-11-24T08:05:31.990 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 34, 2023-11-24T08:05:31.990 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 34, 2023-11-24T08:05:31.990 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 21.479976180000001 2023-11-24T08:05:31.990 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 -- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fca8006ae70 msgr2=0x7fca8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fca8006ae70 0x7fca8006d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fca90006fd0 tx=0x7fca90008040 comp rx=0 tx=0).stop 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 -- 172.21.15.64:0/1433736685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2d70 msgr2=0x7fcaa015ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2d70 0x7fcaa015ab20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fca9800cac0 tx=0x7fca9800cf80 comp rx=0 tx=0).stop 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 -- 172.21.15.64:0/1433736685 shutdown_connections 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fca8006ae70 0x7fca8006d330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaa00b2d70 0x7fcaa015ab20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcaa00b2930 0x7fcaa014c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 --2- 172.21.15.64:0/1433736685 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcaa00b2550 0x7fcaa015dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:31.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 -- 172.21.15.64:0/1433736685 >> 172.21.15.64:0/1433736685 conn(0x7fcaa00b0840 msgr2=0x7fcaa015b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:31.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 -- 172.21.15.64:0/1433736685 shutdown_connections 2023-11-24T08:05:31.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:31.981+0000 7fca9ffff640 1 -- 172.21.15.64:0/1433736685 wait complete. 2023-11-24T08:05:32.000 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:05:32.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.221+0000 7f1a428cf640 1 Processor -- start 2023-11-24T08:05:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.221+0000 7f1a428cf640 1 -- start start 2023-11-24T08:05:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.222+0000 7f1a428cf640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1a3c0b39a0 0x7f1a3c071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.222+0000 7f1a428cf640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0abff0 0x7f1a3c071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.223+0000 7f1a428cf640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c13da30 con 0x7f1a3c15a100 2023-11-24T08:05:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.223+0000 7f1a428cf640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c0ac840 con 0x7f1a3c0b39a0 2023-11-24T08:05:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.223+0000 7f1a428cf640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0ac560 con 0x7f1a3c0abff0 2023-11-24T08:05:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.223+0000 7f1a40e6c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0abff0 0x7f1a3c071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.223+0000 7f1a3b7fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1a3c0b39a0 0x7f1a3c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.223+0000 7f1a40e6c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0abff0 0x7f1a3c071670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:58332/0 (socket says 172.21.15.64:58332) 2023-11-24T08:05:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.223+0000 7f1a40e6c640 1 -- 172.21.15.64:0/2016425550 learned_addr learned my addr 172.21.15.64:0/2016425550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.223+0000 7f1a3bfff640 1 --1- 172.21.15.64:0/2016425550 >> v1:172.21.15.91:6789/0 conn(0x7f1a3c15a100 0x7f1a3c0b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:41950/0 (socket says 172.21.15.64:41950) 2023-11-24T08:05:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.224+0000 7f1a40e6c640 1 -- 172.21.15.64:0/2016425550 >> v2:172.21.15.91:3300/0 conn(0x7f1a3c0b39a0 msgr2=0x7f1a3c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.224+0000 7f1a40e6c640 1 --2- 172.21.15.64:0/2016425550 >> v2:172.21.15.91:3300/0 conn(0x7f1a3c0b39a0 0x7f1a3c071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.224+0000 7f1a40e6c640 1 -- 172.21.15.64:0/2016425550 >> v1:172.21.15.91:6789/0 conn(0x7f1a3c15a100 legacy=0x7f1a3c0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.224+0000 7f1a40e6c640 1 -- 172.21.15.64:0/2016425550 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c070650 con 0x7f1a3c0abff0 2023-11-24T08:05:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.226+0000 7f1a40e6c640 1 --2- 172.21.15.64:0/2016425550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0abff0 0x7f1a3c071670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009870 tx=0x7f1a3402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d15f0a3e2af03d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.226+0000 7f1a3affd640 1 -- 172.21.15.64:0/2016425550 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34030840 con 0x7f1a3c0abff0 2023-11-24T08:05:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.226+0000 7f1a3affd640 1 -- 172.21.15.64:0/2016425550 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a340309a0 con 0x7f1a3c0abff0 2023-11-24T08:05:32.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.226+0000 7f1a3affd640 1 -- 172.21.15.64:0/2016425550 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34030d30 con 0x7f1a3c0abff0 2023-11-24T08:05:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.227+0000 7f1a428cf640 1 -- 172.21.15.64:0/2016425550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0abff0 msgr2=0x7f1a3c071670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.227+0000 7f1a428cf640 1 --2- 172.21.15.64:0/2016425550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0abff0 0x7f1a3c071670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009870 tx=0x7f1a3402b980 comp rx=0 tx=0).stop 2023-11-24T08:05:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.227+0000 7f1a428cf640 1 -- 172.21.15.64:0/2016425550 shutdown_connections 2023-11-24T08:05:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.227+0000 7f1a428cf640 1 --2- 172.21.15.64:0/2016425550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0abff0 0x7f1a3c071670 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.227+0000 7f1a428cf640 1 --2- 172.21.15.64:0/2016425550 >> v2:172.21.15.91:3300/0 conn(0x7f1a3c0b39a0 0x7f1a3c071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.227+0000 7f1a428cf640 1 -- 172.21.15.64:0/2016425550 >> 172.21.15.64:0/2016425550 conn(0x7f1a3c0b0d80 msgr2=0x7f1a3c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.228+0000 7f1a428cf640 1 -- 172.21.15.64:0/2016425550 shutdown_connections 2023-11-24T08:05:32.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.228+0000 7f1a428cf640 1 -- 172.21.15.64:0/2016425550 wait complete. 2023-11-24T08:05:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.230+0000 7f1a428cf640 1 Processor -- start 2023-11-24T08:05:32.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.230+0000 7f1a428cf640 1 -- start start 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.231+0000 7f1a428cf640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0abff0 0x7f1a3c0989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.231+0000 7f1a428cf640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0b39a0 0x7f1a3c150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a428cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c15a100 0x7f1a3c154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a428cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c098f20 con 0x7f1a3c15a100 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a428cf640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c14f290 con 0x7f1a3c0abff0 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a428cf640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c1f44f0 con 0x7f1a3c0b39a0 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a40e6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c15a100 0x7f1a3c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a40e6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c15a100 0x7f1a3c154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47362/0 (socket says 172.21.15.64:47362) 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a40e6c640 1 -- 172.21.15.64:0/1790988180 learned_addr learned my addr 172.21.15.64:0/1790988180 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a3bfff640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0abff0 0x7f1a3c0989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a3b7fe640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0b39a0 0x7f1a3c150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.232+0000 7f1a40e6c640 1 -- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0b39a0 msgr2=0x7f1a3c150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a40e6c640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0b39a0 0x7f1a3c150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a40e6c640 1 -- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0abff0 msgr2=0x7f1a3c0989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a40e6c640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0abff0 0x7f1a3c0989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a40e6c640 1 -- 172.21.15.64:0/1790988180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340094d0 con 0x7f1a3c15a100 2023-11-24T08:05:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a40e6c640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c15a100 0x7f1a3c154280 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1a3402b9c0 tx=0x7f1a34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:32.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a38ff9640 1 -- 172.21.15.64:0/1790988180 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34039a90 con 0x7f1a3c15a100 2023-11-24T08:05:32.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c14f530 con 0x7f1a3c15a100 2023-11-24T08:05:32.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a38ff9640 1 -- 172.21.15.64:0/1790988180 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34039bf0 con 0x7f1a3c15a100 2023-11-24T08:05:32.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.233+0000 7f1a38ff9640 1 -- 172.21.15.64:0/1790988180 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a340433f0 con 0x7f1a3c15a100 2023-11-24T08:05:32.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.234+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c14faf0 con 0x7f1a3c15a100 2023-11-24T08:05:32.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.235+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c067960 con 0x7f1a3c15a100 2023-11-24T08:05:32.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.236+0000 7f1a38ff9640 1 -- 172.21.15.64:0/1790988180 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a34042390 con 0x7f1a3c15a100 2023-11-24T08:05:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.237+0000 7f1a38ff9640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a2806ae40 0x7f1a2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.237+0000 7f1a38ff9640 1 -- 172.21.15.64:0/1790988180 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a340ab7a0 con 0x7f1a3c15a100 2023-11-24T08:05:32.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.237+0000 7f1a3bfff640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a2806ae40 0x7f1a2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:32.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.238+0000 7f1a3bfff640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a2806ae40 0x7f1a2806d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0012e0 tx=0x7f1a2c0016f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.242+0000 7f1a38ff9640 1 -- 172.21.15.64:0/1790988180 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a340767e0 con 0x7f1a3c15a100 2023-11-24T08:05:32.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.474+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1a3c14fd10 con 0x7f1a3c15a100 2023-11-24T08:05:32.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.475+0000 7f1a38ff9640 1 -- 172.21.15.64:0/1790988180 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1a34048060 con 0x7f1a3c15a100 2023-11-24T08:05:32.477 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:05:32.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a2806ae40 msgr2=0x7f1a2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:32.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a2806ae40 0x7f1a2806d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0012e0 tx=0x7f1a2c0016f0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c15a100 msgr2=0x7f1a3c154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c15a100 0x7f1a3c154280 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1a3402b9c0 tx=0x7f1a34004730 comp rx=0 tx=0).stop 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 shutdown_connections 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a2806ae40 0x7f1a2806d300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c15a100 0x7f1a3c154280 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c0b39a0 0x7f1a3c150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 --2- 172.21.15.64:0/1790988180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0abff0 0x7f1a3c0989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 >> 172.21.15.64:0/1790988180 conn(0x7f1a3c0b0d80 msgr2=0x7f1a3c13df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.477+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 shutdown_connections 2023-11-24T08:05:32.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:32.478+0000 7f1a428cf640 1 -- 172.21.15.64:0/1790988180 wait complete. 2023-11-24T08:05:32.498 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:05:32.498 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:05:32.498 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T08:05:32.498 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:05:32.498 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:05:32.498 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:05:32.499 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:05:32.499 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:05:32.499 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:05:32.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:32.499 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T08:05:32.556 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42250sec preferred_lft 42250sec 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3c83:ffff:fee3:836f/64 scope link 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::54d4:76ff:fe60:984a/64 scope link 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:05:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether ae:e6:20:dc:18:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:05:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ace6:20ff:fedc:18d9/64 scope link 2023-11-24T08:05:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:32.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:32.561 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:05:32.658 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T08:05:32.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:32.662 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T08:05:32.719 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T08:05:32.719 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:05:32.719 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:05:32.719 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:05:32.719 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:05:32.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:32.720 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:05:32.720 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:05:32.720 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:05:32.720 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:05:32.720 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:05:32.720 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:05:32.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:32.992 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:05:33.060 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:05:33.061 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:05:33.061 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:05:33.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:33.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-11-24T08:05:33.165 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T08:05:33.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:05:33.166 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:05:33.224 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:05:33.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:05:33.225 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:05:33.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:05:33.383 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:05:33.446 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:05:33.446 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:05:33.446 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:05:33.447 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:05:33.447 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:05:33.447 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:05:33.447 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:05:33.447 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:05:33.447 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:05:33.447 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:05:33.447 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:05:33.448 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:05:33.449 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:05:33.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:33.451 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:05:33.507 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:05:33.508 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T08:05:33.515 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:05:33.594 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:05:33.641 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:05:33.641 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:05:33.641 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:05:33.642 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:05:33.642 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:05:33.642 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:05:33.642 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:05:33.642 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:05:33.643 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:05:33.643 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:05:33.645 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:05:33.645 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:05:33.646 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:05:33.647 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:05:33.647 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:05:33.647 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:05:33.647 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:05:33.648 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:05:33.650 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:05:33.651 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:05:33.652 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:05:33.652 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:05:33.652 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:05:33.653 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:05:33.653 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:05:33.654 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:05:33.655 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:05:33.655 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:05:33.655 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:05:33.655 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:05:33.655 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:05:33.656 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:05:33.656 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:05:33.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:33.658 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:05:33.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-11-24T08:05:33.703+0000 7fe6c8a4c480 -1 init, newargv = 0x55e62a4efb30 newargc=15 2023-11-24T08:05:33.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[102438]: starting ceph client 2023-11-24T08:05:33.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[102438]: starting fuse 2023-11-24T08:05:33.744 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T08:05:33.744 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T08:05:33.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:05:33.744 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T08:05:33.744 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T08:05:33.745 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-11-24T08:05:33.746 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T08:05:33.965 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.102438.asok 2023-11-24T08:05:33.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:05:33.982 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.102438.asok 2023-11-24T08:05:33.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:33.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.102438.asok status 2023-11-24T08:05:34.229 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:05:34.229 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T08:05:34.229 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:05:34.229 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:05:34.230 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-11-24T08:05:34.230 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T08:05:34.230 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T08:05:34.230 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "102438", 2023-11-24T08:05:34.230 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T08:05:34.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:34.231 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T08:05:34.231 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T08:05:34.231 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5439, 2023-11-24T08:05:34.231 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T08:05:34.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T08:05:34.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T08:05:34.231 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5439 2023-11-24T08:05:34.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:34.232 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:05:34.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:05:34.232 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:05:34.232 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3965413684 2023-11-24T08:05:34.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:34.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:34.232 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:05:34.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:05:34.233 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:05:34.233 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3965413684 2023-11-24T08:05:34.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:05:34.233 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5439 192.168.0.1:0/3965413684", 2023-11-24T08:05:34.233 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3965413684", 2023-11-24T08:05:34.233 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T08:05:34.234 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 15, 2023-11-24T08:05:34.234 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 34, 2023-11-24T08:05:34.234 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:05:34.234 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T08:05:34.234 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T08:05:34.234 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:05:34.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:34.244 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T08:05:34.244 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T08:05:34.302 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T08:05:34.303 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T08:05:34.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:34.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:05:34.405 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:05:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.626+0000 7f712b59e640 1 Processor -- start 2023-11-24T08:05:34.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.626+0000 7f712b59e640 1 -- start start 2023-11-24T08:05:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.628+0000 7f712b59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c0cf050 0x7f712c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.628+0000 7f712b59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c140900 con 0x7f712c0cd7b0 2023-11-24T08:05:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.628+0000 7f712b59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c15fa70 con 0x7f712c13dfd0 2023-11-24T08:05:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.628+0000 7f712b59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c15c580 con 0x7f712c0cf050 2023-11-24T08:05:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.628+0000 7f712ad9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f712c0cd7b0 0x7f712c0cdc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54282/0 (socket says 172.21.15.64:54282) 2023-11-24T08:05:34.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.628+0000 7f712ad9d640 1 -- 172.21.15.64:0/2520563132 learned_addr learned my addr 172.21.15.64:0/2520563132 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f7129d9b640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c0cf050 0x7f712c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609708393 0 0) 0x7f712c140900 con 0x7f712c0cd7b0 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7114003520 con 0x7f712c0cd7b0 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 889781237 0 0) 0x7f7114003520 con 0x7f712c0cd7b0 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c0cf050 msgr2=0x7f712c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c0cf050 0x7f712c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 >> v1:172.21.15.91:6789/0 conn(0x7f712c13dfd0 legacy=0x7f712c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c15ee60 con 0x7f712c0cd7b0 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f71240019f0 con 0x7f712c0cd7b0 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 >> v1:172.21.15.64:6789/0 conn(0x7f712c0cd7b0 legacy=0x7f712c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.629+0000 7f712959a640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c0cf050 0x7f7114003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712959a640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114003fe0 0x7f71140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712959a640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7114004d00 0x7f7114005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c15c580 con 0x7f7114004d00 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711400bb40 con 0x7f7114003fe0 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711400bd00 con 0x7f712c0cf050 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712ad9d640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7114004d00 0x7f7114005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712a59c640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c0cf050 0x7f7114003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f7129d9b640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114003fe0 0x7f71140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712ad9d640 1 -- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c0cf050 msgr2=0x7f7114003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712ad9d640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c0cf050 0x7f7114003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712ad9d640 1 -- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114003fe0 msgr2=0x7f71140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712ad9d640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114003fe0 0x7f71140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.630+0000 7f712ad9d640 1 -- 172.21.15.64:0/2520563132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7114003520 con 0x7f7114004d00 2023-11-24T08:05:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.631+0000 7f712ad9d640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7114004d00 0x7f7114005180 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f712400a750 tx=0x7f71140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3f7bd0f2b816c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.632+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124034630 con 0x7f7114004d00 2023-11-24T08:05:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.632+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712402f070 con 0x7f7114004d00 2023-11-24T08:05:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.632+0000 7f712959a640 1 -- 172.21.15.64:0/2520563132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71240349d0 con 0x7f7114004d00 2023-11-24T08:05:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.632+0000 7f712b59e640 1 -- 172.21.15.64:0/2520563132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7114004d00 msgr2=0x7f7114005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.632+0000 7f712b59e640 1 --2- 172.21.15.64:0/2520563132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7114004d00 0x7f7114005180 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f712400a750 tx=0x7f71140057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.632+0000 7f712ad9d640 1 -- 172.21.15.64:0/2520563132 reap_dead start 2023-11-24T08:05:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.632+0000 7f712b59e640 1 -- 172.21.15.64:0/2520563132 shutdown_connections 2023-11-24T08:05:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.632+0000 7f712b59e640 1 -- 172.21.15.64:0/2520563132 >> 172.21.15.64:0/2520563132 conn(0x7f712c06a860 msgr2=0x7f712c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.633+0000 7f712b59e640 1 -- 172.21.15.64:0/2520563132 shutdown_connections 2023-11-24T08:05:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.633+0000 7f712b59e640 1 -- 172.21.15.64:0/2520563132 wait complete. 2023-11-24T08:05:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.634+0000 7f712b59e640 1 Processor -- start 2023-11-24T08:05:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.634+0000 7f712b59e640 1 -- start start 2023-11-24T08:05:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.635+0000 7f712b59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114004d00 0x7f712c154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.635+0000 7f712b59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c071980 0x7f712c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.635+0000 7f712b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f712c071de0 0x7f712c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.635+0000 7f712b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0ae340 con 0x7f712c071de0 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.635+0000 7f712b59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c154b90 con 0x7f7114004d00 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.635+0000 7f712b59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0700d0 con 0x7f712c071980 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f712c071de0 0x7f712c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f712c071de0 0x7f712c0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47388/0 (socket says 172.21.15.64:47388) 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712ad9d640 1 -- 172.21.15.64:0/495923668 learned_addr learned my addr 172.21.15.64:0/495923668 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712a59c640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114004d00 0x7f712c154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f7129d9b640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c071980 0x7f712c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712ad9d640 1 -- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c071980 msgr2=0x7f712c0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712ad9d640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c071980 0x7f712c0cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712ad9d640 1 -- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114004d00 msgr2=0x7f712c154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712ad9d640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114004d00 0x7f712c154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f712ad9d640 1 -- 172.21.15.64:0/495923668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0ad6e0 con 0x7f712c071de0 2023-11-24T08:05:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.636+0000 7f7129d9b640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c071980 0x7f712c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.637+0000 7f712ad9d640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f712c071de0 0x7f712c0ce0d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f712400a750 tx=0x7f712c06dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.637+0000 7f711affd640 1 -- 172.21.15.64:0/495923668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124034640 con 0x7f712c071de0 2023-11-24T08:05:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.637+0000 7f711affd640 1 -- 172.21.15.64:0/495923668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71240347a0 con 0x7f712c071de0 2023-11-24T08:05:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.637+0000 7f711affd640 1 -- 172.21.15.64:0/495923668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124034b10 con 0x7f712c071de0 2023-11-24T08:05:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.637+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c0ad9a0 con 0x7f712c071de0 2023-11-24T08:05:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.637+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c0adf60 con 0x7f712c071de0 2023-11-24T08:05:34.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.638+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c0ae480 con 0x7f712c071de0 2023-11-24T08:05:34.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.639+0000 7f711affd640 1 -- 172.21.15.64:0/495923668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7124031030 con 0x7f712c071de0 2023-11-24T08:05:34.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.640+0000 7f711affd640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f710c06ae40 0x7f710c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:34.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.640+0000 7f711affd640 1 -- 172.21.15.64:0/495923668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71240aba90 con 0x7f712c071de0 2023-11-24T08:05:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.641+0000 7f712a59c640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f710c06ae40 0x7f710c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.642+0000 7f712a59c640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f710c06ae40 0x7f710c06d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f711c007fd0 tx=0x7f711c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.645+0000 7f711affd640 1 -- 172.21.15.64:0/495923668 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f712c0ae480 con 0x7f712c071de0 2023-11-24T08:05:34.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.878+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f712c0ae480 con 0x7f712c071de0 2023-11-24T08:05:34.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.879+0000 7f711affd640 1 -- 172.21.15.64:0/495923668 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f712c0ae480 con 0x7f712c071de0 2023-11-24T08:05:34.881 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:05:34.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.881+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f710c06ae40 msgr2=0x7f710c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:34.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.881+0000 7f712b59e640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f710c06ae40 0x7f710c06d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f711c007fd0 tx=0x7f711c009040 comp rx=0 tx=0).stop 2023-11-24T08:05:34.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f712c071de0 msgr2=0x7f712c0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:34.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f712c071de0 0x7f712c0ce0d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f712400a750 tx=0x7f712c06dbe0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 shutdown_connections 2023-11-24T08:05:34.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f710c06ae40 0x7f710c06d300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f712c071de0 0x7f712c0ce0d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f712c071980 0x7f712c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 --2- 172.21.15.64:0/495923668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7114004d00 0x7f712c154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 >> 172.21.15.64:0/495923668 conn(0x7f712c06a860 msgr2=0x7f712c13f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 shutdown_connections 2023-11-24T08:05:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:34.882+0000 7f712b59e640 1 -- 172.21.15.64:0/495923668 wait complete. 2023-11-24T08:05:34.901 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:05:34.901 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:05:34.901 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2023-11-24T08:05:34.901 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:05:34.901 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:05:34.902 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:05:34.902 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:05:34.902 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:05:34.902 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:05:34.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:34.902 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2023-11-24T08:05:34.964 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:05:34.964 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:05:34.964 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:05:34.965 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:34.965 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:05:34.965 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:34.965 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:05:34.966 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:34.966 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2023-11-24T08:05:34.966 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:05:34.966 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:34.967 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2023-11-24T08:05:34.967 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:05:34.967 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:34.967 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:05:34.967 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:34.967 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:05:34.968 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42237sec preferred_lft 42237sec 2023-11-24T08:05:34.968 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link 2023-11-24T08:05:34.968 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:34.968 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:05:34.968 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:05:34.968 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:05:34.969 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:34.969 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::5073:3fff:fef1:71bb/64 scope link 2023-11-24T08:05:34.969 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:34.969 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:05:34.969 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:05:34.969 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::2497:37ff:fe0c:852a/64 scope link 2023-11-24T08:05:34.970 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:34.970 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:05:34.970 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 32:d9:7a:07:6c:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:05:34.970 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::30d9:7aff:fe07:6ce0/64 scope link 2023-11-24T08:05:34.970 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:05:34.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:34.971 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:05:35.067 INFO:teuthology.orchestra.run.smithi091.stdout:1 2023-11-24T08:05:35.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:35.072 DEBUG:teuthology.orchestra.run.smithi091:> route 2023-11-24T08:05:35.130 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2023-11-24T08:05:35.130 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:05:35.130 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:05:35.130 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:05:35.130 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:05:35.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:35.131 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:05:35.131 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:05:35.132 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:05:35.132 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:05:35.132 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:05:35.132 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:05:35.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:35.310 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:05:35.357 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:05:35.357 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:05:35.357 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:05:35.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:35.357 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2023-11-24T08:05:35.487 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T08:05:35.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:05:35.487 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:05:35.538 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:05:35.538 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:05:35.538 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:05:35.587 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:05:35.671 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:05:35.711 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:05:35.711 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:05:35.711 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:05:35.711 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:05:35.712 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:05:35.712 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:05:35.712 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:05:35.712 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:05:35.712 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:05:35.713 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:05:35.713 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:05:35.713 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:05:35.713 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:05:35.713 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:05:35.713 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:05:35.714 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:05:35.714 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:05:35.714 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:05:35.714 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:05:35.714 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:05:35.714 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:05:35.714 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:05:35.715 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:05:35.715 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:05:35.715 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:05:35.715 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:05:35.715 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:05:35.715 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:05:35.715 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:05:35.716 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:05:35.716 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:05:35.716 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:05:35.716 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:05:35.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:35.717 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:05:35.806 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:05:35.806 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T08:05:35.854 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:05:35.928 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:05:35.968 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:05:35.969 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:05:35.970 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:05:35.971 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:05:35.971 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:05:35.971 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:05:35.971 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:05:35.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:35.972 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:05:36.031 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:2023-11-24T08:05:36.030+0000 7f6967bb8480 -1 init, newargv = 0x557d87e36f10 newargc=15 2023-11-24T08:05:36.031 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[95849]: starting ceph client 2023-11-24T08:05:36.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[95849]: starting fuse 2023-11-24T08:05:36.064 INFO:teuthology.orchestra.run.smithi091.stdout:41 2023-11-24T08:05:36.064 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T08:05:36.064 DEBUG:teuthology.orchestra.run.smithi091:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:05:36.064 DEBUG:teuthology.orchestra.run.smithi091:> import glob 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> import re 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> import os 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> def _find_admin_socket(client_name): 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> files = glob.glob(asok_path) 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:05:36.065 DEBUG:teuthology.orchestra.run.smithi091:> # Given a non-glob path, it better be there 2023-11-24T08:05:36.066 DEBUG:teuthology.orchestra.run.smithi091:> if "*" not in asok_path: 2023-11-24T08:05:36.066 DEBUG:teuthology.orchestra.run.smithi091:> assert(len(files) == 1) 2023-11-24T08:05:36.066 DEBUG:teuthology.orchestra.run.smithi091:> return files[0] 2023-11-24T08:05:36.066 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:05:36.066 DEBUG:teuthology.orchestra.run.smithi091:> for f in files: 2023-11-24T08:05:36.067 DEBUG:teuthology.orchestra.run.smithi091:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:05:36.067 DEBUG:teuthology.orchestra.run.smithi091:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:05:36.067 DEBUG:teuthology.orchestra.run.smithi091:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:05:36.067 DEBUG:teuthology.orchestra.run.smithi091:> contents = proc_f.read() 2023-11-24T08:05:36.067 DEBUG:teuthology.orchestra.run.smithi091:> if mountpoint in contents: 2023-11-24T08:05:36.067 DEBUG:teuthology.orchestra.run.smithi091:> return f 2023-11-24T08:05:36.067 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:05:36.068 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:05:36.068 DEBUG:teuthology.orchestra.run.smithi091:> print(_find_admin_socket("client.1")) 2023-11-24T08:05:36.068 DEBUG:teuthology.orchestra.run.smithi091:> ' 2023-11-24T08:05:36.251 INFO:teuthology.orchestra.run.smithi091.stdout:/var/run/ceph/ceph-client.1.95849.asok 2023-11-24T08:05:36.255 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:05:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:05:36.267 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95849.asok 2023-11-24T08:05:36.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:36.267 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95849.asok status 2023-11-24T08:05:36.487 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2023-11-24T08:05:36.487 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": { 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "entity_id": "1", 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "hostname": "smithi091", 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "pid": "95849", 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "root": "/" 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_count": 0, 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_pinned_count": 0, 2023-11-24T08:05:36.488 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 4751, 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: "inst": { 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: "name": { 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "client", 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: "num": 4751 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 10623327 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: } 2023-11-24T08:05:36.489 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 10623327 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "inst_str": "client.4751 192.168.0.1:0/10623327", 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "addr_str": "192.168.0.1:0/10623327", 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "inode_count": 1, 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_epoch": 15, 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch": 34, 2023-11-24T08:05:36.490 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:05:36.491 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklisted": false, 2023-11-24T08:05:36.491 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs" 2023-11-24T08:05:36.491 INFO:teuthology.orchestra.run.smithi091.stdout:} 2023-11-24T08:05:36.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:36.501 DEBUG:teuthology.orchestra.run.smithi091:> stat --file-system '--printf=%T 2023-11-24T08:05:36.501 DEBUG:teuthology.orchestra.run.smithi091:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T08:05:36.550 INFO:teuthology.orchestra.run.smithi091.stdout:fuseblk 2023-11-24T08:05:36.550 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T08:05:36.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:36.551 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:05:36.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:05:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.922+0000 7f00b3581640 1 Processor -- start 2023-11-24T08:05:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.922+0000 7f00b3581640 1 -- start start 2023-11-24T08:05:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.923+0000 7f00b3581640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f00ac144770 0x7f00ac15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.923+0000 7f00b3581640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac0b35a0 0x7f00ac15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.923+0000 7f00b3581640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac15f930 con 0x7f00ac06c9d0 2023-11-24T08:05:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.923+0000 7f00b3581640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0b43b0 con 0x7f00ac144770 2023-11-24T08:05:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.923+0000 7f00b3581640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0b3b10 con 0x7f00ac0b35a0 2023-11-24T08:05:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.923+0000 7f00b257f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f00ac144770 0x7f00ac15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.923+0000 7f00b257f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f00ac144770 0x7f00ac15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47404/0 (socket says 172.21.15.64:47404) 2023-11-24T08:05:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.923+0000 7f00b257f640 1 -- 172.21.15.64:0/1848051466 learned_addr learned my addr 172.21.15.64:0/1848051466 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.924+0000 7f00b1d7e640 1 --2- 172.21.15.64:0/1848051466 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac0b35a0 0x7f00ac15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.924+0000 7f00b257f640 1 -- 172.21.15.64:0/1848051466 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac0b35a0 msgr2=0x7f00ac15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.924+0000 7f00b257f640 1 --2- 172.21.15.64:0/1848051466 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac0b35a0 0x7f00ac15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.924+0000 7f00b257f640 1 -- 172.21.15.64:0/1848051466 >> v1:172.21.15.91:6789/0 conn(0x7f00ac06c9d0 legacy=0x7f00ac06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.924+0000 7f00b257f640 1 -- 172.21.15.64:0/1848051466 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0b3df0 con 0x7f00ac144770 2023-11-24T08:05:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.925+0000 7f00b257f640 1 --2- 172.21.15.64:0/1848051466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac144770 0x7f00ac15eb60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f00a80098a0 tx=0x7f00a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5420bd3a924cff4d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.926+0000 7f00b157d640 1 -- 172.21.15.64:0/1848051466 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8030840 con 0x7f00ac144770 2023-11-24T08:05:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.926+0000 7f00b157d640 1 -- 172.21.15.64:0/1848051466 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a80309a0 con 0x7f00ac144770 2023-11-24T08:05:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.926+0000 7f00b157d640 1 -- 172.21.15.64:0/1848051466 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8030d30 con 0x7f00ac144770 2023-11-24T08:05:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.926+0000 7f00b3581640 1 -- 172.21.15.64:0/1848051466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac144770 msgr2=0x7f00ac15eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.926+0000 7f00b3581640 1 --2- 172.21.15.64:0/1848051466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac144770 0x7f00ac15eb60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f00a80098a0 tx=0x7f00a802b980 comp rx=0 tx=0).stop 2023-11-24T08:05:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.927+0000 7f00b3581640 1 -- 172.21.15.64:0/1848051466 shutdown_connections 2023-11-24T08:05:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.927+0000 7f00b3581640 1 --2- 172.21.15.64:0/1848051466 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac0b35a0 0x7f00ac15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.927+0000 7f00b3581640 1 --2- 172.21.15.64:0/1848051466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac144770 0x7f00ac15eb60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.927+0000 7f00b3581640 1 -- 172.21.15.64:0/1848051466 >> 172.21.15.64:0/1848051466 conn(0x7f00ac0cec10 msgr2=0x7f00ac0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.927+0000 7f00b3581640 1 -- 172.21.15.64:0/1848051466 shutdown_connections 2023-11-24T08:05:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.927+0000 7f00b3581640 1 -- 172.21.15.64:0/1848051466 wait complete. 2023-11-24T08:05:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.929+0000 7f00b3581640 1 Processor -- start 2023-11-24T08:05:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.929+0000 7f00b3581640 1 -- start start 2023-11-24T08:05:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b3581640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00ac06c9d0 0x7f00ac151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b3581640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac0b35a0 0x7f00ac0a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b3581640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac144770 0x7f00ac0aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b3581640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac142670 con 0x7f00ac0b35a0 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b3581640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac142950 con 0x7f00ac06c9d0 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b3581640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac142c30 con 0x7f00ac144770 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b1d7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac0b35a0 0x7f00ac0a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b1d7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac0b35a0 0x7f00ac0a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47408/0 (socket says 172.21.15.64:47408) 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b1d7e640 1 -- 172.21.15.64:0/1764069737 learned_addr learned my addr 172.21.15.64:0/1764069737 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.930+0000 7f00b257f640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00ac06c9d0 0x7f00ac151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.931+0000 7f00b1d7e640 1 -- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac144770 msgr2=0x7f00ac0aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.931+0000 7f00b1d7e640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac144770 0x7f00ac0aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.931+0000 7f00b1d7e640 1 -- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00ac06c9d0 msgr2=0x7f00ac151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.931+0000 7f00b1d7e640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00ac06c9d0 0x7f00ac151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:36.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.931+0000 7f00b1d7e640 1 -- 172.21.15.64:0/1764069737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a80094d0 con 0x7f00ac0b35a0 2023-11-24T08:05:36.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.931+0000 7f00b1d7e640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac0b35a0 0x7f00ac0a8460 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d800 tx=0x7f009c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:36.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.931+0000 7f00a2ffd640 1 -- 172.21.15.64:0/1764069737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c00b6c0 con 0x7f00ac0b35a0 2023-11-24T08:05:36.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.932+0000 7f00a2ffd640 1 -- 172.21.15.64:0/1764069737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c00b820 con 0x7f00ac0b35a0 2023-11-24T08:05:36.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.932+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac142f10 con 0x7f00ac0b35a0 2023-11-24T08:05:36.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.932+0000 7f00a2ffd640 1 -- 172.21.15.64:0/1764069737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c00bbb0 con 0x7f00ac0b35a0 2023-11-24T08:05:36.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.932+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac0a89a0 con 0x7f00ac0b35a0 2023-11-24T08:05:36.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.933+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac04e840 con 0x7f00ac0b35a0 2023-11-24T08:05:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.934+0000 7f00a2ffd640 1 -- 172.21.15.64:0/1764069737 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f009c00bd10 con 0x7f00ac0b35a0 2023-11-24T08:05:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.935+0000 7f00a2ffd640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f009006ae40 0x7f009006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.935+0000 7f00a2ffd640 1 -- 172.21.15.64:0/1764069737 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f009c089b80 con 0x7f00ac0b35a0 2023-11-24T08:05:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.935+0000 7f00b257f640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f009006ae40 0x7f009006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.936+0000 7f00b257f640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f009006ae40 0x7f009006d300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00a8036000 tx=0x7f00a8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:36.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:36.940+0000 7f00a2ffd640 1 -- 172.21.15.64:0/1764069737 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f009c054b90 con 0x7f00ac0b35a0 2023-11-24T08:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.176+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f00ac0b2460 con 0x7f00ac0b35a0 2023-11-24T08:05:37.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.981+0000 7f00a2ffd640 1 -- 172.21.15.64:0/1764069737 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 136+0+0 (secure 0 0 0) 0x7f009c0581b0 con 0x7f00ac0b35a0 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f009006ae40 msgr2=0x7f009006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f009006ae40 0x7f009006d300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00a8036000 tx=0x7f00a8008030 comp rx=0 tx=0).stop 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac0b35a0 msgr2=0x7f00ac0a8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac0b35a0 0x7f00ac0a8460 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d800 tx=0x7f009c00dcc0 comp rx=0 tx=0).stop 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 shutdown_connections 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f009006ae40 0x7f009006d300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00ac144770 0x7f00ac0aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00ac0b35a0 0x7f00ac0a8460 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 --2- 172.21.15.64:0/1764069737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00ac06c9d0 0x7f00ac151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 >> 172.21.15.64:0/1764069737 conn(0x7f00ac0cec10 msgr2=0x7f00ac070110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 shutdown_connections 2023-11-24T08:05:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:37.986+0000 7f00b3581640 1 -- 172.21.15.64:0/1764069737 wait complete. 2023-11-24T08:05:38.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:05:38.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.232+0000 7fad96d1a640 1 Processor -- start 2023-11-24T08:05:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.232+0000 7fad96d1a640 1 -- start start 2023-11-24T08:05:38.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.234+0000 7fad96d1a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fad901449d0 0x7fad900700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.234+0000 7fad96d1a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 0x7fad90145360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.234+0000 7fad96d1a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad90070a70 con 0x7fad90070610 2023-11-24T08:05:38.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.234+0000 7fad96d1a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad90144ed0 con 0x7fad901449d0 2023-11-24T08:05:38.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.234+0000 7fad96d1a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9006d040 con 0x7fad9006c9e0 2023-11-24T08:05:38.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.235+0000 7fad95517640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fad90070610 0x7fad9006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47188/0 (socket says 172.21.15.64:47188) 2023-11-24T08:05:38.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.235+0000 7fad95517640 1 -- 172.21.15.64:0/615417218 learned_addr learned my addr 172.21.15.64:0/615417218 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:38.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.235+0000 7fad96519640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 0x7fad90145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.235+0000 7fad95d18640 1 --2- 172.21.15.64:0/615417218 >> v2:172.21.15.91:3300/0 conn(0x7fad901449d0 0x7fad900700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.235+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430241752 0 0) 0x7fad90070a70 con 0x7fad90070610 2023-11-24T08:05:38.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.235+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad7c003520 con 0x7fad90070610 2023-11-24T08:05:38.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 731008560 0 0) 0x7fad7c003520 con 0x7fad90070610 2023-11-24T08:05:38.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 msgr2=0x7fad90145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:38.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 0x7fad90145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 >> v2:172.21.15.91:3300/0 conn(0x7fad901449d0 msgr2=0x7fad900700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:38.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 --2- 172.21.15.64:0/615417218 >> v2:172.21.15.91:3300/0 conn(0x7fad901449d0 0x7fad900700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9006d320 con 0x7fad90070610 2023-11-24T08:05:38.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fad80001670 con 0x7fad90070610 2023-11-24T08:05:38.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 >> v1:172.21.15.64:6789/0 conn(0x7fad90070610 legacy=0x7fad9006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 0x7fad7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c004d00 0x7fad7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad90144ed0 con 0x7fad7c0039e0 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9006d040 con 0x7fad7c004d00 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c00ba30 con 0x7fad9006c9e0 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.236+0000 7fad95d18640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.237+0000 7fad95517640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 0x7fad7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.237+0000 7fad96519640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c004d00 0x7fad7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.237+0000 7fad95d18640 1 -- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 msgr2=0x7fad7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.237+0000 7fad95d18640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 0x7fad7c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.237+0000 7fad95d18640 1 -- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c004d00 msgr2=0x7fad7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.237+0000 7fad95d18640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad7c004d00 0x7fad7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.237+0000 7fad95d18640 1 -- 172.21.15.64:0/615417218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c003520 con 0x7fad7c0039e0 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.237+0000 7fad95517640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad9006c9e0 0x7fad7c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.238+0000 7fad95d18640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad7c003e00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00ba30 tx=0x7fad7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420732a3c4ff9816 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.238+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c032680 con 0x7fad7c0039e0 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.238+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c0327e0 con 0x7fad7c0039e0 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.238+0000 7fad94d16640 1 -- 172.21.15.64:0/615417218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c032b70 con 0x7fad7c0039e0 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.239+0000 7fad96d1a640 1 -- 172.21.15.64:0/615417218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 msgr2=0x7fad7c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.239+0000 7fad96d1a640 1 --2- 172.21.15.64:0/615417218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad7c003e00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00ba30 tx=0x7fad7c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.239+0000 7fad96519640 1 -- 172.21.15.64:0/615417218 reap_dead start 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.239+0000 7fad96d1a640 1 -- 172.21.15.64:0/615417218 shutdown_connections 2023-11-24T08:05:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.239+0000 7fad96d1a640 1 -- 172.21.15.64:0/615417218 >> 172.21.15.64:0/615417218 conn(0x7fad900b0840 msgr2=0x7fad900b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.239+0000 7fad96d1a640 1 -- 172.21.15.64:0/615417218 shutdown_connections 2023-11-24T08:05:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.239+0000 7fad96d1a640 1 -- 172.21.15.64:0/615417218 wait complete. 2023-11-24T08:05:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.241+0000 7fad96d1a640 1 Processor -- start 2023-11-24T08:05:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.241+0000 7fad96d1a640 1 -- start start 2023-11-24T08:05:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad96d1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad90156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad96d1a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9015e060 0x7fad900af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad96d1a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad900af5d0 0x7fad9015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad96d1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad900afa50 con 0x7fad7c0039e0 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad96d1a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad90156c00 con 0x7fad9015e060 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad96d1a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad90156ee0 con 0x7fad900af5d0 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad95d18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad90156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad95d18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad90156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36460/0 (socket says 172.21.15.64:36460) 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.242+0000 7fad95d18640 1 -- 172.21.15.64:0/3822629050 learned_addr learned my addr 172.21.15.64:0/3822629050 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad95d18640 1 -- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad900af5d0 msgr2=0x7fad9015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad96519640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad900af5d0 0x7fad9015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad95d18640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad900af5d0 0x7fad9015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad95517640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9015e060 0x7fad900af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad95d18640 1 -- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9015e060 msgr2=0x7fad900af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad95d18640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9015e060 0x7fad900af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad95d18640 1 -- 172.21.15.64:0/3822629050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad901571c0 con 0x7fad7c0039e0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad95d18640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad90156300 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fad8c02bf00 tx=0x7fad8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad867fc640 1 -- 172.21.15.64:0/3822629050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c004220 con 0x7fad7c0039e0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad867fc640 1 -- 172.21.15.64:0/3822629050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c0043a0 con 0x7fad7c0039e0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad867fc640 1 -- 172.21.15.64:0/3822629050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c030aa0 con 0x7fad7c0039e0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.243+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad901574a0 con 0x7fad7c0039e0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.244+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad90156840 con 0x7fad7c0039e0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.245+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad901569d0 con 0x7fad7c0039e0 2023-11-24T08:05:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.246+0000 7fad867fc640 1 -- 172.21.15.64:0/3822629050 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad8c030c00 con 0x7fad7c0039e0 2023-11-24T08:05:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.246+0000 7fad867fc640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad7406ae70 0x7fad7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.247+0000 7fad867fc640 1 -- 172.21.15.64:0/3822629050 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad8c0ac060 con 0x7fad7c0039e0 2023-11-24T08:05:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.247+0000 7fad95517640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad7406ae70 0x7fad7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.248+0000 7fad95517640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad7406ae70 0x7fad7406d330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fad800018e0 tx=0x7fad80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.251+0000 7fad867fc640 1 -- 172.21.15.64:0/3822629050 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad8c0dee90 con 0x7fad7c0039e0 2023-11-24T08:05:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.465+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fad9006d8e0 con 0x7fad7406ae70 2023-11-24T08:05:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.469+0000 7fad867fc640 1 -- 172.21.15.64:0/3822629050 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fad8000d040 con 0x7fad7406ae70 2023-11-24T08:05:38.470 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:05:38.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:05:38.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-11-24T08:05:38.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:05:38.471 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:05:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.471+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad7406ae70 msgr2=0x7fad7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.471+0000 7fad96d1a640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad7406ae70 0x7fad7406d330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fad800018e0 tx=0x7fad80009040 comp rx=0 tx=0).stop 2023-11-24T08:05:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.471+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 msgr2=0x7fad90156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.471+0000 7fad96d1a640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad90156300 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fad8c02bf00 tx=0x7fad8c004730 comp rx=0 tx=0).stop 2023-11-24T08:05:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.472+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 shutdown_connections 2023-11-24T08:05:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.472+0000 7fad96d1a640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad7406ae70 0x7fad7406d330 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.472+0000 7fad96d1a640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad900af5d0 0x7fad9015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.472+0000 7fad96d1a640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad9015e060 0x7fad900af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.472+0000 7fad96d1a640 1 --2- 172.21.15.64:0/3822629050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad7c0039e0 0x7fad90156300 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.472+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 >> 172.21.15.64:0/3822629050 conn(0x7fad900b0840 msgr2=0x7fad9015bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.472+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 shutdown_connections 2023-11-24T08:05:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.472+0000 7fad96d1a640 1 -- 172.21.15.64:0/3822629050 wait complete. 2023-11-24T08:05:38.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:05:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.712+0000 7fd7e6256640 1 Processor -- start 2023-11-24T08:05:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.713+0000 7fd7e6256640 1 -- start start 2023-11-24T08:05:38.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.714+0000 7fd7e6256640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd7e00b3210 0x7fd7e00b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.714+0000 7fd7e6256640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e00b23e0 con 0x7fd7e0160bd0 2023-11-24T08:05:38.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.714+0000 7fd7e6256640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e00d41b0 con 0x7fd7e00b48d0 2023-11-24T08:05:38.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.714+0000 7fd7e6256640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e00cdc70 con 0x7fd7e00b3210 2023-11-24T08:05:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.715+0000 7fd7df7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd7e0160bd0 0x7fd7e00b2b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47204/0 (socket says 172.21.15.64:47204) 2023-11-24T08:05:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.715+0000 7fd7df7fe640 1 -- 172.21.15.64:0/4258224967 learned_addr learned my addr 172.21.15.64:0/4258224967 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.715+0000 7fd7dffff640 1 --2- 172.21.15.64:0/4258224967 >> v2:172.21.15.64:3300/0 conn(0x7fd7e00b3210 0x7fd7e00b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.715+0000 7fd7de7fc640 1 -- 172.21.15.64:0/4258224967 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844273717 0 0) 0x7fd7e00b23e0 con 0x7fd7e0160bd0 2023-11-24T08:05:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.715+0000 7fd7de7fc640 1 -- 172.21.15.64:0/4258224967 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7cc003520 con 0x7fd7e0160bd0 2023-11-24T08:05:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.715+0000 7fd7dffff640 1 -- 172.21.15.64:0/4258224967 >> v1:172.21.15.91:6789/0 conn(0x7fd7e00b48d0 legacy=0x7fd7e0165280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.715+0000 7fd7dffff640 1 -- 172.21.15.64:0/4258224967 >> v1:172.21.15.64:6789/0 conn(0x7fd7e0160bd0 legacy=0x7fd7e00b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.715+0000 7fd7dffff640 1 -- 172.21.15.64:0/4258224967 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e0160fb0 con 0x7fd7e00b3210 2023-11-24T08:05:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.716+0000 7fd7dffff640 1 --2- 172.21.15.64:0/4258224967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b3210 0x7fd7e00b1ea0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00cddb0 tx=0x7fd7d802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3187637e9a726d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7de7fc640 1 -- 172.21.15.64:0/4258224967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d802cbc0 con 0x7fd7e00b3210 2023-11-24T08:05:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7de7fc640 1 -- 172.21.15.64:0/4258224967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d802cd20 con 0x7fd7e00b3210 2023-11-24T08:05:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7de7fc640 1 -- 172.21.15.64:0/4258224967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8034660 con 0x7fd7e00b3210 2023-11-24T08:05:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7e6256640 1 -- 172.21.15.64:0/4258224967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b3210 msgr2=0x7fd7e00b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7e6256640 1 --2- 172.21.15.64:0/4258224967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b3210 0x7fd7e00b1ea0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00cddb0 tx=0x7fd7d802be10 comp rx=0 tx=0).stop 2023-11-24T08:05:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7e6256640 1 -- 172.21.15.64:0/4258224967 shutdown_connections 2023-11-24T08:05:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7e6256640 1 --2- 172.21.15.64:0/4258224967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b3210 0x7fd7e00b1ea0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7e6256640 1 -- 172.21.15.64:0/4258224967 >> 172.21.15.64:0/4258224967 conn(0x7fd7e009c5a0 msgr2=0x7fd7e00d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7e6256640 1 -- 172.21.15.64:0/4258224967 shutdown_connections 2023-11-24T08:05:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.717+0000 7fd7e6256640 1 -- 172.21.15.64:0/4258224967 wait complete. 2023-11-24T08:05:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.719+0000 7fd7e6256640 1 Processor -- start 2023-11-24T08:05:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.719+0000 7fd7e6256640 1 -- start start 2023-11-24T08:05:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7e6256640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7e00b3210 0x7fd7e00b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7e6256640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b48d0 0x7fd7e00b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7e6256640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7e0160bd0 0x7fd7e00b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7e6256640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e015bf20 con 0x7fd7e00b48d0 2023-11-24T08:05:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7e6256640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e015b3a0 con 0x7fd7e0160bd0 2023-11-24T08:05:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7e6256640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e015b680 con 0x7fd7e00b3210 2023-11-24T08:05:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7deffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b48d0 0x7fd7e00b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7deffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b48d0 0x7fd7e00b7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36482/0 (socket says 172.21.15.64:36482) 2023-11-24T08:05:38.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.720+0000 7fd7deffd640 1 -- 172.21.15.64:0/1841061757 learned_addr learned my addr 172.21.15.64:0/1841061757 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:38.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7df7fe640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7e00b3210 0x7fd7e00b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7dffff640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7e0160bd0 0x7fd7e00b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7deffd640 1 -- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7e00b3210 msgr2=0x7fd7e00b89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7deffd640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7e00b3210 0x7fd7e00b89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7deffd640 1 -- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7e0160bd0 msgr2=0x7fd7e00b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7deffd640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7e0160bd0 0x7fd7e00b8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7deffd640 1 -- 172.21.15.64:0/1841061757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d8009a70 con 0x7fd7e00b48d0 2023-11-24T08:05:38.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7df7fe640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7e00b3210 0x7fd7e00b89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:38.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7deffd640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b48d0 0x7fd7e00b7bd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800c3c0 tx=0x7fd7c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:38.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.721+0000 7fd7d7fff640 1 -- 172.21.15.64:0/1841061757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8016070 con 0x7fd7e00b48d0 2023-11-24T08:05:38.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.722+0000 7fd7d7fff640 1 -- 172.21.15.64:0/1841061757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c8007020 con 0x7fd7e00b48d0 2023-11-24T08:05:38.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.722+0000 7fd7d7fff640 1 -- 172.21.15.64:0/1841061757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80073d0 con 0x7fd7e00b48d0 2023-11-24T08:05:38.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.722+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e015b960 con 0x7fd7e00b48d0 2023-11-24T08:05:38.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.722+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e00b8ef0 con 0x7fd7e00b48d0 2023-11-24T08:05:38.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.723+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e00a1210 con 0x7fd7e00b48d0 2023-11-24T08:05:38.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.724+0000 7fd7d7fff640 1 -- 172.21.15.64:0/1841061757 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd7c8007530 con 0x7fd7e00b48d0 2023-11-24T08:05:38.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.725+0000 7fd7d7fff640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd7c006ae40 0x7fd7c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:38.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.725+0000 7fd7d7fff640 1 -- 172.21.15.64:0/1841061757 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7c8088ee0 con 0x7fd7e00b48d0 2023-11-24T08:05:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.726+0000 7fd7df7fe640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd7c006ae40 0x7fd7c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.727+0000 7fd7df7fe640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd7c006ae40 0x7fd7c006d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00011e0 tx=0x7fd7d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.729+0000 7fd7d7fff640 1 -- 172.21.15.64:0/1841061757 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7c8053f00 con 0x7fd7e00b48d0 2023-11-24T08:05:38.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.920+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd7e004e920 con 0x7fd7e00b48d0 2023-11-24T08:05:38.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.921+0000 7fd7d7fff640 1 -- 172.21.15.64:0/1841061757 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd7c8057520 con 0x7fd7e00b48d0 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd7c006ae40 msgr2=0x7fd7c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd7c006ae40 0x7fd7c006d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00011e0 tx=0x7fd7d0009040 comp rx=0 tx=0).stop 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b48d0 msgr2=0x7fd7e00b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b48d0 0x7fd7e00b7bd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800c3c0 tx=0x7fd7c800c880 comp rx=0 tx=0).stop 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 shutdown_connections 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd7c006ae40 0x7fd7c006d300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7e0160bd0 0x7fd7e00b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7e00b48d0 0x7fd7e00b7bd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 --2- 172.21.15.64:0/1841061757 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7e00b3210 0x7fd7e00b89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:38.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 >> 172.21.15.64:0/1841061757 conn(0x7fd7e009c5a0 msgr2=0x7fd7e00b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:38.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 shutdown_connections 2023-11-24T08:05:38.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:38.924+0000 7fd7e6256640 1 -- 172.21.15.64:0/1841061757 wait complete. 2023-11-24T08:05:38.942 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp3948 2023-11-24T08:05:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.154+0000 7f5523fff640 1 Processor -- start 2023-11-24T08:05:39.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.154+0000 7f5523fff640 1 -- start start 2023-11-24T08:05:39.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f5523fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5524144770 0x7f552415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f5523fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55240b35a0 0x7f552415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f5523fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552415f930 con 0x7f552406c9d0 2023-11-24T08:05:39.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f5523fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55240b43b0 con 0x7f5524144770 2023-11-24T08:05:39.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f5523fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240b3b10 con 0x7f55240b35a0 2023-11-24T08:05:39.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f5522ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5524144770 0x7f552415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f5522ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5524144770 0x7f552415eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36494/0 (socket says 172.21.15.64:36494) 2023-11-24T08:05:39.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f55237fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f552406c9d0 0x7f552406ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47216/0 (socket says 172.21.15.64:47216) 2023-11-24T08:05:39.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.157+0000 7f5522ffd640 1 -- 172.21.15.64:0/1495353444 learned_addr learned my addr 172.21.15.64:0/1495353444 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:39.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.158+0000 7f551bfff640 1 --2- 172.21.15.64:0/1495353444 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55240b35a0 0x7f552415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.158+0000 7f5522ffd640 1 -- 172.21.15.64:0/1495353444 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55240b35a0 msgr2=0x7f552415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:39.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.158+0000 7f5522ffd640 1 --2- 172.21.15.64:0/1495353444 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55240b35a0 0x7f552415f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.158+0000 7f5522ffd640 1 -- 172.21.15.64:0/1495353444 >> v1:172.21.15.64:6789/0 conn(0x7f552406c9d0 legacy=0x7f552406ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.158+0000 7f55227fc640 1 -- 172.21.15.64:0/1495353444 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354833265 0 0) 0x7f552415f930 con 0x7f552406c9d0 2023-11-24T08:05:39.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.158+0000 7f5522ffd640 1 -- 172.21.15.64:0/1495353444 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240b3df0 con 0x7f5524144770 2023-11-24T08:05:39.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.159+0000 7f5522ffd640 1 --2- 172.21.15.64:0/1495353444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5524144770 0x7f552415eb60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f55140098a0 tx=0x7f551402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a8429fdebc2b93 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.159+0000 7f55227fc640 1 -- 172.21.15.64:0/1495353444 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514030840 con 0x7f5524144770 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.159+0000 7f55227fc640 1 -- 172.21.15.64:0/1495353444 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55140309a0 con 0x7f5524144770 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f55227fc640 1 -- 172.21.15.64:0/1495353444 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514030d30 con 0x7f5524144770 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f5523fff640 1 -- 172.21.15.64:0/1495353444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5524144770 msgr2=0x7f552415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f5523fff640 1 --2- 172.21.15.64:0/1495353444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5524144770 0x7f552415eb60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f55140098a0 tx=0x7f551402b980 comp rx=0 tx=0).stop 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f5523fff640 1 -- 172.21.15.64:0/1495353444 shutdown_connections 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f5523fff640 1 --2- 172.21.15.64:0/1495353444 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f55240b35a0 0x7f552415f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f5523fff640 1 --2- 172.21.15.64:0/1495353444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5524144770 0x7f552415eb60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f5523fff640 1 -- 172.21.15.64:0/1495353444 >> 172.21.15.64:0/1495353444 conn(0x7f55240cec10 msgr2=0x7f55240cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f5523fff640 1 -- 172.21.15.64:0/1495353444 shutdown_connections 2023-11-24T08:05:39.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.160+0000 7f5523fff640 1 -- 172.21.15.64:0/1495353444 wait complete. 2023-11-24T08:05:39.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.162+0000 7f5523fff640 1 Processor -- start 2023-11-24T08:05:39.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.162+0000 7f5523fff640 1 -- start start 2023-11-24T08:05:39.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.163+0000 7f5523fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f552406c9d0 0x7f55241f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.163+0000 7f5523fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55240b35a0 0x7f552413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.163+0000 7f5523fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5524144770 0x7f5524154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.163+0000 7f5523fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524154380 con 0x7f55240b35a0 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.163+0000 7f5523fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524153040 con 0x7f5524144770 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.163+0000 7f5523fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55241f1af0 con 0x7f552406c9d0 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55240b35a0 0x7f552413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55240b35a0 0x7f552413a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36502/0 (socket says 172.21.15.64:36502) 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 -- 172.21.15.64:0/3397870197 learned_addr learned my addr 172.21.15.64:0/3397870197 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 -- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f552406c9d0 msgr2=0x7f55241f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f552406c9d0 0x7f55241f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 -- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5524144770 msgr2=0x7f5524154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5524144770 0x7f5524154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 -- 172.21.15.64:0/3397870197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140094d0 con 0x7f55240b35a0 2023-11-24T08:05:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.164+0000 7f551bfff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55240b35a0 0x7f552413a5f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f550c00d800 tx=0x7f550c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.165+0000 7f551b7fe640 1 -- 172.21.15.64:0/3397870197 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00b6c0 con 0x7f55240b35a0 2023-11-24T08:05:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.165+0000 7f551b7fe640 1 -- 172.21.15.64:0/3397870197 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550c00b820 con 0x7f55240b35a0 2023-11-24T08:05:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.165+0000 7f551b7fe640 1 -- 172.21.15.64:0/3397870197 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00bbb0 con 0x7f55240b35a0 2023-11-24T08:05:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.165+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552413ab30 con 0x7f55240b35a0 2023-11-24T08:05:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.165+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55241535c0 con 0x7f55240b35a0 2023-11-24T08:05:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.166+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552404e840 con 0x7f55240b35a0 2023-11-24T08:05:39.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.167+0000 7f551b7fe640 1 -- 172.21.15.64:0/3397870197 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f550c00bd10 con 0x7f55240b35a0 2023-11-24T08:05:39.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.168+0000 7f551b7fe640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f550406ae40 0x7f550406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.168+0000 7f551b7fe640 1 -- 172.21.15.64:0/3397870197 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f550c089c30 con 0x7f55240b35a0 2023-11-24T08:05:39.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.168+0000 7f5522ffd640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f550406ae40 0x7f550406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.169+0000 7f5522ffd640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f550406ae40 0x7f550406d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5514036000 tx=0x7f551403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:39.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.172+0000 7f551b7fe640 1 -- 172.21.15.64:0/3397870197 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f550c054c40 con 0x7f55240b35a0 2023-11-24T08:05:39.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.393+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp3948", "target": ["mon-mgr", ""]}) v1 -- 0x7f5524066ef0 con 0x7f550406ae40 2023-11-24T08:05:39.405 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:05:39.403+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:05:39.405 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:05:39.403+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:05:39.405 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:05:39.403+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:05:39.405 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:05:39.403+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:05:39.405 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:05:39.403+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:05:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.443+0000 7f551b7fe640 1 -- 172.21.15.64:0/3397870197 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5524066ef0 con 0x7f550406ae40 2023-11-24T08:05:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.445+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f550406ae40 msgr2=0x7f550406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.445+0000 7f5523fff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f550406ae40 0x7f550406d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5514036000 tx=0x7f551403b040 comp rx=0 tx=0).stop 2023-11-24T08:05:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.445+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55240b35a0 msgr2=0x7f552413a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.445+0000 7f5523fff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55240b35a0 0x7f552413a5f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f550c00d800 tx=0x7f550c00dcc0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.446+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 shutdown_connections 2023-11-24T08:05:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.446+0000 7f5523fff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f550406ae40 0x7f550406d300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.446+0000 7f5523fff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5524144770 0x7f5524154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.446+0000 7f5523fff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55240b35a0 0x7f552413a5f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.446+0000 7f5523fff640 1 --2- 172.21.15.64:0/3397870197 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f552406c9d0 0x7f55241f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.446+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 >> 172.21.15.64:0/3397870197 conn(0x7f55240cec10 msgr2=0x7f552406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.446+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 shutdown_connections 2023-11-24T08:05:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.446+0000 7f5523fff640 1 -- 172.21.15.64:0/3397870197 wait complete. 2023-11-24T08:05:39.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create cephfs subvol_grp3948 subvol_snap7853 2023-11-24T08:05:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.692+0000 7f2708874640 1 Processor -- start 2023-11-24T08:05:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.693+0000 7f2708874640 1 -- start start 2023-11-24T08:05:39.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.694+0000 7f2708874640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f27040d1fd0 0x7f27040ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.694+0000 7f2708874640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27040b2980 con 0x7f27040b33d0 2023-11-24T08:05:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.694+0000 7f2708874640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27040cf220 con 0x7f27040cfa30 2023-11-24T08:05:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.694+0000 7f2708874640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27040cf500 con 0x7f27040d1fd0 2023-11-24T08:05:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.695+0000 7f270259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f27040d1fd0 0x7f27040ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.695+0000 7f270259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f27040d1fd0 0x7f27040ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36506/0 (socket says 172.21.15.64:36506) 2023-11-24T08:05:39.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.695+0000 7f270259c640 1 -- 172.21.15.64:0/1699307092 learned_addr learned my addr 172.21.15.64:0/1699307092 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:39.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.695+0000 7f270259c640 1 -- 172.21.15.64:0/1699307092 >> v1:172.21.15.91:6789/0 conn(0x7f27040cfa30 legacy=0x7f27040cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:39.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.695+0000 7f270259c640 1 -- 172.21.15.64:0/1699307092 >> v1:172.21.15.64:6789/0 conn(0x7f27040b33d0 legacy=0x7f27040b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.695+0000 7f270259c640 1 -- 172.21.15.64:0/1699307092 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040b2bd0 con 0x7f27040d1fd0 2023-11-24T08:05:39.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.695+0000 7f2701d9b640 1 -- 172.21.15.64:0/1699307092 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690211492 0 0) 0x7f27040b2980 con 0x7f27040b33d0 2023-11-24T08:05:39.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.696+0000 7f270259c640 1 --2- 172.21.15.64:0/1699307092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27040d1fd0 0x7f27040ce920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009870 tx=0x7f26f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a3798e1e7523de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:39.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2701d9b640 1 -- 172.21.15.64:0/1699307092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8030860 con 0x7f27040d1fd0 2023-11-24T08:05:39.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2701d9b640 1 -- 172.21.15.64:0/1699307092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f80309c0 con 0x7f27040d1fd0 2023-11-24T08:05:39.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2701d9b640 1 -- 172.21.15.64:0/1699307092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8030d50 con 0x7f27040d1fd0 2023-11-24T08:05:39.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2708874640 1 -- 172.21.15.64:0/1699307092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27040d1fd0 msgr2=0x7f27040ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2708874640 1 --2- 172.21.15.64:0/1699307092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27040d1fd0 0x7f27040ce920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009870 tx=0x7f26f802b9a0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2708874640 1 -- 172.21.15.64:0/1699307092 shutdown_connections 2023-11-24T08:05:39.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2708874640 1 --2- 172.21.15.64:0/1699307092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27040d1fd0 0x7f27040ce920 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2708874640 1 -- 172.21.15.64:0/1699307092 >> 172.21.15.64:0/1699307092 conn(0x7f27040d3020 msgr2=0x7f27040d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:39.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.697+0000 7f2708874640 1 -- 172.21.15.64:0/1699307092 shutdown_connections 2023-11-24T08:05:39.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.698+0000 7f2708874640 1 -- 172.21.15.64:0/1699307092 wait complete. 2023-11-24T08:05:39.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.699+0000 7f2708874640 1 Processor -- start 2023-11-24T08:05:39.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.699+0000 7f2708874640 1 -- start start 2023-11-24T08:05:39.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.700+0000 7f2708874640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27040b33d0 0x7f27040cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.700+0000 7f2708874640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27040cfa30 0x7f270415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.700+0000 7f2708874640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27040d1fd0 0x7f270415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.700+0000 7f2708874640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270415ed70 con 0x7f27040cfa30 2023-11-24T08:05:39.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.700+0000 7f2708874640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2704151d50 con 0x7f27040b33d0 2023-11-24T08:05:39.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.700+0000 7f2708874640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270415e8e0 con 0x7f27040d1fd0 2023-11-24T08:05:39.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27040cfa30 0x7f270415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27040cfa30 0x7f270415a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36510/0 (socket says 172.21.15.64:36510) 2023-11-24T08:05:39.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270259c640 1 -- 172.21.15.64:0/532048376 learned_addr learned my addr 172.21.15.64:0/532048376 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:39.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f2702d9d640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27040b33d0 0x7f27040cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270359e640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27040d1fd0 0x7f270415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270259c640 1 -- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27040d1fd0 msgr2=0x7f270415a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270259c640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27040d1fd0 0x7f270415a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270259c640 1 -- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27040b33d0 msgr2=0x7f27040cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:39.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270259c640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f27040b33d0 0x7f27040cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:39.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270259c640 1 -- 172.21.15.64:0/532048376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80094d0 con 0x7f27040cfa30 2023-11-24T08:05:39.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.701+0000 7f270359e640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f27040d1fd0 0x7f270415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:39.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.702+0000 7f270259c640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27040cfa30 0x7f270415a390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009870 tx=0x7f26f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:39.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.702+0000 7f26eb7fe640 1 -- 172.21.15.64:0/532048376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8030810 con 0x7f27040cfa30 2023-11-24T08:05:39.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.702+0000 7f26eb7fe640 1 -- 172.21.15.64:0/532048376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f8043400 con 0x7f27040cfa30 2023-11-24T08:05:39.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.702+0000 7f26eb7fe640 1 -- 172.21.15.64:0/532048376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f80437b0 con 0x7f27040cfa30 2023-11-24T08:05:39.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.702+0000 7f2708874640 1 -- 172.21.15.64:0/532048376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270415dcd0 con 0x7f27040cfa30 2023-11-24T08:05:39.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.702+0000 7f2708874640 1 -- 172.21.15.64:0/532048376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270415e290 con 0x7f27040cfa30 2023-11-24T08:05:39.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.703+0000 7f2708874640 1 -- 172.21.15.64:0/532048376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27040d0a40 con 0x7f27040cfa30 2023-11-24T08:05:39.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.704+0000 7f26eb7fe640 1 -- 172.21.15.64:0/532048376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26f8043910 con 0x7f27040cfa30 2023-11-24T08:05:39.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.705+0000 7f26eb7fe640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f26f006ae40 0x7f26f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:39.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.706+0000 7f26eb7fe640 1 -- 172.21.15.64:0/532048376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26f803e070 con 0x7f27040cfa30 2023-11-24T08:05:39.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.706+0000 7f2702d9d640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f26f006ae40 0x7f26f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:39.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.707+0000 7f2702d9d640 1 --2- 172.21.15.64:0/532048376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f26f006ae40 0x7f26f006d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004ea0 tx=0x7f26f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:39.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.710+0000 7f26eb7fe640 1 -- 172.21.15.64:0/532048376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26f8077ac0 con 0x7f27040cfa30 2023-11-24T08:05:39.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.930+0000 7f2708874640 1 -- 172.21.15.64:0/532048376 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "cephfs", "group_name": "subvol_grp3948", "snap_name": "subvol_snap7853", "target": ["mon-mgr", ""]}) v1 -- 0x7f270415d7f0 con 0x7f26f006ae40 2023-11-24T08:05:39.935 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:05:39.933+0000 7fe78a34d640 -1 mgr.server reply reply (38) Function not implemented subvolume group snapshots are not supported 2023-11-24T08:05:39.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:39.935+0000 7f26eb7fe640 1 -- 172.21.15.64:0/532048376 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: -38 subvolume group snapshots are not supported) v1 ==== 51+0+0 (secure 0 0 0) 0x7f26f4005420 con 0x7f26f006ae40 2023-11-24T08:05:39.936 INFO:teuthology.orchestra.run.smithi064.stderr:Error ENOSYS: subvolume group snapshots are not supported 2023-11-24T08:05:39.950 DEBUG:teuthology.orchestra.run:got remote process result: 38 2023-11-24T08:05:39.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp3948 2023-11-24T08:05:40.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.182+0000 7fb9f77ec640 1 Processor -- start 2023-11-24T08:05:40.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.182+0000 7fb9f77ec640 1 -- start start 2023-11-24T08:05:40.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.183+0000 7fb9f77ec640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 0x7fb9f00d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.183+0000 7fb9f77ec640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f015d720 con 0x7fb9f0070af0 2023-11-24T08:05:40.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.183+0000 7fb9f77ec640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f006d080 con 0x7fb9f015d340 2023-11-24T08:05:40.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.183+0000 7fb9f77ec640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00d0d90 con 0x7fb9f00ce150 2023-11-24T08:05:40.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.184+0000 7fb9f4d87640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb9f0070af0 0x7fb9f00cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47236/0 (socket says 172.21.15.64:47236) 2023-11-24T08:05:40.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.184+0000 7fb9f4d87640 1 -- 172.21.15.64:0/1082123974 learned_addr learned my addr 172.21.15.64:0/1082123974 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:40.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.184+0000 7fb9f5d89640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 0x7fb9f00d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:40.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.184+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909509829 0 0) 0x7fb9f015d720 con 0x7fb9f0070af0 2023-11-24T08:05:40.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.184+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9dc003520 con 0x7fb9f0070af0 2023-11-24T08:05:40.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2267437005 0 0) 0x7fb9dc003520 con 0x7fb9f0070af0 2023-11-24T08:05:40.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 msgr2=0x7fb9f00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 0x7fb9f00d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 >> v1:172.21.15.91:6789/0 conn(0x7fb9f015d340 legacy=0x7fb9f00cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f006d360 con 0x7fb9f0070af0 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fb9e0001670 con 0x7fb9f0070af0 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 >> v1:172.21.15.64:6789/0 conn(0x7fb9f0070af0 legacy=0x7fb9f00cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9dc0047c0 0x7fb9dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 0x7fb9dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00d0d90 con 0x7fb9dc0039a0 2023-11-24T08:05:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc005510 con 0x7fb9dc0047c0 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc00bec0 con 0x7fb9f00ce150 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.185+0000 7fb9f5588640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f5d89640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 0x7fb9dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f4d87640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9dc0047c0 0x7fb9dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f5588640 1 -- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 msgr2=0x7fb9dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f5588640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 0x7fb9dc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f5588640 1 -- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9dc0047c0 msgr2=0x7fb9dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f5588640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9dc0047c0 0x7fb9dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f5588640 1 -- 172.21.15.64:0/1082123974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc003520 con 0x7fb9dc0039a0 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f5d89640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f00ce150 0x7fb9dc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.186+0000 7fb9f4d87640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9dc0047c0 0x7fb9dc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.187+0000 7fb9f5588640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9dc003dc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec001e50 tx=0x7fb9dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d00af68fbf63139 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:40.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.187+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec003b70 con 0x7fb9dc0039a0 2023-11-24T08:05:40.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.187+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec003cd0 con 0x7fb9dc0039a0 2023-11-24T08:05:40.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.187+0000 7fb9e7fff640 1 -- 172.21.15.64:0/1082123974 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec005310 con 0x7fb9dc0039a0 2023-11-24T08:05:40.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.188+0000 7fb9f77ec640 1 -- 172.21.15.64:0/1082123974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 msgr2=0x7fb9dc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:40.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.188+0000 7fb9f77ec640 1 --2- 172.21.15.64:0/1082123974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9dc003dc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec001e50 tx=0x7fb9dc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.188+0000 7fb9f5d89640 1 -- 172.21.15.64:0/1082123974 reap_dead start 2023-11-24T08:05:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.188+0000 7fb9f77ec640 1 -- 172.21.15.64:0/1082123974 shutdown_connections 2023-11-24T08:05:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.188+0000 7fb9f77ec640 1 -- 172.21.15.64:0/1082123974 >> 172.21.15.64:0/1082123974 conn(0x7fb9f00b2ba0 msgr2=0x7fb9f00b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.188+0000 7fb9f77ec640 1 -- 172.21.15.64:0/1082123974 shutdown_connections 2023-11-24T08:05:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.188+0000 7fb9f77ec640 1 -- 172.21.15.64:0/1082123974 wait complete. 2023-11-24T08:05:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.190+0000 7fb9f77ec640 1 Processor -- start 2023-11-24T08:05:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.190+0000 7fb9f77ec640 1 -- start start 2023-11-24T08:05:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f77ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9f00ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f77ec640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f01601e0 0x7fb9f014e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f77ec640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9f0157560 0x7fb9f0159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f77ec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00c9b60 con 0x7fb9dc0039a0 2023-11-24T08:05:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f77ec640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00ca400 con 0x7fb9f0157560 2023-11-24T08:05:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f77ec640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f01606c0 con 0x7fb9f01601e0 2023-11-24T08:05:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f5588640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9f00ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f5588640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9f00ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36540/0 (socket says 172.21.15.64:36540) 2023-11-24T08:05:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.191+0000 7fb9f5588640 1 -- 172.21.15.64:0/587625489 learned_addr learned my addr 172.21.15.64:0/587625489 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9f5588640 1 -- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f01601e0 msgr2=0x7fb9f014e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9f5588640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f01601e0 0x7fb9f014e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9f5588640 1 -- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9f0157560 msgr2=0x7fb9f0159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9f5588640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9f0157560 0x7fb9f0159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9f5588640 1 -- 172.21.15.64:0/587625489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00c9db0 con 0x7fb9dc0039a0 2023-11-24T08:05:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9f5588640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9f00ca890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec000f80 tx=0x7fb9ec0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9e5ffb640 1 -- 172.21.15.64:0/587625489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec03e070 con 0x7fb9dc0039a0 2023-11-24T08:05:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9e5ffb640 1 -- 172.21.15.64:0/587625489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec003e40 con 0x7fb9dc0039a0 2023-11-24T08:05:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9e5ffb640 1 -- 172.21.15.64:0/587625489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec006d50 con 0x7fb9dc0039a0 2023-11-24T08:05:40.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.192+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f00ca090 con 0x7fb9dc0039a0 2023-11-24T08:05:40.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.193+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f0158ed0 con 0x7fb9dc0039a0 2023-11-24T08:05:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.194+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f0160c40 con 0x7fb9dc0039a0 2023-11-24T08:05:40.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.195+0000 7fb9e5ffb640 1 -- 172.21.15.64:0/587625489 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9ec007030 con 0x7fb9dc0039a0 2023-11-24T08:05:40.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.196+0000 7fb9e5ffb640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb9d406ae70 0x7fb9d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.196+0000 7fb9e5ffb640 1 -- 172.21.15.64:0/587625489 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9ec0ab140 con 0x7fb9dc0039a0 2023-11-24T08:05:40.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.197+0000 7fb9f4d87640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb9d406ae70 0x7fb9d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:40.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.198+0000 7fb9f4d87640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb9d406ae70 0x7fb9d406d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00b6ae0 tx=0x7fb9e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.201+0000 7fb9e5ffb640 1 -- 172.21.15.64:0/587625489 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9ec076160 con 0x7fb9dc0039a0 2023-11-24T08:05:40.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.424+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp3948", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9f014e990 con 0x7fb9d406ae70 2023-11-24T08:05:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.436+0000 7fb9e5ffb640 1 -- 172.21.15.64:0/587625489 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb9f014e990 con 0x7fb9d406ae70 2023-11-24T08:05:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb9d406ae70 msgr2=0x7fb9d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb9d406ae70 0x7fb9d406d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00b6ae0 tx=0x7fb9e0009040 comp rx=0 tx=0).stop 2023-11-24T08:05:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 msgr2=0x7fb9f00ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9f00ca890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec000f80 tx=0x7fb9ec0014a0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 shutdown_connections 2023-11-24T08:05:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb9d406ae70 0x7fb9d406d330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9f0157560 0x7fb9f0159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9f01601e0 0x7fb9f014e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 --2- 172.21.15.64:0/587625489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9dc0039a0 0x7fb9f00ca890 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.439+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 >> 172.21.15.64:0/587625489 conn(0x7fb9f00b2ba0 msgr2=0x7fb9f006b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.440+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 shutdown_connections 2023-11-24T08:05:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.440+0000 7fb9f77ec640 1 -- 172.21.15.64:0/587625489 wait complete. 2023-11-24T08:05:40.461 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:05:40.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:05:40.578 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:05:40.704 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:05:40.704 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:05:40.704 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:05:40.768 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:40.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.996+0000 7fc304d6c640 1 Processor -- start 2023-11-24T08:05:40.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.996+0000 7fc304d6c640 1 -- start start 2023-11-24T08:05:40.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.998+0000 7fc304d6c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3000b2040 0x7fc3000cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.998+0000 7fc304d6c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc30006b760 0x7fc300166ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.998+0000 7fc304d6c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc300160c90 con 0x7fc30006bf60 2023-11-24T08:05:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.998+0000 7fc304d6c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3000b0ba0 con 0x7fc30006b760 2023-11-24T08:05:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.998+0000 7fc304d6c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000ce7a0 con 0x7fc3000b2040 2023-11-24T08:05:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.998+0000 7fc2fffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc30006b760 0x7fc300166ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.999+0000 7fc2fffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc30006b760 0x7fc300166ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:55744/0 (socket says 172.21.15.64:55744) 2023-11-24T08:05:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.999+0000 7fc2fffff640 1 -- 172.21.15.64:0/2137496689 learned_addr learned my addr 172.21.15.64:0/2137496689 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.999+0000 7fc2fffff640 1 -- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3000b2040 msgr2=0x7fc3000cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.999+0000 7fc2f7fff640 1 --2- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3000b2040 0x7fc3000cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:40.999+0000 7fc2fffff640 1 --2- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3000b2040 0x7fc3000cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.000+0000 7fc2fffff640 1 -- 172.21.15.64:0/2137496689 >> v1:172.21.15.91:6789/0 conn(0x7fc30006bf60 legacy=0x7fc30006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.000+0000 7fc2fffff640 1 -- 172.21.15.64:0/2137496689 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000d0350 con 0x7fc30006b760 2023-11-24T08:05:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.000+0000 7fc2f7fff640 1 --2- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3000b2040 0x7fc3000cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.001+0000 7fc2fffff640 1 --2- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc30006b760 0x7fc300166ed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009e50 tx=0x7fc2f802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e45ad77c7e9e393 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.015+0000 7fc2feffd640 1 -- 172.21.15.64:0/2137496689 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802ca70 con 0x7fc30006b760 2023-11-24T08:05:41.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.015+0000 7fc2feffd640 1 -- 172.21.15.64:0/2137496689 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f802cbd0 con 0x7fc30006b760 2023-11-24T08:05:41.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.015+0000 7fc2feffd640 1 -- 172.21.15.64:0/2137496689 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80343f0 con 0x7fc30006b760 2023-11-24T08:05:41.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.016+0000 7fc304d6c640 1 -- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc30006b760 msgr2=0x7fc300166ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.016+0000 7fc304d6c640 1 --2- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc30006b760 0x7fc300166ed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009e50 tx=0x7fc2f802bed0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.016+0000 7fc304d6c640 1 -- 172.21.15.64:0/2137496689 shutdown_connections 2023-11-24T08:05:41.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.016+0000 7fc304d6c640 1 --2- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc30006b760 0x7fc300166ed0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.016+0000 7fc304d6c640 1 --2- 172.21.15.64:0/2137496689 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3000b2040 0x7fc3000cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.016+0000 7fc304d6c640 1 -- 172.21.15.64:0/2137496689 >> 172.21.15.64:0/2137496689 conn(0x7fc3000d0e50 msgr2=0x7fc3000d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.016+0000 7fc304d6c640 1 -- 172.21.15.64:0/2137496689 shutdown_connections 2023-11-24T08:05:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.016+0000 7fc304d6c640 1 -- 172.21.15.64:0/2137496689 wait complete. 2023-11-24T08:05:41.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.019+0000 7fc304d6c640 1 Processor -- start 2023-11-24T08:05:41.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.020+0000 7fc304d6c640 1 -- start start 2023-11-24T08:05:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.021+0000 7fc304d6c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc30006b760 0x7fc30015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.021+0000 7fc304d6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc30006bf60 0x7fc3000aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.021+0000 7fc304d6c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3000b2040 0x7fc3000af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.021+0000 7fc304d6c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30015b850 con 0x7fc30006bf60 2023-11-24T08:05:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.021+0000 7fc304d6c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30015ff80 con 0x7fc3000b2040 2023-11-24T08:05:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.021+0000 7fc304d6c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30015fca0 con 0x7fc30006b760 2023-11-24T08:05:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.022+0000 7fc2f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc30006bf60 0x7fc3000aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.022+0000 7fc2f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc30006bf60 0x7fc3000aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36552/0 (socket says 172.21.15.64:36552) 2023-11-24T08:05:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.022+0000 7fc2f7fff640 1 -- 172.21.15.64:0/3258693306 learned_addr learned my addr 172.21.15.64:0/3258693306 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:41.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.022+0000 7fc2fffff640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3000b2040 0x7fc3000af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.022+0000 7fc2ff7fe640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc30006b760 0x7fc30015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.022+0000 7fc2f7fff640 1 -- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc30006b760 msgr2=0x7fc30015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.022+0000 7fc2f7fff640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc30006b760 0x7fc30015bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.022+0000 7fc2f7fff640 1 -- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3000b2040 msgr2=0x7fc3000af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.023+0000 7fc2f7fff640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3000b2040 0x7fc3000af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.023+0000 7fc2f7fff640 1 -- 172.21.15.64:0/3258693306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8009b30 con 0x7fc30006bf60 2023-11-24T08:05:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.023+0000 7fc2fffff640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3000b2040 0x7fc3000af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.023+0000 7fc2f7fff640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc30006bf60 0x7fc3000aedc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c7f0 tx=0x7fc2e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.023+0000 7fc2fcff9640 1 -- 172.21.15.64:0/3258693306 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8007a30 con 0x7fc30006bf60 2023-11-24T08:05:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.023+0000 7fc2fcff9640 1 -- 172.21.15.64:0/3258693306 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e8007b90 con 0x7fc30006bf60 2023-11-24T08:05:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.023+0000 7fc2fcff9640 1 -- 172.21.15.64:0/3258693306 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e800ce40 con 0x7fc30006bf60 2023-11-24T08:05:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.023+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30015b190 con 0x7fc30006bf60 2023-11-24T08:05:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.024+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30015f990 con 0x7fc30006bf60 2023-11-24T08:05:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.025+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30015b990 con 0x7fc30006bf60 2023-11-24T08:05:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.026+0000 7fc2fcff9640 1 -- 172.21.15.64:0/3258693306 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc2e80113f0 con 0x7fc30006bf60 2023-11-24T08:05:41.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.027+0000 7fc2fcff9640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc2e006aed0 0x7fc2e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.027+0000 7fc2fcff9640 1 -- 172.21.15.64:0/3258693306 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2e8089890 con 0x7fc30006bf60 2023-11-24T08:05:41.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.027+0000 7fc2ff7fe640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc2e006aed0 0x7fc2e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.028+0000 7fc2ff7fe640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc2e006aed0 0x7fc2e006d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0007fd0 tx=0x7fc2f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.032+0000 7fc2fcff9640 1 -- 172.21.15.64:0/3258693306 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc30015b990 con 0x7fc30006bf60 2023-11-24T08:05:41.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.231+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc30015b990 con 0x7fc30006bf60 2023-11-24T08:05:41.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.233+0000 7fc2fcff9640 1 -- 172.21.15.64:0/3258693306 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fc30015b990 con 0x7fc30006bf60 2023-11-24T08:05:41.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:41.234 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:21.012478+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:05:17.535151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:05:18.499872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.235+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc2e006aed0 msgr2=0x7fc2e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.235+0000 7fc304d6c640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc2e006aed0 0x7fc2e006d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0007fd0 tx=0x7fc2f0009040 comp rx=0 tx=0).stop 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.235+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc30006bf60 msgr2=0x7fc3000aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.235+0000 7fc304d6c640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc30006bf60 0x7fc3000aedc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c7f0 tx=0x7fc2e800cbf0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.236+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 shutdown_connections 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.236+0000 7fc304d6c640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc2e006aed0 0x7fc2e006d390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.236+0000 7fc304d6c640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3000b2040 0x7fc3000af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.236+0000 7fc304d6c640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc30006bf60 0x7fc3000aedc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.236+0000 7fc304d6c640 1 --2- 172.21.15.64:0/3258693306 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc30006b760 0x7fc30015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.236+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 >> 172.21.15.64:0/3258693306 conn(0x7fc3000d0e50 msgr2=0x7fc3000cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.236+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 shutdown_connections 2023-11-24T08:05:41.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.236+0000 7fc304d6c640 1 -- 172.21.15.64:0/3258693306 wait complete. 2023-11-24T08:05:41.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:05:17.535151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:05:18.499872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:05:41.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:05:41.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.473+0000 7feafe95c640 1 Processor -- start 2023-11-24T08:05:41.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.474+0000 7feafe95c640 1 -- start start 2023-11-24T08:05:41.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.475+0000 7feafe95c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feaf8144770 0x7feaf815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.475+0000 7feafe95c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7feaf815f0a0 0x7feaf80ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.475+0000 7feafe95c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf815f930 con 0x7feaf80b35a0 2023-11-24T08:05:41.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.475+0000 7feafe95c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf80b43b0 con 0x7feaf815f0a0 2023-11-24T08:05:41.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.475+0000 7feafe95c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf815f530 con 0x7feaf8144770 2023-11-24T08:05:41.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.476+0000 7feafd159640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7feaf80b35a0 0x7feaf806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47252/0 (socket says 172.21.15.64:47252) 2023-11-24T08:05:41.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.476+0000 7feafd159640 1 -- 172.21.15.64:0/502346213 learned_addr learned my addr 172.21.15.64:0/502346213 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:41.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.476+0000 7feafe15b640 1 --2- 172.21.15.64:0/502346213 >> v2:172.21.15.91:3300/0 conn(0x7feaf815f0a0 0x7feaf80ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.476+0000 7feafd95a640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feaf8144770 0x7feaf815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.476+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538941386 0 0) 0x7feaf815f930 con 0x7feaf80b35a0 2023-11-24T08:05:41.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.476+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae4003520 con 0x7feaf80b35a0 2023-11-24T08:05:41.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533252300 0 0) 0x7feae4003520 con 0x7feaf80b35a0 2023-11-24T08:05:41.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feaf8144770 msgr2=0x7feaf815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:41.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feaf8144770 0x7feaf815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 >> v2:172.21.15.91:3300/0 conn(0x7feaf815f0a0 msgr2=0x7feaf80ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 --2- 172.21.15.64:0/502346213 >> v2:172.21.15.91:3300/0 conn(0x7feaf815f0a0 0x7feaf80ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80ce9f0 con 0x7feaf80b35a0 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7feae8001670 con 0x7feaf80b35a0 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 >> v1:172.21.15.64:6789/0 conn(0x7feaf80b35a0 legacy=0x7feaf806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feae40039e0 0x7feae4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae40047c0 0x7feae4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feaf8144770 0x7feae4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80b43b0 con 0x7feae40047c0 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf815f530 con 0x7feae40039e0 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae400fd90 con 0x7feaf8144770 2023-11-24T08:05:41.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafd159640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae40047c0 0x7feae4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafd95a640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feae40039e0 0x7feae4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.477+0000 7feafe15b640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feaf8144770 0x7feae4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.478+0000 7feafd159640 1 -- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feaf8144770 msgr2=0x7feae4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:41.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.478+0000 7feafd159640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feaf8144770 0x7feae4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.478+0000 7feafd159640 1 -- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feae40039e0 msgr2=0x7feae4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:41.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.478+0000 7feafd159640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feae40039e0 0x7feae4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.478+0000 7feafd159640 1 -- 172.21.15.64:0/502346213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae4003520 con 0x7feae40047c0 2023-11-24T08:05:41.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.479+0000 7feafd159640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae40047c0 0x7feae4004c40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feae800c270 tx=0x7feae40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c81439ba2f8270d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.479+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8005ad0 con 0x7feae40047c0 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.479+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8005c30 con 0x7feae40047c0 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.479+0000 7feafc958640 1 -- 172.21.15.64:0/502346213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae802c8a0 con 0x7feae40047c0 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.480+0000 7feafe95c640 1 -- 172.21.15.64:0/502346213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae40047c0 msgr2=0x7feae4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.480+0000 7feafe95c640 1 --2- 172.21.15.64:0/502346213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae40047c0 0x7feae4004c40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feae800c270 tx=0x7feae40057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.480+0000 7feafe15b640 1 -- 172.21.15.64:0/502346213 reap_dead start 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.480+0000 7feafe95c640 1 -- 172.21.15.64:0/502346213 shutdown_connections 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.480+0000 7feafe95c640 1 -- 172.21.15.64:0/502346213 >> 172.21.15.64:0/502346213 conn(0x7feaf80cec10 msgr2=0x7feaf80cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.480+0000 7feafe95c640 1 -- 172.21.15.64:0/502346213 shutdown_connections 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.480+0000 7feafe95c640 1 -- 172.21.15.64:0/502346213 wait complete. 2023-11-24T08:05:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.482+0000 7feafe95c640 1 Processor -- start 2023-11-24T08:05:41.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.482+0000 7feafe95c640 1 -- start start 2023-11-24T08:05:41.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafe95c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feae40047c0 0x7feaf81ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafe95c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf80a9070 0x7feaf8142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafe95c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feaf80a9820 0x7feaf81f1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafe95c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf81431d0 con 0x7feaf80a9070 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafe95c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf81eec40 con 0x7feaf80a9820 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafe95c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8140cb0 con 0x7feae40047c0 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafd159640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf80a9070 0x7feaf8142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafd159640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf80a9070 0x7feaf8142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36564/0 (socket says 172.21.15.64:36564) 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafd159640 1 -- 172.21.15.64:0/3374269264 learned_addr learned my addr 172.21.15.64:0/3374269264 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.483+0000 7feafd95a640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feae40047c0 0x7feaf81ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feafd159640 1 -- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feae40047c0 msgr2=0x7feaf81ee700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feafd159640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feae40047c0 0x7feaf81ee700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feafd159640 1 -- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feaf80a9820 msgr2=0x7feaf81f1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feafd159640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feaf80a9820 0x7feaf81f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feafd159640 1 -- 172.21.15.64:0/3374269264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf81403c0 con 0x7feaf80a9070 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feafd159640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf80a9070 0x7feaf8142980 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7feae800c270 tx=0x7feae8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feaee7fc640 1 -- 172.21.15.64:0/3374269264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8039070 con 0x7feaf80a9070 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feaee7fc640 1 -- 172.21.15.64:0/3374269264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae802ccb0 con 0x7feaf80a9070 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feaee7fc640 1 -- 172.21.15.64:0/3374269264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae803c040 con 0x7feaf80a9070 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf8140660 con 0x7feaf80a9070 2023-11-24T08:05:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.484+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf8142ec0 con 0x7feaf80a9070 2023-11-24T08:05:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.486+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf8140e20 con 0x7feaf80a9070 2023-11-24T08:05:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.486+0000 7feaee7fc640 1 -- 172.21.15.64:0/3374269264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7feae8032070 con 0x7feaf80a9070 2023-11-24T08:05:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.487+0000 7feaee7fc640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feadc06af00 0x7feadc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.487+0000 7feaee7fc640 1 -- 172.21.15.64:0/3374269264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feae80aa800 con 0x7feaf80a9070 2023-11-24T08:05:41.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.488+0000 7feafd95a640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feadc06af00 0x7feadc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.489+0000 7feafd95a640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feadc06af00 0x7feadc06d3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feaf40097f0 tx=0x7feaf4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.492+0000 7feaee7fc640 1 -- 172.21.15.64:0/3374269264 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feae80759e0 con 0x7feaf80a9070 2023-11-24T08:05:41.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.687+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feaf804e840 con 0x7feaf80a9070 2023-11-24T08:05:41.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.688+0000 7feaee7fc640 1 -- 172.21.15.64:0/3374269264 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7feae8079000 con 0x7feaf80a9070 2023-11-24T08:05:41.689 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:05:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.690+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feadc06af00 msgr2=0x7feadc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.690+0000 7feafe95c640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feadc06af00 0x7feadc06d3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feaf40097f0 tx=0x7feaf4008040 comp rx=0 tx=0).stop 2023-11-24T08:05:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.690+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf80a9070 msgr2=0x7feaf8142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.690+0000 7feafe95c640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf80a9070 0x7feaf8142980 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7feae800c270 tx=0x7feae8001a00 comp rx=0 tx=0).stop 2023-11-24T08:05:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.691+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 shutdown_connections 2023-11-24T08:05:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.691+0000 7feafe95c640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feadc06af00 0x7feadc06d3c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.691+0000 7feafe95c640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feaf80a9820 0x7feaf81f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.691+0000 7feafe95c640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf80a9070 0x7feaf8142980 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.691+0000 7feafe95c640 1 --2- 172.21.15.64:0/3374269264 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7feae40047c0 0x7feaf81ee700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.691+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 >> 172.21.15.64:0/3374269264 conn(0x7feaf80cec10 msgr2=0x7feaf806c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.691+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 shutdown_connections 2023-11-24T08:05:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.691+0000 7feafe95c640 1 -- 172.21.15.64:0/3374269264 wait complete. 2023-11-24T08:05:41.711 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:05:41.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.938+0000 7f578329b640 1 Processor -- start 2023-11-24T08:05:41.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.938+0000 7f578329b640 1 -- start start 2023-11-24T08:05:41.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.939+0000 7f578329b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f577c15b350 0x7f577c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.939+0000 7f578329b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c15b730 con 0x7f577c09cb30 2023-11-24T08:05:41.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.939+0000 7f578329b640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c0ac740 con 0x7f577c0adb60 2023-11-24T08:05:41.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.939+0000 7f578329b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c0ac460 con 0x7f577c15b350 2023-11-24T08:05:41.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.940+0000 7f5781037640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f577c15b350 0x7f577c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.940+0000 7f5780836640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f577c09cb30 0x7f577c0b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47254/0 (socket says 172.21.15.64:47254) 2023-11-24T08:05:41.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.940+0000 7f5780836640 1 -- 172.21.15.64:0/2341314858 learned_addr learned my addr 172.21.15.64:0/2341314858 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:41.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.940+0000 7f5781037640 1 -- 172.21.15.64:0/2341314858 >> v1:172.21.15.91:6789/0 conn(0x7f577c0adb60 legacy=0x7f577c0adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:41.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.940+0000 7f5781037640 1 -- 172.21.15.64:0/2341314858 >> v1:172.21.15.64:6789/0 conn(0x7f577c09cb30 legacy=0x7f577c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.940+0000 7f5781037640 1 -- 172.21.15.64:0/2341314858 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0b2a50 con 0x7f577c15b350 2023-11-24T08:05:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.941+0000 7f5781037640 1 --2- 172.21.15.64:0/2341314858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c15b350 0x7f577c0abe80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f57780097a0 tx=0x7f577802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18273169899d9f6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.941+0000 7f576ffff640 1 -- 172.21.15.64:0/2341314858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778030710 con 0x7f577c15b350 2023-11-24T08:05:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f576ffff640 1 -- 172.21.15.64:0/2341314858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577802e050 con 0x7f577c15b350 2023-11-24T08:05:41.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f576ffff640 1 -- 172.21.15.64:0/2341314858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778030a90 con 0x7f577c15b350 2023-11-24T08:05:41.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f578329b640 1 -- 172.21.15.64:0/2341314858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c15b350 msgr2=0x7f577c0abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f578329b640 1 --2- 172.21.15.64:0/2341314858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c15b350 0x7f577c0abe80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f57780097a0 tx=0x7f577802b710 comp rx=0 tx=0).stop 2023-11-24T08:05:41.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f578329b640 1 -- 172.21.15.64:0/2341314858 shutdown_connections 2023-11-24T08:05:41.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f578329b640 1 --2- 172.21.15.64:0/2341314858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c15b350 0x7f577c0abe80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f578329b640 1 -- 172.21.15.64:0/2341314858 >> 172.21.15.64:0/2341314858 conn(0x7f577c0b0ba0 msgr2=0x7f577c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:41.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f578329b640 1 -- 172.21.15.64:0/2341314858 shutdown_connections 2023-11-24T08:05:41.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.942+0000 7f578329b640 1 -- 172.21.15.64:0/2341314858 wait complete. 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.944+0000 7f578329b640 1 Processor -- start 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.944+0000 7f578329b640 1 -- start start 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.945+0000 7f578329b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f577c09cb30 0x7f577c13f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.945+0000 7f578329b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0adb60 0x7f577c157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.945+0000 7f578329b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f577c15b350 0x7f577c157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.945+0000 7f578329b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c09ad40 con 0x7f577c0adb60 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.945+0000 7f578329b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c158490 con 0x7f577c09cb30 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.945+0000 7f578329b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c1399d0 con 0x7f577c15b350 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.945+0000 7f5780836640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0adb60 0x7f577c157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5780836640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0adb60 0x7f577c157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36582/0 (socket says 172.21.15.64:36582) 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5780836640 1 -- 172.21.15.64:0/3471809284 learned_addr learned my addr 172.21.15.64:0/3471809284 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5781037640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f577c09cb30 0x7f577c13f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5781838640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f577c15b350 0x7f577c157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5780836640 1 -- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f577c15b350 msgr2=0x7f577c157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5780836640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f577c15b350 0x7f577c157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5780836640 1 -- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f577c09cb30 msgr2=0x7f577c13f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5780836640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f577c09cb30 0x7f577c13f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5780836640 1 -- 172.21.15.64:0/3471809284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778009400 con 0x7f577c0adb60 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5781838640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f577c15b350 0x7f577c157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.946+0000 7f5780836640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0adb60 0x7f577c157a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f577000c870 tx=0x7f577000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.947+0000 7f576dffb640 1 -- 172.21.15.64:0/3471809284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57700074e0 con 0x7f577c0adb60 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.947+0000 7f576dffb640 1 -- 172.21.15.64:0/3471809284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5770007640 con 0x7f577c0adb60 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.947+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c139cb0 con 0x7f577c0adb60 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.947+0000 7f576dffb640 1 -- 172.21.15.64:0/3471809284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770004a20 con 0x7f577c0adb60 2023-11-24T08:05:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.947+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c13a270 con 0x7f577c0adb60 2023-11-24T08:05:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.948+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c13a490 con 0x7f577c0adb60 2023-11-24T08:05:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.949+0000 7f576dffb640 1 -- 172.21.15.64:0/3471809284 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5770004b80 con 0x7f577c0adb60 2023-11-24T08:05:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.950+0000 7f576dffb640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f576006af00 0x7f576006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.950+0000 7f576dffb640 1 -- 172.21.15.64:0/3471809284 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5770088d80 con 0x7f577c0adb60 2023-11-24T08:05:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.951+0000 7f5781037640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f576006af00 0x7f576006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.952+0000 7f5781037640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f576006af00 0x7f576006d3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5778034000 tx=0x7f5778008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:41.954+0000 7f576dffb640 1 -- 172.21.15.64:0/3471809284 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5770053dd0 con 0x7f577c0adb60 2023-11-24T08:05:42.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.152+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f577c15b730 con 0x7f577c0adb60 2023-11-24T08:05:42.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.153+0000 7f576dffb640 1 -- 172.21.15.64:0/3471809284 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f577c15b730 con 0x7f577c0adb60 2023-11-24T08:05:42.154 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:05:42.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f576006af00 msgr2=0x7f576006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f576006af00 0x7f576006d3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5778034000 tx=0x7f5778008030 comp rx=0 tx=0).stop 2023-11-24T08:05:42.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0adb60 msgr2=0x7f577c157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0adb60 0x7f577c157a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f577000c870 tx=0x7f577000cd30 comp rx=0 tx=0).stop 2023-11-24T08:05:42.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 shutdown_connections 2023-11-24T08:05:42.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f576006af00 0x7f576006d3c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f577c15b350 0x7f577c157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0adb60 0x7f577c157a10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 --2- 172.21.15.64:0/3471809284 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f577c09cb30 0x7f577c13f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 >> 172.21.15.64:0/3471809284 conn(0x7f577c0b0ba0 msgr2=0x7f577c159900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:42.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 shutdown_connections 2023-11-24T08:05:42.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.155+0000 7f578329b640 1 -- 172.21.15.64:0/3471809284 wait complete. 2023-11-24T08:05:42.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:05:42.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.406+0000 7f0aa6f3c640 1 Processor -- start 2023-11-24T08:05:42.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.407+0000 7f0aa6f3c640 1 -- start start 2023-11-24T08:05:42.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.408+0000 7f0aa6f3c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0aa00d1d90 0x7f0aa00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.408+0000 7f0aa6f3c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aa00d0fa0 0x7f0aa00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.408+0000 7f0aa6f3c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa00c6bf0 con 0x7f0aa015d3b0 2023-11-24T08:05:42.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.408+0000 7f0aa6f3c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa00d13f0 con 0x7f0aa00d0fa0 2023-11-24T08:05:42.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.408+0000 7f0aa6f3c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa00cedc0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.409+0000 7f0aa673b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aa00d0fa0 0x7f0aa00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.409+0000 7f0aa673b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aa00d0fa0 0x7f0aa00b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36590/0 (socket says 172.21.15.64:36590) 2023-11-24T08:05:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.409+0000 7f0aa673b640 1 -- 172.21.15.64:0/13677231 learned_addr learned my addr 172.21.15.64:0/13677231 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.409+0000 7f0aa5739640 1 --2- 172.21.15.64:0/13677231 >> v2:172.21.15.91:3300/0 conn(0x7f0aa00d1d90 0x7f0aa00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.409+0000 7f0aa673b640 1 -- 172.21.15.64:0/13677231 >> v2:172.21.15.91:3300/0 conn(0x7f0aa00d1d90 msgr2=0x7f0aa00ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.409+0000 7f0aa673b640 1 --2- 172.21.15.64:0/13677231 >> v2:172.21.15.91:3300/0 conn(0x7f0aa00d1d90 0x7f0aa00ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.409+0000 7f0aa673b640 1 -- 172.21.15.64:0/13677231 >> v1:172.21.15.91:6789/0 conn(0x7f0aa015d3b0 legacy=0x7f0aa00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.409+0000 7f0aa673b640 1 -- 172.21.15.64:0/13677231 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa00c6e40 con 0x7f0aa00d0fa0 2023-11-24T08:05:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.410+0000 7f0aa673b640 1 --2- 172.21.15.64:0/13677231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d0fa0 0x7f0aa00b3ca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00ca100 tx=0x7f0a9802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286568119191b64b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.411+0000 7f0aa4f38640 1 -- 172.21.15.64:0/13677231 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a980317e0 con 0x7f0aa00d0fa0 2023-11-24T08:05:42.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.411+0000 7f0aa4f38640 1 -- 172.21.15.64:0/13677231 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a98031940 con 0x7f0aa00d0fa0 2023-11-24T08:05:42.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.411+0000 7f0aa4f38640 1 -- 172.21.15.64:0/13677231 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98031cd0 con 0x7f0aa00d0fa0 2023-11-24T08:05:42.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.411+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/13677231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d0fa0 msgr2=0x7f0aa00b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.411+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/13677231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d0fa0 0x7f0aa00b3ca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00ca100 tx=0x7f0a9802b880 comp rx=0 tx=0).stop 2023-11-24T08:05:42.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.412+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/13677231 shutdown_connections 2023-11-24T08:05:42.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.412+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/13677231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d0fa0 0x7f0aa00b3ca0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.412+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/13677231 >> v2:172.21.15.91:3300/0 conn(0x7f0aa00d1d90 0x7f0aa00ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.412+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/13677231 >> 172.21.15.64:0/13677231 conn(0x7f0aa00d30a0 msgr2=0x7f0aa00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:42.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.412+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/13677231 shutdown_connections 2023-11-24T08:05:42.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.412+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/13677231 wait complete. 2023-11-24T08:05:42.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.413+0000 7f0aa6f3c640 1 Processor -- start 2023-11-24T08:05:42.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.414+0000 7f0aa6f3c640 1 -- start start 2023-11-24T08:05:42.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.414+0000 7f0aa6f3c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0aa00d0fa0 0x7f0aa0159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.414+0000 7f0aa6f3c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d1d90 0x7f0aa006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa6f3c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0aa015d3b0 0x7f0aa006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa6f3c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa0159690 con 0x7f0aa00d1d90 2023-11-24T08:05:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa6f3c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa00b5e70 con 0x7f0aa00d0fa0 2023-11-24T08:05:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa6f3c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa00b60c0 con 0x7f0aa015d3b0 2023-11-24T08:05:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5739640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d1d90 0x7f0aa006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5739640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d1d90 0x7f0aa006c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36594/0 (socket says 172.21.15.64:36594) 2023-11-24T08:05:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5739640 1 -- 172.21.15.64:0/2153623231 learned_addr learned my addr 172.21.15.64:0/2153623231 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:42.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5f3a640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0aa00d0fa0 0x7f0aa0159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa673b640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0aa015d3b0 0x7f0aa006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5739640 1 -- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0aa015d3b0 msgr2=0x7f0aa006c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5739640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0aa015d3b0 0x7f0aa006c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5739640 1 -- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0aa00d0fa0 msgr2=0x7f0aa0159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5739640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0aa00d0fa0 0x7f0aa0159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa5739640 1 -- 172.21.15.64:0/2153623231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a980094d0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.415+0000 7f0aa673b640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0aa015d3b0 0x7f0aa006c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.416+0000 7f0aa5739640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d1d90 0x7f0aa006c3a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d800 tx=0x7f0a9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.416+0000 7f0a9e7fc640 1 -- 172.21.15.64:0/2153623231 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400b6c0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.416+0000 7f0a9e7fc640 1 -- 172.21.15.64:0/2153623231 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9400b820 con 0x7f0aa00d1d90 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.416+0000 7f0a9e7fc640 1 -- 172.21.15.64:0/2153623231 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400bbb0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.416+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa00b63a0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.416+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa00b6960 con 0x7f0aa00d1d90 2023-11-24T08:05:42.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.418+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa004e840 con 0x7f0aa00d1d90 2023-11-24T08:05:42.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.418+0000 7f0a9e7fc640 1 -- 172.21.15.64:0/2153623231 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0a9400bd10 con 0x7f0aa00d1d90 2023-11-24T08:05:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.419+0000 7f0a9e7fc640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a8c06aed0 0x7f0a8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.419+0000 7f0a9e7fc640 1 -- 172.21.15.64:0/2153623231 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a94089be0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.420+0000 7f0aa5f3a640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a8c06aed0 0x7f0a8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.421+0000 7f0aa5f3a640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a8c06aed0 0x7f0a8c06d390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009f20 tx=0x7f0a900094d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:42.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.424+0000 7f0a9e7fc640 1 -- 172.21.15.64:0/2153623231 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a94054ca0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.617+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0aa00665c0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.618+0000 7f0a9e7fc640 1 -- 172.21.15.64:0/2153623231 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a940582c0 con 0x7f0aa00d1d90 2023-11-24T08:05:42.619 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:05:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a8c06aed0 msgr2=0x7f0a8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a8c06aed0 0x7f0a8c06d390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009f20 tx=0x7f0a900094d0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d1d90 msgr2=0x7f0aa006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d1d90 0x7f0aa006c3a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d800 tx=0x7f0a9400dcc0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 shutdown_connections 2023-11-24T08:05:42.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0a8c06aed0 0x7f0a8c06d390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0aa015d3b0 0x7f0aa006c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa00d1d90 0x7f0aa006c3a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 --2- 172.21.15.64:0/2153623231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0aa00d0fa0 0x7f0aa0159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 >> 172.21.15.64:0/2153623231 conn(0x7f0aa00d30a0 msgr2=0x7f0aa00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 shutdown_connections 2023-11-24T08:05:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.620+0000 7f0aa6f3c640 1 -- 172.21.15.64:0/2153623231 wait complete. 2023-11-24T08:05:42.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:05:42.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.856+0000 7f659f425640 1 Processor -- start 2023-11-24T08:05:42.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.856+0000 7f659f425640 1 -- start start 2023-11-24T08:05:42.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.857+0000 7f659f425640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6598146aa0 0x7f659806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659f425640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f659806cb20 0x7f6598071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659f425640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980adf70 con 0x7f65981466c0 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659f425640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659806cfd0 con 0x7f6598146aa0 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659f425640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65980accc0 con 0x7f659806cb20 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659e423640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f65981466c0 0x7f659815e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47264/0 (socket says 172.21.15.64:47264) 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659e423640 1 -- 172.21.15.64:0/1421839080 learned_addr learned my addr 172.21.15.64:0/1421839080 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659dc22640 1 --2- 172.21.15.64:0/1421839080 >> v2:172.21.15.64:3300/0 conn(0x7f6598146aa0 0x7f659806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659d421640 1 -- 172.21.15.64:0/1421839080 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872018666 0 0) 0x7f65980adf70 con 0x7f65981466c0 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659d421640 1 -- 172.21.15.64:0/1421839080 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6584003520 con 0x7f65981466c0 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.858+0000 7f659ec24640 1 --2- 172.21.15.64:0/1421839080 >> v2:172.21.15.91:3300/0 conn(0x7f659806cb20 0x7f6598071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.859+0000 7f659dc22640 1 -- 172.21.15.64:0/1421839080 >> v2:172.21.15.91:3300/0 conn(0x7f659806cb20 msgr2=0x7f6598071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:42.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.859+0000 7f659dc22640 1 --2- 172.21.15.64:0/1421839080 >> v2:172.21.15.91:3300/0 conn(0x7f659806cb20 0x7f6598071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.859+0000 7f659dc22640 1 -- 172.21.15.64:0/1421839080 >> v1:172.21.15.64:6789/0 conn(0x7f65981466c0 legacy=0x7f659815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.859+0000 7f659d421640 1 -- 172.21.15.64:0/1421839080 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3030378832 0 0) 0x7f6584003520 con 0x7f65981466c0 2023-11-24T08:05:42.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.859+0000 7f659dc22640 1 -- 172.21.15.64:0/1421839080 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980acfa0 con 0x7f6598146aa0 2023-11-24T08:05:42.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.860+0000 7f659dc22640 1 --2- 172.21.15.64:0/1421839080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f659806c5e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6598070a40 tx=0x7f658802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43aae68e0398f89 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:42.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.860+0000 7f659d421640 1 -- 172.21.15.64:0/1421839080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594001670 con 0x7f6598146aa0 2023-11-24T08:05:42.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.860+0000 7f659d421640 1 -- 172.21.15.64:0/1421839080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588033440 con 0x7f6598146aa0 2023-11-24T08:05:42.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.860+0000 7f659d421640 1 -- 172.21.15.64:0/1421839080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880337b0 con 0x7f6598146aa0 2023-11-24T08:05:42.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.861+0000 7f659f425640 1 -- 172.21.15.64:0/1421839080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 msgr2=0x7f659806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.861+0000 7f659f425640 1 --2- 172.21.15.64:0/1421839080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f659806c5e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6598070a40 tx=0x7f658802b7a0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.861+0000 7f659f425640 1 -- 172.21.15.64:0/1421839080 shutdown_connections 2023-11-24T08:05:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.861+0000 7f659f425640 1 --2- 172.21.15.64:0/1421839080 >> v2:172.21.15.91:3300/0 conn(0x7f659806cb20 0x7f6598071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.861+0000 7f659f425640 1 --2- 172.21.15.64:0/1421839080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f659806c5e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.861+0000 7f659f425640 1 -- 172.21.15.64:0/1421839080 >> 172.21.15.64:0/1421839080 conn(0x7f659806a620 msgr2=0x7f659806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.861+0000 7f659f425640 1 -- 172.21.15.64:0/1421839080 shutdown_connections 2023-11-24T08:05:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.861+0000 7f659f425640 1 -- 172.21.15.64:0/1421839080 wait complete. 2023-11-24T08:05:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.863+0000 7f659f425640 1 Processor -- start 2023-11-24T08:05:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.863+0000 7f659f425640 1 -- start start 2023-11-24T08:05:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659f425640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f659806cb20 0x7f659815c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659f425640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65981466c0 0x7f6598160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659f425640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f6598161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659f425640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6598143730 con 0x7f6598146aa0 2023-11-24T08:05:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659f425640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659815cce0 con 0x7f659806cb20 2023-11-24T08:05:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659f425640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6598142690 con 0x7f65981466c0 2023-11-24T08:05:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659ec24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f6598161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659ec24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f6598161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36606/0 (socket says 172.21.15.64:36606) 2023-11-24T08:05:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659ec24640 1 -- 172.21.15.64:0/3477329426 learned_addr learned my addr 172.21.15.64:0/3477329426 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.864+0000 7f659e423640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f659806cb20 0x7f659815c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f659ec24640 1 -- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65981466c0 msgr2=0x7f6598160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f659ec24640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65981466c0 0x7f6598160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f659ec24640 1 -- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f659806cb20 msgr2=0x7f659815c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f659ec24640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f659806cb20 0x7f659815c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f659ec24640 1 -- 172.21.15.64:0/3477329426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6588009400 con 0x7f6598146aa0 2023-11-24T08:05:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f659ec24640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f6598161260 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6590007e40 tx=0x7f659000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:42.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f658effd640 1 -- 172.21.15.64:0/3477329426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590014620 con 0x7f6598146aa0 2023-11-24T08:05:42.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f658effd640 1 -- 172.21.15.64:0/3477329426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659000ee90 con 0x7f6598146aa0 2023-11-24T08:05:42.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f658effd640 1 -- 172.21.15.64:0/3477329426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900149e0 con 0x7f6598146aa0 2023-11-24T08:05:42.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6598142970 con 0x7f6598146aa0 2023-11-24T08:05:42.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.865+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6598142f30 con 0x7f6598146aa0 2023-11-24T08:05:42.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.866+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659804e840 con 0x7f6598146aa0 2023-11-24T08:05:42.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.867+0000 7f658effd640 1 -- 172.21.15.64:0/3477329426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6590014b40 con 0x7f6598146aa0 2023-11-24T08:05:42.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.868+0000 7f658effd640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f657c06aed0 0x7f657c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:42.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.868+0000 7f658effd640 1 -- 172.21.15.64:0/3477329426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f659008a070 con 0x7f6598146aa0 2023-11-24T08:05:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.869+0000 7f659e423640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f657c06aed0 0x7f657c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.870+0000 7f659e423640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f657c06aed0 0x7f657c06d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f65940017a0 tx=0x7f6594009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:42.872+0000 7f658effd640 1 -- 172.21.15.64:0/3477329426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65900bce90 con 0x7f6598146aa0 2023-11-24T08:05:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.083+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f65980b4040 con 0x7f6598146aa0 2023-11-24T08:05:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.084+0000 7f658effd640 1 -- 172.21.15.64:0/3477329426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f65900550c0 con 0x7f6598146aa0 2023-11-24T08:05:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:05:43.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f657c06aed0 msgr2=0x7f657c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:43.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f657c06aed0 0x7f657c06d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f65940017a0 tx=0x7f6594009040 comp rx=0 tx=0).stop 2023-11-24T08:05:43.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 msgr2=0x7f6598161260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:43.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f6598161260 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6590007e40 tx=0x7f659000e4b0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 shutdown_connections 2023-11-24T08:05:43.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f657c06aed0 0x7f657c06d390 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6598146aa0 0x7f6598161260 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f65981466c0 0x7f6598160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 --2- 172.21.15.64:0/3477329426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f659806cb20 0x7f659815c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 >> 172.21.15.64:0/3477329426 conn(0x7f659806a620 msgr2=0x7f6598143cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:43.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 shutdown_connections 2023-11-24T08:05:43.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.088+0000 7f659f425640 1 -- 172.21.15.64:0/3477329426 wait complete. 2023-11-24T08:05:43.108 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T08:05:43.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:43.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T08:05:43.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[102438]: fuse finished with error 0 and tester_r 0 2023-11-24T08:05:43.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:05:43.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:43.204 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:05:43.308 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:05:43.308 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:05:43.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:43.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.600+0000 7f2b23fff640 1 Processor -- start 2023-11-24T08:05:43.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.601+0000 7f2b23fff640 1 -- start start 2023-11-24T08:05:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.602+0000 7f2b23fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2b2415d370 0x7f2b240d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.602+0000 7f2b23fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2b240d00d0 0x7f2b240b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.602+0000 7f2b23fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b240afd30 con 0x7f2b240b1f00 2023-11-24T08:05:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.602+0000 7f2b23fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b240d0c30 con 0x7f2b240d00d0 2023-11-24T08:05:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.602+0000 7f2b23fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b240cef60 con 0x7f2b2415d370 2023-11-24T08:05:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b237fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2b240d00d0 0x7f2b240b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b237fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2b240d00d0 0x7f2b240b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36618/0 (socket says 172.21.15.64:36618) 2023-11-24T08:05:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b237fe640 1 -- 172.21.15.64:0/3269568535 learned_addr learned my addr 172.21.15.64:0/3269568535 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b22ffd640 1 --2- 172.21.15.64:0/3269568535 >> v2:172.21.15.91:3300/0 conn(0x7f2b2415d370 0x7f2b240d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b237fe640 1 -- 172.21.15.64:0/3269568535 >> v2:172.21.15.91:3300/0 conn(0x7f2b2415d370 msgr2=0x7f2b240d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b237fe640 1 --2- 172.21.15.64:0/3269568535 >> v2:172.21.15.91:3300/0 conn(0x7f2b2415d370 0x7f2b240d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b237fe640 1 -- 172.21.15.64:0/3269568535 >> v1:172.21.15.91:6789/0 conn(0x7f2b240b1f00 legacy=0x7f2b240cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b237fe640 1 -- 172.21.15.64:0/3269568535 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b240d0ef0 con 0x7f2b240d00d0 2023-11-24T08:05:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.603+0000 7f2b22ffd640 1 --2- 172.21.15.64:0/3269568535 >> v2:172.21.15.91:3300/0 conn(0x7f2b2415d370 0x7f2b240d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.605+0000 7f2b237fe640 1 --2- 172.21.15.64:0/3269568535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b240b04a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2b2408b8b0 tx=0x7f2b1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b8893b13934440 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.605+0000 7f2b21ffb640 1 -- 172.21.15.64:0/3269568535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c031870 con 0x7f2b240d00d0 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.605+0000 7f2b21ffb640 1 -- 172.21.15.64:0/3269568535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c0319d0 con 0x7f2b240d00d0 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.605+0000 7f2b21ffb640 1 -- 172.21.15.64:0/3269568535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c031d60 con 0x7f2b240d00d0 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.606+0000 7f2b23fff640 1 -- 172.21.15.64:0/3269568535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 msgr2=0x7f2b240b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.606+0000 7f2b23fff640 1 --2- 172.21.15.64:0/3269568535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b240b04a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2b2408b8b0 tx=0x7f2b1c02b910 comp rx=0 tx=0).stop 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.606+0000 7f2b23fff640 1 -- 172.21.15.64:0/3269568535 shutdown_connections 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.606+0000 7f2b23fff640 1 --2- 172.21.15.64:0/3269568535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b240b04a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.606+0000 7f2b23fff640 1 --2- 172.21.15.64:0/3269568535 >> v2:172.21.15.91:3300/0 conn(0x7f2b2415d370 0x7f2b240d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.606+0000 7f2b23fff640 1 -- 172.21.15.64:0/3269568535 >> 172.21.15.64:0/3269568535 conn(0x7f2b2409c3d0 msgr2=0x7f2b240d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.606+0000 7f2b23fff640 1 -- 172.21.15.64:0/3269568535 shutdown_connections 2023-11-24T08:05:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.606+0000 7f2b23fff640 1 -- 172.21.15.64:0/3269568535 wait complete. 2023-11-24T08:05:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.608+0000 7f2b23fff640 1 Processor -- start 2023-11-24T08:05:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.608+0000 7f2b23fff640 1 -- start start 2023-11-24T08:05:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b23fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b240b1f00 0x7f2b241f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b241f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b23fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b2415d370 0x7f2b240cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b23fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240c7110 con 0x7f2b240d00d0 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b23fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b241f7010 con 0x7f2b2415d370 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b23fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b241f3a60 con 0x7f2b240b1f00 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b227fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b241f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b227fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b241f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36624/0 (socket says 172.21.15.64:36624) 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.609+0000 7f2b227fc640 1 -- 172.21.15.64:0/2181326339 learned_addr learned my addr 172.21.15.64:0/2181326339 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b237fe640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b2415d370 0x7f2b240cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b22ffd640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b240b1f00 0x7f2b241f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b227fc640 1 -- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b240b1f00 msgr2=0x7f2b241f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b227fc640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b240b1f00 0x7f2b241f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b227fc640 1 -- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b2415d370 msgr2=0x7f2b240cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b227fc640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b2415d370 0x7f2b240cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b227fc640 1 -- 172.21.15.64:0/2181326339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c0094d0 con 0x7f2b240d00d0 2023-11-24T08:05:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b22ffd640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b240b1f00 0x7f2b241f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b227fc640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b241f2440 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800c400 tx=0x7f2b1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b0b7fe640 1 -- 172.21.15.64:0/2181326339 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180052d0 con 0x7f2b240d00d0 2023-11-24T08:05:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b0b7fe640 1 -- 172.21.15.64:0/2181326339 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b18005430 con 0x7f2b240d00d0 2023-11-24T08:05:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b241f2ca0 con 0x7f2b240d00d0 2023-11-24T08:05:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.610+0000 7f2b0b7fe640 1 -- 172.21.15.64:0/2181326339 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18003830 con 0x7f2b240d00d0 2023-11-24T08:05:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.611+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b241f3260 con 0x7f2b240d00d0 2023-11-24T08:05:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.612+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2404e840 con 0x7f2b240d00d0 2023-11-24T08:05:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.613+0000 7f2b0b7fe640 1 -- 172.21.15.64:0/2181326339 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2b18003990 con 0x7f2b240d00d0 2023-11-24T08:05:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.614+0000 7f2b0b7fe640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b1006aed0 0x7f2b1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.614+0000 7f2b0b7fe640 1 -- 172.21.15.64:0/2181326339 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b18088ca0 con 0x7f2b240d00d0 2023-11-24T08:05:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.614+0000 7f2b22ffd640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b1006aed0 0x7f2b1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.616+0000 7f2b22ffd640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b1006aed0 0x7f2b1006d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2b140094c0 tx=0x7f2b1400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.618+0000 7f2b0b7fe640 1 -- 172.21.15.64:0/2181326339 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b18053c60 con 0x7f2b240d00d0 2023-11-24T08:05:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.804+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b240b3ae0 con 0x7f2b240d00d0 2023-11-24T08:05:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.806+0000 7f2b0b7fe640 1 -- 172.21.15.64:0/2181326339 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f2b18017090 con 0x7f2b240d00d0 2023-11-24T08:05:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:43.808 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:21.012478+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:05:17.535151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:05:18.499872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.808+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b1006aed0 msgr2=0x7f2b1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.808+0000 7f2b23fff640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b1006aed0 0x7f2b1006d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2b140094c0 tx=0x7f2b1400b040 comp rx=0 tx=0).stop 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.808+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 msgr2=0x7f2b241f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.808+0000 7f2b23fff640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b241f2440 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800c400 tx=0x7f2b1800c8c0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.808+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 shutdown_connections 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.809+0000 7f2b23fff640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b1006aed0 0x7f2b1006d390 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.809+0000 7f2b23fff640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b2415d370 0x7f2b240cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.809+0000 7f2b23fff640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b240d00d0 0x7f2b241f2440 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.809+0000 7f2b23fff640 1 --2- 172.21.15.64:0/2181326339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b240b1f00 0x7f2b241f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.809+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 >> 172.21.15.64:0/2181326339 conn(0x7f2b2409c3d0 msgr2=0x7f2b2406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.809+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 shutdown_connections 2023-11-24T08:05:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:43.809+0000 7f2b23fff640 1 -- 172.21.15.64:0/2181326339 wait complete. 2023-11-24T08:05:43.831 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:05:17.535151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:05:18.499872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:05:43.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:05:44.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.062+0000 7f251cc39640 1 Processor -- start 2023-11-24T08:05:44.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.062+0000 7f251cc39640 1 -- start start 2023-11-24T08:05:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.063+0000 7f251cc39640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f25181435d0 0x7f2518071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.063+0000 7f251cc39640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f25181439b0 0x7f25180b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f251cc39640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 0x7f2518070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f251cc39640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25180b43e0 con 0x7f25181439b0 2023-11-24T08:05:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f251cc39640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2518070d40 con 0x7f25181435d0 2023-11-24T08:05:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f251cc39640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2518071020 con 0x7f25180b3f70 2023-11-24T08:05:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f2516ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f25181439b0 0x7f25180b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f2516ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f25181439b0 0x7f25180b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36636/0 (socket says 172.21.15.64:36636) 2023-11-24T08:05:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f2516ffd640 1 -- 172.21.15.64:0/282255379 learned_addr learned my addr 172.21.15.64:0/282255379 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f2516ffd640 1 -- 172.21.15.64:0/282255379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 msgr2=0x7f2518070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f2516ffd640 1 --2- 172.21.15.64:0/282255379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 0x7f2518070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f2516ffd640 1 -- 172.21.15.64:0/282255379 >> v2:172.21.15.91:3300/0 conn(0x7f25181435d0 msgr2=0x7f2518071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f2516ffd640 1 --2- 172.21.15.64:0/282255379 >> v2:172.21.15.91:3300/0 conn(0x7f25181435d0 0x7f2518071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.064+0000 7f2516ffd640 1 -- 172.21.15.64:0/282255379 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2518071300 con 0x7f25181439b0 2023-11-24T08:05:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.065+0000 7f2516ffd640 1 --2- 172.21.15.64:0/282255379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180b3a30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f25180cfc10 tx=0x7f250c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179df64aa8b1ec8f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f25167fc640 1 -- 172.21.15.64:0/282255379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c031870 con 0x7f25181439b0 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f25167fc640 1 -- 172.21.15.64:0/282255379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c0319d0 con 0x7f25181439b0 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f25167fc640 1 -- 172.21.15.64:0/282255379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c031d60 con 0x7f25181439b0 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 -- 172.21.15.64:0/282255379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 msgr2=0x7f25180b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 --2- 172.21.15.64:0/282255379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180b3a30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f25180cfc10 tx=0x7f250c02b910 comp rx=0 tx=0).stop 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 -- 172.21.15.64:0/282255379 shutdown_connections 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 --2- 172.21.15.64:0/282255379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 0x7f2518070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 --2- 172.21.15.64:0/282255379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180b3a30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 --2- 172.21.15.64:0/282255379 >> v2:172.21.15.91:3300/0 conn(0x7f25181435d0 0x7f2518071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 -- 172.21.15.64:0/282255379 >> 172.21.15.64:0/282255379 conn(0x7f25180cec10 msgr2=0x7f25180cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:44.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 -- 172.21.15.64:0/282255379 shutdown_connections 2023-11-24T08:05:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.066+0000 7f251cc39640 1 -- 172.21.15.64:0/282255379 wait complete. 2023-11-24T08:05:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.068+0000 7f251cc39640 1 Processor -- start 2023-11-24T08:05:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.068+0000 7f251cc39640 1 -- start start 2023-11-24T08:05:44.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.069+0000 7f251cc39640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 0x7f2518158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.069+0000 7f251cc39640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25181435d0 0x7f25180ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.069+0000 7f251cc39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.069+0000 7f251cc39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25180adae0 con 0x7f25181439b0 2023-11-24T08:05:44.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.069+0000 7f251cc39640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25180ac870 con 0x7f25181435d0 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.069+0000 7f251cc39640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25180ce870 con 0x7f25180b3f70 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2517fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2517fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36638/0 (socket says 172.21.15.64:36638) 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2517fff640 1 -- 172.21.15.64:0/2221621217 learned_addr learned my addr 172.21.15.64:0/2221621217 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2516ffd640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25181435d0 0x7f25180ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f25177fe640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 0x7f2518158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2517fff640 1 -- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 msgr2=0x7f2518158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2517fff640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 0x7f2518158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2517fff640 1 -- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25181435d0 msgr2=0x7f25180ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2517fff640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25181435d0 0x7f25180ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f2517fff640 1 -- 172.21.15.64:0/2221621217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c0094d0 con 0x7f25181439b0 2023-11-24T08:05:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.070+0000 7f25177fe640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 0x7f2518158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.071+0000 7f2517fff640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180ad5a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f251000b970 tx=0x7f251000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.071+0000 7f24fffff640 1 -- 172.21.15.64:0/2221621217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251000cb40 con 0x7f25181439b0 2023-11-24T08:05:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.071+0000 7f24fffff640 1 -- 172.21.15.64:0/2221621217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251000ccc0 con 0x7f25181439b0 2023-11-24T08:05:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.071+0000 7f24fffff640 1 -- 172.21.15.64:0/2221621217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510014680 con 0x7f25181439b0 2023-11-24T08:05:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.071+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25180acac0 con 0x7f25181439b0 2023-11-24T08:05:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.071+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25180ad080 con 0x7f25181439b0 2023-11-24T08:05:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.073+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25180ad270 con 0x7f25181439b0 2023-11-24T08:05:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.073+0000 7f24fffff640 1 -- 172.21.15.64:0/2221621217 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f251001d3b0 con 0x7f25181439b0 2023-11-24T08:05:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.074+0000 7f24fffff640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f250406aed0 0x7f250406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.074+0000 7f24fffff640 1 -- 172.21.15.64:0/2221621217 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25100895e0 con 0x7f25181439b0 2023-11-24T08:05:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.075+0000 7f25177fe640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f250406aed0 0x7f250406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.076+0000 7f25177fe640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f250406aed0 0x7f250406d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2508006fd0 tx=0x7f2508008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:44.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.079+0000 7f24fffff640 1 -- 172.21.15.64:0/2221621217 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25100546a0 con 0x7f25181439b0 2023-11-24T08:05:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.270+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f251804e840 con 0x7f25181439b0 2023-11-24T08:05:44.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.270+0000 7f24fffff640 1 -- 172.21.15.64:0/2221621217 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2510019020 con 0x7f25181439b0 2023-11-24T08:05:44.272 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:05:44.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f250406aed0 msgr2=0x7f250406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f250406aed0 0x7f250406d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2508006fd0 tx=0x7f2508008040 comp rx=0 tx=0).stop 2023-11-24T08:05:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 msgr2=0x7f25180ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180ad5a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f251000b970 tx=0x7f251000be30 comp rx=0 tx=0).stop 2023-11-24T08:05:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 shutdown_connections 2023-11-24T08:05:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f250406aed0 0x7f250406d390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25181439b0 0x7f25180ad5a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25181435d0 0x7f25180ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 --2- 172.21.15.64:0/2221621217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f25180b3f70 0x7f2518158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 >> 172.21.15.64:0/2221621217 conn(0x7f25180cec10 msgr2=0x7f25180b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 shutdown_connections 2023-11-24T08:05:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.273+0000 7f251cc39640 1 -- 172.21.15.64:0/2221621217 wait complete. 2023-11-24T08:05:44.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:05:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.521+0000 7f1851c27640 1 Processor -- start 2023-11-24T08:05:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.522+0000 7f1851c27640 1 -- start start 2023-11-24T08:05:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.523+0000 7f1851c27640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f184c15efa0 0x7f184c161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.523+0000 7f1851c27640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c0b3080 0x7f184c06c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.523+0000 7f1851c27640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184c06c9a0 con 0x7f184c06da40 2023-11-24T08:05:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.523+0000 7f1851c27640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184c06cc80 con 0x7f184c15efa0 2023-11-24T08:05:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.523+0000 7f1851c27640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c06cf60 con 0x7f184c0b3080 2023-11-24T08:05:44.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.524+0000 7f1850c25640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f184c15efa0 0x7f184c161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.524+0000 7f184bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f184c06da40 0x7f184c0b2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47266/0 (socket says 172.21.15.64:47266) 2023-11-24T08:05:44.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.524+0000 7f184bfff640 1 -- 172.21.15.64:0/3902419007 learned_addr learned my addr 172.21.15.64:0/3902419007 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.524+0000 7f1851426640 1 --2- 172.21.15.64:0/3902419007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c0b3080 0x7f184c06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.525+0000 7f1850c25640 1 -- 172.21.15.64:0/3902419007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c0b3080 msgr2=0x7f184c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.525+0000 7f1850c25640 1 --2- 172.21.15.64:0/3902419007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c0b3080 0x7f184c06c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.525+0000 7f1850c25640 1 -- 172.21.15.64:0/3902419007 >> v1:172.21.15.64:6789/0 conn(0x7f184c06da40 legacy=0x7f184c0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.525+0000 7f1850c25640 1 -- 172.21.15.64:0/3902419007 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c06d240 con 0x7f184c15efa0 2023-11-24T08:05:44.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.525+0000 7f1851426640 1 --2- 172.21.15.64:0/3902419007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c0b3080 0x7f184c06c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:44.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.526+0000 7f1850c25640 1 --2- 172.21.15.64:0/3902419007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c15efa0 0x7f184c161ac0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f18400097a0 tx=0x7f184002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73f117a9e2f75a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:44.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.526+0000 7f184b7fe640 1 -- 172.21.15.64:0/3902419007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840031780 con 0x7f184c15efa0 2023-11-24T08:05:44.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f184b7fe640 1 -- 172.21.15.64:0/3902419007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18400318e0 con 0x7f184c15efa0 2023-11-24T08:05:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f184b7fe640 1 -- 172.21.15.64:0/3902419007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840031c70 con 0x7f184c15efa0 2023-11-24T08:05:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f1851c27640 1 -- 172.21.15.64:0/3902419007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c15efa0 msgr2=0x7f184c161ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f1851c27640 1 --2- 172.21.15.64:0/3902419007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c15efa0 0x7f184c161ac0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f18400097a0 tx=0x7f184002b820 comp rx=0 tx=0).stop 2023-11-24T08:05:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f1851c27640 1 -- 172.21.15.64:0/3902419007 shutdown_connections 2023-11-24T08:05:44.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f1851c27640 1 --2- 172.21.15.64:0/3902419007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c0b3080 0x7f184c06c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f1851c27640 1 --2- 172.21.15.64:0/3902419007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c15efa0 0x7f184c161ac0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f1851c27640 1 -- 172.21.15.64:0/3902419007 >> 172.21.15.64:0/3902419007 conn(0x7f184c0cec10 msgr2=0x7f184c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f1851c27640 1 -- 172.21.15.64:0/3902419007 shutdown_connections 2023-11-24T08:05:44.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.527+0000 7f1851c27640 1 -- 172.21.15.64:0/3902419007 wait complete. 2023-11-24T08:05:44.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.529+0000 7f1851c27640 1 Processor -- start 2023-11-24T08:05:44.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.529+0000 7f1851c27640 1 -- start start 2023-11-24T08:05:44.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.530+0000 7f1851c27640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c06da40 0x7f184c148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.530+0000 7f1851c27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c0b3080 0x7f184c1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.530+0000 7f1851c27640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f184c15efa0 0x7f184c0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.530+0000 7f1851c27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0b06e0 con 0x7f184c0b3080 2023-11-24T08:05:44.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.530+0000 7f1851c27640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0b0120 con 0x7f184c15efa0 2023-11-24T08:05:44.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.530+0000 7f1851c27640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0b0400 con 0x7f184c06da40 2023-11-24T08:05:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c0b3080 0x7f184c1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c0b3080 0x7f184c1547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36642/0 (socket says 172.21.15.64:36642) 2023-11-24T08:05:44.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 -- 172.21.15.64:0/170540377 learned_addr learned my addr 172.21.15.64:0/170540377 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f1850c25640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c06da40 0x7f184c148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f1851426640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f184c15efa0 0x7f184c0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 -- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c06da40 msgr2=0x7f184c148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c06da40 0x7f184c148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 -- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f184c15efa0 msgr2=0x7f184c0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f184c15efa0 0x7f184c0af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 -- 172.21.15.64:0/170540377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840009400 con 0x7f184c0b3080 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f1850c25640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c06da40 0x7f184c148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f1851426640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f184c15efa0 0x7f184c0af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.531+0000 7f184bfff640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c0b3080 0x7f184c1547c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f183c00c370 tx=0x7f183c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.532+0000 7f18497fa640 1 -- 172.21.15.64:0/170540377 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c005ce0 con 0x7f184c0b3080 2023-11-24T08:05:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.532+0000 7f18497fa640 1 -- 172.21.15.64:0/170540377 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c005e40 con 0x7f184c0b3080 2023-11-24T08:05:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.532+0000 7f18497fa640 1 -- 172.21.15.64:0/170540377 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c007310 con 0x7f184c0b3080 2023-11-24T08:05:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.532+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c153fc0 con 0x7f184c0b3080 2023-11-24T08:05:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.532+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c154d00 con 0x7f184c0b3080 2023-11-24T08:05:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.533+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184c154ef0 con 0x7f184c0b3080 2023-11-24T08:05:44.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.534+0000 7f18497fa640 1 -- 172.21.15.64:0/170540377 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f183c007470 con 0x7f184c0b3080 2023-11-24T08:05:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.535+0000 7f18497fa640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f183806aed0 0x7f183806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.535+0000 7f18497fa640 1 -- 172.21.15.64:0/170540377 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f183c088b30 con 0x7f184c0b3080 2023-11-24T08:05:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.536+0000 7f1850c25640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f183806aed0 0x7f183806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.538+0000 7f1850c25640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f183806aed0 0x7f183806d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1840036000 tx=0x7f184003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.540+0000 7f18497fa640 1 -- 172.21.15.64:0/170540377 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f183c053c80 con 0x7f184c0b3080 2023-11-24T08:05:44.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.738+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f184c1544e0 con 0x7f184c0b3080 2023-11-24T08:05:44.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.739+0000 7f18497fa640 1 -- 172.21.15.64:0/170540377 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f183c0170a0 con 0x7f184c0b3080 2023-11-24T08:05:44.741 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:05:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.741+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f183806aed0 msgr2=0x7f183806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.741+0000 7f1851c27640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f183806aed0 0x7f183806d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1840036000 tx=0x7f184003d040 comp rx=0 tx=0).stop 2023-11-24T08:05:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.741+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c0b3080 msgr2=0x7f184c1547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c0b3080 0x7f184c1547c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f183c00c370 tx=0x7f183c00c830 comp rx=0 tx=0).stop 2023-11-24T08:05:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 shutdown_connections 2023-11-24T08:05:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f183806aed0 0x7f183806d390 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f184c15efa0 0x7f184c0af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c0b3080 0x7f184c1547c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 --2- 172.21.15.64:0/170540377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f184c06da40 0x7f184c148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:44.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 >> 172.21.15.64:0/170540377 conn(0x7f184c0cec10 msgr2=0x7f184c162c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:44.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 shutdown_connections 2023-11-24T08:05:44.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.742+0000 7f1851c27640 1 -- 172.21.15.64:0/170540377 wait complete. 2023-11-24T08:05:44.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:05:44.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.995+0000 7f3c3ddff640 1 Processor -- start 2023-11-24T08:05:44.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.995+0000 7f3c3ddff640 1 -- start start 2023-11-24T08:05:44.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.997+0000 7f3c3ddff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3c380b4140 0x7f3c38071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.997+0000 7f3c3ddff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c381439b0 0x7f3c38070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:44.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.997+0000 7f3c3ddff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c38070d40 con 0x7f3c381435d0 2023-11-24T08:05:44.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.997+0000 7f3c3ddff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c38071020 con 0x7f3c380b4140 2023-11-24T08:05:44.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.997+0000 7f3c3ddff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38071300 con 0x7f3c381439b0 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.997+0000 7f3c37fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3c380b4140 0x7f3c38071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.997+0000 7f3c3cdfd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3c381435d0 0x7f3c380b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47270/0 (socket says 172.21.15.64:47270) 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.997+0000 7f3c3cdfd640 1 -- 172.21.15.64:0/65437750 learned_addr learned my addr 172.21.15.64:0/65437750 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.998+0000 7f3c3d5fe640 1 --2- 172.21.15.64:0/65437750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c381439b0 0x7f3c38070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.998+0000 7f3c37fff640 1 -- 172.21.15.64:0/65437750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c381439b0 msgr2=0x7f3c38070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.998+0000 7f3c37fff640 1 --2- 172.21.15.64:0/65437750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c381439b0 0x7f3c38070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.998+0000 7f3c37fff640 1 -- 172.21.15.64:0/65437750 >> v1:172.21.15.64:6789/0 conn(0x7f3c381435d0 legacy=0x7f3c380b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.998+0000 7f3c37fff640 1 -- 172.21.15.64:0/65437750 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c380715e0 con 0x7f3c380b4140 2023-11-24T08:05:45.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.998+0000 7f3c3d5fe640 1 --2- 172.21.15.64:0/65437750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c381439b0 0x7f3c38070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:45.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:44.999+0000 7f3c37fff640 1 --2- 172.21.15.64:0/65437750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c380b4140 0x7f3c38071de0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3c240097a0 tx=0x7f3c2402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669efe90863b38c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:45.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.000+0000 7f3c377fe640 1 -- 172.21.15.64:0/65437750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2402f8a0 con 0x7f3c380b4140 2023-11-24T08:05:45.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.000+0000 7f3c377fe640 1 -- 172.21.15.64:0/65437750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2402fa00 con 0x7f3c380b4140 2023-11-24T08:05:45.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.000+0000 7f3c377fe640 1 -- 172.21.15.64:0/65437750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2402fd90 con 0x7f3c380b4140 2023-11-24T08:05:45.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.000+0000 7f3c3ddff640 1 -- 172.21.15.64:0/65437750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c380b4140 msgr2=0x7f3c38071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.000+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/65437750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c380b4140 0x7f3c38071de0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3c240097a0 tx=0x7f3c2402dae0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.001+0000 7f3c3ddff640 1 -- 172.21.15.64:0/65437750 shutdown_connections 2023-11-24T08:05:45.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.001+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/65437750 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c381439b0 0x7f3c38070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.001+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/65437750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c380b4140 0x7f3c38071de0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.001+0000 7f3c3ddff640 1 -- 172.21.15.64:0/65437750 >> 172.21.15.64:0/65437750 conn(0x7f3c380cec10 msgr2=0x7f3c380cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:45.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.001+0000 7f3c3ddff640 1 -- 172.21.15.64:0/65437750 shutdown_connections 2023-11-24T08:05:45.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.001+0000 7f3c3ddff640 1 -- 172.21.15.64:0/65437750 wait complete. 2023-11-24T08:05:45.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.002+0000 7f3c3ddff640 1 Processor -- start 2023-11-24T08:05:45.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.003+0000 7f3c3ddff640 1 -- start start 2023-11-24T08:05:45.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c3ddff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c380b4140 0x7f3c3813a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c3ddff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c381435d0 0x7f3c3815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c3ddff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c381439b0 0x7f3c3813dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c3ddff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381f36d0 con 0x7f3c381435d0 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c3ddff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c380b3540 con 0x7f3c381439b0 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c3ddff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3815cfc0 con 0x7f3c380b4140 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c381435d0 0x7f3c3815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c381435d0 0x7f3c3815ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36664/0 (socket says 172.21.15.64:36664) 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c37fff640 1 -- 172.21.15.64:0/4044207307 learned_addr learned my addr 172.21.15.64:0/4044207307 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c3cdfd640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c380b4140 0x7f3c3813a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c37fff640 1 -- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c380b4140 msgr2=0x7f3c3813a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c37fff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c380b4140 0x7f3c3813a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c37fff640 1 -- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c381439b0 msgr2=0x7f3c3813dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c3d5fe640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c381439b0 0x7f3c3813dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c37fff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c381439b0 0x7f3c3813dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.004+0000 7f3c37fff640 1 -- 172.21.15.64:0/4044207307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24009400 con 0x7f3c381435d0 2023-11-24T08:05:45.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.005+0000 7f3c37fff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c381435d0 0x7f3c3815ca80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3c2402e0f0 tx=0x7f3c240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:45.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.005+0000 7f3c357fa640 1 -- 172.21.15.64:0/4044207307 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24030ab0 con 0x7f3c381435d0 2023-11-24T08:05:45.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.005+0000 7f3c357fa640 1 -- 172.21.15.64:0/4044207307 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c24030c30 con 0x7f3c381435d0 2023-11-24T08:05:45.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.005+0000 7f3c357fa640 1 -- 172.21.15.64:0/4044207307 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24042410 con 0x7f3c381435d0 2023-11-24T08:05:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.005+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c381f2d30 con 0x7f3c381435d0 2023-11-24T08:05:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.005+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c381f32f0 con 0x7f3c381435d0 2023-11-24T08:05:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.006+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c380722f0 con 0x7f3c381435d0 2023-11-24T08:05:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.007+0000 7f3c357fa640 1 -- 172.21.15.64:0/4044207307 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3c24039400 con 0x7f3c381435d0 2023-11-24T08:05:45.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.008+0000 7f3c357fa640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c1c06aed0 0x7f3c1c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.008+0000 7f3c357fa640 1 -- 172.21.15.64:0/4044207307 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c240ab3e0 con 0x7f3c381435d0 2023-11-24T08:05:45.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.009+0000 7f3c3cdfd640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c1c06aed0 0x7f3c1c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.010+0000 7f3c3cdfd640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c1c06aed0 0x7f3c1c06d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005880 tx=0x7f3c2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:45.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.013+0000 7f3c357fa640 1 -- 172.21.15.64:0/4044207307 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c24076320 con 0x7f3c381435d0 2023-11-24T08:05:45.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:05:45.100 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:05:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.211+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c3804e840 con 0x7f3c381435d0 2023-11-24T08:05:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.212+0000 7f3c357fa640 1 -- 172.21.15.64:0/4044207307 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3c24079940 con 0x7f3c381435d0 2023-11-24T08:05:45.214 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c1c06aed0 msgr2=0x7f3c1c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c1c06aed0 0x7f3c1c06d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005880 tx=0x7f3c2c009040 comp rx=0 tx=0).stop 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c381435d0 msgr2=0x7f3c3815ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c381435d0 0x7f3c3815ca80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3c2402e0f0 tx=0x7f3c240046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 shutdown_connections 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3c1c06aed0 0x7f3c1c06d390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3c381439b0 0x7f3c3813dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c381435d0 0x7f3c3815ca80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 --2- 172.21.15.64:0/4044207307 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3c380b4140 0x7f3c3813a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.214+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 >> 172.21.15.64:0/4044207307 conn(0x7f3c380cec10 msgr2=0x7f3c3813d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:45.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.215+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 shutdown_connections 2023-11-24T08:05:45.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.215+0000 7f3c3ddff640 1 -- 172.21.15.64:0/4044207307 wait complete. 2023-11-24T08:05:45.236 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:05:45.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.510+0000 7fa88d4aa640 1 Processor -- start 2023-11-24T08:05:45.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.510+0000 7fa88d4aa640 1 -- start start 2023-11-24T08:05:45.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.512+0000 7fa88d4aa640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa888070550 0x7fa888160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.512+0000 7fa88d4aa640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa888161940 con 0x7fa8880b2550 2023-11-24T08:05:45.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.512+0000 7fa88d4aa640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8880b3040 con 0x7fa888161530 2023-11-24T08:05:45.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.512+0000 7fa88d4aa640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa888070cf0 con 0x7fa888070550 2023-11-24T08:05:45.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.512+0000 7fa887fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa8880b2550 0x7fa8880b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47274/0 (socket says 172.21.15.64:47274) 2023-11-24T08:05:45.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.512+0000 7fa887fff640 1 -- 172.21.15.64:0/3914117787 learned_addr learned my addr 172.21.15.64:0/3914117787 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:45.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.512+0000 7fa8877fe640 1 --2- 172.21.15.64:0/3914117787 >> v2:172.21.15.64:3300/0 conn(0x7fa888070550 0x7fa888160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.513+0000 7fa886ffd640 1 -- 172.21.15.64:0/3914117787 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162246465 0 0) 0x7fa888161940 con 0x7fa8880b2550 2023-11-24T08:05:45.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.513+0000 7fa886ffd640 1 -- 172.21.15.64:0/3914117787 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa870003520 con 0x7fa8880b2550 2023-11-24T08:05:45.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.513+0000 7fa8877fe640 1 -- 172.21.15.64:0/3914117787 >> v1:172.21.15.91:6789/0 conn(0x7fa888161530 legacy=0x7fa8880ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:45.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.513+0000 7fa8877fe640 1 -- 172.21.15.64:0/3914117787 >> v1:172.21.15.64:6789/0 conn(0x7fa8880b2550 legacy=0x7fa8880b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.513+0000 7fa8877fe640 1 -- 172.21.15.64:0/3914117787 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa888070fd0 con 0x7fa888070550 2023-11-24T08:05:45.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.514+0000 7fa8877fe640 1 --2- 172.21.15.64:0/3914117787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa888070550 0x7fa888160ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0097a0 tx=0x7fa87c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad6b4830072976b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:45.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.514+0000 7fa886ffd640 1 -- 172.21.15.64:0/3914117787 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c030610 con 0x7fa888070550 2023-11-24T08:05:45.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.514+0000 7fa886ffd640 1 -- 172.21.15.64:0/3914117787 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c030770 con 0x7fa888070550 2023-11-24T08:05:45.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.514+0000 7fa886ffd640 1 -- 172.21.15.64:0/3914117787 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c030b00 con 0x7fa888070550 2023-11-24T08:05:45.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.515+0000 7fa88d4aa640 1 -- 172.21.15.64:0/3914117787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa888070550 msgr2=0x7fa888160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.515+0000 7fa88d4aa640 1 --2- 172.21.15.64:0/3914117787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa888070550 0x7fa888160ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0097a0 tx=0x7fa87c02b960 comp rx=0 tx=0).stop 2023-11-24T08:05:45.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.515+0000 7fa88d4aa640 1 -- 172.21.15.64:0/3914117787 shutdown_connections 2023-11-24T08:05:45.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.515+0000 7fa88d4aa640 1 --2- 172.21.15.64:0/3914117787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa888070550 0x7fa888160ff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.515+0000 7fa88d4aa640 1 -- 172.21.15.64:0/3914117787 >> 172.21.15.64:0/3914117787 conn(0x7fa8880b0840 msgr2=0x7fa8880b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:45.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.515+0000 7fa88d4aa640 1 -- 172.21.15.64:0/3914117787 shutdown_connections 2023-11-24T08:05:45.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.515+0000 7fa88d4aa640 1 -- 172.21.15.64:0/3914117787 wait complete. 2023-11-24T08:05:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.517+0000 7fa88d4aa640 1 Processor -- start 2023-11-24T08:05:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.517+0000 7fa88d4aa640 1 -- start start 2023-11-24T08:05:45.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa88d4aa640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa888070550 0x7fa8881f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa88d4aa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8880b2550 0x7fa888146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa88d4aa640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa888161530 0x7fa8881468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa88d4aa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88814a770 con 0x7fa8880b2550 2023-11-24T08:05:45.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa88d4aa640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880ad420 con 0x7fa888161530 2023-11-24T08:05:45.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa88d4aa640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880ad700 con 0x7fa888070550 2023-11-24T08:05:45.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8880b2550 0x7fa888146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8880b2550 0x7fa888146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36676/0 (socket says 172.21.15.64:36676) 2023-11-24T08:05:45.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.518+0000 7fa8877fe640 1 -- 172.21.15.64:0/1048361951 learned_addr learned my addr 172.21.15.64:0/1048361951 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:45.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa8877fe640 1 -- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa888070550 msgr2=0x7fa8881f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa887fff640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa888070550 0x7fa8881f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa8877fe640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa888070550 0x7fa8881f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa8877fe640 1 -- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa888161530 msgr2=0x7fa8881468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa88cca9640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa888161530 0x7fa8881468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa8877fe640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa888161530 0x7fa8881468a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa8877fe640 1 -- 172.21.15.64:0/1048361951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c009400 con 0x7fa8880b2550 2023-11-24T08:05:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa887fff640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa888070550 0x7fa8881f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.519+0000 7fa8877fe640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8880b2550 0x7fa888146360 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa87c02bf70 tx=0x7fa87c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.520+0000 7fa884ff9640 1 -- 172.21.15.64:0/1048361951 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c03e070 con 0x7fa8880b2550 2023-11-24T08:05:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.520+0000 7fa884ff9640 1 -- 172.21.15.64:0/1048361951 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c030e10 con 0x7fa8880b2550 2023-11-24T08:05:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.520+0000 7fa884ff9640 1 -- 172.21.15.64:0/1048361951 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c039c30 con 0x7fa8880b2550 2023-11-24T08:05:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.520+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8880ad9e0 con 0x7fa8880b2550 2023-11-24T08:05:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.520+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8880adfa0 con 0x7fa8880b2550 2023-11-24T08:05:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.521+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88804e840 con 0x7fa8880b2550 2023-11-24T08:05:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.522+0000 7fa884ff9640 1 -- 172.21.15.64:0/1048361951 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa87c04b400 con 0x7fa8880b2550 2023-11-24T08:05:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.523+0000 7fa884ff9640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa87406af00 0x7fa87406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.523+0000 7fa884ff9640 1 -- 172.21.15.64:0/1048361951 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa87c0abbb0 con 0x7fa8880b2550 2023-11-24T08:05:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.523+0000 7fa887fff640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa87406af00 0x7fa87406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:45.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.524+0000 7fa887fff640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa87406af00 0x7fa87406d3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa8780011e0 tx=0x7fa878009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.527+0000 7fa884ff9640 1 -- 172.21.15.64:0/1048361951 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa87c076a60 con 0x7fa8880b2550 2023-11-24T08:05:45.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.733+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa8880b2a60 con 0x7fa8880b2550 2023-11-24T08:05:45.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.734+0000 7fa884ff9640 1 -- 172.21.15.64:0/1048361951 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa87c034080 con 0x7fa8880b2550 2023-11-24T08:05:45.736 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:05:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa87406af00 msgr2=0x7fa87406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa87406af00 0x7fa87406d3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa8780011e0 tx=0x7fa878009040 comp rx=0 tx=0).stop 2023-11-24T08:05:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8880b2550 msgr2=0x7fa888146360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8880b2550 0x7fa888146360 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa87c02bf70 tx=0x7fa87c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 shutdown_connections 2023-11-24T08:05:45.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa87406af00 0x7fa87406d3c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa888161530 0x7fa8881468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8880b2550 0x7fa888146360 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 --2- 172.21.15.64:0/1048361951 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa888070550 0x7fa8881f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:45.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 >> 172.21.15.64:0/1048361951 conn(0x7fa8880b0840 msgr2=0x7fa8880aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:45.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 shutdown_connections 2023-11-24T08:05:45.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:45.737+0000 7fa88d4aa640 1 -- 172.21.15.64:0/1048361951 wait complete. 2023-11-24T08:05:45.758 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi091.front.sepia.ceph.com... 2023-11-24T08:05:45.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:45.759 DEBUG:teuthology.orchestra.run.smithi091:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T08:05:45.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[95849]: fuse finished with error 0 and tester_r 0 2023-11-24T08:05:45.857 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:05:45.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:45.857 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:05:46.034 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:05:46.035 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:05:46.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:05:46.099 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:05:46.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:46.100 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T08:05:46.156 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T08:05:46.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:05:46.157 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:05:46.157 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:05:46.212 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:05:46.213 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:05:46.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:05:46.213 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T08:05:46.259 INFO:teuthology.orchestra.run.smithi091.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T08:05:46.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:05:46.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.494+0000 7f1f253b3640 1 Processor -- start 2023-11-24T08:05:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.495+0000 7f1f253b3640 1 -- start start 2023-11-24T08:05:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.496+0000 7f1f253b3640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1f2015d5b0 0x7f1f200d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.496+0000 7f1f253b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200b1650 0x7f1f200d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.496+0000 7f1f253b3640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f200cf0c0 con 0x7f1f200d14a0 2023-11-24T08:05:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.496+0000 7f1f253b3640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200b01b0 con 0x7f1f2015d5b0 2023-11-24T08:05:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.496+0000 7f1f253b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2015d990 con 0x7f1f200b1650 2023-11-24T08:05:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.497+0000 7f1f1f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1f200d14a0 0x7f1f200cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47286/0 (socket says 172.21.15.64:47286) 2023-11-24T08:05:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.497+0000 7f1f1f7fe640 1 -- 172.21.15.64:0/2738383081 learned_addr learned my addr 172.21.15.64:0/2738383081 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.497+0000 7f1f24bb2640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200b1650 0x7f1f200d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.497+0000 7f1f1ffff640 1 --2- 172.21.15.64:0/2738383081 >> v2:172.21.15.91:3300/0 conn(0x7f1f2015d5b0 0x7f1f200d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.497+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576834328 0 0) 0x7f1f200cf0c0 con 0x7f1f200d14a0 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.497+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f0c003520 con 0x7f1f200d14a0 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495744714 0 0) 0x7f1f0c003520 con 0x7f1f200d14a0 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200b1650 msgr2=0x7f1f200d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200b1650 0x7f1f200d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 >> v2:172.21.15.91:3300/0 conn(0x7f1f2015d5b0 msgr2=0x7f1f200d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 --2- 172.21.15.64:0/2738383081 >> v2:172.21.15.91:3300/0 conn(0x7f1f2015d5b0 0x7f1f200d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200d1e40 con 0x7f1f200d14a0 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f1f08001670 con 0x7f1f200d14a0 2023-11-24T08:05:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 >> v1:172.21.15.64:6789/0 conn(0x7f1f200d14a0 legacy=0x7f1f200cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200b1650 0x7f1f0c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c003f60 0x7f1f0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f0c004d00 0x7f1f0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200b01b0 con 0x7f1f0c004d00 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2015d990 con 0x7f1f0c003f60 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c00b9b0 con 0x7f1f200b1650 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f24bb2640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f0c004d00 0x7f1f0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.498+0000 7f1f1f7fe640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c003f60 0x7f1f0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.499+0000 7f1f1ffff640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200b1650 0x7f1f0c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.499+0000 7f1f24bb2640 1 -- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200b1650 msgr2=0x7f1f0c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.499+0000 7f1f24bb2640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200b1650 0x7f1f0c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.499+0000 7f1f24bb2640 1 -- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c003f60 msgr2=0x7f1f0c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.499+0000 7f1f24bb2640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c003f60 0x7f1f0c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.499+0000 7f1f24bb2640 1 -- 172.21.15.64:0/2738383081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c003520 con 0x7f1f0c004d00 2023-11-24T08:05:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.499+0000 7f1f1f7fe640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c003f60 0x7f1f0c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.500+0000 7f1f24bb2640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f0c004d00 0x7f1f0c005180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009a90 tx=0x7f1f0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f59f0ad162ac9a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.500+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1802c680 con 0x7f1f0c004d00 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.500+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1802c7e0 con 0x7f1f0c004d00 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.500+0000 7f1f1effd640 1 -- 172.21.15.64:0/2738383081 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1802cb90 con 0x7f1f0c004d00 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.501+0000 7f1f253b3640 1 -- 172.21.15.64:0/2738383081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f0c004d00 msgr2=0x7f1f0c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.501+0000 7f1f253b3640 1 --2- 172.21.15.64:0/2738383081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f0c004d00 0x7f1f0c005180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009a90 tx=0x7f1f0c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.501+0000 7f1f24bb2640 1 -- 172.21.15.64:0/2738383081 reap_dead start 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.501+0000 7f1f253b3640 1 -- 172.21.15.64:0/2738383081 shutdown_connections 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.501+0000 7f1f253b3640 1 -- 172.21.15.64:0/2738383081 >> 172.21.15.64:0/2738383081 conn(0x7f1f2009c3b0 msgr2=0x7f1f200d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.501+0000 7f1f253b3640 1 -- 172.21.15.64:0/2738383081 shutdown_connections 2023-11-24T08:05:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.501+0000 7f1f253b3640 1 -- 172.21.15.64:0/2738383081 wait complete. 2023-11-24T08:05:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.503+0000 7f1f253b3640 1 Processor -- start 2023-11-24T08:05:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.503+0000 7f1f253b3640 1 -- start start 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f253b3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c004d00 0x7f1f201f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f253b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200c6ba0 0x7f1f2015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f253b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200cc150 0x7f1f200cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f253b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f201f6eb0 con 0x7f1f200c6ba0 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f253b3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200ccb10 con 0x7f1f0c004d00 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f253b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200b8840 con 0x7f1f200cc150 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f1f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200c6ba0 0x7f1f2015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f1f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200c6ba0 0x7f1f2015a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36692/0 (socket says 172.21.15.64:36692) 2023-11-24T08:05:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.504+0000 7f1f1f7fe640 1 -- 172.21.15.64:0/949816357 learned_addr learned my addr 172.21.15.64:0/949816357 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:46.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1ffff640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c004d00 0x7f1f201f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f24bb2640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200cc150 0x7f1f200cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1f7fe640 1 -- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200cc150 msgr2=0x7f1f200cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1f7fe640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200cc150 0x7f1f200cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1f7fe640 1 -- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c004d00 msgr2=0x7f1f201f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1f7fe640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c004d00 0x7f1f201f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1f7fe640 1 -- 172.21.15.64:0/949816357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2015ab30 con 0x7f1f200c6ba0 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1f7fe640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200c6ba0 0x7f1f2015a5f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800dae0 tx=0x7f1f08007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1cff9640 1 -- 172.21.15.64:0/949816357 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080074e0 con 0x7f1f200c6ba0 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1cff9640 1 -- 172.21.15.64:0/949816357 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f08007640 con 0x7f1f200c6ba0 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f1cff9640 1 -- 172.21.15.64:0/949816357 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080049e0 con 0x7f1f200c6ba0 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f200c70f0 con 0x7f1f200c6ba0 2023-11-24T08:05:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.505+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f200c76b0 con 0x7f1f200c6ba0 2023-11-24T08:05:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.507+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2004e840 con 0x7f1f200c6ba0 2023-11-24T08:05:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.508+0000 7f1f1cff9640 1 -- 172.21.15.64:0/949816357 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1f08004b40 con 0x7f1f200c6ba0 2023-11-24T08:05:46.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.508+0000 7f1f1cff9640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1f0006aed0 0x7f1f0006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.509+0000 7f1f1cff9640 1 -- 172.21.15.64:0/949816357 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f08089360 con 0x7f1f200c6ba0 2023-11-24T08:05:46.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.509+0000 7f1f1ffff640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1f0006aed0 0x7f1f0006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.510+0000 7f1f1ffff640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1f0006aed0 0x7f1f0006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f10003a50 tx=0x7f1f1000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.513+0000 7f1f1cff9640 1 -- 172.21.15.64:0/949816357 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f08054210 con 0x7f1f200c6ba0 2023-11-24T08:05:46.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.746+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f200678d0 con 0x7f1f200c6ba0 2023-11-24T08:05:46.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.747+0000 7f1f1cff9640 1 -- 172.21.15.64:0/949816357 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f1f08057830 con 0x7f1f200c6ba0 2023-11-24T08:05:46.749 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:46.749 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1893080737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":1893080737},{"type":"v1","addr":"172.21.15.91:6837","nonce":1893080737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2845212044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2845212044},{"type":"v1","addr":"172.21.15.91:6835","nonce":2845212044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2382740551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2382740551},{"type":"v1","addr":"172.21.15.64:6833","nonce":2382740551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:05:19.478049+0000","modified":"2023-11-24T08:05:22.529755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6835/2394348469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2394348469},{"type":"v1","addr":"172.21.15.64:6835","nonce":2394348469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:05:46.749 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-11-24T08:05:46.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1f0006aed0 msgr2=0x7f1f0006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:46.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1f0006aed0 0x7f1f0006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f10003a50 tx=0x7f1f1000b040 comp rx=0 tx=0).stop 2023-11-24T08:05:46.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200c6ba0 msgr2=0x7f1f2015a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:46.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200c6ba0 0x7f1f2015a5f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800dae0 tx=0x7f1f08007e50 comp rx=0 tx=0).stop 2023-11-24T08:05:46.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 shutdown_connections 2023-11-24T08:05:46.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1f0006aed0 0x7f1f0006d390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1f200cc150 0x7f1f200cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200c6ba0 0x7f1f2015a5f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 --2- 172.21.15.64:0/949816357 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1f0c004d00 0x7f1f201f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.749+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 >> 172.21.15.64:0/949816357 conn(0x7f1f2009c3b0 msgr2=0x7f1f200d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:46.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.750+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 shutdown_connections 2023-11-24T08:05:46.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.750+0000 7f1f253b3640 1 -- 172.21.15.64:0/949816357 wait complete. 2023-11-24T08:05:46.772 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.993+0000 7fa3b0cea640 1 Processor -- start 2023-11-24T08:05:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.993+0000 7fa3b0cea640 1 -- start start 2023-11-24T08:05:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.995+0000 7fa3b0cea640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3ac157770 0x7fa3ac06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.995+0000 7fa3b0cea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 0x7fa3ac15d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.995+0000 7fa3b0cea640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ac161670 con 0x7fa3ac15ddb0 2023-11-24T08:05:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.995+0000 7fa3b0cea640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac06d500 con 0x7fa3ac157770 2023-11-24T08:05:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.995+0000 7fa3b0cea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac071730 con 0x7fa3ac06d080 2023-11-24T08:05:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.995+0000 7fa3ab7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3ac157770 0x7fa3ac06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.995+0000 7fa3ab7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3ac157770 0x7fa3ac06cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36696/0 (socket says 172.21.15.64:36696) 2023-11-24T08:05:46.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.995+0000 7fa3ab7fe640 1 -- 172.21.15.64:0/3270838032 learned_addr learned my addr 172.21.15.64:0/3270838032 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:46.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.996+0000 7fa3aaffd640 1 --2- 172.21.15.64:0/3270838032 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 0x7fa3ac15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:46.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.996+0000 7fa3ab7fe640 1 -- 172.21.15.64:0/3270838032 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 msgr2=0x7fa3ac15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.996+0000 7fa3ab7fe640 1 --2- 172.21.15.64:0/3270838032 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 0x7fa3ac15d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.996+0000 7fa3ab7fe640 1 -- 172.21.15.64:0/3270838032 >> v1:172.21.15.91:6789/0 conn(0x7fa3ac15ddb0 legacy=0x7fa3ac0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.996+0000 7fa3ab7fe640 1 -- 172.21.15.64:0/3270838032 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac15e610 con 0x7fa3ac157770 2023-11-24T08:05:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.997+0000 7fa3ab7fe640 1 --2- 172.21.15.64:0/3270838032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac06cb40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009870 tx=0x7fa39c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a62c29a0a2a963 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.997+0000 7fa3aa7fc640 1 -- 172.21.15.64:0/3270838032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c031640 con 0x7fa3ac157770 2023-11-24T08:05:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.997+0000 7fa3aa7fc640 1 -- 172.21.15.64:0/3270838032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c0317a0 con 0x7fa3ac157770 2023-11-24T08:05:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.997+0000 7fa3aa7fc640 1 -- 172.21.15.64:0/3270838032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c031b30 con 0x7fa3ac157770 2023-11-24T08:05:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.998+0000 7fa3b0cea640 1 -- 172.21.15.64:0/3270838032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 msgr2=0x7fa3ac06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.998+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/3270838032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac06cb40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009870 tx=0x7fa39c02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.998+0000 7fa3b0cea640 1 -- 172.21.15.64:0/3270838032 shutdown_connections 2023-11-24T08:05:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.998+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/3270838032 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 0x7fa3ac15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.998+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/3270838032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac06cb40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.998+0000 7fa3b0cea640 1 -- 172.21.15.64:0/3270838032 >> 172.21.15.64:0/3270838032 conn(0x7fa3ac06a860 msgr2=0x7fa3ac06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.998+0000 7fa3b0cea640 1 -- 172.21.15.64:0/3270838032 shutdown_connections 2023-11-24T08:05:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:46.998+0000 7fa3b0cea640 1 -- 172.21.15.64:0/3270838032 wait complete. 2023-11-24T08:05:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.000+0000 7fa3b0cea640 1 Processor -- start 2023-11-24T08:05:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.000+0000 7fa3b0cea640 1 -- start start 2023-11-24T08:05:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3b0cea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 0x7fa3ac154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3b0cea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3b0cea640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3ac15ddb0 0x7fa3ac0af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3b0cea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0cb1e0 con 0x7fa3ac157770 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3b0cea640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0cb4c0 con 0x7fa3ac15ddb0 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3b0cea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0cb7a0 con 0x7fa3ac06d080 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3aaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3aaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36702/0 (socket says 172.21.15.64:36702) 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3aaffd640 1 -- 172.21.15.64:0/2698443053 learned_addr learned my addr 172.21.15.64:0/2698443053 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.001+0000 7fa3ab7fe640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 0x7fa3ac154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.002+0000 7fa3abfff640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3ac15ddb0 0x7fa3ac0af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.002+0000 7fa3aaffd640 1 -- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 msgr2=0x7fa3ac154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.002+0000 7fa3aaffd640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 0x7fa3ac154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.002+0000 7fa3aaffd640 1 -- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3ac15ddb0 msgr2=0x7fa3ac0af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.002+0000 7fa3aaffd640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3ac15ddb0 0x7fa3ac0af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.002+0000 7fa3aaffd640 1 -- 172.21.15.64:0/2698443053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c0094d0 con 0x7fa3ac157770 2023-11-24T08:05:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.002+0000 7fa3aaffd640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac148680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000d800 tx=0x7fa3a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.006+0000 7fa393fff640 1 -- 172.21.15.64:0/2698443053 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a000b6c0 con 0x7fa3ac157770 2023-11-24T08:05:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.006+0000 7fa393fff640 1 -- 172.21.15.64:0/2698443053 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a000b820 con 0x7fa3ac157770 2023-11-24T08:05:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.006+0000 7fa393fff640 1 -- 172.21.15.64:0/2698443053 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a000bbb0 con 0x7fa3ac157770 2023-11-24T08:05:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.007+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac0cba80 con 0x7fa3ac157770 2023-11-24T08:05:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.007+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac1526c0 con 0x7fa3ac157770 2023-11-24T08:05:47.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.008+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ac04e840 con 0x7fa3ac157770 2023-11-24T08:05:47.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.009+0000 7fa393fff640 1 -- 172.21.15.64:0/2698443053 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa3a000bd10 con 0x7fa3ac157770 2023-11-24T08:05:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.010+0000 7fa393fff640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa39806aed0 0x7fa39806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.010+0000 7fa393fff640 1 -- 172.21.15.64:0/2698443053 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3a0089ca0 con 0x7fa3ac157770 2023-11-24T08:05:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.010+0000 7fa3ab7fe640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa39806aed0 0x7fa39806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.011+0000 7fa3ab7fe640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa39806aed0 0x7fa39806d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa39c036000 tx=0x7fa39c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.015+0000 7fa393fff640 1 -- 172.21.15.64:0/2698443053 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3a0054c60 con 0x7fa3ac157770 2023-11-24T08:05:47.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.210+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3ac15daa0 con 0x7fa3ac157770 2023-11-24T08:05:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.212+0000 7fa393fff640 1 -- 172.21.15.64:0/2698443053 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fa3a0058280 con 0x7fa3ac157770 2023-11-24T08:05:47.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:47.214 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:21.012478+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:05:17.535151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:05:18.499872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:47.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.214+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa39806aed0 msgr2=0x7fa39806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.214+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa39806aed0 0x7fa39806d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa39c036000 tx=0x7fa39c03c040 comp rx=0 tx=0).stop 2023-11-24T08:05:47.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.214+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 msgr2=0x7fa3ac148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.214+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac148680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000d800 tx=0x7fa3a000dcc0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.215+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 shutdown_connections 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.215+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa39806aed0 0x7fa39806d390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.215+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3ac15ddb0 0x7fa3ac0af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.215+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3ac157770 0x7fa3ac148680 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.215+0000 7fa3b0cea640 1 --2- 172.21.15.64:0/2698443053 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3ac06d080 0x7fa3ac154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.215+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 >> 172.21.15.64:0/2698443053 conn(0x7fa3ac06a860 msgr2=0x7fa3ac1555b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.215+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 shutdown_connections 2023-11-24T08:05:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.215+0000 7fa3b0cea640 1 -- 172.21.15.64:0/2698443053 wait complete. 2023-11-24T08:05:47.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:05:17.535151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:05:18.499872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:05:47.237 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:05:47.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.464+0000 7fb7d1839640 1 Processor -- start 2023-11-24T08:05:47.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.464+0000 7fb7d1839640 1 -- start start 2023-11-24T08:05:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.466+0000 7fb7d1839640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb7cc144770 0x7fb7cc15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.466+0000 7fb7d1839640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7cc0b35a0 0x7fb7cc15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.466+0000 7fb7d1839640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc15f930 con 0x7fb7cc06c9d0 2023-11-24T08:05:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.466+0000 7fb7d1839640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b43b0 con 0x7fb7cc144770 2023-11-24T08:05:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.466+0000 7fb7d1839640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b3b10 con 0x7fb7cc0b35a0 2023-11-24T08:05:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.466+0000 7fb7d1038640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb7cc06c9d0 0x7fb7cc06ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56416/0 (socket says 172.21.15.64:56416) 2023-11-24T08:05:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.466+0000 7fb7d1038640 1 -- 172.21.15.64:0/123301126 learned_addr learned my addr 172.21.15.64:0/123301126 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:47.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cbfff640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7cc0b35a0 0x7fb7cc15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7d0837640 1 --2- 172.21.15.64:0/123301126 >> v2:172.21.15.91:3300/0 conn(0x7fb7cc144770 0x7fb7cc15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507708435 0 0) 0x7fb7cc15f930 con 0x7fb7cc06c9d0 2023-11-24T08:05:47.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7bc003520 con 0x7fb7cc06c9d0 2023-11-24T08:05:47.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253715162 0 0) 0x7fb7bc003520 con 0x7fb7cc06c9d0 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7cc0b35a0 msgr2=0x7fb7cc15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7cc0b35a0 0x7fb7cc15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 >> v2:172.21.15.91:3300/0 conn(0x7fb7cc144770 msgr2=0x7fb7cc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 --2- 172.21.15.64:0/123301126 >> v2:172.21.15.91:3300/0 conn(0x7fb7cc144770 0x7fb7cc15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0b3df0 con 0x7fb7cc06c9d0 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fb7c40019f0 con 0x7fb7cc06c9d0 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.467+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 >> v1:172.21.15.64:6789/0 conn(0x7fb7cc06c9d0 legacy=0x7fb7cc06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7cb7fe640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7bc0039e0 0x7fb7bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7cb7fe640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7cc0b35a0 0x7fb7bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7cb7fe640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7bc004d00 0x7fb7bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b43b0 con 0x7fb7bc0039e0 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b3b10 con 0x7fb7bc004d00 2023-11-24T08:05:47.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc00b9b0 con 0x7fb7cc0b35a0 2023-11-24T08:05:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7d0837640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7bc0039e0 0x7fb7bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7cbfff640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7cc0b35a0 0x7fb7bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7d1038640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7bc004d00 0x7fb7bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7d0837640 1 -- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7cc0b35a0 msgr2=0x7fb7bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7d0837640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7cc0b35a0 0x7fb7bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7d0837640 1 -- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7bc004d00 msgr2=0x7fb7bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7d0837640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7bc004d00 0x7fb7bc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.468+0000 7fb7d0837640 1 -- 172.21.15.64:0/123301126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc003520 con 0x7fb7bc0039e0 2023-11-24T08:05:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.470+0000 7fb7d0837640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7bc0039e0 0x7fb7bc003e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009ac0 tx=0x7fb7bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dede12b40fd6df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.470+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c002c710 con 0x7fb7bc0039e0 2023-11-24T08:05:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.470+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c002c870 con 0x7fb7bc0039e0 2023-11-24T08:05:47.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.470+0000 7fb7cb7fe640 1 -- 172.21.15.64:0/123301126 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c002cc00 con 0x7fb7bc0039e0 2023-11-24T08:05:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.471+0000 7fb7d1839640 1 -- 172.21.15.64:0/123301126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7bc0039e0 msgr2=0x7fb7bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.471+0000 7fb7d1839640 1 --2- 172.21.15.64:0/123301126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7bc0039e0 0x7fb7bc003e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009ac0 tx=0x7fb7bc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.471+0000 7fb7d1038640 1 -- 172.21.15.64:0/123301126 reap_dead start 2023-11-24T08:05:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.471+0000 7fb7d1839640 1 -- 172.21.15.64:0/123301126 shutdown_connections 2023-11-24T08:05:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.471+0000 7fb7d1839640 1 -- 172.21.15.64:0/123301126 >> 172.21.15.64:0/123301126 conn(0x7fb7cc0cec10 msgr2=0x7fb7cc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.471+0000 7fb7d1839640 1 -- 172.21.15.64:0/123301126 shutdown_connections 2023-11-24T08:05:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.471+0000 7fb7d1839640 1 -- 172.21.15.64:0/123301126 wait complete. 2023-11-24T08:05:47.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.473+0000 7fb7d1839640 1 Processor -- start 2023-11-24T08:05:47.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.473+0000 7fb7d1839640 1 -- start start 2023-11-24T08:05:47.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.474+0000 7fb7d1839640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7bc0039e0 0x7fb7cc13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.474+0000 7fb7d1839640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7cc0c3c30 0x7fb7cc0c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.474+0000 7fb7d1839640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc1417a0 0x7fb7cc141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.474+0000 7fb7d1839640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc15bd20 con 0x7fb7cc1417a0 2023-11-24T08:05:47.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.474+0000 7fb7d1839640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc15c000 con 0x7fb7cc0c3c30 2023-11-24T08:05:47.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.474+0000 7fb7d1839640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc15c2e0 con 0x7fb7bc0039e0 2023-11-24T08:05:47.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.474+0000 7fb7d1038640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc1417a0 0x7fb7cc141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d1038640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc1417a0 0x7fb7cc141c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35062/0 (socket says 172.21.15.64:35062) 2023-11-24T08:05:47.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d1038640 1 -- 172.21.15.64:0/987339714 learned_addr learned my addr 172.21.15.64:0/987339714 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:47.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d0837640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7bc0039e0 0x7fb7cc13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7cbfff640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7cc0c3c30 0x7fb7cc0c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d1038640 1 -- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7bc0039e0 msgr2=0x7fb7cc13e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d1038640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7bc0039e0 0x7fb7cc13e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d1038640 1 -- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7cc0c3c30 msgr2=0x7fb7cc0c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d1038640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7cc0c3c30 0x7fb7cc0c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d1038640 1 -- 172.21.15.64:0/987339714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc15c5c0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d0837640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7bc0039e0 0x7fb7cc13e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7cbfff640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7cc0c3c30 0x7fb7cc0c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.475+0000 7fb7d1038640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc1417a0 0x7fb7cc141c20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0cfc40 tx=0x7fb7c400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.476+0000 7fb7c97fa640 1 -- 172.21.15.64:0/987339714 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400eb90 con 0x7fb7cc1417a0 2023-11-24T08:05:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.476+0000 7fb7c97fa640 1 -- 172.21.15.64:0/987339714 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c400ecf0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.476+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc15c8a0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.476+0000 7fb7c97fa640 1 -- 172.21.15.64:0/987339714 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4013660 con 0x7fb7cc1417a0 2023-11-24T08:05:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.476+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc13eaa0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.477+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc15cdc0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.478+0000 7fb7c97fa640 1 -- 172.21.15.64:0/987339714 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb7c40137c0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.479+0000 7fb7c97fa640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb7b006af00 0x7fb7b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.479+0000 7fb7c97fa640 1 -- 172.21.15.64:0/987339714 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7c4089aa0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.480+0000 7fb7d0837640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb7b006af00 0x7fb7b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.481+0000 7fb7d0837640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb7b006af00 0x7fb7b006d3c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0036000 tx=0x7fb7c0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.483+0000 7fb7c97fa640 1 -- 172.21.15.64:0/987339714 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7cc15cdc0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.675+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7cc15cdc0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.676+0000 7fb7c97fa640 1 -- 172.21.15.64:0/987339714 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7cc15cdc0 con 0x7fb7cc1417a0 2023-11-24T08:05:47.678 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:05:47.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb7b006af00 msgr2=0x7fb7b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb7b006af00 0x7fb7b006d3c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0036000 tx=0x7fb7c0030000 comp rx=0 tx=0).stop 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc1417a0 msgr2=0x7fb7cc141c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc1417a0 0x7fb7cc141c20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0cfc40 tx=0x7fb7c400e530 comp rx=0 tx=0).stop 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 shutdown_connections 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb7b006af00 0x7fb7b006d3c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc1417a0 0x7fb7cc141c20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb7cc0c3c30 0x7fb7cc0c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 --2- 172.21.15.64:0/987339714 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb7bc0039e0 0x7fb7cc13e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.678+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 >> 172.21.15.64:0/987339714 conn(0x7fb7cc0cec10 msgr2=0x7fb7cc06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.679+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 shutdown_connections 2023-11-24T08:05:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.679+0000 7fb7d1839640 1 -- 172.21.15.64:0/987339714 wait complete. 2023-11-24T08:05:47.701 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:05:47.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.927+0000 7ff19a51a640 1 Processor -- start 2023-11-24T08:05:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.928+0000 7ff19a51a640 1 -- start start 2023-11-24T08:05:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.929+0000 7ff19a51a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff1941435d0 0x7ff194071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.929+0000 7ff19a51a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1941439b0 0x7ff1940b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.930+0000 7ff19a51a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff194070d40 con 0x7ff1940b3f70 2023-11-24T08:05:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.930+0000 7ff19a51a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1940b4400 con 0x7ff1941435d0 2023-11-24T08:05:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.930+0000 7ff19a51a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff194071020 con 0x7ff1941439b0 2023-11-24T08:05:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.930+0000 7ff199d19640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff1940b3f70 0x7ff194070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56426/0 (socket says 172.21.15.64:56426) 2023-11-24T08:05:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.930+0000 7ff199d19640 1 -- 172.21.15.64:0/118053705 learned_addr learned my addr 172.21.15.64:0/118053705 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.930+0000 7ff198d17640 1 --2- 172.21.15.64:0/118053705 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1941439b0 0x7ff1940b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.930+0000 7ff199518640 1 --2- 172.21.15.64:0/118053705 >> v2:172.21.15.91:3300/0 conn(0x7ff1941435d0 0x7ff194071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.931+0000 7ff18bfff640 1 -- 172.21.15.64:0/118053705 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587650814 0 0) 0x7ff194070d40 con 0x7ff1940b3f70 2023-11-24T08:05:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.931+0000 7ff18bfff640 1 -- 172.21.15.64:0/118053705 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff180005730 con 0x7ff1940b3f70 2023-11-24T08:05:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.931+0000 7ff199518640 1 -- 172.21.15.64:0/118053705 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1941439b0 msgr2=0x7ff1940b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.931+0000 7ff199518640 1 --2- 172.21.15.64:0/118053705 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1941439b0 0x7ff1940b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.931+0000 7ff199518640 1 -- 172.21.15.64:0/118053705 >> v1:172.21.15.64:6789/0 conn(0x7ff1940b3f70 legacy=0x7ff194070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.931+0000 7ff18bfff640 1 -- 172.21.15.64:0/118053705 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 807037019 0 0) 0x7ff180005730 con 0x7ff1940b3f70 2023-11-24T08:05:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.931+0000 7ff199518640 1 -- 172.21.15.64:0/118053705 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194071300 con 0x7ff1941435d0 2023-11-24T08:05:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.933+0000 7ff199518640 1 --2- 172.21.15.64:0/118053705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 0x7ff194071de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff190009dc0 tx=0x7ff19002bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d30f1627be0613 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.933+0000 7ff18bfff640 1 -- 172.21.15.64:0/118053705 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c001670 con 0x7ff1941435d0 2023-11-24T08:05:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.933+0000 7ff18bfff640 1 -- 172.21.15.64:0/118053705 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19002c880 con 0x7ff1941435d0 2023-11-24T08:05:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.933+0000 7ff18bfff640 1 -- 172.21.15.64:0/118053705 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19002cc10 con 0x7ff1941435d0 2023-11-24T08:05:47.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.933+0000 7ff19a51a640 1 -- 172.21.15.64:0/118053705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 msgr2=0x7ff194071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.933+0000 7ff19a51a640 1 --2- 172.21.15.64:0/118053705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 0x7ff194071de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff190009dc0 tx=0x7ff19002bd50 comp rx=0 tx=0).stop 2023-11-24T08:05:47.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.934+0000 7ff19a51a640 1 -- 172.21.15.64:0/118053705 shutdown_connections 2023-11-24T08:05:47.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.934+0000 7ff19a51a640 1 --2- 172.21.15.64:0/118053705 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1941439b0 0x7ff1940b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.934+0000 7ff19a51a640 1 --2- 172.21.15.64:0/118053705 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 0x7ff194071de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.934+0000 7ff19a51a640 1 -- 172.21.15.64:0/118053705 >> 172.21.15.64:0/118053705 conn(0x7ff1940cec10 msgr2=0x7ff1940cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:47.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.934+0000 7ff19a51a640 1 -- 172.21.15.64:0/118053705 shutdown_connections 2023-11-24T08:05:47.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.934+0000 7ff19a51a640 1 -- 172.21.15.64:0/118053705 wait complete. 2023-11-24T08:05:47.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.935+0000 7ff19a51a640 1 Processor -- start 2023-11-24T08:05:47.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.936+0000 7ff19a51a640 1 -- start start 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff19a51a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1940b3f70 0x7ff19413aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff19a51a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 0x7ff19415aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff19a51a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1941439b0 0x7ff194159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff19a51a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19415b010 con 0x7ff1941439b0 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff19a51a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19415ce90 con 0x7ff1941435d0 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff19a51a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff194159d10 con 0x7ff1940b3f70 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff199d19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1941439b0 0x7ff194159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff199d19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1941439b0 0x7ff194159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35078/0 (socket says 172.21.15.64:35078) 2023-11-24T08:05:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff199d19640 1 -- 172.21.15.64:0/4251960116 learned_addr learned my addr 172.21.15.64:0/4251960116 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff199518640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1940b3f70 0x7ff19413aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.937+0000 7ff198d17640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 0x7ff19415aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff199d19640 1 -- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1940b3f70 msgr2=0x7ff19413aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff199d19640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1940b3f70 0x7ff19413aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff199d19640 1 -- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 msgr2=0x7ff19415aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff199d19640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 0x7ff19415aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff199d19640 1 -- 172.21.15.64:0/4251960116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009a70 con 0x7ff1941439b0 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff198d17640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 0x7ff19415aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff199d19640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1941439b0 0x7ff194159450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c700 tx=0x7ff18c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff189ffb640 1 -- 172.21.15.64:0/4251960116 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c005ad0 con 0x7ff1941439b0 2023-11-24T08:05:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff189ffb640 1 -- 172.21.15.64:0/4251960116 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c005c30 con 0x7ff1941439b0 2023-11-24T08:05:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff194159ff0 con 0x7ff1941439b0 2023-11-24T08:05:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff189ffb640 1 -- 172.21.15.64:0/4251960116 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c007070 con 0x7ff1941439b0 2023-11-24T08:05:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.938+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19415a5b0 con 0x7ff1941439b0 2023-11-24T08:05:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.940+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19404e840 con 0x7ff1941439b0 2023-11-24T08:05:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.940+0000 7ff189ffb640 1 -- 172.21.15.64:0/4251960116 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff18c0071d0 con 0x7ff1941439b0 2023-11-24T08:05:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.941+0000 7ff189ffb640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff17806aed0 0x7ff17806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.941+0000 7ff189ffb640 1 -- 172.21.15.64:0/4251960116 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff18c089380 con 0x7ff1941439b0 2023-11-24T08:05:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.942+0000 7ff199518640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff17806aed0 0x7ff17806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.943+0000 7ff199518640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff17806aed0 0x7ff17806d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff190036000 tx=0x7ff190030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:47.946+0000 7ff189ffb640 1 -- 172.21.15.64:0/4251960116 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff18c054440 con 0x7ff1941439b0 2023-11-24T08:05:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.143+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff19406c790 con 0x7ff1941439b0 2023-11-24T08:05:48.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.144+0000 7ff189ffb640 1 -- 172.21.15.64:0/4251960116 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff18c057a60 con 0x7ff1941439b0 2023-11-24T08:05:48.146 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:05:48.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff17806aed0 msgr2=0x7ff17806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff17806aed0 0x7ff17806d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff190036000 tx=0x7ff190030000 comp rx=0 tx=0).stop 2023-11-24T08:05:48.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1941439b0 msgr2=0x7ff194159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1941439b0 0x7ff194159450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c700 tx=0x7ff18c00cbc0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 shutdown_connections 2023-11-24T08:05:48.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff17806aed0 0x7ff17806d390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1941439b0 0x7ff194159450 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1941435d0 0x7ff19415aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 --2- 172.21.15.64:0/4251960116 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1940b3f70 0x7ff19413aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 >> 172.21.15.64:0/4251960116 conn(0x7ff1940cec10 msgr2=0x7ff1940ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:48.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 shutdown_connections 2023-11-24T08:05:48.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.147+0000 7ff19a51a640 1 -- 172.21.15.64:0/4251960116 wait complete. 2023-11-24T08:05:48.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:05:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.393+0000 7f99b0a1d640 1 Processor -- start 2023-11-24T08:05:48.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.394+0000 7f99b0a1d640 1 -- start start 2023-11-24T08:05:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.395+0000 7f99b0a1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99ac0ad450 0x7f99ac154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.395+0000 7f99b0a1d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f99ac070850 0x7f99ac071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99b0a1d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac0ce650 con 0x7f99ac153ea0 2023-11-24T08:05:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99b0a1d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0ad8e0 con 0x7f99ac0ad450 2023-11-24T08:05:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99b0a1d640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac1547c0 con 0x7f99ac070850 2023-11-24T08:05:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99aa59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99ac0ad450 0x7f99ac154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99aa59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99ac0ad450 0x7f99ac154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35090/0 (socket says 172.21.15.64:35090) 2023-11-24T08:05:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99aa59c640 1 -- 172.21.15.64:0/165696458 learned_addr learned my addr 172.21.15.64:0/165696458 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99ab59e640 1 --2- 172.21.15.64:0/165696458 >> v2:172.21.15.91:3300/0 conn(0x7f99ac070850 0x7f99ac071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99aa59c640 1 -- 172.21.15.64:0/165696458 >> v2:172.21.15.91:3300/0 conn(0x7f99ac070850 msgr2=0x7f99ac071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99aa59c640 1 --2- 172.21.15.64:0/165696458 >> v2:172.21.15.91:3300/0 conn(0x7f99ac070850 0x7f99ac071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99aa59c640 1 -- 172.21.15.64:0/165696458 >> v1:172.21.15.91:6789/0 conn(0x7f99ac153ea0 legacy=0x7f99ac15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.396+0000 7f99aa59c640 1 -- 172.21.15.64:0/165696458 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac15e0a0 con 0x7f99ac0ad450 2023-11-24T08:05:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.398+0000 7f99aa59c640 1 --2- 172.21.15.64:0/165696458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac154280 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f99ac157e50 tx=0x7f99a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8502e64beeb1c9f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.398+0000 7f99a9d9b640 1 -- 172.21.15.64:0/165696458 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0031870 con 0x7f99ac0ad450 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.398+0000 7f99a9d9b640 1 -- 172.21.15.64:0/165696458 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a00319d0 con 0x7f99ac0ad450 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.398+0000 7f99a9d9b640 1 -- 172.21.15.64:0/165696458 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0031d60 con 0x7f99ac0ad450 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.399+0000 7f99b0a1d640 1 -- 172.21.15.64:0/165696458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 msgr2=0x7f99ac154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.399+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/165696458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac154280 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f99ac157e50 tx=0x7f99a002b910 comp rx=0 tx=0).stop 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.399+0000 7f99b0a1d640 1 -- 172.21.15.64:0/165696458 shutdown_connections 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.399+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/165696458 >> v2:172.21.15.91:3300/0 conn(0x7f99ac070850 0x7f99ac071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.399+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/165696458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac154280 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.399+0000 7f99b0a1d640 1 -- 172.21.15.64:0/165696458 >> 172.21.15.64:0/165696458 conn(0x7f99ac0cec10 msgr2=0x7f99ac0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.399+0000 7f99b0a1d640 1 -- 172.21.15.64:0/165696458 shutdown_connections 2023-11-24T08:05:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.399+0000 7f99b0a1d640 1 -- 172.21.15.64:0/165696458 wait complete. 2023-11-24T08:05:48.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.401+0000 7f99b0a1d640 1 Processor -- start 2023-11-24T08:05:48.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.401+0000 7f99b0a1d640 1 -- start start 2023-11-24T08:05:48.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99b0a1d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99ac070850 0x7f99ac1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99b0a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac0b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99b0a1d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99ac153ea0 0x7f99ac148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99b0a1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac148fa0 con 0x7f99ac0ad450 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99b0a1d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac152b60 con 0x7f99ac153ea0 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99b0a1d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac151a10 con 0x7f99ac070850 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99aa59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac0b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99aa59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac0b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35094/0 (socket says 172.21.15.64:35094) 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99aa59c640 1 -- 172.21.15.64:0/3818355787 learned_addr learned my addr 172.21.15.64:0/3818355787 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99aad9d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99ac070850 0x7f99ac1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99ab59e640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99ac153ea0 0x7f99ac148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99aa59c640 1 -- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99ac070850 msgr2=0x7f99ac1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99aa59c640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99ac070850 0x7f99ac1513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.402+0000 7f99aa59c640 1 -- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99ac153ea0 msgr2=0x7f99ac148a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99aa59c640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99ac153ea0 0x7f99ac148a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99aa59c640 1 -- 172.21.15.64:0/3818355787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00094d0 con 0x7f99ac0ad450 2023-11-24T08:05:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99aad9d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99ac070850 0x7f99ac1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99ab59e640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99ac153ea0 0x7f99ac148a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99aa59c640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac0b09b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f99a00097f0 tx=0x7f99a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99937fe640 1 -- 172.21.15.64:0/3818355787 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a003adf0 con 0x7f99ac0ad450 2023-11-24T08:05:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99937fe640 1 -- 172.21.15.64:0/3818355787 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a00443f0 con 0x7f99ac0ad450 2023-11-24T08:05:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99937fe640 1 -- 172.21.15.64:0/3818355787 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a00447a0 con 0x7f99ac0ad450 2023-11-24T08:05:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac151cf0 con 0x7f99ac0ad450 2023-11-24T08:05:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.403+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac1522b0 con 0x7f99ac0ad450 2023-11-24T08:05:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.404+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac1524a0 con 0x7f99ac0ad450 2023-11-24T08:05:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.405+0000 7f99937fe640 1 -- 172.21.15.64:0/3818355787 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f99a004d450 con 0x7f99ac0ad450 2023-11-24T08:05:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.407+0000 7f99937fe640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999806af00 0x7f999806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.407+0000 7f99937fe640 1 -- 172.21.15.64:0/3818355787 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99a002d040 con 0x7f99ac0ad450 2023-11-24T08:05:48.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.407+0000 7f99aad9d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999806af00 0x7f999806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.409+0000 7f99aad9d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999806af00 0x7f999806d3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f999c00a830 tx=0x7f999c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.412+0000 7f99937fe640 1 -- 172.21.15.64:0/3818355787 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99ac1524a0 con 0x7f99ac0ad450 2023-11-24T08:05:48.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.612+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99ac1524a0 con 0x7f99ac0ad450 2023-11-24T08:05:48.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.613+0000 7f99937fe640 1 -- 172.21.15.64:0/3818355787 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99ac1524a0 con 0x7f99ac0ad450 2023-11-24T08:05:48.615 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:05:48.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999806af00 msgr2=0x7f999806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999806af00 0x7f999806d3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f999c00a830 tx=0x7f999c008040 comp rx=0 tx=0).stop 2023-11-24T08:05:48.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 msgr2=0x7f99ac0b09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac0b09b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f99a00097f0 tx=0x7f99a00046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 shutdown_connections 2023-11-24T08:05:48.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999806af00 0x7f999806d3c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99ac153ea0 0x7f99ac148a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99ac0ad450 0x7f99ac0b09b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 --2- 172.21.15.64:0/3818355787 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99ac070850 0x7f99ac1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 >> 172.21.15.64:0/3818355787 conn(0x7f99ac0cec10 msgr2=0x7f99ac0ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:48.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 shutdown_connections 2023-11-24T08:05:48.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.616+0000 7f99b0a1d640 1 -- 172.21.15.64:0/3818355787 wait complete. 2023-11-24T08:05:48.637 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:48.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.861+0000 7f0329839640 1 Processor -- start 2023-11-24T08:05:48.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.861+0000 7f0329839640 1 -- start start 2023-11-24T08:05:48.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.862+0000 7f0329839640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cd7f0 0x7f03240cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.862+0000 7f0329839640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f03240d1900 0x7f03240cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.862+0000 7f0329839640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03240b3c30 con 0x7f03240cbb10 2023-11-24T08:05:48.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.862+0000 7f0329839640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03240cb530 con 0x7f03240d1900 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.862+0000 7f0329839640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03240d1d60 con 0x7f03240cd7f0 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.862+0000 7f0323fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f03240d1900 0x7f03240cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.863+0000 7f0323fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f03240d1900 0x7f03240cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35110/0 (socket says 172.21.15.64:35110) 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.863+0000 7f0323fff640 1 -- 172.21.15.64:0/1197220373 learned_addr learned my addr 172.21.15.64:0/1197220373 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.863+0000 7f0328837640 1 --2- 172.21.15.64:0/1197220373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cd7f0 0x7f03240cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.863+0000 7f0323fff640 1 -- 172.21.15.64:0/1197220373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cd7f0 msgr2=0x7f03240cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.863+0000 7f0323fff640 1 --2- 172.21.15.64:0/1197220373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cd7f0 0x7f03240cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.863+0000 7f0323fff640 1 -- 172.21.15.64:0/1197220373 >> v1:172.21.15.91:6789/0 conn(0x7f03240cbb10 legacy=0x7f03240cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.863+0000 7f0323fff640 1 -- 172.21.15.64:0/1197220373 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03240ce110 con 0x7f03240d1900 2023-11-24T08:05:48.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.864+0000 7f0323fff640 1 --2- 172.21.15.64:0/1197220373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240cdbd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f03180098a0 tx=0x7f031802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1df6319fa5d482 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:48.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f03237fe640 1 -- 172.21.15.64:0/1197220373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318030840 con 0x7f03240d1900 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f03237fe640 1 -- 172.21.15.64:0/1197220373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03180309a0 con 0x7f03240d1900 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f03237fe640 1 -- 172.21.15.64:0/1197220373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318030d30 con 0x7f03240d1900 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f0329839640 1 -- 172.21.15.64:0/1197220373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 msgr2=0x7f03240cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f0329839640 1 --2- 172.21.15.64:0/1197220373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240cdbd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f03180098a0 tx=0x7f031802b980 comp rx=0 tx=0).stop 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f0329839640 1 -- 172.21.15.64:0/1197220373 shutdown_connections 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f0329839640 1 --2- 172.21.15.64:0/1197220373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240cdbd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f0329839640 1 --2- 172.21.15.64:0/1197220373 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cd7f0 0x7f03240cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.865+0000 7f0329839640 1 -- 172.21.15.64:0/1197220373 >> 172.21.15.64:0/1197220373 conn(0x7f03240d0de0 msgr2=0x7f03240d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.866+0000 7f0329839640 1 -- 172.21.15.64:0/1197220373 shutdown_connections 2023-11-24T08:05:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.866+0000 7f0329839640 1 -- 172.21.15.64:0/1197220373 wait complete. 2023-11-24T08:05:48.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.867+0000 7f0329839640 1 Processor -- start 2023-11-24T08:05:48.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.868+0000 7f0329839640 1 -- start start 2023-11-24T08:05:48.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.868+0000 7f0329839640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cbb10 0x7f03240c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.868+0000 7f0329839640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03240cd7f0 0x7f0324158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329839640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329839640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03241f7bd0 con 0x7f03240d1900 2023-11-24T08:05:48.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329839640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03240c79b0 con 0x7f03240cd7f0 2023-11-24T08:05:48.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329839640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03241f6f50 con 0x7f03240cbb10 2023-11-24T08:05:48.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329038640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329038640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240b5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35114/0 (socket says 172.21.15.64:35114) 2023-11-24T08:05:48.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329038640 1 -- 172.21.15.64:0/2411773683 learned_addr learned my addr 172.21.15.64:0/2411773683 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:48.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0328837640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cbb10 0x7f03240c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0323fff640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03240cd7f0 0x7f0324158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329038640 1 -- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cbb10 msgr2=0x7f03240c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329038640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cbb10 0x7f03240c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329038640 1 -- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03240cd7f0 msgr2=0x7f0324158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329038640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03240cd7f0 0x7f0324158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.869+0000 7f0329038640 1 -- 172.21.15.64:0/2411773683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180094d0 con 0x7f03240d1900 2023-11-24T08:05:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.870+0000 7f0329038640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240b5ef0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f031c001eb0 tx=0x7f031c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.870+0000 7f03217fa640 1 -- 172.21.15.64:0/2411773683 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c0052d0 con 0x7f03240d1900 2023-11-24T08:05:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.870+0000 7f03217fa640 1 -- 172.21.15.64:0/2411773683 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c005430 con 0x7f03240d1900 2023-11-24T08:05:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.870+0000 7f03217fa640 1 -- 172.21.15.64:0/2411773683 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c003810 con 0x7f03240d1900 2023-11-24T08:05:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.870+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03241f7230 con 0x7f03240d1900 2023-11-24T08:05:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.870+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03241f77f0 con 0x7f03240d1900 2023-11-24T08:05:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.871+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03241f7d10 con 0x7f03240d1900 2023-11-24T08:05:48.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.872+0000 7f03217fa640 1 -- 172.21.15.64:0/2411773683 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f031c003970 con 0x7f03240d1900 2023-11-24T08:05:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.873+0000 7f03217fa640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f031006af00 0x7f031006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.873+0000 7f03217fa640 1 -- 172.21.15.64:0/2411773683 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f031c088b30 con 0x7f03240d1900 2023-11-24T08:05:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.874+0000 7f0328837640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f031006af00 0x7f031006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.875+0000 7f0328837640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f031006af00 0x7f031006d3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0314009a60 tx=0x7f0314003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:48.878+0000 7f03217fa640 1 -- 172.21.15.64:0/2411773683 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03241f7d10 con 0x7f03240d1900 2023-11-24T08:05:49.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.113+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03241f7d10 con 0x7f03240d1900 2023-11-24T08:05:49.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.115+0000 7f03217fa640 1 -- 172.21.15.64:0/2411773683 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f03241f7d10 con 0x7f03240d1900 2023-11-24T08:05:49.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:49.117 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1893080737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":1893080737},{"type":"v1","addr":"172.21.15.91:6837","nonce":1893080737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2845212044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2845212044},{"type":"v1","addr":"172.21.15.91:6835","nonce":2845212044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2382740551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2382740551},{"type":"v1","addr":"172.21.15.64:6833","nonce":2382740551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:05:19.478049+0000","modified":"2023-11-24T08:05:22.529755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6835/2394348469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2394348469},{"type":"v1","addr":"172.21.15.64:6835","nonce":2394348469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:05:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-11-24T08:05:49.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f031006af00 msgr2=0x7f031006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f031006af00 0x7f031006d3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0314009a60 tx=0x7f0314003990 comp rx=0 tx=0).stop 2023-11-24T08:05:49.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 msgr2=0x7f03240b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240b5ef0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f031c001eb0 tx=0x7f031c00c890 comp rx=0 tx=0).stop 2023-11-24T08:05:49.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 shutdown_connections 2023-11-24T08:05:49.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f031006af00 0x7f031006d3c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03240d1900 0x7f03240b5ef0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f03240cd7f0 0x7f0324158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 --2- 172.21.15.64:0/2411773683 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03240cbb10 0x7f03240c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 >> 172.21.15.64:0/2411773683 conn(0x7f03240d0de0 msgr2=0x7f032406ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:49.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 shutdown_connections 2023-11-24T08:05:49.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.118+0000 7f0329839640 1 -- 172.21.15.64:0/2411773683 wait complete. 2023-11-24T08:05:49.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.361+0000 7facccd88640 1 Processor -- start 2023-11-24T08:05:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.361+0000 7facccd88640 1 -- start start 2023-11-24T08:05:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.363+0000 7facccd88640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc8157770 0x7facc806cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.363+0000 7facccd88640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7facc815df80 0x7facc8070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.363+0000 7facccd88640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc8161670 con 0x7facc806d080 2023-11-24T08:05:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.363+0000 7facccd88640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc815e410 con 0x7facc815df80 2023-11-24T08:05:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.363+0000 7facccd88640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc806d500 con 0x7facc8157770 2023-11-24T08:05:49.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.364+0000 7facc77fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc8157770 0x7facc806cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.364+0000 7facc7fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7facc815df80 0x7facc8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.364+0000 7facc77fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc8157770 0x7facc806cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:57194/0 (socket says 172.21.15.64:57194) 2023-11-24T08:05:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.364+0000 7facc77fe640 1 -- 172.21.15.64:0/3943986555 learned_addr learned my addr 172.21.15.64:0/3943986555 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.365+0000 7facc7fff640 1 -- 172.21.15.64:0/3943986555 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc8157770 msgr2=0x7facc806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.365+0000 7facc7fff640 1 --2- 172.21.15.64:0/3943986555 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc8157770 0x7facc806cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.365+0000 7facc7fff640 1 -- 172.21.15.64:0/3943986555 >> v1:172.21.15.91:6789/0 conn(0x7facc806d080 legacy=0x7facc815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.365+0000 7facc7fff640 1 -- 172.21.15.64:0/3943986555 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc815e610 con 0x7facc815df80 2023-11-24T08:05:49.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.366+0000 7facc7fff640 1 --2- 172.21.15.64:0/3943986555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc815df80 0x7facc8070720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7facc0009d90 tx=0x7facc002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed06a049e7f3f8a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.367+0000 7facc6ffd640 1 -- 172.21.15.64:0/3943986555 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc002cc50 con 0x7facc815df80 2023-11-24T08:05:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.367+0000 7facc6ffd640 1 -- 172.21.15.64:0/3943986555 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc002cdb0 con 0x7facc815df80 2023-11-24T08:05:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.367+0000 7facc6ffd640 1 -- 172.21.15.64:0/3943986555 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0034660 con 0x7facc815df80 2023-11-24T08:05:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.368+0000 7facccd88640 1 -- 172.21.15.64:0/3943986555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc815df80 msgr2=0x7facc8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.368+0000 7facccd88640 1 --2- 172.21.15.64:0/3943986555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc815df80 0x7facc8070720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7facc0009d90 tx=0x7facc002bea0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.368+0000 7facccd88640 1 -- 172.21.15.64:0/3943986555 shutdown_connections 2023-11-24T08:05:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.368+0000 7facccd88640 1 --2- 172.21.15.64:0/3943986555 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc815df80 0x7facc8070720 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.368+0000 7facccd88640 1 --2- 172.21.15.64:0/3943986555 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc8157770 0x7facc806cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.368+0000 7facccd88640 1 -- 172.21.15.64:0/3943986555 >> 172.21.15.64:0/3943986555 conn(0x7facc806a860 msgr2=0x7facc806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.368+0000 7facccd88640 1 -- 172.21.15.64:0/3943986555 shutdown_connections 2023-11-24T08:05:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.368+0000 7facccd88640 1 -- 172.21.15.64:0/3943986555 wait complete. 2023-11-24T08:05:49.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.370+0000 7facccd88640 1 Processor -- start 2023-11-24T08:05:49.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.370+0000 7facccd88640 1 -- start start 2023-11-24T08:05:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facccd88640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc806d080 0x7facc80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facccd88640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc8157770 0x7facc813e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facccd88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc815df80 0x7facc80b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facccd88640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc814ff90 con 0x7facc815df80 2023-11-24T08:05:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facccd88640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc80cc190 con 0x7facc8157770 2023-11-24T08:05:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facccd88640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc813eb90 con 0x7facc806d080 2023-11-24T08:05:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facc7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc815df80 0x7facc80b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facc7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc815df80 0x7facc80b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35122/0 (socket says 172.21.15.64:35122) 2023-11-24T08:05:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.371+0000 7facc7fff640 1 -- 172.21.15.64:0/1242988362 learned_addr learned my addr 172.21.15.64:0/1242988362 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facbffff640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc8157770 0x7facc813e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc77fe640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc806d080 0x7facc80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc7fff640 1 -- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc806d080 msgr2=0x7facc80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc7fff640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc806d080 0x7facc80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc7fff640 1 -- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc8157770 msgr2=0x7facc813e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc7fff640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc8157770 0x7facc813e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc7fff640 1 -- 172.21.15.64:0/1242988362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0009a70 con 0x7facc815df80 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc7fff640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc815df80 0x7facc80b2fe0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7facc002c4b0 tx=0x7facc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc4ff9640 1 -- 172.21.15.64:0/1242988362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc003e610 con 0x7facc815df80 2023-11-24T08:05:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.372+0000 7facc4ff9640 1 -- 172.21.15.64:0/1242988362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc003e770 con 0x7facc815df80 2023-11-24T08:05:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.373+0000 7facc4ff9640 1 -- 172.21.15.64:0/1242988362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc003eb20 con 0x7facc815df80 2023-11-24T08:05:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.373+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc80caeb0 con 0x7facc815df80 2023-11-24T08:05:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.373+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc80cb450 con 0x7facc815df80 2023-11-24T08:05:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.374+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc804e840 con 0x7facc815df80 2023-11-24T08:05:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.375+0000 7facc4ff9640 1 -- 172.21.15.64:0/1242988362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7facc003b050 con 0x7facc815df80 2023-11-24T08:05:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.376+0000 7facc4ff9640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faca806aed0 0x7faca806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.376+0000 7facc4ff9640 1 -- 172.21.15.64:0/1242988362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facc00abf30 con 0x7facc815df80 2023-11-24T08:05:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.377+0000 7facc77fe640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faca806aed0 0x7faca806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.378+0000 7facc77fe640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faca806aed0 0x7faca806d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7facb8009b40 tx=0x7facb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:49.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.381+0000 7facc4ff9640 1 -- 172.21.15.64:0/1242988362 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7facc0076f80 con 0x7facc815df80 2023-11-24T08:05:49.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.575+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facc8070500 con 0x7facc815df80 2023-11-24T08:05:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.579+0000 7facc4ff9640 1 -- 172.21.15.64:0/1242988362 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16521 (secure 0 0 0) 0x7facc0039090 con 0x7facc815df80 2023-11-24T08:05:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:49.581 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:21.012478+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:05:17.535151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:05:18.499872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:49.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faca806aed0 msgr2=0x7faca806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faca806aed0 0x7faca806d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7facb8009b40 tx=0x7facb8008040 comp rx=0 tx=0).stop 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc815df80 msgr2=0x7facc80b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc815df80 0x7facc80b2fe0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7facc002c4b0 tx=0x7facc0004a40 comp rx=0 tx=0).stop 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 shutdown_connections 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faca806aed0 0x7faca806d390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc815df80 0x7facc80b2fe0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7facc8157770 0x7facc813e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 --2- 172.21.15.64:0/1242988362 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7facc806d080 0x7facc80cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.581+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 >> 172.21.15.64:0/1242988362 conn(0x7facc806a860 msgr2=0x7facc8156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:49.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.582+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 shutdown_connections 2023-11-24T08:05:49.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.582+0000 7facccd88640 1 -- 172.21.15.64:0/1242988362 wait complete. 2023-11-24T08:05:49.604 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:05:49.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:49.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.832+0000 7f8c2539e640 1 Processor -- start 2023-11-24T08:05:49.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.832+0000 7f8c2539e640 1 -- start start 2023-11-24T08:05:49.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.833+0000 7f8c2539e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c200d14a0 0x7f8c200d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.833+0000 7f8c2539e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.833+0000 7f8c2539e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c200cf0c0 con 0x7f8c2015d5b0 2023-11-24T08:05:49.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.833+0000 7f8c2539e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c200b01b0 con 0x7f8c200d14a0 2023-11-24T08:05:49.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.833+0000 7f8c2539e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2015d990 con 0x7f8c200b0640 2023-11-24T08:05:49.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.833+0000 7f8c1ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8c2015d5b0 0x7f8c200d3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56440/0 (socket says 172.21.15.64:56440) 2023-11-24T08:05:49.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.833+0000 7f8c1ffff640 1 -- 172.21.15.64:0/3239463565 learned_addr learned my addr 172.21.15.64:0/3239463565 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:49.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c177fe640 1 --2- 172.21.15.64:0/3239463565 >> v2:172.21.15.64:3300/0 conn(0x7f8c200d14a0 0x7f8c200d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c24b9d640 1 --2- 172.21.15.64:0/3239463565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c1f7fe640 1 -- 172.21.15.64:0/3239463565 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160283165 0 0) 0x7f8c200cf0c0 con 0x7f8c2015d5b0 2023-11-24T08:05:49.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c1f7fe640 1 -- 172.21.15.64:0/3239463565 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c0c003520 con 0x7f8c2015d5b0 2023-11-24T08:05:49.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c177fe640 1 -- 172.21.15.64:0/3239463565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 msgr2=0x7f8c200b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:49.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c177fe640 1 --2- 172.21.15.64:0/3239463565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c177fe640 1 -- 172.21.15.64:0/3239463565 >> v1:172.21.15.64:6789/0 conn(0x7f8c2015d5b0 legacy=0x7f8c200d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c177fe640 1 -- 172.21.15.64:0/3239463565 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200d1e40 con 0x7f8c200d14a0 2023-11-24T08:05:49.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.834+0000 7f8c24b9d640 1 --2- 172.21.15.64:0/3239463565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:49.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.836+0000 7f8c177fe640 1 --2- 172.21.15.64:0/3239463565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200d1900 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097a0 tx=0x7f8c0802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5af95c93e69de3e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:49.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.836+0000 7f8c1f7fe640 1 -- 172.21.15.64:0/3239463565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0802d040 con 0x7f8c200d14a0 2023-11-24T08:05:49.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.836+0000 7f8c1f7fe640 1 -- 172.21.15.64:0/3239463565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08031810 con 0x7f8c200d14a0 2023-11-24T08:05:49.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.836+0000 7f8c1f7fe640 1 -- 172.21.15.64:0/3239463565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08031ba0 con 0x7f8c200d14a0 2023-11-24T08:05:49.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.836+0000 7f8c2539e640 1 -- 172.21.15.64:0/3239463565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 msgr2=0x7f8c200d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.836+0000 7f8c2539e640 1 --2- 172.21.15.64:0/3239463565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200d1900 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097a0 tx=0x7f8c0802b8b0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.837+0000 7f8c2539e640 1 -- 172.21.15.64:0/3239463565 shutdown_connections 2023-11-24T08:05:49.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.837+0000 7f8c2539e640 1 --2- 172.21.15.64:0/3239463565 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.837+0000 7f8c2539e640 1 --2- 172.21.15.64:0/3239463565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200d1900 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.837+0000 7f8c2539e640 1 -- 172.21.15.64:0/3239463565 >> 172.21.15.64:0/3239463565 conn(0x7f8c2009c3b0 msgr2=0x7f8c200d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:49.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.837+0000 7f8c2539e640 1 -- 172.21.15.64:0/3239463565 shutdown_connections 2023-11-24T08:05:49.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.837+0000 7f8c2539e640 1 -- 172.21.15.64:0/3239463565 wait complete. 2023-11-24T08:05:49.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.839+0000 7f8c2539e640 1 Processor -- start 2023-11-24T08:05:49.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.839+0000 7f8c2539e640 1 -- start start 2023-11-24T08:05:49.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c2539e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c2539e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c2539e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c2015d5b0 0x7f8c2015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c2539e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200c7a00 con 0x7f8c200d14a0 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c2539e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200c97c0 con 0x7f8c2015d5b0 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c2539e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c201f3920 con 0x7f8c200b0640 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35130/0 (socket says 172.21.15.64:35130) 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c177fe640 1 -- 172.21.15.64:0/986204853 learned_addr learned my addr 172.21.15.64:0/986204853 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c1ffff640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c24b9d640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c2015d5b0 0x7f8c2015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c177fe640 1 -- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 msgr2=0x7f8c200cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c177fe640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c177fe640 1 -- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c2015d5b0 msgr2=0x7f8c2015eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c177fe640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c2015d5b0 0x7f8c2015eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c177fe640 1 -- 172.21.15.64:0/986204853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08009400 con 0x7f8c200d14a0 2023-11-24T08:05:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.840+0000 7f8c24b9d640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c2015d5b0 0x7f8c2015eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.841+0000 7f8c177fe640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200c86f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c080096f0 tx=0x7f8c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:49.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.841+0000 7f8c1d7fa640 1 -- 172.21.15.64:0/986204853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0803ad60 con 0x7f8c200d14a0 2023-11-24T08:05:49.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.841+0000 7f8c1d7fa640 1 -- 172.21.15.64:0/986204853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c080423f0 con 0x7f8c200d14a0 2023-11-24T08:05:49.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.841+0000 7f8c1d7fa640 1 -- 172.21.15.64:0/986204853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080427a0 con 0x7f8c200d14a0 2023-11-24T08:05:49.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.841+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c200c7c10 con 0x7f8c200d14a0 2023-11-24T08:05:49.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.841+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c200c81d0 con 0x7f8c200d14a0 2023-11-24T08:05:49.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.842+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c200c83c0 con 0x7f8c200d14a0 2023-11-24T08:05:49.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.843+0000 7f8c1d7fa640 1 -- 172.21.15.64:0/986204853 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8c0804b450 con 0x7f8c200d14a0 2023-11-24T08:05:49.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.844+0000 7f8c1d7fa640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c0006aed0 0x7f8c0006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:49.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.844+0000 7f8c1d7fa640 1 -- 172.21.15.64:0/986204853 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c080ab660 con 0x7f8c200d14a0 2023-11-24T08:05:49.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.845+0000 7f8c1ffff640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c0006aed0 0x7f8c0006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:49.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.846+0000 7f8c1ffff640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c0006aed0 0x7f8c0006d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8c100011e0 tx=0x7f8c10009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:49.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:49.849+0000 7f8c1d7fa640 1 -- 172.21.15.64:0/986204853 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c080765a0 con 0x7f8c200d14a0 2023-11-24T08:05:50.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.083+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c2004e840 con 0x7f8c200d14a0 2023-11-24T08:05:50.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.084+0000 7f8c1d7fa640 1 -- 172.21.15.64:0/986204853 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f8c08079bc0 con 0x7f8c200d14a0 2023-11-24T08:05:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:50.087 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1893080737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":1893080737},{"type":"v1","addr":"172.21.15.91:6837","nonce":1893080737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2845212044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2845212044},{"type":"v1","addr":"172.21.15.91:6835","nonce":2845212044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2382740551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2382740551},{"type":"v1","addr":"172.21.15.64:6833","nonce":2382740551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:05:19.478049+0000","modified":"2023-11-24T08:05:22.529755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6835/2394348469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2394348469},{"type":"v1","addr":"172.21.15.64:6835","nonce":2394348469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:05:50.087 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-11-24T08:05:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c0006aed0 msgr2=0x7f8c0006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c0006aed0 0x7f8c0006d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8c100011e0 tx=0x7f8c10009040 comp rx=0 tx=0).stop 2023-11-24T08:05:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 msgr2=0x7f8c200c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200c86f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c080096f0 tx=0x7f8c080046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 shutdown_connections 2023-11-24T08:05:50.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8c0006aed0 0x7f8c0006d390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8c2015d5b0 0x7f8c2015eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c200d14a0 0x7f8c200c86f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 --2- 172.21.15.64:0/986204853 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8c200b0640 0x7f8c200cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 >> 172.21.15.64:0/986204853 conn(0x7f8c2009c3b0 msgr2=0x7f8c200d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 shutdown_connections 2023-11-24T08:05:50.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.087+0000 7f8c2539e640 1 -- 172.21.15.64:0/986204853 wait complete. 2023-11-24T08:05:50.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.333+0000 7f9a2f59e640 1 Processor -- start 2023-11-24T08:05:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.333+0000 7f9a2f59e640 1 -- start start 2023-11-24T08:05:50.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.335+0000 7f9a2f59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9a3013dfd0 0x7f9a3015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.335+0000 7f9a2f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a300ce7c0 con 0x7f9a300cf050 2023-11-24T08:05:50.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.335+0000 7f9a2f59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a30140900 con 0x7f9a300cf500 2023-11-24T08:05:50.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.335+0000 7f9a2f59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3006cf50 con 0x7f9a3013dfd0 2023-11-24T08:05:50.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.335+0000 7f9a2dd9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9a300cf050 0x7f9a3006c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56450/0 (socket says 172.21.15.64:56450) 2023-11-24T08:05:50.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.335+0000 7f9a2dd9b640 1 -- 172.21.15.64:0/3131289814 learned_addr learned my addr 172.21.15.64:0/3131289814 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:50.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.335+0000 7f9a2e59c640 1 --2- 172.21.15.64:0/3131289814 >> v2:172.21.15.64:3300/0 conn(0x7f9a3013dfd0 0x7f9a3015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.336+0000 7f9a2d59a640 1 -- 172.21.15.64:0/3131289814 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252700950 0 0) 0x7f9a300ce7c0 con 0x7f9a300cf050 2023-11-24T08:05:50.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.336+0000 7f9a2d59a640 1 -- 172.21.15.64:0/3131289814 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a18003520 con 0x7f9a300cf050 2023-11-24T08:05:50.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.336+0000 7f9a2e59c640 1 -- 172.21.15.64:0/3131289814 >> v1:172.21.15.91:6789/0 conn(0x7f9a300cf500 legacy=0x7f9a3006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.336+0000 7f9a2d59a640 1 -- 172.21.15.64:0/3131289814 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2028692189 0 0) 0x7f9a18003520 con 0x7f9a300cf050 2023-11-24T08:05:50.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.336+0000 7f9a2e59c640 1 -- 172.21.15.64:0/3131289814 >> v1:172.21.15.64:6789/0 conn(0x7f9a300cf050 legacy=0x7f9a3006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.336+0000 7f9a2e59c640 1 -- 172.21.15.64:0/3131289814 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3015fa70 con 0x7f9a3013dfd0 2023-11-24T08:05:50.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.338+0000 7f9a2e59c640 1 --2- 172.21.15.64:0/3131289814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015be70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9a200097a0 tx=0x7f9a2002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dad3d161a5740a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.338+0000 7f9a2d59a640 1 -- 172.21.15.64:0/3131289814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2002f050 con 0x7f9a3013dfd0 2023-11-24T08:05:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.338+0000 7f9a2d59a640 1 -- 172.21.15.64:0/3131289814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a20031500 con 0x7f9a3013dfd0 2023-11-24T08:05:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.338+0000 7f9a2d59a640 1 -- 172.21.15.64:0/3131289814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a24001670 con 0x7f9a3013dfd0 2023-11-24T08:05:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.338+0000 7f9a2f59e640 1 -- 172.21.15.64:0/3131289814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 msgr2=0x7f9a3015be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.338+0000 7f9a2f59e640 1 --2- 172.21.15.64:0/3131289814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015be70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9a200097a0 tx=0x7f9a2002b820 comp rx=0 tx=0).stop 2023-11-24T08:05:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.339+0000 7f9a2f59e640 1 -- 172.21.15.64:0/3131289814 shutdown_connections 2023-11-24T08:05:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.339+0000 7f9a2f59e640 1 --2- 172.21.15.64:0/3131289814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015be70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.339+0000 7f9a2f59e640 1 -- 172.21.15.64:0/3131289814 >> 172.21.15.64:0/3131289814 conn(0x7f9a3006a860 msgr2=0x7f9a3006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.339+0000 7f9a2f59e640 1 -- 172.21.15.64:0/3131289814 shutdown_connections 2023-11-24T08:05:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.339+0000 7f9a2f59e640 1 -- 172.21.15.64:0/3131289814 wait complete. 2023-11-24T08:05:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.341+0000 7f9a2f59e640 1 Processor -- start 2023-11-24T08:05:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.341+0000 7f9a2f59e640 1 -- start start 2023-11-24T08:05:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a300cf050 0x7f9a301f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a300cf500 0x7f9a3013d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a30150b30 con 0x7f9a3013dfd0 2023-11-24T08:05:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301f1c50 con 0x7f9a300cf500 2023-11-24T08:05:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301f1f30 con 0x7f9a300cf050 2023-11-24T08:05:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35148/0 (socket says 172.21.15.64:35148) 2023-11-24T08:05:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2ed9d640 1 -- 172.21.15.64:0/1892374598 learned_addr learned my addr 172.21.15.64:0/1892374598 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2dd9b640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a300cf500 0x7f9a3013d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.342+0000 7f9a2e59c640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a300cf050 0x7f9a301f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.343+0000 7f9a2ed9d640 1 -- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a300cf050 msgr2=0x7f9a301f13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.343+0000 7f9a2ed9d640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a300cf050 0x7f9a301f13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.343+0000 7f9a2ed9d640 1 -- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a300cf500 msgr2=0x7f9a3013d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.343+0000 7f9a2ed9d640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a300cf500 0x7f9a3013d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.343+0000 7f9a2ed9d640 1 -- 172.21.15.64:0/1892374598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20009400 con 0x7f9a3013dfd0 2023-11-24T08:05:50.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.343+0000 7f9a2ed9d640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015bc40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800c7a0 tx=0x7f9a2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:50.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.343+0000 7f9a1effd640 1 -- 172.21.15.64:0/1892374598 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28003b70 con 0x7f9a3013dfd0 2023-11-24T08:05:50.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.344+0000 7f9a1effd640 1 -- 172.21.15.64:0/1892374598 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28003cf0 con 0x7f9a3013dfd0 2023-11-24T08:05:50.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.344+0000 7f9a1effd640 1 -- 172.21.15.64:0/1892374598 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28003ea0 con 0x7f9a3013dfd0 2023-11-24T08:05:50.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.344+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a301f2210 con 0x7f9a3013dfd0 2023-11-24T08:05:50.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.344+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a301f1930 con 0x7f9a3013dfd0 2023-11-24T08:05:50.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.345+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a30150fc0 con 0x7f9a3013dfd0 2023-11-24T08:05:50.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.346+0000 7f9a1effd640 1 -- 172.21.15.64:0/1892374598 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9a2800aa90 con 0x7f9a3013dfd0 2023-11-24T08:05:50.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.347+0000 7f9a1effd640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9a1006aed0 0x7f9a1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.347+0000 7f9a1effd640 1 -- 172.21.15.64:0/1892374598 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a280894c0 con 0x7f9a3013dfd0 2023-11-24T08:05:50.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.347+0000 7f9a2e59c640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9a1006aed0 0x7f9a1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.348+0000 7f9a2e59c640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9a1006aed0 0x7f9a1006d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9a20035000 tx=0x7f9a20008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:50.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.351+0000 7f9a1effd640 1 -- 172.21.15.64:0/1892374598 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a30150fc0 con 0x7f9a3013dfd0 2023-11-24T08:05:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.588+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a30150fc0 con 0x7f9a3013dfd0 2023-11-24T08:05:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.590+0000 7f9a1effd640 1 -- 172.21.15.64:0/1892374598 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f9a30150fc0 con 0x7f9a3013dfd0 2023-11-24T08:05:50.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:50.592 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1893080737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":1893080737},{"type":"v1","addr":"172.21.15.91:6837","nonce":1893080737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2845212044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2845212044},{"type":"v1","addr":"172.21.15.91:6835","nonce":2845212044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2382740551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2382740551},{"type":"v1","addr":"172.21.15.64:6833","nonce":2382740551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:05:19.478049+0000","modified":"2023-11-24T08:05:22.529755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.64:6835/2394348469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2394348469},{"type":"v1","addr":"172.21.15.64:6835","nonce":2394348469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:05:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.592+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9a1006aed0 msgr2=0x7f9a1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.592+0000 7f9a2f59e640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9a1006aed0 0x7f9a1006d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9a20035000 tx=0x7f9a20008030 comp rx=0 tx=0).stop 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.592+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 msgr2=0x7f9a3015bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015bc40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800c7a0 tx=0x7f9a2800cc60 comp rx=0 tx=0).stop 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 shutdown_connections 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9a1006aed0 0x7f9a1006d390 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a3013dfd0 0x7f9a3015bc40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9a300cf500 0x7f9a3013d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 --2- 172.21.15.64:0/1892374598 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9a300cf050 0x7f9a301f13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 >> 172.21.15.64:0/1892374598 conn(0x7f9a3006a860 msgr2=0x7f9a3013e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 shutdown_connections 2023-11-24T08:05:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.593+0000 7f9a2f59e640 1 -- 172.21.15.64:0/1892374598 wait complete. 2023-11-24T08:05:50.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.837+0000 7f96b1c8c640 1 Processor -- start 2023-11-24T08:05:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.838+0000 7f96b1c8c640 1 -- start start 2023-11-24T08:05:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.839+0000 7f96b1c8c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96ac157770 0x7f96ac06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.839+0000 7f96b1c8c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac06d080 0x7f96ac15d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.839+0000 7f96b1c8c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ac161670 con 0x7f96ac15ddb0 2023-11-24T08:05:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.839+0000 7f96b1c8c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ac06d500 con 0x7f96ac157770 2023-11-24T08:05:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.839+0000 7f96b1c8c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac071730 con 0x7f96ac06d080 2023-11-24T08:05:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96b148b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f96ac15ddb0 0x7f96ac0ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56460/0 (socket says 172.21.15.64:56460) 2023-11-24T08:05:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96b148b640 1 -- 172.21.15.64:0/800433859 learned_addr learned my addr 172.21.15.64:0/800433859 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96b0c8a640 1 --2- 172.21.15.64:0/800433859 >> v2:172.21.15.64:3300/0 conn(0x7f96ac157770 0x7f96ac06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96a3fff640 1 --2- 172.21.15.64:0/800433859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac06d080 0x7f96ac15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96b0c8a640 1 -- 172.21.15.64:0/800433859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac06d080 msgr2=0x7f96ac15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96b0c8a640 1 --2- 172.21.15.64:0/800433859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac06d080 0x7f96ac15d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96b0c8a640 1 -- 172.21.15.64:0/800433859 >> v1:172.21.15.64:6789/0 conn(0x7f96ac15ddb0 legacy=0x7f96ac0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96a37fe640 1 -- 172.21.15.64:0/800433859 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658270962 0 0) 0x7f96ac161670 con 0x7f96ac15ddb0 2023-11-24T08:05:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.840+0000 7f96b0c8a640 1 -- 172.21.15.64:0/800433859 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac15e610 con 0x7f96ac157770 2023-11-24T08:05:50.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.841+0000 7f96b0c8a640 1 --2- 172.21.15.64:0/800433859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac157770 0x7f96ac06cb40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f96a80097a0 tx=0x7f96a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a94b4aa4779d60 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:50.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96a37fe640 1 -- 172.21.15.64:0/800433859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8031830 con 0x7f96ac157770 2023-11-24T08:05:50.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96a37fe640 1 -- 172.21.15.64:0/800433859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8031990 con 0x7f96ac157770 2023-11-24T08:05:50.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96a37fe640 1 -- 172.21.15.64:0/800433859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8031d20 con 0x7f96ac157770 2023-11-24T08:05:50.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96b1c8c640 1 -- 172.21.15.64:0/800433859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac157770 msgr2=0x7f96ac06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/800433859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac157770 0x7f96ac06cb40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f96a80097a0 tx=0x7f96a802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96b1c8c640 1 -- 172.21.15.64:0/800433859 shutdown_connections 2023-11-24T08:05:50.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/800433859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac06d080 0x7f96ac15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/800433859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac157770 0x7f96ac06cb40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.842+0000 7f96b1c8c640 1 -- 172.21.15.64:0/800433859 >> 172.21.15.64:0/800433859 conn(0x7f96ac06a860 msgr2=0x7f96ac06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:50.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.843+0000 7f96b1c8c640 1 -- 172.21.15.64:0/800433859 shutdown_connections 2023-11-24T08:05:50.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.843+0000 7f96b1c8c640 1 -- 172.21.15.64:0/800433859 wait complete. 2023-11-24T08:05:50.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.844+0000 7f96b1c8c640 1 Processor -- start 2023-11-24T08:05:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.844+0000 7f96b1c8c640 1 -- start start 2023-11-24T08:05:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.845+0000 7f96b1c8c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96ac06d080 0x7f96ac15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.845+0000 7f96b1c8c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac157770 0x7f96ac1f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.845+0000 7f96b1c8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac15ddb0 0x7f96ac159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.845+0000 7f96b1c8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac15ca80 con 0x7f96ac15ddb0 2023-11-24T08:05:50.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.845+0000 7f96b1c8c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac1f7960 con 0x7f96ac06d080 2023-11-24T08:05:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b1c8c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac15be70 con 0x7f96ac157770 2023-11-24T08:05:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b148b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac15ddb0 0x7f96ac159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b148b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac15ddb0 0x7f96ac159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35158/0 (socket says 172.21.15.64:35158) 2023-11-24T08:05:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b148b640 1 -- 172.21.15.64:0/2367900533 learned_addr learned my addr 172.21.15.64:0/2367900533 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b0c8a640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96ac06d080 0x7f96ac15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96a3fff640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac157770 0x7f96ac1f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b148b640 1 -- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac157770 msgr2=0x7f96ac1f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b148b640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac157770 0x7f96ac1f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b148b640 1 -- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96ac06d080 msgr2=0x7f96ac15b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:50.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b148b640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96ac06d080 0x7f96ac15b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:50.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b148b640 1 -- 172.21.15.64:0/2367900533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a8009400 con 0x7f96ac15ddb0 2023-11-24T08:05:50.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.846+0000 7f96b0c8a640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96ac06d080 0x7f96ac15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:05:50.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.847+0000 7f96b148b640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac15ddb0 0x7f96ac159a50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c7b0 tx=0x7f96a400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:50.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.847+0000 7f96a17fa640 1 -- 172.21.15.64:0/2367900533 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400a640 con 0x7f96ac15ddb0 2023-11-24T08:05:50.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.847+0000 7f96a17fa640 1 -- 172.21.15.64:0/2367900533 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a400a7c0 con 0x7f96ac15ddb0 2023-11-24T08:05:50.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.847+0000 7f96a17fa640 1 -- 172.21.15.64:0/2367900533 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400ab70 con 0x7f96ac15ddb0 2023-11-24T08:05:50.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.847+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac15c150 con 0x7f96ac15ddb0 2023-11-24T08:05:50.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.847+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ac15c710 con 0x7f96ac15ddb0 2023-11-24T08:05:50.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.848+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac15cc30 con 0x7f96ac15ddb0 2023-11-24T08:05:50.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.849+0000 7f96a17fa640 1 -- 172.21.15.64:0/2367900533 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f96a400acd0 con 0x7f96ac15ddb0 2023-11-24T08:05:50.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.850+0000 7f96a17fa640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f969806aed0 0x7f969806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:50.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.850+0000 7f96a17fa640 1 -- 172.21.15.64:0/2367900533 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96a4088b50 con 0x7f96ac15ddb0 2023-11-24T08:05:50.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.851+0000 7f96b0c8a640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f969806aed0 0x7f969806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:50.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.852+0000 7f96b0c8a640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f969806aed0 0x7f969806d390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f96a8036000 tx=0x7f96a803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:50.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:50.855+0000 7f96a17fa640 1 -- 172.21.15.64:0/2367900533 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96ac15cc30 con 0x7f96ac15ddb0 2023-11-24T08:05:51.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.061+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96ac15cc30 con 0x7f96ac15ddb0 2023-11-24T08:05:51.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.062+0000 7f96a17fa640 1 -- 172.21.15.64:0/2367900533 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f96ac15cc30 con 0x7f96ac15ddb0 2023-11-24T08:05:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:51.065 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:21.012478+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:05:17.535151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:05:18.499872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:51.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f969806aed0 msgr2=0x7f969806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f969806aed0 0x7f969806d390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f96a8036000 tx=0x7f96a803c040 comp rx=0 tx=0).stop 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac15ddb0 msgr2=0x7f96ac159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac15ddb0 0x7f96ac159a50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c7b0 tx=0x7f96a400cc70 comp rx=0 tx=0).stop 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 shutdown_connections 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f969806aed0 0x7f969806d390 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96ac15ddb0 0x7f96ac159a50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96ac157770 0x7f96ac1f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 --2- 172.21.15.64:0/2367900533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f96ac06d080 0x7f96ac15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.065+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 >> 172.21.15.64:0/2367900533 conn(0x7f96ac06a860 msgr2=0x7f96ac155340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.066+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 shutdown_connections 2023-11-24T08:05:51.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.066+0000 7f96b1c8c640 1 -- 172.21.15.64:0/2367900533 wait complete. 2023-11-24T08:05:51.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:05:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.301+0000 7f71b3cbb640 1 Processor -- start 2023-11-24T08:05:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.301+0000 7f71b3cbb640 1 -- start start 2023-11-24T08:05:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.303+0000 7f71b3cbb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.303+0000 7f71b3cbb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f71ac06dc30 0x7f71ac0b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.303+0000 7f71b3cbb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac0ac530 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.303+0000 7f71b3cbb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0b2080 con 0x7f71ac06dc30 2023-11-24T08:05:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.303+0000 7f71b3cbb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0b2360 con 0x7f71ac15b340 2023-11-24T08:05:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.303+0000 7f71b2258640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f71ac06dc30 0x7f71ac0b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.303+0000 7f71b2258640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f71ac06dc30 0x7f71ac0b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35166/0 (socket says 172.21.15.64:35166) 2023-11-24T08:05:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.303+0000 7f71b2258640 1 -- 172.21.15.64:0/2127674087 learned_addr learned my addr 172.21.15.64:0/2127674087 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:51.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.304+0000 7f71b1a57640 1 --2- 172.21.15.64:0/2127674087 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.304+0000 7f71b2258640 1 -- 172.21.15.64:0/2127674087 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 msgr2=0x7f71ac0abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:51.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.304+0000 7f71b2258640 1 --2- 172.21.15.64:0/2127674087 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.304+0000 7f71b2258640 1 -- 172.21.15.64:0/2127674087 >> v1:172.21.15.91:6789/0 conn(0x7f71ac0cbcb0 legacy=0x7f71ac06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.304+0000 7f71b2258640 1 -- 172.21.15.64:0/2127674087 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0b2640 con 0x7f71ac06dc30 2023-11-24T08:05:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.305+0000 7f71b2258640 1 --2- 172.21.15.64:0/2127674087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac06dc30 0x7f71ac0b3120 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e50 tx=0x7f71a402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a8ce75667440db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.305+0000 7f71b0a55640 1 -- 172.21.15.64:0/2127674087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a402cb00 con 0x7f71ac06dc30 2023-11-24T08:05:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.305+0000 7f71b0a55640 1 -- 172.21.15.64:0/2127674087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a402cc60 con 0x7f71ac06dc30 2023-11-24T08:05:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.305+0000 7f71b0a55640 1 -- 172.21.15.64:0/2127674087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40343f0 con 0x7f71ac06dc30 2023-11-24T08:05:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.306+0000 7f71b3cbb640 1 -- 172.21.15.64:0/2127674087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac06dc30 msgr2=0x7f71ac0b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.306+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/2127674087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac06dc30 0x7f71ac0b3120 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e50 tx=0x7f71a402bf60 comp rx=0 tx=0).stop 2023-11-24T08:05:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.306+0000 7f71b3cbb640 1 -- 172.21.15.64:0/2127674087 shutdown_connections 2023-11-24T08:05:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.306+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/2127674087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac06dc30 0x7f71ac0b3120 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.306+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/2127674087 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.306+0000 7f71b3cbb640 1 -- 172.21.15.64:0/2127674087 >> 172.21.15.64:0/2127674087 conn(0x7f71ac0b0d80 msgr2=0x7f71ac0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.306+0000 7f71b3cbb640 1 -- 172.21.15.64:0/2127674087 shutdown_connections 2023-11-24T08:05:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.306+0000 7f71b3cbb640 1 -- 172.21.15.64:0/2127674087 wait complete. 2023-11-24T08:05:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.308+0000 7f71b3cbb640 1 Processor -- start 2023-11-24T08:05:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.308+0000 7f71b3cbb640 1 -- start start 2023-11-24T08:05:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b3cbb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71ac06dc30 0x7f71ac15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b3cbb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0cbcb0 0x7f71ac0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b3cbb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b3cbb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac15e540 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b3cbb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac15e0b0 con 0x7f71ac06dc30 2023-11-24T08:05:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b3cbb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac14dbc0 con 0x7f71ac15b340 2023-11-24T08:05:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b1256640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0cbcb0 0x7f71ac0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b1256640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0cbcb0 0x7f71ac0c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35176/0 (socket says 172.21.15.64:35176) 2023-11-24T08:05:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.309+0000 7f71b1256640 1 -- 172.21.15.64:0/3173132446 learned_addr learned my addr 172.21.15.64:0/3173132446 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b1a57640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71ac06dc30 0x7f71ac15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b2258640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b1256640 1 -- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 msgr2=0x7f71ac0c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b1256640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b1256640 1 -- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71ac06dc30 msgr2=0x7f71ac15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b1256640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71ac06dc30 0x7f71ac15ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b1256640 1 -- 172.21.15.64:0/3173132446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4009b30 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b2258640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b1a57640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71ac06dc30 0x7f71ac15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71b1256640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0cbcb0 0x7f71ac0c8eb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f719c00c3f0 tx=0x7f719c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.310+0000 7f71a27fc640 1 -- 172.21.15.64:0/3173132446 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c0052d0 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.311+0000 7f71a27fc640 1 -- 172.21.15.64:0/3173132446 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c005450 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.311+0000 7f71a27fc640 1 -- 172.21.15.64:0/3173132446 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c003830 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.311+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac15cbb0 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.311+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac15d780 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.312+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac074770 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.313+0000 7f71a27fc640 1 -- 172.21.15.64:0/3173132446 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f719c003990 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.314+0000 7f71a27fc640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f719006aed0 0x7f719006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.314+0000 7f71a27fc640 1 -- 172.21.15.64:0/3173132446 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f719c088bd0 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.314+0000 7f71b1a57640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f719006aed0 0x7f719006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.315+0000 7f71b1a57640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f719006aed0 0x7f719006d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71a8006fd0 tx=0x7f71a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:51.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.319+0000 7f71a27fc640 1 -- 172.21.15.64:0/3173132446 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f719c053c20 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.554+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f71ac15e1f0 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.557 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-11-24T08:05:51.555+0000 7fb13dee4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:05:51.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.558+0000 7f71a27fc640 1 -- 172.21.15.64:0/3173132446 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f719c017090 con 0x7f71ac0cbcb0 2023-11-24T08:05:51.560 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:05:51.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.562+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f719006aed0 msgr2=0x7f719006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.562+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f719006aed0 0x7f719006d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71a8006fd0 tx=0x7f71a8008040 comp rx=0 tx=0).stop 2023-11-24T08:05:51.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.562+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0cbcb0 msgr2=0x7f71ac0c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.562+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0cbcb0 0x7f71ac0c8eb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f719c00c3f0 tx=0x7f719c00c8b0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.563+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 shutdown_connections 2023-11-24T08:05:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.563+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f719006aed0 0x7f719006d390 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.563+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71ac15b340 0x7f71ac0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.563+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0cbcb0 0x7f71ac0c8eb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.563+0000 7f71b3cbb640 1 --2- 172.21.15.64:0/3173132446 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71ac06dc30 0x7f71ac15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.563+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 >> 172.21.15.64:0/3173132446 conn(0x7f71ac0b0d80 msgr2=0x7f71ac0aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.563+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 shutdown_connections 2023-11-24T08:05:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.563+0000 7f71b3cbb640 1 -- 172.21.15.64:0/3173132446 wait complete. 2023-11-24T08:05:51.574 INFO:tasks.ceph.mds.c.smithi064.stderr: -3463> 2023-11-24T08:05:21.236+0000 7f6aa7c15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:05:51.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:05:51.650 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T08:05:51.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.777+0000 7f860a9a0640 1 Processor -- start 2023-11-24T08:05:51.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.778+0000 7f860a9a0640 1 -- start start 2023-11-24T08:05:51.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.779+0000 7f860a9a0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f86040cfa30 0x7f86040cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.779+0000 7f860a9a0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 0x7f86040ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.779+0000 7f860a9a0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86040b2980 con 0x7f86040d0e00 2023-11-24T08:05:51.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.779+0000 7f860a9a0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040d0350 con 0x7f86040cfa30 2023-11-24T08:05:51.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.779+0000 7f860a9a0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040cf220 con 0x7f86040d1fd0 2023-11-24T08:05:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.779+0000 7f860a19f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f86040d0e00 0x7f86040d1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56466/0 (socket says 172.21.15.64:56466) 2023-11-24T08:05:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.779+0000 7f860a19f640 1 -- 172.21.15.64:0/3960738715 learned_addr learned my addr 172.21.15.64:0/3960738715 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.780+0000 7f860999e640 1 --2- 172.21.15.64:0/3960738715 >> v2:172.21.15.64:3300/0 conn(0x7f86040cfa30 0x7f86040cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.780+0000 7f860919d640 1 --2- 172.21.15.64:0/3960738715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 0x7f86040ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.780+0000 7f860899c640 1 -- 172.21.15.64:0/3960738715 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375662604 0 0) 0x7f86040b2980 con 0x7f86040d0e00 2023-11-24T08:05:51.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.780+0000 7f860999e640 1 -- 172.21.15.64:0/3960738715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 msgr2=0x7f86040ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:51.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.780+0000 7f860999e640 1 --2- 172.21.15.64:0/3960738715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 0x7f86040ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.780+0000 7f860999e640 1 -- 172.21.15.64:0/3960738715 >> v1:172.21.15.64:6789/0 conn(0x7f86040d0e00 legacy=0x7f86040d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.780+0000 7f860999e640 1 -- 172.21.15.64:0/3960738715 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040cf500 con 0x7f86040cfa30 2023-11-24T08:05:51.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.781+0000 7f860999e640 1 --2- 172.21.15.64:0/3960738715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86040cfe10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d90 tx=0x7f85f402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0215bc9dbc1b32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:51.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860899c640 1 -- 172.21.15.64:0/3960738715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f402cc50 con 0x7f86040cfa30 2023-11-24T08:05:51.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860899c640 1 -- 172.21.15.64:0/3960738715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f402cdb0 con 0x7f86040cfa30 2023-11-24T08:05:51.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860899c640 1 -- 172.21.15.64:0/3960738715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4034660 con 0x7f86040cfa30 2023-11-24T08:05:51.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860a9a0640 1 -- 172.21.15.64:0/3960738715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 msgr2=0x7f86040cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3960738715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86040cfe10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d90 tx=0x7f85f402bea0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860a9a0640 1 -- 172.21.15.64:0/3960738715 shutdown_connections 2023-11-24T08:05:51.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3960738715 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 0x7f86040ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3960738715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86040cfe10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860a9a0640 1 -- 172.21.15.64:0/3960738715 >> 172.21.15.64:0/3960738715 conn(0x7f86040d3020 msgr2=0x7f86040d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:51.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860a9a0640 1 -- 172.21.15.64:0/3960738715 shutdown_connections 2023-11-24T08:05:51.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.782+0000 7f860a9a0640 1 -- 172.21.15.64:0/3960738715 wait complete. 2023-11-24T08:05:51.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.784+0000 7f860a9a0640 1 Processor -- start 2023-11-24T08:05:51.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.784+0000 7f860a9a0640 1 -- start start 2023-11-24T08:05:51.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.785+0000 7f860a9a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86041f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.785+0000 7f860a9a0640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86040d0e00 0x7f86040cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.785+0000 7f860a9a0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 0x7f86040ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.785+0000 7f860a9a0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040ca0f0 con 0x7f86040cfa30 2023-11-24T08:05:51.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.785+0000 7f860a9a0640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040b8020 con 0x7f86040d0e00 2023-11-24T08:05:51.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.785+0000 7f860a9a0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040b8300 con 0x7f86040d1fd0 2023-11-24T08:05:51.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860999e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86041f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860999e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86041f2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35212/0 (socket says 172.21.15.64:35212) 2023-11-24T08:05:51.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860999e640 1 -- 172.21.15.64:0/3840198401 learned_addr learned my addr 172.21.15.64:0/3840198401 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:51.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860919d640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86040d0e00 0x7f86040cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860a19f640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 0x7f86040ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860999e640 1 -- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 msgr2=0x7f86040ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860999e640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 0x7f86040ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860999e640 1 -- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86040d0e00 msgr2=0x7f86040cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:51.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860999e640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86040d0e00 0x7f86040cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:51.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.786+0000 7f860999e640 1 -- 172.21.15.64:0/3840198401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4009a70 con 0x7f86040cfa30 2023-11-24T08:05:51.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.787+0000 7f860999e640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86041f2f70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009eb0 tx=0x7f85f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:51.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.787+0000 7f86027fc640 1 -- 172.21.15.64:0/3840198401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4003990 con 0x7f86040cfa30 2023-11-24T08:05:51.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.787+0000 7f86027fc640 1 -- 172.21.15.64:0/3840198401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4003b10 con 0x7f86040cfa30 2023-11-24T08:05:51.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.787+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86040b85e0 con 0x7f86040cfa30 2023-11-24T08:05:51.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.787+0000 7f86027fc640 1 -- 172.21.15.64:0/3840198401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4004020 con 0x7f86040cfa30 2023-11-24T08:05:51.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.787+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8604152910 con 0x7f86040cfa30 2023-11-24T08:05:51.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.788+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86040d0a40 con 0x7f86040cfa30 2023-11-24T08:05:51.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.789+0000 7f86027fc640 1 -- 172.21.15.64:0/3840198401 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f85f4004180 con 0x7f86040cfa30 2023-11-24T08:05:51.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.790+0000 7f86027fc640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f006aed0 0x7f85f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:51.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.790+0000 7f86027fc640 1 -- 172.21.15.64:0/3840198401 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f85f40ac480 con 0x7f86040cfa30 2023-11-24T08:05:51.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.791+0000 7f860919d640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f006aed0 0x7f85f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:51.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.792+0000 7f860919d640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f006aed0 0x7f85f006d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f85f8006fd0 tx=0x7f85f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:51.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:51.795+0000 7f86027fc640 1 -- 172.21.15.64:0/3840198401 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85f40773e0 con 0x7f86040cfa30 2023-11-24T08:05:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.031+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8604066d20 con 0x7f86040cfa30 2023-11-24T08:05:52.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.562+0000 7f86027fc640 1 -- 172.21.15.64:0/3840198401 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85f404aed0 con 0x7f86040cfa30 2023-11-24T08:05:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f006aed0 msgr2=0x7f85f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f006aed0 0x7f85f006d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f85f8006fd0 tx=0x7f85f8008040 comp rx=0 tx=0).stop 2023-11-24T08:05:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 msgr2=0x7f86041f2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86041f2f70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009eb0 tx=0x7f85f4004a40 comp rx=0 tx=0).stop 2023-11-24T08:05:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 shutdown_connections 2023-11-24T08:05:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f85f006aed0 0x7f85f006d390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86040d1fd0 0x7f86040ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f86040d0e00 0x7f86040cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 --2- 172.21.15.64:0/3840198401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86040cfa30 0x7f86041f2f70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.565+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 >> 172.21.15.64:0/3840198401 conn(0x7f86040d3020 msgr2=0x7f86040d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:52.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.566+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 shutdown_connections 2023-11-24T08:05:52.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.566+0000 7f860a9a0640 1 -- 172.21.15.64:0/3840198401 wait complete. 2023-11-24T08:05:52.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:05:52.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.814+0000 7fcb4f067640 1 Processor -- start 2023-11-24T08:05:52.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.814+0000 7fcb4f067640 1 -- start start 2023-11-24T08:05:52.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.816+0000 7fcb4f067640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb481438e0 0x7fcb4806a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:52.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.816+0000 7fcb4f067640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcb4806de10 0x7fcb480b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:52.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.816+0000 7fcb4f067640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4806ade0 con 0x7fcb4806a980 2023-11-24T08:05:52.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.816+0000 7fcb4f067640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb48160db0 con 0x7fcb481438e0 2023-11-24T08:05:52.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.816+0000 7fcb4f067640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb48161000 con 0x7fcb4806de10 2023-11-24T08:05:52.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.816+0000 7fcb4e065640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb481438e0 0x7fcb4806a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:52.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.817+0000 7fcb4e065640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb481438e0 0x7fcb4806a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35228/0 (socket says 172.21.15.64:35228) 2023-11-24T08:05:52.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.817+0000 7fcb4e065640 1 -- 172.21.15.64:0/1563979774 learned_addr learned my addr 172.21.15.64:0/1563979774 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.817+0000 7fcb4e866640 1 --2- 172.21.15.64:0/1563979774 >> v2:172.21.15.91:3300/0 conn(0x7fcb4806de10 0x7fcb480b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.817+0000 7fcb4e065640 1 -- 172.21.15.64:0/1563979774 >> v2:172.21.15.91:3300/0 conn(0x7fcb4806de10 msgr2=0x7fcb480b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.817+0000 7fcb4e065640 1 --2- 172.21.15.64:0/1563979774 >> v2:172.21.15.91:3300/0 conn(0x7fcb4806de10 0x7fcb480b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.817+0000 7fcb4e065640 1 -- 172.21.15.64:0/1563979774 >> v1:172.21.15.91:6789/0 conn(0x7fcb4806a980 legacy=0x7fcb4806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.817+0000 7fcb4e065640 1 -- 172.21.15.64:0/1563979774 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb481612e0 con 0x7fcb481438e0 2023-11-24T08:05:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.819+0000 7fcb4e065640 1 --2- 172.21.15.64:0/1563979774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb481438e0 0x7fcb4806a440 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb480cfdb0 tx=0x7fcb3802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f9117265ef6658 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.819+0000 7fcb4d063640 1 -- 172.21.15.64:0/1563979774 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3802cb20 con 0x7fcb481438e0 2023-11-24T08:05:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.819+0000 7fcb4d063640 1 -- 172.21.15.64:0/1563979774 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3802cc80 con 0x7fcb481438e0 2023-11-24T08:05:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.819+0000 7fcb4d063640 1 -- 172.21.15.64:0/1563979774 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380343f0 con 0x7fcb481438e0 2023-11-24T08:05:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.820+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563979774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb481438e0 msgr2=0x7fcb4806a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.820+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563979774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb481438e0 0x7fcb4806a440 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb480cfdb0 tx=0x7fcb3802bf80 comp rx=0 tx=0).stop 2023-11-24T08:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.820+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563979774 shutdown_connections 2023-11-24T08:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.820+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563979774 >> v2:172.21.15.91:3300/0 conn(0x7fcb4806de10 0x7fcb480b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.820+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563979774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb481438e0 0x7fcb4806a440 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.820+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563979774 >> 172.21.15.64:0/1563979774 conn(0x7fcb4806be90 msgr2=0x7fcb480cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.820+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563979774 shutdown_connections 2023-11-24T08:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.820+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563979774 wait complete. 2023-11-24T08:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.822+0000 7fcb4f067640 1 Processor -- start 2023-11-24T08:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.822+0000 7fcb4f067640 1 -- start start 2023-11-24T08:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4f067640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb4806a980 0x7fcb48142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4f067640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4806de10 0x7fcb4813b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4f067640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb481438e0 0x7fcb48139330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4f067640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4813daa0 con 0x7fcb4806de10 2023-11-24T08:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4f067640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb48142b40 con 0x7fcb481438e0 2023-11-24T08:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4f067640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4813a570 con 0x7fcb4806a980 2023-11-24T08:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4d864640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4806de10 0x7fcb4813b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4d864640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4806de10 0x7fcb4813b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35238/0 (socket says 172.21.15.64:35238) 2023-11-24T08:05:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.823+0000 7fcb4d864640 1 -- 172.21.15.64:0/1563640201 learned_addr learned my addr 172.21.15.64:0/1563640201 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb4d864640 1 -- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb4806a980 msgr2=0x7fcb48142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb4d864640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb4806a980 0x7fcb48142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb4d864640 1 -- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb481438e0 msgr2=0x7fcb48139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb4d864640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb481438e0 0x7fcb48139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb4d864640 1 -- 172.21.15.64:0/1563640201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38009b30 con 0x7fcb4806de10 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb4d864640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4806de10 0x7fcb4813b1e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00c450 tx=0x7fcb3c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb46ffd640 1 -- 172.21.15.64:0/1563640201 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0052d0 con 0x7fcb4806de10 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb46ffd640 1 -- 172.21.15.64:0/1563640201 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c005430 con 0x7fcb4806de10 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb46ffd640 1 -- 172.21.15.64:0/1563640201 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c012870 con 0x7fcb4806de10 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4813a850 con 0x7fcb4806de10 2023-11-24T08:05:52.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.824+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4813df90 con 0x7fcb4806de10 2023-11-24T08:05:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.826+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4813e1b0 con 0x7fcb4806de10 2023-11-24T08:05:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.827+0000 7fcb46ffd640 1 -- 172.21.15.64:0/1563640201 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcb3c0129d0 con 0x7fcb4806de10 2023-11-24T08:05:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.827+0000 7fcb46ffd640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb3406af00 0x7fcb3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.828+0000 7fcb46ffd640 1 -- 172.21.15.64:0/1563640201 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcb3c089110 con 0x7fcb4806de10 2023-11-24T08:05:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.828+0000 7fcb4e065640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb3406af00 0x7fcb3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:52.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.829+0000 7fcb4e065640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb3406af00 0x7fcb3406d3c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcb38035000 tx=0x7fcb3802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:52.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:52.832+0000 7fcb46ffd640 1 -- 172.21.15.64:0/1563640201 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb3c054200 con 0x7fcb4806de10 2023-11-24T08:05:53.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.025+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcb4813aa40 con 0x7fcb4806de10 2023-11-24T08:05:53.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.565+0000 7fcb46ffd640 1 -- 172.21.15.64:0/1563640201 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcb4813aa40 con 0x7fcb4806de10 2023-11-24T08:05:53.568 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:05:53.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.570+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb3406af00 msgr2=0x7fcb3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:53.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.570+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb3406af00 0x7fcb3406d3c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcb38035000 tx=0x7fcb3802e000 comp rx=0 tx=0).stop 2023-11-24T08:05:53.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.570+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4806de10 msgr2=0x7fcb4813b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:53.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.570+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4806de10 0x7fcb4813b1e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00c450 tx=0x7fcb3c00c910 comp rx=0 tx=0).stop 2023-11-24T08:05:53.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.571+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 shutdown_connections 2023-11-24T08:05:53.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.571+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb3406af00 0x7fcb3406d3c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.571+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb481438e0 0x7fcb48139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.571+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4806de10 0x7fcb4813b1e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.571+0000 7fcb4f067640 1 --2- 172.21.15.64:0/1563640201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb4806a980 0x7fcb48142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.571+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 >> 172.21.15.64:0/1563640201 conn(0x7fcb4806be90 msgr2=0x7fcb4815f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:53.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.571+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 shutdown_connections 2023-11-24T08:05:53.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.572+0000 7fcb4f067640 1 -- 172.21.15.64:0/1563640201 wait complete. 2023-11-24T08:05:53.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:05:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.816+0000 7fae95e94640 1 Processor -- start 2023-11-24T08:05:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.816+0000 7fae95e94640 1 -- start start 2023-11-24T08:05:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae95e94640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae90161b80 0x7fae9006b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae95e94640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae9006b990 0x7fae9006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae95e94640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae90161ff0 con 0x7fae9015d3c0 2023-11-24T08:05:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae95e94640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9006bdc0 con 0x7fae90161b80 2023-11-24T08:05:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae95e94640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae900b3ff0 con 0x7fae9006b990 2023-11-24T08:05:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae8effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae90161b80 0x7fae9006b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae8effd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae90161b80 0x7fae9006b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35254/0 (socket says 172.21.15.64:35254) 2023-11-24T08:05:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae8effd640 1 -- 172.21.15.64:0/640350139 learned_addr learned my addr 172.21.15.64:0/640350139 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.818+0000 7fae8ffff640 1 --2- 172.21.15.64:0/640350139 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae9006b990 0x7fae9006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.819+0000 7fae8effd640 1 -- 172.21.15.64:0/640350139 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae9006b990 msgr2=0x7fae9006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.819+0000 7fae8effd640 1 --2- 172.21.15.64:0/640350139 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae9006b990 0x7fae9006cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.819+0000 7fae8effd640 1 -- 172.21.15.64:0/640350139 >> v1:172.21.15.91:6789/0 conn(0x7fae9015d3c0 legacy=0x7fae900b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.819+0000 7fae8effd640 1 -- 172.21.15.64:0/640350139 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae900b3270 con 0x7fae90161b80 2023-11-24T08:05:53.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.820+0000 7fae8effd640 1 --2- 172.21.15.64:0/640350139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae90161b80 0x7fae9006b450 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fae84009870 tx=0x7fae8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bf1a86a7d42dd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:53.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.820+0000 7fae8e7fc640 1 -- 172.21.15.64:0/640350139 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84031640 con 0x7fae90161b80 2023-11-24T08:05:53.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.820+0000 7fae8e7fc640 1 -- 172.21.15.64:0/640350139 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae840317a0 con 0x7fae90161b80 2023-11-24T08:05:53.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.820+0000 7fae8e7fc640 1 -- 172.21.15.64:0/640350139 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84031b30 con 0x7fae90161b80 2023-11-24T08:05:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.821+0000 7fae95e94640 1 -- 172.21.15.64:0/640350139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae90161b80 msgr2=0x7fae9006b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.821+0000 7fae95e94640 1 --2- 172.21.15.64:0/640350139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae90161b80 0x7fae9006b450 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fae84009870 tx=0x7fae8402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.821+0000 7fae95e94640 1 -- 172.21.15.64:0/640350139 shutdown_connections 2023-11-24T08:05:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.821+0000 7fae95e94640 1 --2- 172.21.15.64:0/640350139 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae9006b990 0x7fae9006cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.821+0000 7fae95e94640 1 --2- 172.21.15.64:0/640350139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae90161b80 0x7fae9006b450 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.821+0000 7fae95e94640 1 -- 172.21.15.64:0/640350139 >> 172.21.15.64:0/640350139 conn(0x7fae900cfd60 msgr2=0x7fae90070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.821+0000 7fae95e94640 1 -- 172.21.15.64:0/640350139 shutdown_connections 2023-11-24T08:05:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.821+0000 7fae95e94640 1 -- 172.21.15.64:0/640350139 wait complete. 2023-11-24T08:05:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.823+0000 7fae95e94640 1 Processor -- start 2023-11-24T08:05:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.823+0000 7fae95e94640 1 -- start start 2023-11-24T08:05:53.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae95e94640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae9006b990 0x7fae90147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:53.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae95e94640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae9015d3c0 0x7fae900afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae95e94640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae90161b80 0x7fae900aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae95e94640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae901480a0 con 0x7fae9006b990 2023-11-24T08:05:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae95e94640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae900c7f20 con 0x7fae9015d3c0 2023-11-24T08:05:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae95e94640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae90160aa0 con 0x7fae90161b80 2023-11-24T08:05:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae8f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae9006b990 0x7fae90147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae8f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae9006b990 0x7fae90147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35262/0 (socket says 172.21.15.64:35262) 2023-11-24T08:05:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.824+0000 7fae8f7fe640 1 -- 172.21.15.64:0/2735593794 learned_addr learned my addr 172.21.15.64:0/2735593794 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8effd640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae9015d3c0 0x7fae900afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8f7fe640 1 -- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae90161b80 msgr2=0x7fae900aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8ffff640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae90161b80 0x7fae900aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8f7fe640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae90161b80 0x7fae900aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8f7fe640 1 -- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae9015d3c0 msgr2=0x7fae900afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8f7fe640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae9015d3c0 0x7fae900afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8f7fe640 1 -- 172.21.15.64:0/2735593794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae840094d0 con 0x7fae9006b990 2023-11-24T08:05:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8effd640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae9015d3c0 0x7fae900afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae8f7fe640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae9006b990 0x7fae90147b60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c460 tx=0x7fae8000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae77fff640 1 -- 172.21.15.64:0/2735593794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80011790 con 0x7fae9006b990 2023-11-24T08:05:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae77fff640 1 -- 172.21.15.64:0/2735593794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80011930 con 0x7fae9006b990 2023-11-24T08:05:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae77fff640 1 -- 172.21.15.64:0/2735593794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80011ce0 con 0x7fae9006b990 2023-11-24T08:05:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.825+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae90148430 con 0x7fae9006b990 2023-11-24T08:05:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.826+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae901486d0 con 0x7fae9006b990 2023-11-24T08:05:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.827+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae90072620 con 0x7fae9006b990 2023-11-24T08:05:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.828+0000 7fae77fff640 1 -- 172.21.15.64:0/2735593794 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fae8001c460 con 0x7fae9006b990 2023-11-24T08:05:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.829+0000 7fae77fff640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae7c06aed0 0x7fae7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.829+0000 7fae77fff640 1 -- 172.21.15.64:0/2735593794 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fae800894c0 con 0x7fae9006b990 2023-11-24T08:05:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.829+0000 7fae8effd640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae7c06aed0 0x7fae7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:53.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.830+0000 7fae8effd640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae7c06aed0 0x7fae7c06d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae84036000 tx=0x7fae8403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:53.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:53.834+0000 7fae77fff640 1 -- 172.21.15.64:0/2735593794 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae80054670 con 0x7fae9006b990 2023-11-24T08:05:54.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.027+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae9004e920 con 0x7fae9006b990 2023-11-24T08:05:54.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.578+0000 7fae77fff640 1 -- 172.21.15.64:0/2735593794 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7fae80057c90 con 0x7fae9006b990 2023-11-24T08:05:54.581 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-11-24T08:05:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae7c06aed0 msgr2=0x7fae7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae7c06aed0 0x7fae7c06d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae84036000 tx=0x7fae8403c040 comp rx=0 tx=0).stop 2023-11-24T08:05:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae9006b990 msgr2=0x7fae90147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae9006b990 0x7fae90147b60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c460 tx=0x7fae8000c920 comp rx=0 tx=0).stop 2023-11-24T08:05:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 shutdown_connections 2023-11-24T08:05:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fae7c06aed0 0x7fae7c06d390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fae90161b80 0x7fae900aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fae9015d3c0 0x7fae900afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 --2- 172.21.15.64:0/2735593794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae9006b990 0x7fae90147b60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 >> 172.21.15.64:0/2735593794 conn(0x7fae900cfd60 msgr2=0x7fae900ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:54.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.582+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 shutdown_connections 2023-11-24T08:05:54.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.583+0000 7fae95e94640 1 -- 172.21.15.64:0/2735593794 wait complete. 2023-11-24T08:05:54.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:05:54.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.836+0000 7f2409f33640 1 Processor -- start 2023-11-24T08:05:54.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.836+0000 7f2409f33640 1 -- start start 2023-11-24T08:05:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.837+0000 7f2409f33640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2404160b10 0x7f240406e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.837+0000 7f2409f33640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f240406cad0 0x7f24040b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.837+0000 7f2409f33640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2404071780 con 0x7f2404072280 2023-11-24T08:05:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.838+0000 7f2409f33640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24040b2320 con 0x7f2404160b10 2023-11-24T08:05:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.838+0000 7f2409f33640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240406df30 con 0x7f240406cad0 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.838+0000 7f24037fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2404160b10 0x7f240406e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.838+0000 7f23fbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2404072280 0x7f240406c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56478/0 (socket says 172.21.15.64:56478) 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.838+0000 7f23fbfff640 1 -- 172.21.15.64:0/3688987172 learned_addr learned my addr 172.21.15.64:0/3688987172 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.838+0000 7f2403fff640 1 --2- 172.21.15.64:0/3688987172 >> v2:172.21.15.91:3300/0 conn(0x7f240406cad0 0x7f24040b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.839+0000 7f24037fe640 1 -- 172.21.15.64:0/3688987172 >> v2:172.21.15.91:3300/0 conn(0x7f240406cad0 msgr2=0x7f24040b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.839+0000 7f24037fe640 1 --2- 172.21.15.64:0/3688987172 >> v2:172.21.15.91:3300/0 conn(0x7f240406cad0 0x7f24040b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.839+0000 7f24037fe640 1 -- 172.21.15.64:0/3688987172 >> v1:172.21.15.64:6789/0 conn(0x7f2404072280 legacy=0x7f240406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.839+0000 7f24037fe640 1 -- 172.21.15.64:0/3688987172 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24040711c0 con 0x7f2404160b10 2023-11-24T08:05:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.839+0000 7f2402ffd640 1 -- 172.21.15.64:0/3688987172 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277092966 0 0) 0x7f2404071780 con 0x7f2404072280 2023-11-24T08:05:54.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.840+0000 7f24037fe640 1 --2- 172.21.15.64:0/3688987172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404160b10 0x7f240406e420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f24040d0090 tx=0x7f23f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dc75277fb24e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:54.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.840+0000 7f2402ffd640 1 -- 172.21.15.64:0/3688987172 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f402e070 con 0x7f2404160b10 2023-11-24T08:05:54.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.840+0000 7f2402ffd640 1 -- 172.21.15.64:0/3688987172 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f40315a0 con 0x7f2404160b10 2023-11-24T08:05:54.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.840+0000 7f2402ffd640 1 -- 172.21.15.64:0/3688987172 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4031930 con 0x7f2404160b10 2023-11-24T08:05:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.841+0000 7f2409f33640 1 -- 172.21.15.64:0/3688987172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404160b10 msgr2=0x7f240406e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.841+0000 7f2409f33640 1 --2- 172.21.15.64:0/3688987172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404160b10 0x7f240406e420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f24040d0090 tx=0x7f23f402b840 comp rx=0 tx=0).stop 2023-11-24T08:05:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.841+0000 7f2409f33640 1 -- 172.21.15.64:0/3688987172 shutdown_connections 2023-11-24T08:05:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.841+0000 7f2409f33640 1 --2- 172.21.15.64:0/3688987172 >> v2:172.21.15.91:3300/0 conn(0x7f240406cad0 0x7f24040b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.841+0000 7f2409f33640 1 --2- 172.21.15.64:0/3688987172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404160b10 0x7f240406e420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.841+0000 7f2409f33640 1 -- 172.21.15.64:0/3688987172 >> 172.21.15.64:0/3688987172 conn(0x7f24040b0ba0 msgr2=0x7f24040b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.841+0000 7f2409f33640 1 -- 172.21.15.64:0/3688987172 shutdown_connections 2023-11-24T08:05:54.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.841+0000 7f2409f33640 1 -- 172.21.15.64:0/3688987172 wait complete. 2023-11-24T08:05:54.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.843+0000 7f2409f33640 1 Processor -- start 2023-11-24T08:05:54.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.843+0000 7f2409f33640 1 -- start start 2023-11-24T08:05:54.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f2409f33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f240406cad0 0x7f24041f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:54.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f2409f33640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2404072280 0x7f24040ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:54.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f2409f33640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2404160b10 0x7f240415ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f2409f33640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24040c9750 con 0x7f240406cad0 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f2409f33640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24040ad640 con 0x7f2404160b10 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f2409f33640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24040ad8e0 con 0x7f2404072280 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f24037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f240406cad0 0x7f24041f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f24037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f240406cad0 0x7f24041f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35278/0 (socket says 172.21.15.64:35278) 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f24037fe640 1 -- 172.21.15.64:0/2874030257 learned_addr learned my addr 172.21.15.64:0/2874030257 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f24037fe640 1 -- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2404072280 msgr2=0x7f24040ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f24037fe640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2404072280 0x7f24040ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f24037fe640 1 -- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2404160b10 msgr2=0x7f240415ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.844+0000 7f24037fe640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2404160b10 0x7f240415ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.845+0000 7f24037fe640 1 -- 172.21.15.64:0/2874030257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f4009400 con 0x7f240406cad0 2023-11-24T08:05:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.845+0000 7f24037fe640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f240406cad0 0x7f24041f35c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f23f4034040 tx=0x7f23f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.845+0000 7f2400ff9640 1 -- 172.21.15.64:0/2874030257 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4032840 con 0x7f240406cad0 2023-11-24T08:05:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.845+0000 7f2400ff9640 1 -- 172.21.15.64:0/2874030257 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f40329c0 con 0x7f240406cad0 2023-11-24T08:05:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.845+0000 7f2400ff9640 1 -- 172.21.15.64:0/2874030257 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4032d70 con 0x7f240406cad0 2023-11-24T08:05:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.845+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24040c8e20 con 0x7f240406cad0 2023-11-24T08:05:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.845+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24040c93e0 con 0x7f240406cad0 2023-11-24T08:05:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.846+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240409d200 con 0x7f240406cad0 2023-11-24T08:05:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.847+0000 7f2400ff9640 1 -- 172.21.15.64:0/2874030257 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f23f403b3f0 con 0x7f240406cad0 2023-11-24T08:05:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.848+0000 7f2400ff9640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f23e406aed0 0x7f23e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.848+0000 7f2400ff9640 1 -- 172.21.15.64:0/2874030257 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f23f40ab420 con 0x7f240406cad0 2023-11-24T08:05:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.849+0000 7f23fbfff640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f23e406aed0 0x7f23e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.850+0000 7f23fbfff640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f23e406aed0 0x7f23e406d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f23ec007fd0 tx=0x7f23ec007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:54.853+0000 7f2400ff9640 1 -- 172.21.15.64:0/2874030257 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23f40766d0 con 0x7f240406cad0 2023-11-24T08:05:55.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.047+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f240404e920 con 0x7f240406cad0 2023-11-24T08:05:55.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.052+0000 7f2400ff9640 1 -- 172.21.15.64:0/2874030257 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f23f4079cf0 con 0x7f240406cad0 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.054+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f23e406aed0 msgr2=0x7f23e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.054+0000 7f2409f33640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f23e406aed0 0x7f23e406d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f23ec007fd0 tx=0x7f23ec007920 comp rx=0 tx=0).stop 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f240406cad0 msgr2=0x7f24041f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f240406cad0 0x7f24041f35c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f23f4034040 tx=0x7f23f40046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 shutdown_connections 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f23e406aed0 0x7f23e406d390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2404160b10 0x7f240415ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2404072280 0x7f24040ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 --2- 172.21.15.64:0/2874030257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f240406cad0 0x7f24041f35c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 >> 172.21.15.64:0/2874030257 conn(0x7f24040b0ba0 msgr2=0x7f24040abe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 shutdown_connections 2023-11-24T08:05:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.055+0000 7f2409f33640 1 -- 172.21.15.64:0/2874030257 wait complete. 2023-11-24T08:05:55.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T08:05:55.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.289+0000 7f093bfff640 1 Processor -- start 2023-11-24T08:05:55.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.289+0000 7f093bfff640 1 -- start start 2023-11-24T08:05:55.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.291+0000 7f093bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f093c13dfd0 0x7f093c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.291+0000 7f093bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.291+0000 7f093bfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0ce7c0 con 0x7f093c0cf050 2023-11-24T08:05:55.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.291+0000 7f093bfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c140900 con 0x7f093c13dfd0 2023-11-24T08:05:55.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.291+0000 7f093bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c15fa70 con 0x7f093c0cf500 2023-11-24T08:05:55.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.291+0000 7f093affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f093c13dfd0 0x7f093c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.291+0000 7f093affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f093c13dfd0 0x7f093c15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35292/0 (socket says 172.21.15.64:35292) 2023-11-24T08:05:55.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.291+0000 7f093affd640 1 -- 172.21.15.64:0/2227702708 learned_addr learned my addr 172.21.15.64:0/2227702708 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:55.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.292+0000 7f093affd640 1 -- 172.21.15.64:0/2227702708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 msgr2=0x7f093c06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:55.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.292+0000 7f093b7fe640 1 --2- 172.21.15.64:0/2227702708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.292+0000 7f093affd640 1 --2- 172.21.15.64:0/2227702708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c06c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.292+0000 7f093affd640 1 -- 172.21.15.64:0/2227702708 >> v1:172.21.15.91:6789/0 conn(0x7f093c0cf050 legacy=0x7f093c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:55.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.292+0000 7f093affd640 1 -- 172.21.15.64:0/2227702708 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0cf950 con 0x7f093c13dfd0 2023-11-24T08:05:55.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.293+0000 7f093affd640 1 --2- 172.21.15.64:0/2227702708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c13dfd0 0x7f093c15be70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f093c0cfc10 tx=0x7f092c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8eab5f2cef7603c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:55.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.293+0000 7f093a7fc640 1 -- 172.21.15.64:0/2227702708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c031830 con 0x7f093c13dfd0 2023-11-24T08:05:55.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.293+0000 7f093a7fc640 1 -- 172.21.15.64:0/2227702708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c031990 con 0x7f093c13dfd0 2023-11-24T08:05:55.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.293+0000 7f093a7fc640 1 -- 172.21.15.64:0/2227702708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c031d20 con 0x7f093c13dfd0 2023-11-24T08:05:55.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.294+0000 7f093bfff640 1 -- 172.21.15.64:0/2227702708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c13dfd0 msgr2=0x7f093c15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.294+0000 7f093bfff640 1 --2- 172.21.15.64:0/2227702708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c13dfd0 0x7f093c15be70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f093c0cfc10 tx=0x7f092c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.294+0000 7f093bfff640 1 -- 172.21.15.64:0/2227702708 shutdown_connections 2023-11-24T08:05:55.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.294+0000 7f093bfff640 1 --2- 172.21.15.64:0/2227702708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.294+0000 7f093bfff640 1 --2- 172.21.15.64:0/2227702708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c13dfd0 0x7f093c15be70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.294+0000 7f093bfff640 1 -- 172.21.15.64:0/2227702708 >> 172.21.15.64:0/2227702708 conn(0x7f093c06a860 msgr2=0x7f093c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:55.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.294+0000 7f093bfff640 1 -- 172.21.15.64:0/2227702708 shutdown_connections 2023-11-24T08:05:55.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.294+0000 7f093bfff640 1 -- 172.21.15.64:0/2227702708 wait complete. 2023-11-24T08:05:55.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.296+0000 7f093bfff640 1 Processor -- start 2023-11-24T08:05:55.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.296+0000 7f093bfff640 1 -- start start 2023-11-24T08:05:55.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0cf050 0x7f093c154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c130d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f093c13dfd0 0x7f093c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c156a90 con 0x7f093c0cf050 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c155bc0 con 0x7f093c13dfd0 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c155e10 con 0x7f093c0cf500 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0cf050 0x7f093c154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0cf050 0x7f093c154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35308/0 (socket says 172.21.15.64:35308) 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093affd640 1 -- 172.21.15.64:0/2504550175 learned_addr learned my addr 172.21.15.64:0/2504550175 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f0933fff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c130d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093affd640 1 -- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 msgr2=0x7f093c130d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093affd640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c130d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093affd640 1 -- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f093c13dfd0 msgr2=0x7f093c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.297+0000 7f093affd640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f093c13dfd0 0x7f093c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.298+0000 7f093affd640 1 -- 172.21.15.64:0/2504550175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0094d0 con 0x7f093c0cf050 2023-11-24T08:05:55.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.298+0000 7f0933fff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c130d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:55.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.298+0000 7f093affd640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0cf050 0x7f093c154cf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f092c02b910 tx=0x7f092c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:55.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.298+0000 7f09337fe640 1 -- 172.21.15.64:0/2504550175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c03aa60 con 0x7f093c0cf050 2023-11-24T08:05:55.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.298+0000 7f09337fe640 1 -- 172.21.15.64:0/2504550175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c03abc0 con 0x7f093c0cf050 2023-11-24T08:05:55.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.298+0000 7f09337fe640 1 -- 172.21.15.64:0/2504550175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0443f0 con 0x7f093c0cf050 2023-11-24T08:05:55.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.298+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c1560f0 con 0x7f093c0cf050 2023-11-24T08:05:55.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.298+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c1566b0 con 0x7f093c0cf050 2023-11-24T08:05:55.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.299+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c04e840 con 0x7f093c0cf050 2023-11-24T08:05:55.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.300+0000 7f09337fe640 1 -- 172.21.15.64:0/2504550175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f092c044550 con 0x7f093c0cf050 2023-11-24T08:05:55.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.301+0000 7f09337fe640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f091c06aed0 0x7f091c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.301+0000 7f09337fe640 1 -- 172.21.15.64:0/2504550175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f092c0ab8a0 con 0x7f093c0cf050 2023-11-24T08:05:55.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.302+0000 7f0933fff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f091c06aed0 0x7f091c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.303+0000 7f0933fff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f091c06aed0 0x7f091c06d390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f093c15eb20 tx=0x7f0924008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:55.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.306+0000 7f09337fe640 1 -- 172.21.15.64:0/2504550175 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f092c0769c0 con 0x7f093c0cf050 2023-11-24T08:05:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.574+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f093c0ce9a0 con 0x7f093c0cf050 2023-11-24T08:05:55.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.581+0000 7f09337fe640 1 -- 172.21.15.64:0/2504550175 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7f092c035070 con 0x7f093c0cf050 2023-11-24T08:05:55.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f091c06aed0 msgr2=0x7f091c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f091c06aed0 0x7f091c06d390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f093c15eb20 tx=0x7f0924008040 comp rx=0 tx=0).stop 2023-11-24T08:05:55.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0cf050 msgr2=0x7f093c154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0cf050 0x7f093c154cf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f092c02b910 tx=0x7f092c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 shutdown_connections 2023-11-24T08:05:55.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f091c06aed0 0x7f091c06d390 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f093c13dfd0 0x7f093c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f093c0cf500 0x7f093c130d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 --2- 172.21.15.64:0/2504550175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0cf050 0x7f093c154cf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 >> 172.21.15.64:0/2504550175 conn(0x7f093c06a860 msgr2=0x7f093c06ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:55.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 shutdown_connections 2023-11-24T08:05:55.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.584+0000 7f093bfff640 1 -- 172.21.15.64:0/2504550175 wait complete. 2023-11-24T08:05:55.605 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... ok 2023-11-24T08:05:55.605 INFO:tasks.cephfs_test_runner:Starting test: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) Validate cleaning of stale subvolume snapshot metadata. 2023-11-24T08:05:55.605 INFO:tasks.cephfs_test_runner:test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:05:55.606 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:05:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.834+0000 7fe0f3fff640 1 Processor -- start 2023-11-24T08:05:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.834+0000 7fe0f3fff640 1 -- start start 2023-11-24T08:05:55.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.835+0000 7fe0f3fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe0f415d5b0 0x7fe0f40d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f3fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0f40b1650 0x7fe0f40d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f3fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f40cf0c0 con 0x7fe0f40d14a0 2023-11-24T08:05:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f3fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f40b01b0 con 0x7fe0f415d5b0 2023-11-24T08:05:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f3fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f415d990 con 0x7fe0f40b1650 2023-11-24T08:05:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f27fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe0f40d14a0 0x7fe0f40cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56494/0 (socket says 172.21.15.64:56494) 2023-11-24T08:05:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f27fc640 1 -- 172.21.15.64:0/3440296182 learned_addr learned my addr 172.21.15.64:0/3440296182 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638711751 0 0) 0x7fe0f40cf0c0 con 0x7fe0f40d14a0 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f37fe640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0f40b1650 0x7fe0f40d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0e0003520 con 0x7fe0f40d14a0 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.836+0000 7fe0f2ffd640 1 --2- 172.21.15.64:0/3440296182 >> v2:172.21.15.91:3300/0 conn(0x7fe0f415d5b0 0x7fe0f40d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855662954 0 0) 0x7fe0e0003520 con 0x7fe0f40d14a0 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0f40b1650 msgr2=0x7fe0f40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0f40b1650 0x7fe0f40d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 >> v2:172.21.15.91:3300/0 conn(0x7fe0f415d5b0 msgr2=0x7fe0f40d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 --2- 172.21.15.64:0/3440296182 >> v2:172.21.15.91:3300/0 conn(0x7fe0f415d5b0 0x7fe0f40d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f40d1e40 con 0x7fe0f40d14a0 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fe0dc001670 con 0x7fe0f40d14a0 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 >> v1:172.21.15.64:6789/0 conn(0x7fe0f40d14a0 legacy=0x7fe0f40cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0f40b1650 0x7fe0e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0e0004010 0x7fe0e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0e0004d00 0x7fe0e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f40b01b0 con 0x7fe0e0004d00 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f415d990 con 0x7fe0e0004010 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e000bb10 con 0x7fe0f40b1650 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f37fe640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0e0004d00 0x7fe0e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f2ffd640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0f40b1650 0x7fe0e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f27fc640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0e0004010 0x7fe0e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f37fe640 1 -- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0f40b1650 msgr2=0x7fe0e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:55.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f37fe640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0f40b1650 0x7fe0e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f37fe640 1 -- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0e0004010 msgr2=0x7fe0e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:55.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.837+0000 7fe0f37fe640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0e0004010 0x7fe0e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.838+0000 7fe0f37fe640 1 -- 172.21.15.64:0/3440296182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e0003520 con 0x7fe0e0004d00 2023-11-24T08:05:55.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.838+0000 7fe0f27fc640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0e0004010 0x7fe0e00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:55.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f37fe640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0e0004d00 0x7fe0e0005180 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40b1ae0 tx=0x7fe0e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc76a4a3b08e51ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:55.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0325f0 con 0x7fe0e0004d00 2023-11-24T08:05:55.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec032750 con 0x7fe0e0004d00 2023-11-24T08:05:55.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f1ffb640 1 -- 172.21.15.64:0/3440296182 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec032ae0 con 0x7fe0e0004d00 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f3fff640 1 -- 172.21.15.64:0/3440296182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0e0004d00 msgr2=0x7fe0e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f3fff640 1 --2- 172.21.15.64:0/3440296182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0e0004d00 0x7fe0e0005180 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40b1ae0 tx=0x7fe0e00057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f37fe640 1 -- 172.21.15.64:0/3440296182 reap_dead start 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f3fff640 1 -- 172.21.15.64:0/3440296182 shutdown_connections 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.839+0000 7fe0f3fff640 1 -- 172.21.15.64:0/3440296182 >> 172.21.15.64:0/3440296182 conn(0x7fe0f409c3b0 msgr2=0x7fe0f40d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.840+0000 7fe0f3fff640 1 -- 172.21.15.64:0/3440296182 shutdown_connections 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.840+0000 7fe0f3fff640 1 -- 172.21.15.64:0/3440296182 wait complete. 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.841+0000 7fe0f3fff640 1 Processor -- start 2023-11-24T08:05:55.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.841+0000 7fe0f3fff640 1 -- start start 2023-11-24T08:05:55.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.842+0000 7fe0f3fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0e0004d00 0x7fe0f40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.842+0000 7fe0f3fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0f415ff60 0x7fe0f41605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.842+0000 7fe0f3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f4152140 0x7fe0f40c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.842+0000 7fe0f3fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f40c7a20 con 0x7fe0f4152140 2023-11-24T08:05:55.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.842+0000 7fe0f3fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f40c9450 con 0x7fe0f415ff60 2023-11-24T08:05:55.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.842+0000 7fe0f3fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f40c9730 con 0x7fe0e0004d00 2023-11-24T08:05:55.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f4152140 0x7fe0f40c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f4152140 0x7fe0f40c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35336/0 (socket says 172.21.15.64:35336) 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 -- 172.21.15.64:0/73879173 learned_addr learned my addr 172.21.15.64:0/73879173 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f27fc640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0f415ff60 0x7fe0f41605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f2ffd640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0e0004d00 0x7fe0f40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 -- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0e0004d00 msgr2=0x7fe0f40ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0e0004d00 0x7fe0f40ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 -- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0f415ff60 msgr2=0x7fe0f41605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0f415ff60 0x7fe0f41605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 -- 172.21.15.64:0/73879173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f40c9aa0 con 0x7fe0f4152140 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.843+0000 7fe0f37fe640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f4152140 0x7fe0f40c6ba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec02bf40 tx=0x7fe0ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.844+0000 7fe0ebfff640 1 -- 172.21.15.64:0/73879173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec03b630 con 0x7fe0f4152140 2023-11-24T08:05:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.844+0000 7fe0ebfff640 1 -- 172.21.15.64:0/73879173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec03b790 con 0x7fe0f4152140 2023-11-24T08:05:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.844+0000 7fe0ebfff640 1 -- 172.21.15.64:0/73879173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec03bb40 con 0x7fe0f4152140 2023-11-24T08:05:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.844+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f41525c0 con 0x7fe0f4152140 2023-11-24T08:05:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.844+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f40c70e0 con 0x7fe0f4152140 2023-11-24T08:05:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.845+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f40c92b0 con 0x7fe0f4152140 2023-11-24T08:05:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.846+0000 7fe0ebfff640 1 -- 172.21.15.64:0/73879173 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe0ec03bca0 con 0x7fe0f4152140 2023-11-24T08:05:55.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.847+0000 7fe0ebfff640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe0d406aed0 0x7fe0d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:55.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.847+0000 7fe0ebfff640 1 -- 172.21.15.64:0/73879173 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0ec0ab410 con 0x7fe0f4152140 2023-11-24T08:05:55.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.847+0000 7fe0f2ffd640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe0d406aed0 0x7fe0d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.849+0000 7fe0f2ffd640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe0d406aed0 0x7fe0d406d390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40042f0 tx=0x7fe0e400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:55.851+0000 7fe0ebfff640 1 -- 172.21.15.64:0/73879173 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0f40c92b0 con 0x7fe0f4152140 2023-11-24T08:05:56.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.048+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0f40b34d0 con 0x7fe0f4152140 2023-11-24T08:05:56.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.050+0000 7fe0ebfff640 1 -- 172.21.15.64:0/73879173 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12938 (secure 0 0 0) 0x7fe0f40b34d0 con 0x7fe0f4152140 2023-11-24T08:05:56.052 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:56.052 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":37,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:05:54.567452+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6835/2394348469":"2023-11-25T08:05:51.556073+0000","172.21.15.64:6834/2394348469":"2023-11-25T08:05:51.556073+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:05:56.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.052+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe0d406aed0 msgr2=0x7fe0d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.052+0000 7fe0f3fff640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe0d406aed0 0x7fe0d406d390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40042f0 tx=0x7fe0e400b040 comp rx=0 tx=0).stop 2023-11-24T08:05:56.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.052+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f4152140 msgr2=0x7fe0f40c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.052+0000 7fe0f3fff640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f4152140 0x7fe0f40c6ba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec02bf40 tx=0x7fe0ec004730 comp rx=0 tx=0).stop 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.053+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 shutdown_connections 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.053+0000 7fe0f3fff640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe0d406aed0 0x7fe0d406d390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.053+0000 7fe0f3fff640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0f4152140 0x7fe0f40c6ba0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.053+0000 7fe0f3fff640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe0f415ff60 0x7fe0f41605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.053+0000 7fe0f3fff640 1 --2- 172.21.15.64:0/73879173 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe0e0004d00 0x7fe0f40ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.053+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 >> 172.21.15.64:0/73879173 conn(0x7fe0f409c3b0 msgr2=0x7fe0f415be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.053+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 shutdown_connections 2023-11-24T08:05:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.053+0000 7fe0f3fff640 1 -- 172.21.15.64:0/73879173 wait complete. 2023-11-24T08:05:56.076 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:05:56.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:05:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.305+0000 7f654a1db640 1 Processor -- start 2023-11-24T08:05:56.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.306+0000 7f654a1db640 1 -- start start 2023-11-24T08:05:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.307+0000 7f654a1db640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6544153ea0 0x7f65440ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.307+0000 7f654a1db640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6544154280 0x7f654415d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f654a1db640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65440ce650 con 0x7f6544070850 2023-11-24T08:05:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f654a1db640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6544154710 con 0x7f6544154280 2023-11-24T08:05:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f654a1db640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654415ddd0 con 0x7f6544153ea0 2023-11-24T08:05:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65489d8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6544154280 0x7f654415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65489d8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6544154280 0x7f654415d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35340/0 (socket says 172.21.15.64:35340) 2023-11-24T08:05:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65489d8640 1 -- 172.21.15.64:0/2862551050 learned_addr learned my addr 172.21.15.64:0/2862551050 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65491d9640 1 --2- 172.21.15.64:0/2862551050 >> v2:172.21.15.91:3300/0 conn(0x7f6544153ea0 0x7f65440ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65489d8640 1 -- 172.21.15.64:0/2862551050 >> v2:172.21.15.91:3300/0 conn(0x7f6544153ea0 msgr2=0x7f65440ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65489d8640 1 --2- 172.21.15.64:0/2862551050 >> v2:172.21.15.91:3300/0 conn(0x7f6544153ea0 0x7f65440ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65489d8640 1 -- 172.21.15.64:0/2862551050 >> v1:172.21.15.91:6789/0 conn(0x7f6544070850 legacy=0x7f6544071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65489d8640 1 -- 172.21.15.64:0/2862551050 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654415e0a0 con 0x7f6544154280 2023-11-24T08:05:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.308+0000 7f65491d9640 1 --2- 172.21.15.64:0/2862551050 >> v2:172.21.15.91:3300/0 conn(0x7f6544153ea0 0x7f65440ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.310+0000 7f65489d8640 1 --2- 172.21.15.64:0/2862551050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544154280 0x7f654415d890 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f65440cfc10 tx=0x7f653402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5684254f214b952d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.310+0000 7f653bfff640 1 -- 172.21.15.64:0/2862551050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534031870 con 0x7f6544154280 2023-11-24T08:05:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.310+0000 7f653bfff640 1 -- 172.21.15.64:0/2862551050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65340319d0 con 0x7f6544154280 2023-11-24T08:05:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.310+0000 7f653bfff640 1 -- 172.21.15.64:0/2862551050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534031d60 con 0x7f6544154280 2023-11-24T08:05:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.311+0000 7f654a1db640 1 -- 172.21.15.64:0/2862551050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544154280 msgr2=0x7f654415d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.311+0000 7f654a1db640 1 --2- 172.21.15.64:0/2862551050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544154280 0x7f654415d890 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f65440cfc10 tx=0x7f653402b910 comp rx=0 tx=0).stop 2023-11-24T08:05:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.311+0000 7f654a1db640 1 -- 172.21.15.64:0/2862551050 shutdown_connections 2023-11-24T08:05:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.311+0000 7f654a1db640 1 --2- 172.21.15.64:0/2862551050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544154280 0x7f654415d890 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.311+0000 7f654a1db640 1 --2- 172.21.15.64:0/2862551050 >> v2:172.21.15.91:3300/0 conn(0x7f6544153ea0 0x7f65440ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.311+0000 7f654a1db640 1 -- 172.21.15.64:0/2862551050 >> 172.21.15.64:0/2862551050 conn(0x7f65440cec10 msgr2=0x7f65440cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.311+0000 7f654a1db640 1 -- 172.21.15.64:0/2862551050 shutdown_connections 2023-11-24T08:05:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.311+0000 7f654a1db640 1 -- 172.21.15.64:0/2862551050 wait complete. 2023-11-24T08:05:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.313+0000 7f654a1db640 1 Processor -- start 2023-11-24T08:05:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.313+0000 7f654a1db640 1 -- start start 2023-11-24T08:05:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f654a1db640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6544070850 0x7f6544147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f654a1db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544153ea0 0x7f6544159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f654a1db640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6544154280 0x7f654415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f654a1db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654415cd50 con 0x7f6544153ea0 2023-11-24T08:05:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f654a1db640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654415c770 con 0x7f6544070850 2023-11-24T08:05:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f654a1db640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6544147d90 con 0x7f6544154280 2023-11-24T08:05:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f65489d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544153ea0 0x7f6544159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f65489d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544153ea0 0x7f6544159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35350/0 (socket says 172.21.15.64:35350) 2023-11-24T08:05:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.314+0000 7f65489d8640 1 -- 172.21.15.64:0/2412315904 learned_addr learned my addr 172.21.15.64:0/2412315904 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f65489d8640 1 -- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6544154280 msgr2=0x7f654415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f65489d8640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6544154280 0x7f654415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f65489d8640 1 -- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6544070850 msgr2=0x7f6544147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f65489d8640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6544070850 0x7f6544147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f65489d8640 1 -- 172.21.15.64:0/2412315904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65340094d0 con 0x7f6544153ea0 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f65489d8640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544153ea0 0x7f6544159b30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f653402b950 tx=0x7f65340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f6539ffb640 1 -- 172.21.15.64:0/2412315904 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534031840 con 0x7f6544153ea0 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f6539ffb640 1 -- 172.21.15.64:0/2412315904 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65340443f0 con 0x7f6544153ea0 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f6539ffb640 1 -- 172.21.15.64:0/2412315904 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65340447a0 con 0x7f6544153ea0 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.315+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6544157550 con 0x7f6544153ea0 2023-11-24T08:05:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.316+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6544157830 con 0x7f6544153ea0 2023-11-24T08:05:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.317+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6544066e80 con 0x7f6544153ea0 2023-11-24T08:05:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.318+0000 7f6539ffb640 1 -- 172.21.15.64:0/2412315904 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6534044900 con 0x7f6544153ea0 2023-11-24T08:05:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.318+0000 7f6539ffb640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f652806af00 0x7f652806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.319+0000 7f6539ffb640 1 -- 172.21.15.64:0/2412315904 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f653402d040 con 0x7f6544153ea0 2023-11-24T08:05:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.319+0000 7f65491d9640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f652806af00 0x7f652806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.320+0000 7f65491d9640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f652806af00 0x7f652806d3c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.323+0000 7f6539ffb640 1 -- 172.21.15.64:0/2412315904 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6534076fd0 con 0x7f6544153ea0 2023-11-24T08:05:56.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.517+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f654415cf00 con 0x7f6544153ea0 2023-11-24T08:05:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.518+0000 7f6539ffb640 1 -- 172.21.15.64:0/2412315904 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f654415cf00 con 0x7f6544153ea0 2023-11-24T08:05:56.520 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:05:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f652806af00 msgr2=0x7f652806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f652806af00 0x7f652806d3c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540008040 comp rx=0 tx=0).stop 2023-11-24T08:05:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544153ea0 msgr2=0x7f6544159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544153ea0 0x7f6544159b30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f653402b950 tx=0x7f65340046c0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 shutdown_connections 2023-11-24T08:05:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f652806af00 0x7f652806d3c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6544154280 0x7f654415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544153ea0 0x7f6544159b30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 --2- 172.21.15.64:0/2412315904 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6544070850 0x7f6544147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 >> 172.21.15.64:0/2412315904 conn(0x7f65440cec10 msgr2=0x7f6544160180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 shutdown_connections 2023-11-24T08:05:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.521+0000 7f654a1db640 1 -- 172.21.15.64:0/2412315904 wait complete. 2023-11-24T08:05:56.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T08:05:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.766+0000 7f85294bb640 1 Processor -- start 2023-11-24T08:05:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.766+0000 7f85294bb640 1 -- start start 2023-11-24T08:05:56.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.768+0000 7f85294bb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f852415ed20 0x7f8524141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.768+0000 7f85294bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f852415f100 0x7f8524071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.768+0000 7f85294bb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852406cae0 con 0x7f8524071650 2023-11-24T08:05:56.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.768+0000 7f85294bb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8524071e40 con 0x7f852415ed20 2023-11-24T08:05:56.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.768+0000 7f85294bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852406c1b0 con 0x7f852415f100 2023-11-24T08:05:56.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.769+0000 7f8523fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f852415ed20 0x7f8524141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.769+0000 7f8523fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f852415ed20 0x7f8524141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35352/0 (socket says 172.21.15.64:35352) 2023-11-24T08:05:56.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.769+0000 7f8523fff640 1 -- 172.21.15.64:0/357014379 learned_addr learned my addr 172.21.15.64:0/357014379 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:56.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.769+0000 7f851bfff640 1 --2- 172.21.15.64:0/357014379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f852415f100 0x7f8524071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.769+0000 7f8523fff640 1 -- 172.21.15.64:0/357014379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f852415f100 msgr2=0x7f8524071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:56.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.769+0000 7f8523fff640 1 --2- 172.21.15.64:0/357014379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f852415f100 0x7f8524071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.769+0000 7f8523fff640 1 -- 172.21.15.64:0/357014379 >> v1:172.21.15.91:6789/0 conn(0x7f8524071650 legacy=0x7f852406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:56.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.769+0000 7f8523fff640 1 -- 172.21.15.64:0/357014379 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852406c490 con 0x7f852415ed20 2023-11-24T08:05:56.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.770+0000 7f8523fff640 1 --2- 172.21.15.64:0/357014379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415ed20 0x7f8524141e80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8514009870 tx=0x7f851402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd99e87fb079d6bd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:56.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.770+0000 7f85237fe640 1 -- 172.21.15.64:0/357014379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514031640 con 0x7f852415ed20 2023-11-24T08:05:56.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85237fe640 1 -- 172.21.15.64:0/357014379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85140317a0 con 0x7f852415ed20 2023-11-24T08:05:56.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85237fe640 1 -- 172.21.15.64:0/357014379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514031b30 con 0x7f852415ed20 2023-11-24T08:05:56.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85294bb640 1 -- 172.21.15.64:0/357014379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415ed20 msgr2=0x7f8524141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85294bb640 1 --2- 172.21.15.64:0/357014379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415ed20 0x7f8524141e80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8514009870 tx=0x7f851402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85294bb640 1 -- 172.21.15.64:0/357014379 shutdown_connections 2023-11-24T08:05:56.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85294bb640 1 --2- 172.21.15.64:0/357014379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f852415f100 0x7f8524071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85294bb640 1 --2- 172.21.15.64:0/357014379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415ed20 0x7f8524141e80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85294bb640 1 -- 172.21.15.64:0/357014379 >> 172.21.15.64:0/357014379 conn(0x7f85240b0840 msgr2=0x7f85240b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:56.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85294bb640 1 -- 172.21.15.64:0/357014379 shutdown_connections 2023-11-24T08:05:56.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.771+0000 7f85294bb640 1 -- 172.21.15.64:0/357014379 wait complete. 2023-11-24T08:05:56.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.773+0000 7f85294bb640 1 Processor -- start 2023-11-24T08:05:56.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.773+0000 7f85294bb640 1 -- start start 2023-11-24T08:05:56.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.774+0000 7f85294bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8524071650 0x7f852415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.774+0000 7f85294bb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f852415ed20 0x7f85241e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.774+0000 7f85294bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415f100 0x7f85241e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.774+0000 7f85294bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85241e97b0 con 0x7f852415f100 2023-11-24T08:05:56.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.774+0000 7f85294bb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8524133e00 con 0x7f852415ed20 2023-11-24T08:05:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.774+0000 7f85294bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85241e74d0 con 0x7f8524071650 2023-11-24T08:05:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415f100 0x7f85241e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415f100 0x7f85241e6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35360/0 (socket says 172.21.15.64:35360) 2023-11-24T08:05:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 -- 172.21.15.64:0/1897267282 learned_addr learned my addr 172.21.15.64:0/1897267282 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f851bfff640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f852415ed20 0x7f85241e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8523fff640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8524071650 0x7f852415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 -- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8524071650 msgr2=0x7f852415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8524071650 0x7f852415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 -- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f852415ed20 msgr2=0x7f85241e6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:56.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f852415ed20 0x7f85241e6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:56.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 -- 172.21.15.64:0/1897267282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140094d0 con 0x7f852415f100 2023-11-24T08:05:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.775+0000 7f8528cba640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415f100 0x7f85241e6f90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c780 tx=0x7f851c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.776+0000 7f85217fa640 1 -- 172.21.15.64:0/1897267282 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c004b60 con 0x7f852415f100 2023-11-24T08:05:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.776+0000 7f85217fa640 1 -- 172.21.15.64:0/1897267282 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00ce90 con 0x7f852415f100 2023-11-24T08:05:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.776+0000 7f85217fa640 1 -- 172.21.15.64:0/1897267282 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00a7f0 con 0x7f852415f100 2023-11-24T08:05:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.776+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85241e8e10 con 0x7f852415f100 2023-11-24T08:05:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.776+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85241e93d0 con 0x7f852415f100 2023-11-24T08:05:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.777+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8524067d10 con 0x7f852415f100 2023-11-24T08:05:56.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.778+0000 7f85217fa640 1 -- 172.21.15.64:0/1897267282 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f851c00a950 con 0x7f852415f100 2023-11-24T08:05:56.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.779+0000 7f85217fa640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f850406aed0 0x7f850406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:56.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.780+0000 7f85217fa640 1 -- 172.21.15.64:0/1897267282 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f851c089530 con 0x7f852415f100 2023-11-24T08:05:56.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.780+0000 7f8523fff640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f850406aed0 0x7f850406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:56.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.781+0000 7f8523fff640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f850406aed0 0x7f850406d390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8514036000 tx=0x7f851403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:56.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:56.784+0000 7f85217fa640 1 -- 172.21.15.64:0/1897267282 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f851c0547f0 con 0x7f852415f100 2023-11-24T08:05:57.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.046+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f852404e840 con 0x7f852415f100 2023-11-24T08:05:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.984+0000 7f85217fa640 1 -- 172.21.15.64:0/1897267282 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v131) v1 ==== 175+0+0 (secure 0 0 0) 0x7f851c057e10 con 0x7f852415f100 2023-11-24T08:05:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.987+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f850406aed0 msgr2=0x7f850406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.987+0000 7f85294bb640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f850406aed0 0x7f850406d390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8514036000 tx=0x7f851403c040 comp rx=0 tx=0).stop 2023-11-24T08:05:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.987+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415f100 msgr2=0x7f85241e6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.987+0000 7f85294bb640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415f100 0x7f85241e6f90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c780 tx=0x7f851c00cc40 comp rx=0 tx=0).stop 2023-11-24T08:05:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.988+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 shutdown_connections 2023-11-24T08:05:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.988+0000 7f85294bb640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f850406aed0 0x7f850406d390 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.988+0000 7f85294bb640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852415f100 0x7f85241e6f90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.988+0000 7f85294bb640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f852415ed20 0x7f85241e6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.988+0000 7f85294bb640 1 --2- 172.21.15.64:0/1897267282 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8524071650 0x7f852415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.988+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 >> 172.21.15.64:0/1897267282 conn(0x7f85240b0840 msgr2=0x7f85240b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.988+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 shutdown_connections 2023-11-24T08:05:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:57.988+0000 7f85294bb640 1 -- 172.21.15.64:0/1897267282 wait complete. 2023-11-24T08:05:58.011 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:05:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.240+0000 7f05f7fff640 1 Processor -- start 2023-11-24T08:05:58.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.240+0000 7f05f7fff640 1 -- start start 2023-11-24T08:05:58.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05f815d1c0 0x7f05f80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:58.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f7fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f80cd5f0 0x7f05f8070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:58.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f7fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f806b1f0 con 0x7f05f8070f80 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f7fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f8071430 con 0x7f05f815d1c0 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f7fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f815d5a0 con 0x7f05f80cd5f0 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05f815d1c0 0x7f05f80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f77fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f05f8070f80 0x7f05f806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37314/0 (socket says 172.21.15.64:37314) 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05f815d1c0 0x7f05f80cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52624/0 (socket says 172.21.15.64:52624) 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f77fe640 1 -- 172.21.15.64:0/4179918394 learned_addr learned my addr 172.21.15.64:0/4179918394 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f67fc640 1 --2- 172.21.15.64:0/4179918394 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f80cd5f0 0x7f05f8070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f6ffd640 1 -- 172.21.15.64:0/4179918394 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f80cd5f0 msgr2=0x7f05f8070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.242+0000 7f05f6ffd640 1 --2- 172.21.15.64:0/4179918394 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f80cd5f0 0x7f05f8070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.243+0000 7f05f6ffd640 1 -- 172.21.15.64:0/4179918394 >> v1:172.21.15.64:6789/0 conn(0x7f05f8070f80 legacy=0x7f05f806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.243+0000 7f05f6ffd640 1 -- 172.21.15.64:0/4179918394 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f806a5e0 con 0x7f05f815d1c0 2023-11-24T08:05:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.244+0000 7f05f6ffd640 1 --2- 172.21.15.64:0/4179918394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f815d1c0 0x7f05f80cd0b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f05e00097d0 tx=0x7f05e002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272af3394ce124f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.244+0000 7f05f5ffb640 1 -- 172.21.15.64:0/4179918394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e002d040 con 0x7f05f815d1c0 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.244+0000 7f05f5ffb640 1 -- 172.21.15.64:0/4179918394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e0031810 con 0x7f05f815d1c0 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.244+0000 7f05f5ffb640 1 -- 172.21.15.64:0/4179918394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0031ba0 con 0x7f05f815d1c0 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.244+0000 7f05f7fff640 1 -- 172.21.15.64:0/4179918394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f815d1c0 msgr2=0x7f05f80cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.244+0000 7f05f7fff640 1 --2- 172.21.15.64:0/4179918394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f815d1c0 0x7f05f80cd0b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f05e00097d0 tx=0x7f05e002b8b0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.245+0000 7f05f7fff640 1 -- 172.21.15.64:0/4179918394 shutdown_connections 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.245+0000 7f05f7fff640 1 --2- 172.21.15.64:0/4179918394 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f80cd5f0 0x7f05f8070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.245+0000 7f05f7fff640 1 --2- 172.21.15.64:0/4179918394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f815d1c0 0x7f05f80cd0b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.245+0000 7f05f7fff640 1 -- 172.21.15.64:0/4179918394 >> 172.21.15.64:0/4179918394 conn(0x7f05f80cf4e0 msgr2=0x7f05f80cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.245+0000 7f05f7fff640 1 -- 172.21.15.64:0/4179918394 shutdown_connections 2023-11-24T08:05:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.245+0000 7f05f7fff640 1 -- 172.21.15.64:0/4179918394 wait complete. 2023-11-24T08:05:58.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.246+0000 7f05f7fff640 1 Processor -- start 2023-11-24T08:05:58.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.247+0000 7f05f7fff640 1 -- start start 2023-11-24T08:05:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f7fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f8070f80 0x7f05f8151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f80cd5f0 0x7f05f8152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f7fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f05f815d1c0 0x7f05f8142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f7fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8152c90 con 0x7f05f80cd5f0 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f7fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8152f70 con 0x7f05f815d1c0 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f7fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8153250 con 0x7f05f8070f80 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f80cd5f0 0x7f05f8152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f80cd5f0 0x7f05f8152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52626/0 (socket says 172.21.15.64:52626) 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f67fc640 1 -- 172.21.15.64:0/3918931222 learned_addr learned my addr 172.21.15.64:0/3918931222 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f67fc640 1 -- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f8070f80 msgr2=0x7f05f8151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f67fc640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f8070f80 0x7f05f8151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f67fc640 1 -- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f05f815d1c0 msgr2=0x7f05f8142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f67fc640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f05f815d1c0 0x7f05f8142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.248+0000 7f05f67fc640 1 -- 172.21.15.64:0/3918931222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0009400 con 0x7f05f80cd5f0 2023-11-24T08:05:58.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.249+0000 7f05f67fc640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f80cd5f0 0x7f05f8152240 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f05e800ce10 tx=0x7f05e800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:58.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.249+0000 7f05ef7fe640 1 -- 172.21.15.64:0/3918931222 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8003b90 con 0x7f05f80cd5f0 2023-11-24T08:05:58.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.249+0000 7f05ef7fe640 1 -- 172.21.15.64:0/3918931222 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8003cf0 con 0x7f05f80cd5f0 2023-11-24T08:05:58.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.249+0000 7f05ef7fe640 1 -- 172.21.15.64:0/3918931222 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8003ea0 con 0x7f05f80cd5f0 2023-11-24T08:05:58.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.249+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f8153530 con 0x7f05f80cd5f0 2023-11-24T08:05:58.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.249+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f8151bc0 con 0x7f05f80cd5f0 2023-11-24T08:05:58.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.250+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f804e840 con 0x7f05f80cd5f0 2023-11-24T08:05:58.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.251+0000 7f05ef7fe640 1 -- 172.21.15.64:0/3918931222 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f05e8004240 con 0x7f05f80cd5f0 2023-11-24T08:05:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.252+0000 7f05ef7fe640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f05d806af00 0x7f05d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.252+0000 7f05ef7fe640 1 -- 172.21.15.64:0/3918931222 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f05e8012070 con 0x7f05f80cd5f0 2023-11-24T08:05:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.253+0000 7f05f6ffd640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f05d806af00 0x7f05d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.254+0000 7f05f6ffd640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f05d806af00 0x7f05d806d3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05e0036000 tx=0x7f05e0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.256+0000 7f05ef7fe640 1 -- 172.21.15.64:0/3918931222 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05e80550f0 con 0x7f05f80cd5f0 2023-11-24T08:05:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.449+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f05f80af460 con 0x7f05f80cd5f0 2023-11-24T08:05:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.456+0000 7f05ef7fe640 1 -- 172.21.15.64:0/3918931222 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f05e8058710 con 0x7f05f80cd5f0 2023-11-24T08:05:58.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f05d806af00 msgr2=0x7f05d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:58.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f05d806af00 0x7f05d806d3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05e0036000 tx=0x7f05e0030000 comp rx=0 tx=0).stop 2023-11-24T08:05:58.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f80cd5f0 msgr2=0x7f05f8152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:58.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f80cd5f0 0x7f05f8152240 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f05e800ce10 tx=0x7f05e800a660 comp rx=0 tx=0).stop 2023-11-24T08:05:58.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 shutdown_connections 2023-11-24T08:05:58.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f05d806af00 0x7f05d806d3c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f05f815d1c0 0x7f05f8142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f80cd5f0 0x7f05f8152240 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 --2- 172.21.15.64:0/3918931222 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f05f8070f80 0x7f05f8151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 >> 172.21.15.64:0/3918931222 conn(0x7f05f80cf4e0 msgr2=0x7f05f815b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 shutdown_connections 2023-11-24T08:05:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:58.459+0000 7f05f7fff640 1 -- 172.21.15.64:0/3918931222 wait complete. 2023-11-24T08:05:58.480 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:05:58.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:05:58.578 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:05:58.703 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:05:58.704 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:05:58.704 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:05:58.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:05:58.767 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:05:58.767 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:05:58.827 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:05:58.827 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:05:58.827 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:05:58.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:05:58.890 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:05:58.890 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:05:58.950 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:05:58.950 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:05:58.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:05:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.183+0000 7fe2a2135640 1 Processor -- start 2023-11-24T08:05:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.183+0000 7fe2a2135640 1 -- start start 2023-11-24T08:05:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.185+0000 7fe2a2135640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 0x7fe29c0d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.185+0000 7fe2a2135640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c0cf0c0 con 0x7fe29c0d14a0 2023-11-24T08:05:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.185+0000 7fe2a2135640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c0b01b0 con 0x7fe29c0b1650 2023-11-24T08:05:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.185+0000 7fe2a2135640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0d05c0 con 0x7fe29c15d5b0 2023-11-24T08:05:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.185+0000 7fe2a0932640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe29c0d14a0 0x7fe29c0cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37326/0 (socket says 172.21.15.64:37326) 2023-11-24T08:05:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.185+0000 7fe2a0932640 1 -- 172.21.15.64:0/2089409728 learned_addr learned my addr 172.21.15.64:0/2089409728 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.185+0000 7fe2a1133640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 0x7fe29c0d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428555316 0 0) 0x7fe29c0cf0c0 con 0x7fe29c0d14a0 2023-11-24T08:05:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe288003520 con 0x7fe29c0d14a0 2023-11-24T08:05:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 746025447 0 0) 0x7fe288003520 con 0x7fe29c0d14a0 2023-11-24T08:05:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 msgr2=0x7fe29c0d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 0x7fe29c0d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 >> v1:172.21.15.91:6789/0 conn(0x7fe29c0b1650 legacy=0x7fe29c0cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c15d990 con 0x7fe29c0d14a0 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fe28c001670 con 0x7fe29c0d14a0 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 >> v1:172.21.15.64:6789/0 conn(0x7fe29c0d14a0 legacy=0x7fe29c0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe288003a10 0x7fe288003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe288004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 0x7fe288005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0d05c0 con 0x7fe2880047c0 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28800ba80 con 0x7fe288003a10 2023-11-24T08:05:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28800bc40 con 0x7fe29c15d5b0 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.186+0000 7fe2a0932640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe288004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a1133640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe288003a10 0x7fe288003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a1934640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 0x7fe288005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a0932640 1 -- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 msgr2=0x7fe288005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a0932640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 0x7fe288005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a0932640 1 -- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe288003a10 msgr2=0x7fe288003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a0932640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe288003a10 0x7fe288003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a0932640 1 -- 172.21.15.64:0/2089409728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288003520 con 0x7fe2880047c0 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a1934640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c15d5b0 0x7fe288005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:59.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.187+0000 7fe2a1133640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe288003a10 0x7fe288003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.188+0000 7fe2a0932640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe288004c40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c240 tx=0x7fe2880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc85a68d62e06290 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.188+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c005ad0 con 0x7fe2880047c0 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.188+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c005c30 con 0x7fe2880047c0 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.188+0000 7fe293fff640 1 -- 172.21.15.64:0/2089409728 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c02c8b0 con 0x7fe2880047c0 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.189+0000 7fe2a2135640 1 -- 172.21.15.64:0/2089409728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 msgr2=0x7fe288004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.189+0000 7fe2a2135640 1 --2- 172.21.15.64:0/2089409728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe288004c40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c240 tx=0x7fe2880057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.189+0000 7fe2a1934640 1 -- 172.21.15.64:0/2089409728 reap_dead start 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.189+0000 7fe2a2135640 1 -- 172.21.15.64:0/2089409728 shutdown_connections 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.189+0000 7fe2a2135640 1 -- 172.21.15.64:0/2089409728 >> 172.21.15.64:0/2089409728 conn(0x7fe29c09c3b0 msgr2=0x7fe29c0d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.189+0000 7fe2a2135640 1 -- 172.21.15.64:0/2089409728 shutdown_connections 2023-11-24T08:05:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.189+0000 7fe2a2135640 1 -- 172.21.15.64:0/2089409728 wait complete. 2023-11-24T08:05:59.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.191+0000 7fe2a2135640 1 Processor -- start 2023-11-24T08:05:59.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.191+0000 7fe2a2135640 1 -- start start 2023-11-24T08:05:59.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a2135640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe29c0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a2135640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe29c15a680 0x7fe29c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a2135640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c0b7580 0x7fe29c0b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a2135640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0b70f0 con 0x7fe2880047c0 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a2135640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0b64e0 con 0x7fe29c15a680 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a2135640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0b67c0 con 0x7fe29c0b7580 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a1133640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe29c0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a1133640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe29c0b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52638/0 (socket says 172.21.15.64:52638) 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.192+0000 7fe2a1133640 1 -- 172.21.15.64:0/2982418653 learned_addr learned my addr 172.21.15.64:0/2982418653 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a1934640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c0b7580 0x7fe29c0b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a0932640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe29c15a680 0x7fe29c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a1133640 1 -- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c0b7580 msgr2=0x7fe29c0b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a1133640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c0b7580 0x7fe29c0b7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a1133640 1 -- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe29c15a680 msgr2=0x7fe29c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a1133640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe29c15a680 0x7fe29c0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a1133640 1 -- 172.21.15.64:0/2982418653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c0b6aa0 con 0x7fe2880047c0 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a1934640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c0b7580 0x7fe29c0b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a0932640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe29c15a680 0x7fe29c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a1133640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe29c0b5620 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe2980108b0 tx=0x7fe298010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe291ffb640 1 -- 172.21.15.64:0/2982418653 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29800eb60 con 0x7fe2880047c0 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe291ffb640 1 -- 172.21.15.64:0/2982418653 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29800ecc0 con 0x7fe2880047c0 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe291ffb640 1 -- 172.21.15.64:0/2982418653 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29801c400 con 0x7fe2880047c0 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c0b6d80 con 0x7fe2880047c0 2023-11-24T08:05:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.193+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c15ab00 con 0x7fe2880047c0 2023-11-24T08:05:59.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.195+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c04e840 con 0x7fe2880047c0 2023-11-24T08:05:59.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.195+0000 7fe291ffb640 1 -- 172.21.15.64:0/2982418653 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe29801c560 con 0x7fe2880047c0 2023-11-24T08:05:59.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.197+0000 7fe291ffb640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe28006aed0 0x7fe28006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.197+0000 7fe291ffb640 1 -- 172.21.15.64:0/2982418653 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe2980899f0 con 0x7fe2880047c0 2023-11-24T08:05:59.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.197+0000 7fe2a0932640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe28006aed0 0x7fe28006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.199+0000 7fe2a0932640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe28006aed0 0x7fe28006d390 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe28c034000 tx=0x7fe28c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:59.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.201+0000 7fe291ffb640 1 -- 172.21.15.64:0/2982418653 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe298054df0 con 0x7fe2880047c0 2023-11-24T08:05:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.432+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe29c0678d0 con 0x7fe2880047c0 2023-11-24T08:05:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.433+0000 7fe291ffb640 1 -- 172.21.15.64:0/2982418653 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fe298021e90 con 0x7fe2880047c0 2023-11-24T08:05:59.435 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:05:59.435 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1893080737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":1893080737},{"type":"v1","addr":"172.21.15.91:6837","nonce":1893080737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2845212044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2845212044},{"type":"v1","addr":"172.21.15.91:6835","nonce":2845212044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2382740551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2382740551},{"type":"v1","addr":"172.21.15.64:6833","nonce":2382740551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1213960755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1213960755},{"type":"v1","addr":"172.21.15.64:6835","nonce":1213960755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2023-11-24T08:05:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 18 2023-11-24T08:05:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe28006aed0 msgr2=0x7fe28006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe28006aed0 0x7fe28006d390 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe28c034000 tx=0x7fe28c009030 comp rx=0 tx=0).stop 2023-11-24T08:05:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 msgr2=0x7fe29c0b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe29c0b5620 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe2980108b0 tx=0x7fe298010d70 comp rx=0 tx=0).stop 2023-11-24T08:05:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 shutdown_connections 2023-11-24T08:05:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe28006aed0 0x7fe28006d390 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe29c0b7580 0x7fe29c0b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe29c15a680 0x7fe29c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 --2- 172.21.15.64:0/2982418653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2880047c0 0x7fe29c0b5620 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 >> 172.21.15.64:0/2982418653 conn(0x7fe29c09c3b0 msgr2=0x7fe29c0d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 shutdown_connections 2023-11-24T08:05:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.436+0000 7fe2a2135640 1 -- 172.21.15.64:0/2982418653 wait complete. 2023-11-24T08:05:59.457 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:05:59.457 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:05:59.458 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:05:59.458 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:59.458 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:05:59.458 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:05:59.458 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:05:59.458 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:59.458 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:05:59.459 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:05:59.459 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:05:59.459 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:59.459 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:05:59.459 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:05:59.459 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:05:59.459 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:05:59.473 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:05:59.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:05:59.478 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:05:59.478 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:05:59.482 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:05:59.482 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:05:59.489 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:05:59.489 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:05:59.526 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:05:59.526 DEBUG:teuthology.parallel:result is None 2023-11-24T08:05:59.533 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:05:59.534 DEBUG:teuthology.parallel:result is None 2023-11-24T08:05:59.536 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:05:59.537 DEBUG:teuthology.parallel:result is None 2023-11-24T08:05:59.544 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:05:59.544 DEBUG:teuthology.parallel:result is None 2023-11-24T08:05:59.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:05:59.694 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:05:59.695 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T08:05:59.705 INFO:tasks.ceph.mds.d.smithi091.stdout:starting mds.d at 2023-11-24T08:05:59.706 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2023-11-24T08:05:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.740+0000 7faf3718a640 1 Processor -- start 2023-11-24T08:05:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.740+0000 7faf3718a640 1 -- start start 2023-11-24T08:05:59.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.741+0000 7faf3718a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7faf300b6e70 0x7faf300d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.741+0000 7faf3718a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf300d0b40 con 0x7faf3015d770 2023-11-24T08:05:59.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.741+0000 7faf3718a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3015db50 con 0x7faf300d3270 2023-11-24T08:05:59.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.741+0000 7faf3718a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf300d3c30 con 0x7faf300b6e70 2023-11-24T08:05:59.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.742+0000 7faf34f26640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7faf3015d770 0x7faf300d2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37330/0 (socket says 172.21.15.64:37330) 2023-11-24T08:05:59.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.742+0000 7faf34f26640 1 -- 172.21.15.64:0/2620480050 learned_addr learned my addr 172.21.15.64:0/2620480050 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:05:59.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.742+0000 7faf35727640 1 --2- 172.21.15.64:0/2620480050 >> v2:172.21.15.91:3300/0 conn(0x7faf300b6e70 0x7faf300d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.742+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172277680 0 0) 0x7faf300d0b40 con 0x7faf3015d770 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.742+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf20003520 con 0x7faf3015d770 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3657146137 0 0) 0x7faf20003520 con 0x7faf3015d770 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 >> v2:172.21.15.91:3300/0 conn(0x7faf300b6e70 msgr2=0x7faf300d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2620480050 >> v2:172.21.15.91:3300/0 conn(0x7faf300b6e70 0x7faf300d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 >> v1:172.21.15.91:6789/0 conn(0x7faf300d3270 legacy=0x7faf300b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf300d2910 con 0x7faf3015d770 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7faf24001670 con 0x7faf3015d770 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 >> v1:172.21.15.64:6789/0 conn(0x7faf3015d770 legacy=0x7faf300d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf20003970 0x7faf20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf200047c0 0x7faf20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20005180 0x7faf2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf300d3c30 con 0x7faf20005180 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2000c020 con 0x7faf200047c0 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2000c1e0 con 0x7faf20003970 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf34f26640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf20003970 0x7faf20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf35727640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20005180 0x7faf2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.743+0000 7faf2ffff640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf200047c0 0x7faf20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.744+0000 7faf2ffff640 1 -- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf20003970 msgr2=0x7faf20003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.744+0000 7faf2ffff640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf20003970 0x7faf20003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.744+0000 7faf2ffff640 1 -- 172.21.15.64:0/2620480050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20005180 msgr2=0x7faf2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.744+0000 7faf2ffff640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20005180 0x7faf2000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.744+0000 7faf2ffff640 1 -- 172.21.15.64:0/2620480050 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20003520 con 0x7faf200047c0 2023-11-24T08:05:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.744+0000 7faf35727640 1 -- 172.21.15.64:0/2620480050 reap_dead start 2023-11-24T08:05:59.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.745+0000 7faf2ffff640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf200047c0 0x7faf20004c40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf1c001720 tx=0x7faf200057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=948df9f197f3383b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.745+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c0052d0 con 0x7faf200047c0 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.745+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c02e050 con 0x7faf200047c0 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.745+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2620480050 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c003830 con 0x7faf200047c0 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.746+0000 7faf3718a640 1 -- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf200047c0 msgr2=0x7faf20004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.746+0000 7faf3718a640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf200047c0 0x7faf20004c40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf1c001720 tx=0x7faf200057f0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.746+0000 7faf3718a640 1 -- 172.21.15.64:0/2620480050 shutdown_connections 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.746+0000 7faf3718a640 1 --2- 172.21.15.64:0/2620480050 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf200047c0 0x7faf20004c40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.746+0000 7faf3718a640 1 -- 172.21.15.64:0/2620480050 >> 172.21.15.64:0/2620480050 conn(0x7faf300721a0 msgr2=0x7faf3006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.746+0000 7faf3718a640 1 -- 172.21.15.64:0/2620480050 shutdown_connections 2023-11-24T08:05:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.746+0000 7faf3718a640 1 -- 172.21.15.64:0/2620480050 wait complete. 2023-11-24T08:05:59.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.748+0000 7faf3718a640 1 Processor -- start 2023-11-24T08:05:59.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.748+0000 7faf3718a640 1 -- start start 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf3718a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf200047c0 0x7faf3015c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf3718a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf3015dce0 0x7faf3015e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf3718a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf300c79c0 0x7faf300c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf3718a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3015e980 con 0x7faf300c79c0 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf3718a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3015ec60 con 0x7faf3015dce0 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf3718a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3015ef40 con 0x7faf200047c0 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf35727640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf300c79c0 0x7faf300c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf35727640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf300c79c0 0x7faf300c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52696/0 (socket says 172.21.15.64:52696) 2023-11-24T08:05:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.749+0000 7faf35727640 1 -- 172.21.15.64:0/2521678893 learned_addr learned my addr 172.21.15.64:0/2521678893 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:05:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf2ffff640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf3015dce0 0x7faf3015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf34f26640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf200047c0 0x7faf3015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf35727640 1 -- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf200047c0 msgr2=0x7faf3015c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf35727640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf200047c0 0x7faf3015c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf35727640 1 -- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf3015dce0 msgr2=0x7faf3015e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf35727640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf3015dce0 0x7faf3015e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf35727640 1 -- 172.21.15.64:0/2521678893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3015f220 con 0x7faf300c79c0 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf35727640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf300c79c0 0x7faf300c7e40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faf300cfa00 tx=0x7faf28011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2521678893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800e040 con 0x7faf300c79c0 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2521678893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2800f040 con 0x7faf300c79c0 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2521678893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800faa0 con 0x7faf300c79c0 2023-11-24T08:05:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.750+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf30154260 con 0x7faf300c79c0 2023-11-24T08:05:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.751+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3015cad0 con 0x7faf300c79c0 2023-11-24T08:05:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.752+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf301543f0 con 0x7faf300c79c0 2023-11-24T08:05:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.753+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2521678893 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7faf2800fc00 con 0x7faf300c79c0 2023-11-24T08:05:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.754+0000 7faf2d7fa640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faf1406aed0 0x7faf1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:05:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.754+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2521678893 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faf28089cb0 con 0x7faf300c79c0 2023-11-24T08:05:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.755+0000 7faf34f26640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faf1406aed0 0x7faf1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:05:59.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.755+0000 7faf34f26640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faf1406aed0 0x7faf1406d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf30068d90 tx=0x7faf24009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:05:59.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.758+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2521678893 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf28054ff0 con 0x7faf300c79c0 2023-11-24T08:05:59.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:05:59.965+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faf30074770 con 0x7faf300c79c0 2023-11-24T08:06:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.022+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2521678893 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7faf28003e60 con 0x7faf300c79c0 2023-11-24T08:06:00.024 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.025+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faf1406aed0 msgr2=0x7faf1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.025+0000 7faf3718a640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faf1406aed0 0x7faf1406d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf30068d90 tx=0x7faf24009040 comp rx=0 tx=0).stop 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.025+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf300c79c0 msgr2=0x7faf300c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.025+0000 7faf3718a640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf300c79c0 0x7faf300c7e40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faf300cfa00 tx=0x7faf28011f20 comp rx=0 tx=0).stop 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.026+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 shutdown_connections 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.026+0000 7faf3718a640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7faf1406aed0 0x7faf1406d390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.026+0000 7faf3718a640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf300c79c0 0x7faf300c7e40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.026+0000 7faf3718a640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7faf3015dce0 0x7faf3015e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.026+0000 7faf3718a640 1 --2- 172.21.15.64:0/2521678893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7faf200047c0 0x7faf3015c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.026+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 >> 172.21.15.64:0/2521678893 conn(0x7faf300721a0 msgr2=0x7faf3015a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:00.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.026+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 shutdown_connections 2023-11-24T08:06:00.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.026+0000 7faf3718a640 1 -- 172.21.15.64:0/2521678893 wait complete. 2023-11-24T08:06:00.050 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:06:00.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.261+0000 7fb02359e640 1 Processor -- start 2023-11-24T08:06:00.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.261+0000 7fb02359e640 1 -- start start 2023-11-24T08:06:00.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb02359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb024161630 0x7fb024070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb02359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 0x7fb02406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb02359e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb024071bc0 con 0x7fb02415d540 2023-11-24T08:06:00.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb02359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb024161d80 con 0x7fb024161630 2023-11-24T08:06:00.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb02359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0240cb150 con 0x7fb0240cad30 2023-11-24T08:06:00.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb01bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb024161630 0x7fb024070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb02259c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb02415d540 0x7fb024160f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37338/0 (socket says 172.21.15.64:37338) 2023-11-24T08:06:00.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb01bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb024161630 0x7fb024070fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52710/0 (socket says 172.21.15.64:52710) 2023-11-24T08:06:00.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.263+0000 7fb02259c640 1 -- 172.21.15.64:0/3886137011 learned_addr learned my addr 172.21.15.64:0/3886137011 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:00.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.264+0000 7fb022d9d640 1 --2- 172.21.15.64:0/3886137011 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 0x7fb02406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.264+0000 7fb01bfff640 1 -- 172.21.15.64:0/3886137011 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 msgr2=0x7fb02406cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:00.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.264+0000 7fb01bfff640 1 --2- 172.21.15.64:0/3886137011 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 0x7fb02406cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.264+0000 7fb01bfff640 1 -- 172.21.15.64:0/3886137011 >> v1:172.21.15.64:6789/0 conn(0x7fb02415d540 legacy=0x7fb024160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.264+0000 7fb01bfff640 1 -- 172.21.15.64:0/3886137011 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024071600 con 0x7fb024161630 2023-11-24T08:06:00.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.265+0000 7fb01bfff640 1 --2- 172.21.15.64:0/3886137011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024161630 0x7fb024070fd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0097a0 tx=0x7fb00c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4829a78d2cc8e660 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:00.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.265+0000 7fb021d9b640 1 -- 172.21.15.64:0/3886137011 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0316f0 con 0x7fb024161630 2023-11-24T08:06:00.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb021d9b640 1 -- 172.21.15.64:0/3886137011 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c031850 con 0x7fb024161630 2023-11-24T08:06:00.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb021d9b640 1 -- 172.21.15.64:0/3886137011 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c031be0 con 0x7fb024161630 2023-11-24T08:06:00.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb02359e640 1 -- 172.21.15.64:0/3886137011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024161630 msgr2=0x7fb024070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb02359e640 1 --2- 172.21.15.64:0/3886137011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024161630 0x7fb024070fd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0097a0 tx=0x7fb00c02b790 comp rx=0 tx=0).stop 2023-11-24T08:06:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb02359e640 1 -- 172.21.15.64:0/3886137011 shutdown_connections 2023-11-24T08:06:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb02359e640 1 --2- 172.21.15.64:0/3886137011 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 0x7fb02406cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb02359e640 1 --2- 172.21.15.64:0/3886137011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024161630 0x7fb024070fd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb02359e640 1 -- 172.21.15.64:0/3886137011 >> 172.21.15.64:0/3886137011 conn(0x7fb0240b0840 msgr2=0x7fb0240b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb02359e640 1 -- 172.21.15.64:0/3886137011 shutdown_connections 2023-11-24T08:06:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.266+0000 7fb02359e640 1 -- 172.21.15.64:0/3886137011 wait complete. 2023-11-24T08:06:00.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.268+0000 7fb02359e640 1 Processor -- start 2023-11-24T08:06:00.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.268+0000 7fb02359e640 1 -- start start 2023-11-24T08:06:00.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.269+0000 7fb02359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 0x7fb02415c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.269+0000 7fb02359e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb024161630 0x7fb02413f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.269+0000 7fb02359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024149780 0x7fb0240af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.269+0000 7fb02359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02413faf0 con 0x7fb024149780 2023-11-24T08:06:00.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.269+0000 7fb02359e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02414d6a0 con 0x7fb024161630 2023-11-24T08:06:00.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.269+0000 7fb02359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02414d8f0 con 0x7fb0240cad30 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024149780 0x7fb0240af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024149780 0x7fb0240af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52722/0 (socket says 172.21.15.64:52722) 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 -- 172.21.15.64:0/479821983 learned_addr learned my addr 172.21.15.64:0/479821983 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb02259c640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 0x7fb02415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 -- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 msgr2=0x7fb02415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 0x7fb02415c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 -- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb024161630 msgr2=0x7fb02413f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb024161630 0x7fb02413f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 -- 172.21.15.64:0/479821983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c009400 con 0x7fb024149780 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.270+0000 7fb022d9d640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024149780 0x7fb0240af900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb01c007e40 tx=0x7fb01c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:00.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.271+0000 7fb01affd640 1 -- 172.21.15.64:0/479821983 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c014620 con 0x7fb024149780 2023-11-24T08:06:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.271+0000 7fb01affd640 1 -- 172.21.15.64:0/479821983 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c00ee90 con 0x7fb024149780 2023-11-24T08:06:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.271+0000 7fb01affd640 1 -- 172.21.15.64:0/479821983 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0149c0 con 0x7fb024149780 2023-11-24T08:06:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.271+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02414dbd0 con 0x7fb024149780 2023-11-24T08:06:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.271+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02414e190 con 0x7fb024149780 2023-11-24T08:06:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.272+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02404e840 con 0x7fb024149780 2023-11-24T08:06:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.273+0000 7fb01affd640 1 -- 172.21.15.64:0/479821983 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb01c014b20 con 0x7fb024149780 2023-11-24T08:06:00.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.274+0000 7fb01affd640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00406af00 0x7fb00406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.274+0000 7fb01affd640 1 -- 172.21.15.64:0/479821983 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb01c089db0 con 0x7fb024149780 2023-11-24T08:06:00.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.275+0000 7fb02259c640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00406af00 0x7fb00406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.276+0000 7fb02259c640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00406af00 0x7fb00406d3c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb014005880 tx=0x7fb014009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:00.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.278+0000 7fb01affd640 1 -- 172.21.15.64:0/479821983 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb01c08e070 con 0x7fb024149780 2023-11-24T08:06:00.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.535+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb02406c210 con 0x7fb024149780 2023-11-24T08:06:00.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.536+0000 7fb01affd640 1 -- 172.21.15.64:0/479821983 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb01c0550c0 con 0x7fb024149780 2023-11-24T08:06:00.538 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:00.538 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:06:00.538 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-11-24T08:06:00.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.538 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-11-24T08:06:00.538 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllV/IJxAAMEWwBM8H9uSLw5Bd93bScA==", 2023-11-24T08:06:00.538 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBl35hoEBAADvW80b1cRhhgBRkYgNaftg==", 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.539 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBlZ1rGORAAjjOrPcL663guE4Q9MKA+6g==", 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.540 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlqV+1IRAAaLczF6+eDIuMM6X5Jgni/A==", 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:00.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCuWGBlsrGpIBAAVTJlhEtf7rPd1J4wUZRWTg==", 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.542 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCxWGBl+u/pLBAAwh85K8evI69aY9Nij5dBRQ==", 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC1WGBl1kGECxAAtoQ9UI319UpAU7OwZehzew==", 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-11-24T08:06:00.544 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC4WGBlSlE2JRAAjPT1020nD4Dnrw1hrB2bKQ==", 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC/WGBlrRSGKRAAaiWZZvvd7O8PuipXF1oAUQ==", 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.545 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDDWGBls+mkBxAAJZXdXFq1+8QC5dBE14+YJA==", 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:00.546 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDGWGBl71+WIBAA/NNCMwpi0ycNJvXZFHKuIA==", 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:00.547 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJWGBl11BPKxAAnXhK4EqlIyvbHCpHzPCDUg==", 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA==", 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.549 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlsyr3DBAASDVvIWUkg06i7YARod5spw==", 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-11-24T08:06:00.550 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlhsnfABAA5sZEDm1Uyin3dIOEoMX+gg==", 2023-11-24T08:06:00.551 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-11-24T08:06:00.552 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlT9iOFxAAiupeG8QnYzYA8pJYftfxNA==", 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCnWGBlKY7iFBAAmkZ+pYWFmZ7aPH06BiE0+w==", 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.553 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllqymGhAA+wKOSJeuBn60R4gZJ3edoA==", 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:06:00.554 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBloKAEDhAAFSBzeCcgrTtmsfXrxKqCRQ==", 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:06:00.555 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:00.556 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.556 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:00.556 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T08:06:00.556 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:06:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.538+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00406af00 msgr2=0x7fb00406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.538+0000 7fb02359e640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00406af00 0x7fb00406d3c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb014005880 tx=0x7fb014009040 comp rx=0 tx=0).stop 2023-11-24T08:06:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.538+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024149780 msgr2=0x7fb0240af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.538+0000 7fb02359e640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024149780 0x7fb0240af900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb01c007e40 tx=0x7fb01c00e4b0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.539+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 shutdown_connections 2023-11-24T08:06:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.539+0000 7fb02359e640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb00406af00 0x7fb00406d3c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.539+0000 7fb02359e640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb024149780 0x7fb0240af900 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.539+0000 7fb02359e640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb024161630 0x7fb02413f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.539+0000 7fb02359e640 1 --2- 172.21.15.64:0/479821983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb0240cad30 0x7fb02415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.539+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 >> 172.21.15.64:0/479821983 conn(0x7fb0240b0840 msgr2=0x7fb0240acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.539+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 shutdown_connections 2023-11-24T08:06:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.539+0000 7fb02359e640 1 -- 172.21.15.64:0/479821983 wait complete. 2023-11-24T08:06:00.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:00.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.776+0000 7ff36e46d640 1 Processor -- start 2023-11-24T08:06:00.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.776+0000 7ff36e46d640 1 -- start start 2023-11-24T08:06:00.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.777+0000 7ff36e46d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff36815d5b0 0x7ff3680d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.777+0000 7ff36e46d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3680cf0c0 con 0x7ff3680d14a0 2023-11-24T08:06:00.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.778+0000 7ff36e46d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3680b01b0 con 0x7ff3680b1650 2023-11-24T08:06:00.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.778+0000 7ff36e46d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3680d05c0 con 0x7ff36815d5b0 2023-11-24T08:06:00.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.778+0000 7ff36d46b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff36815d5b0 0x7ff3680d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.778+0000 7ff36cc6a640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff3680d14a0 0x7ff3680cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37340/0 (socket says 172.21.15.64:37340) 2023-11-24T08:06:00.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.778+0000 7ff36cc6a640 1 -- 172.21.15.64:0/3575012942 learned_addr learned my addr 172.21.15.64:0/3575012942 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:00.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.778+0000 7ff35bfff640 1 -- 172.21.15.64:0/3575012942 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224917580 0 0) 0x7ff3680cf0c0 con 0x7ff3680d14a0 2023-11-24T08:06:00.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.778+0000 7ff36d46b640 1 -- 172.21.15.64:0/3575012942 >> v1:172.21.15.91:6789/0 conn(0x7ff3680b1650 legacy=0x7ff3680cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.778+0000 7ff36d46b640 1 -- 172.21.15.64:0/3575012942 >> v1:172.21.15.64:6789/0 conn(0x7ff3680d14a0 legacy=0x7ff3680cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.779+0000 7ff36d46b640 1 -- 172.21.15.64:0/3575012942 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36815d990 con 0x7ff36815d5b0 2023-11-24T08:06:00.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.780+0000 7ff36d46b640 1 --2- 172.21.15.64:0/3575012942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff36815d5b0 0x7ff3680d3990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff3640097a0 tx=0x7ff36402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eea888d53b2dd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:00.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.780+0000 7ff35bfff640 1 -- 172.21.15.64:0/3575012942 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364030610 con 0x7ff36815d5b0 2023-11-24T08:06:00.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.780+0000 7ff35bfff640 1 -- 172.21.15.64:0/3575012942 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364030770 con 0x7ff36815d5b0 2023-11-24T08:06:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.780+0000 7ff35bfff640 1 -- 172.21.15.64:0/3575012942 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364030b00 con 0x7ff36815d5b0 2023-11-24T08:06:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.780+0000 7ff36e46d640 1 -- 172.21.15.64:0/3575012942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff36815d5b0 msgr2=0x7ff3680d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.780+0000 7ff36e46d640 1 --2- 172.21.15.64:0/3575012942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff36815d5b0 0x7ff3680d3990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff3640097a0 tx=0x7ff36402b960 comp rx=0 tx=0).stop 2023-11-24T08:06:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.781+0000 7ff36e46d640 1 -- 172.21.15.64:0/3575012942 shutdown_connections 2023-11-24T08:06:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.781+0000 7ff36e46d640 1 --2- 172.21.15.64:0/3575012942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff36815d5b0 0x7ff3680d3990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.781+0000 7ff36e46d640 1 -- 172.21.15.64:0/3575012942 >> 172.21.15.64:0/3575012942 conn(0x7ff36809c3b0 msgr2=0x7ff3680d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:00.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.781+0000 7ff36e46d640 1 -- 172.21.15.64:0/3575012942 shutdown_connections 2023-11-24T08:06:00.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.781+0000 7ff36e46d640 1 -- 172.21.15.64:0/3575012942 wait complete. 2023-11-24T08:06:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.782+0000 7ff36e46d640 1 Processor -- start 2023-11-24T08:06:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.783+0000 7ff36e46d640 1 -- start start 2023-11-24T08:06:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.783+0000 7ff36e46d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3680b1650 0x7ff36814ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.783+0000 7ff36e46d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3680d14a0 0x7ff3680b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.783+0000 7ff36e46d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff36815d5b0 0x7ff3681f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36e46d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36814bd60 con 0x7ff3680d14a0 2023-11-24T08:06:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36e46d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680b7570 con 0x7ff3680b1650 2023-11-24T08:06:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36e46d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680b7850 con 0x7ff36815d5b0 2023-11-24T08:06:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36cc6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3680d14a0 0x7ff3680b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36cc6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3680d14a0 0x7ff3680b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52752/0 (socket says 172.21.15.64:52752) 2023-11-24T08:06:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36cc6a640 1 -- 172.21.15.64:0/3344155602 learned_addr learned my addr 172.21.15.64:0/3344155602 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36d46b640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3680b1650 0x7ff36814ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36cc6a640 1 -- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff36815d5b0 msgr2=0x7ff3681f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36dc6c640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff36815d5b0 0x7ff3681f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36cc6a640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff36815d5b0 0x7ff3681f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36cc6a640 1 -- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3680b1650 msgr2=0x7ff36814ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36cc6a640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3680b1650 0x7ff36814ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36cc6a640 1 -- 172.21.15.64:0/3344155602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff364009400 con 0x7ff3680d14a0 2023-11-24T08:06:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36d46b640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3680b1650 0x7ff36814ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.784+0000 7ff36dc6c640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff36815d5b0 0x7ff3681f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.785+0000 7ff36cc6a640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3680d14a0 0x7ff3680b5c90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00c3f0 tx=0x7ff35c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.785+0000 7ff359ffb640 1 -- 172.21.15.64:0/3344155602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0074e0 con 0x7ff3680d14a0 2023-11-24T08:06:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.785+0000 7ff359ffb640 1 -- 172.21.15.64:0/3344155602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c007640 con 0x7ff3680d14a0 2023-11-24T08:06:00.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.785+0000 7ff359ffb640 1 -- 172.21.15.64:0/3344155602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c004a00 con 0x7ff3680d14a0 2023-11-24T08:06:00.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.785+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3680b64e0 con 0x7ff3680d14a0 2023-11-24T08:06:00.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.785+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3680b6aa0 con 0x7ff3680d14a0 2023-11-24T08:06:00.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.786+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3680d3e50 con 0x7ff3680d14a0 2023-11-24T08:06:00.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.787+0000 7ff359ffb640 1 -- 172.21.15.64:0/3344155602 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff35c004b60 con 0x7ff3680d14a0 2023-11-24T08:06:00.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.788+0000 7ff359ffb640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff34c06af00 0x7ff34c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:00.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.788+0000 7ff359ffb640 1 -- 172.21.15.64:0/3344155602 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff35c0891a0 con 0x7ff3680d14a0 2023-11-24T08:06:00.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.789+0000 7ff36d46b640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff34c06af00 0x7ff34c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:00.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.790+0000 7ff36d46b640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff34c06af00 0x7ff34c06d3c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff364036000 tx=0x7ff36403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:00.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.792+0000 7ff359ffb640 1 -- 172.21.15.64:0/3344155602 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff35c0544b0 con 0x7ff3680d14a0 2023-11-24T08:06:00.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.987+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff36804e840 con 0x7ff3680d14a0 2023-11-24T08:06:00.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.989+0000 7ff359ffb640 1 -- 172.21.15.64:0/3344155602 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12809 (secure 0 0 0) 0x7ff35c057ad0 con 0x7ff3680d14a0 2023-11-24T08:06:00.991 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:00.992 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":41,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:00.037540+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:00.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff34c06af00 msgr2=0x7ff34c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff34c06af00 0x7ff34c06d3c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff364036000 tx=0x7ff36403b040 comp rx=0 tx=0).stop 2023-11-24T08:06:00.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3680d14a0 msgr2=0x7ff3680b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:00.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3680d14a0 0x7ff3680b5c90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00c3f0 tx=0x7ff35c00c8b0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 shutdown_connections 2023-11-24T08:06:00.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff34c06af00 0x7ff34c06d3c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff36815d5b0 0x7ff3681f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3680d14a0 0x7ff3680b5c90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 --2- 172.21.15.64:0/3344155602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff3680b1650 0x7ff36814ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 >> 172.21.15.64:0/3344155602 conn(0x7ff36809c3b0 msgr2=0x7ff36815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 shutdown_connections 2023-11-24T08:06:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:00.992+0000 7ff36e46d640 1 -- 172.21.15.64:0/3344155602 wait complete. 2023-11-24T08:06:01.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:06:01.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:06:01.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.234+0000 7f2c4ea2e640 1 Processor -- start 2023-11-24T08:06:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.234+0000 7f2c4ea2e640 1 -- start start 2023-11-24T08:06:01.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.235+0000 7f2c4ea2e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c4815dfe0 0x7f2c480c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.235+0000 7f2c4ea2e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2c4806a440 0x7f2c4806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.235+0000 7f2c4ea2e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c480cee40 con 0x7f2c48070990 2023-11-24T08:06:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4ea2e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4815d820 con 0x7f2c4815dfe0 2023-11-24T08:06:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4ea2e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4806e1e0 con 0x7f2c4806a440 2023-11-24T08:06:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4d22b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c4815dfe0 0x7f2c480c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4da2c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2c48070990 0x7f2c48070d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37356/0 (socket says 172.21.15.64:37356) 2023-11-24T08:06:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4da2c640 1 -- 172.21.15.64:0/1017436750 learned_addr learned my addr 172.21.15.64:0/1017436750 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4e22d640 1 --2- 172.21.15.64:0/1017436750 >> v2:172.21.15.91:3300/0 conn(0x7f2c4806a440 0x7f2c4806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4d22b640 1 -- 172.21.15.64:0/1017436750 >> v2:172.21.15.91:3300/0 conn(0x7f2c4806a440 msgr2=0x7f2c4806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4d22b640 1 --2- 172.21.15.64:0/1017436750 >> v2:172.21.15.91:3300/0 conn(0x7f2c4806a440 0x7f2c4806a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4d22b640 1 -- 172.21.15.64:0/1017436750 >> v1:172.21.15.64:6789/0 conn(0x7f2c48070990 legacy=0x7f2c48070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4d22b640 1 -- 172.21.15.64:0/1017436750 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4806e4c0 con 0x7f2c4815dfe0 2023-11-24T08:06:01.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.236+0000 7f2c4e22d640 1 --2- 172.21.15.64:0/1017436750 >> v2:172.21.15.91:3300/0 conn(0x7f2c4806a440 0x7f2c4806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.238+0000 7f2c4d22b640 1 --2- 172.21.15.64:0/1017436750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c480c43a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2c480c51a0 tx=0x7f2c3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaf400b16723e3b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.238+0000 7f2c4ca2a640 1 -- 172.21.15.64:0/1017436750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3802e070 con 0x7f2c4815dfe0 2023-11-24T08:06:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.238+0000 7f2c4ca2a640 1 -- 172.21.15.64:0/1017436750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c380315a0 con 0x7f2c4815dfe0 2023-11-24T08:06:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.238+0000 7f2c4ca2a640 1 -- 172.21.15.64:0/1017436750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38031930 con 0x7f2c4815dfe0 2023-11-24T08:06:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.239+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/1017436750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 msgr2=0x7f2c480c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.239+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/1017436750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c480c43a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2c480c51a0 tx=0x7f2c3802b840 comp rx=0 tx=0).stop 2023-11-24T08:06:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.239+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/1017436750 shutdown_connections 2023-11-24T08:06:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.239+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/1017436750 >> v2:172.21.15.91:3300/0 conn(0x7f2c4806a440 0x7f2c4806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.239+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/1017436750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c480c43a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.239+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/1017436750 >> 172.21.15.64:0/1017436750 conn(0x7f2c480cf500 msgr2=0x7f2c480cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.239+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/1017436750 shutdown_connections 2023-11-24T08:06:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.239+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/1017436750 wait complete. 2023-11-24T08:06:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.241+0000 7f2c4ea2e640 1 Processor -- start 2023-11-24T08:06:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.241+0000 7f2c4ea2e640 1 -- start start 2023-11-24T08:06:01.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4ea2e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c4806a440 0x7f2c48159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4ea2e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c48070990 0x7f2c4814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4ea2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c48158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4ea2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4814c3e0 con 0x7f2c4815dfe0 2023-11-24T08:06:01.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4ea2e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48158d20 con 0x7f2c48070990 2023-11-24T08:06:01.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4ea2e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48159000 con 0x7f2c4806a440 2023-11-24T08:06:01.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4e22d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c48158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4e22d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c48158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52768/0 (socket says 172.21.15.64:52768) 2023-11-24T08:06:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4e22d640 1 -- 172.21.15.64:0/2163762046 learned_addr learned my addr 172.21.15.64:0/2163762046 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4d22b640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c48070990 0x7f2c4814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.242+0000 7f2c4da2c640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c4806a440 0x7f2c48159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c4e22d640 1 -- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c4806a440 msgr2=0x7f2c48159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c4e22d640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c4806a440 0x7f2c48159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c4e22d640 1 -- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c48070990 msgr2=0x7f2c4814bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c4e22d640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c48070990 0x7f2c4814bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c4e22d640 1 -- 172.21.15.64:0/2163762046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c38009400 con 0x7f2c4815dfe0 2023-11-24T08:06:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c4e22d640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c48158440 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000cef0 tx=0x7f2c4000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c3e7fc640 1 -- 172.21.15.64:0/2163762046 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000d840 con 0x7f2c4815dfe0 2023-11-24T08:06:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c3e7fc640 1 -- 172.21.15.64:0/2163762046 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4000d9c0 con 0x7f2c4815dfe0 2023-11-24T08:06:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4815b550 con 0x7f2c4815dfe0 2023-11-24T08:06:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c3e7fc640 1 -- 172.21.15.64:0/2163762046 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000dd70 con 0x7f2c4815dfe0 2023-11-24T08:06:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.243+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4815bb10 con 0x7f2c4815dfe0 2023-11-24T08:06:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.245+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4806bf10 con 0x7f2c4815dfe0 2023-11-24T08:06:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.245+0000 7f2c3e7fc640 1 -- 172.21.15.64:0/2163762046 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2c4001e3b0 con 0x7f2c4815dfe0 2023-11-24T08:06:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.246+0000 7f2c3e7fc640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c2c06aed0 0x7f2c2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.246+0000 7f2c3e7fc640 1 -- 172.21.15.64:0/2163762046 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2c40089dd0 con 0x7f2c4815dfe0 2023-11-24T08:06:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.247+0000 7f2c4da2c640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c2c06aed0 0x7f2c2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.248+0000 7f2c4da2c640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c2c06aed0 0x7f2c2c06d390 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005880 tx=0x7f2c44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:01.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.251+0000 7f2c3e7fc640 1 -- 172.21.15.64:0/2163762046 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c400552a0 con 0x7f2c4815dfe0 2023-11-24T08:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.447+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c4815bd00 con 0x7f2c4815dfe0 2023-11-24T08:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.448+0000 7f2c3e7fc640 1 -- 172.21.15.64:0/2163762046 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c4001a0a0 con 0x7f2c4815dfe0 2023-11-24T08:06:01.449 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:06:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.450+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c2c06aed0 msgr2=0x7f2c2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.450+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c2c06aed0 0x7f2c2c06d390 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005880 tx=0x7f2c44009040 comp rx=0 tx=0).stop 2023-11-24T08:06:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.450+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 msgr2=0x7f2c48158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.450+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c48158440 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000cef0 tx=0x7f2c4000a710 comp rx=0 tx=0).stop 2023-11-24T08:06:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.451+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 shutdown_connections 2023-11-24T08:06:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.451+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c2c06aed0 0x7f2c2c06d390 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.451+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4815dfe0 0x7f2c48158440 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.451+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c48070990 0x7f2c4814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.451+0000 7f2c4ea2e640 1 --2- 172.21.15.64:0/2163762046 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c4806a440 0x7f2c48159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.451+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 >> 172.21.15.64:0/2163762046 conn(0x7f2c480cf500 msgr2=0x7f2c4815f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:01.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.451+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 shutdown_connections 2023-11-24T08:06:01.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.451+0000 7f2c4ea2e640 1 -- 172.21.15.64:0/2163762046 wait complete. 2023-11-24T08:06:01.470 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:06:01.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.687+0000 7fa52e57c640 1 Processor -- start 2023-11-24T08:06:01.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.687+0000 7fa52e57c640 1 -- start start 2023-11-24T08:06:01.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.689+0000 7fa52e57c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa52813dfd0 0x7fa52815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.689+0000 7fa52e57c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280cf500 0x7fa52806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.689+0000 7fa52e57c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5280ce7c0 con 0x7fa5280cf050 2023-11-24T08:06:01.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.689+0000 7fa52e57c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa528140900 con 0x7fa52813dfd0 2023-11-24T08:06:01.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.689+0000 7fa52e57c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52815fa70 con 0x7fa5280cf500 2023-11-24T08:06:01.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.689+0000 7fa52cd79640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa5280cf050 0x7fa52806c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37366/0 (socket says 172.21.15.64:37366) 2023-11-24T08:06:01.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.689+0000 7fa52cd79640 1 -- 172.21.15.64:0/3105767129 learned_addr learned my addr 172.21.15.64:0/3105767129 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:01.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa52d57a640 1 --2- 172.21.15.64:0/3105767129 >> v2:172.21.15.91:3300/0 conn(0x7fa52813dfd0 0x7fa52815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa52dd7b640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280cf500 0x7fa52806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892791516 0 0) 0x7fa5280ce7c0 con 0x7fa5280cf050 2023-11-24T08:06:01.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa514003520 con 0x7fa5280cf050 2023-11-24T08:06:01.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108097764 0 0) 0x7fa514003520 con 0x7fa5280cf050 2023-11-24T08:06:01.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280cf500 msgr2=0x7fa52806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:01.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280cf500 0x7fa52806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 >> v2:172.21.15.91:3300/0 conn(0x7fa52813dfd0 msgr2=0x7fa52815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:01.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 --2- 172.21.15.64:0/3105767129 >> v2:172.21.15.91:3300/0 conn(0x7fa52813dfd0 0x7fa52815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5280cf950 con 0x7fa5280cf050 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fa518001690 con 0x7fa5280cf050 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 >> v1:172.21.15.64:6789/0 conn(0x7fa5280cf050 legacy=0x7fa52806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140039e0 0x7fa514003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5140047c0 0x7fa514004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280cf500 0x7fa514005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528140900 con 0x7fa5140047c0 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52815fa70 con 0x7fa5140039e0 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.690+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51400b980 con 0x7fa5280cf500 2023-11-24T08:06:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.691+0000 7fa52cd79640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5140047c0 0x7fa514004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.691+0000 7fa52d57a640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140039e0 0x7fa514003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.691+0000 7fa52dd7b640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280cf500 0x7fa514005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.691+0000 7fa52cd79640 1 -- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280cf500 msgr2=0x7fa514005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.691+0000 7fa52cd79640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280cf500 0x7fa514005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.691+0000 7fa52cd79640 1 -- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140039e0 msgr2=0x7fa514003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.691+0000 7fa52cd79640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140039e0 0x7fa514003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.691+0000 7fa52cd79640 1 -- 172.21.15.64:0/3105767129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa514003520 con 0x7fa5140047c0 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.692+0000 7fa52cd79640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5140047c0 0x7fa514004c40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c210 tx=0x7fa5140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05b9a867a909288 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.693+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518005ad0 con 0x7fa5140047c0 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.693+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa518005c30 con 0x7fa5140047c0 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.693+0000 7fa51ffff640 1 -- 172.21.15.64:0/3105767129 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51802c890 con 0x7fa5140047c0 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.693+0000 7fa52e57c640 1 -- 172.21.15.64:0/3105767129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5140047c0 msgr2=0x7fa514004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.693+0000 7fa52e57c640 1 --2- 172.21.15.64:0/3105767129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5140047c0 0x7fa514004c40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c210 tx=0x7fa5140057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.693+0000 7fa52dd7b640 1 -- 172.21.15.64:0/3105767129 reap_dead start 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.693+0000 7fa52e57c640 1 -- 172.21.15.64:0/3105767129 shutdown_connections 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.693+0000 7fa52e57c640 1 -- 172.21.15.64:0/3105767129 >> 172.21.15.64:0/3105767129 conn(0x7fa52806a860 msgr2=0x7fa52806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.694+0000 7fa52e57c640 1 -- 172.21.15.64:0/3105767129 shutdown_connections 2023-11-24T08:06:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.694+0000 7fa52e57c640 1 -- 172.21.15.64:0/3105767129 wait complete. 2023-11-24T08:06:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.696+0000 7fa52e57c640 1 Processor -- start 2023-11-24T08:06:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.696+0000 7fa52e57c640 1 -- start start 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52e57c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140047c0 0x7fa5280b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52e57c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280b3060 0x7fa5281581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52e57c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa528155040 0x7fa528071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52e57c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528158720 con 0x7fa528155040 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52e57c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528157420 con 0x7fa5140047c0 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52e57c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa528157700 con 0x7fa5280b3060 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52dd7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa528155040 0x7fa528071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52dd7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa528155040 0x7fa528071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52788/0 (socket says 172.21.15.64:52788) 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.697+0000 7fa52dd7b640 1 -- 172.21.15.64:0/3944301456 learned_addr learned my addr 172.21.15.64:0/3944301456 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52dd7b640 1 -- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280b3060 msgr2=0x7fa5281581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52d57a640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140047c0 0x7fa5280b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52cd79640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280b3060 0x7fa5281581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52dd7b640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280b3060 0x7fa5281581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52dd7b640 1 -- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140047c0 msgr2=0x7fa5280b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52dd7b640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140047c0 0x7fa5280b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52dd7b640 1 -- 172.21.15.64:0/3944301456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5281579e0 con 0x7fa528155040 2023-11-24T08:06:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52cd79640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280b3060 0x7fa5281581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa52dd7b640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa528155040 0x7fa528071120 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa5280cfd30 tx=0x7fa5200115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.698+0000 7fa51dffb640 1 -- 172.21.15.64:0/3944301456 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000e070 con 0x7fa528155040 2023-11-24T08:06:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.699+0000 7fa51dffb640 1 -- 172.21.15.64:0/3944301456 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa520011e10 con 0x7fa528155040 2023-11-24T08:06:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.699+0000 7fa51dffb640 1 -- 172.21.15.64:0/3944301456 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520003990 con 0x7fa528155040 2023-11-24T08:06:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.699+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa528157cc0 con 0x7fa528155040 2023-11-24T08:06:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.699+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5280b3790 con 0x7fa528155040 2023-11-24T08:06:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.700+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52804e840 con 0x7fa528155040 2023-11-24T08:06:01.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.701+0000 7fa51dffb640 1 -- 172.21.15.64:0/3944301456 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa520003af0 con 0x7fa528155040 2023-11-24T08:06:01.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.702+0000 7fa51dffb640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa50c06aed0 0x7fa50c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:01.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.702+0000 7fa51dffb640 1 -- 172.21.15.64:0/3944301456 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa5200899c0 con 0x7fa528155040 2023-11-24T08:06:01.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.703+0000 7fa52d57a640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa50c06aed0 0x7fa50c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:01.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.704+0000 7fa52d57a640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa50c06aed0 0x7fa50c06d390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa524009950 tx=0x7fa524008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:01.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.706+0000 7fa51dffb640 1 -- 172.21.15.64:0/3944301456 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa520054cc0 con 0x7fa528155040 2023-11-24T08:06:01.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.904+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa5280ce9a0 con 0x7fa528155040 2023-11-24T08:06:01.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.904+0000 7fa51dffb640 1 -- 172.21.15.64:0/3944301456 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa5200582e0 con 0x7fa528155040 2023-11-24T08:06:01.906 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:01.907 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:06:01.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:01.907 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-11-24T08:06:01.907 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-11-24T08:06:01.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:01.907 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:06:01.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa50c06aed0 msgr2=0x7fa50c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa50c06aed0 0x7fa50c06d390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa524009950 tx=0x7fa524008040 comp rx=0 tx=0).stop 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa528155040 msgr2=0x7fa528071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa528155040 0x7fa528071120 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa5280cfd30 tx=0x7fa5200115a0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 shutdown_connections 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa50c06aed0 0x7fa50c06d390 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa528155040 0x7fa528071120 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5280b3060 0x7fa5281581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 --2- 172.21.15.64:0/3944301456 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5140047c0 0x7fa5280b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:01.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.907+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 >> 172.21.15.64:0/3944301456 conn(0x7fa52806a860 msgr2=0x7fa5280cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:01.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.908+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 shutdown_connections 2023-11-24T08:06:01.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:01.908+0000 7fa52e57c640 1 -- 172.21.15.64:0/3944301456 wait complete. 2023-11-24T08:06:01.928 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:06:01.929 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:06:02.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.161+0000 7fa41359e640 1 Processor -- start 2023-11-24T08:06:02.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.161+0000 7fa41359e640 1 -- start start 2023-11-24T08:06:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.162+0000 7fa41359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa41415d3b0 0x7fa4140ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.162+0000 7fa41359e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa4140d0fa0 0x7fa4140d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.162+0000 7fa41359e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140c6bf0 con 0x7fa4140d1bc0 2023-11-24T08:06:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.162+0000 7fa41359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140d1430 con 0x7fa41415d3b0 2023-11-24T08:06:02.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.162+0000 7fa41359e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140d1fa0 con 0x7fa4140d0fa0 2023-11-24T08:06:02.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa41259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa41415d3b0 0x7fa4140ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:02.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa41259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa41415d3b0 0x7fa4140ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52800/0 (socket says 172.21.15.64:52800) 2023-11-24T08:06:02.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa41259c640 1 -- 172.21.15.64:0/2963184382 learned_addr learned my addr 172.21.15.64:0/2963184382 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa412d9d640 1 --1- 172.21.15.64:0/2963184382 >> v1:172.21.15.64:6789/0 conn(0x7fa4140d1bc0 0x7fa4140b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37380/0 (socket says 172.21.15.64:37380) 2023-11-24T08:06:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa411d9b640 1 --2- 172.21.15.64:0/2963184382 >> v2:172.21.15.91:3300/0 conn(0x7fa4140d0fa0 0x7fa4140d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa41259c640 1 -- 172.21.15.64:0/2963184382 >> v2:172.21.15.91:3300/0 conn(0x7fa4140d0fa0 msgr2=0x7fa4140d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa41259c640 1 --2- 172.21.15.64:0/2963184382 >> v2:172.21.15.91:3300/0 conn(0x7fa4140d0fa0 0x7fa4140d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa41259c640 1 -- 172.21.15.64:0/2963184382 >> v1:172.21.15.64:6789/0 conn(0x7fa4140d1bc0 legacy=0x7fa4140b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:02.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.163+0000 7fa41259c640 1 -- 172.21.15.64:0/2963184382 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140cedc0 con 0x7fa41415d3b0 2023-11-24T08:06:02.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.165+0000 7fa41259c640 1 --2- 172.21.15.64:0/2963184382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4140ce6c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa41408c4a0 tx=0x7fa40402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbee0b7742b09711 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:02.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.165+0000 7fa41159a640 1 -- 172.21.15.64:0/2963184382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404031870 con 0x7fa41415d3b0 2023-11-24T08:06:02.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.165+0000 7fa41159a640 1 -- 172.21.15.64:0/2963184382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4040319d0 con 0x7fa41415d3b0 2023-11-24T08:06:02.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.165+0000 7fa41159a640 1 -- 172.21.15.64:0/2963184382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404031d60 con 0x7fa41415d3b0 2023-11-24T08:06:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.166+0000 7fa41359e640 1 -- 172.21.15.64:0/2963184382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 msgr2=0x7fa4140ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.166+0000 7fa41359e640 1 --2- 172.21.15.64:0/2963184382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4140ce6c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa41408c4a0 tx=0x7fa40402b910 comp rx=0 tx=0).stop 2023-11-24T08:06:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.166+0000 7fa41359e640 1 -- 172.21.15.64:0/2963184382 shutdown_connections 2023-11-24T08:06:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.166+0000 7fa41359e640 1 --2- 172.21.15.64:0/2963184382 >> v2:172.21.15.91:3300/0 conn(0x7fa4140d0fa0 0x7fa4140d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.166+0000 7fa41359e640 1 --2- 172.21.15.64:0/2963184382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4140ce6c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.166+0000 7fa41359e640 1 -- 172.21.15.64:0/2963184382 >> 172.21.15.64:0/2963184382 conn(0x7fa4140d30a0 msgr2=0x7fa4140d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.166+0000 7fa41359e640 1 -- 172.21.15.64:0/2963184382 shutdown_connections 2023-11-24T08:06:02.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.166+0000 7fa41359e640 1 -- 172.21.15.64:0/2963184382 wait complete. 2023-11-24T08:06:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.168+0000 7fa41359e640 1 Processor -- start 2023-11-24T08:06:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.168+0000 7fa41359e640 1 -- start start 2023-11-24T08:06:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa41359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4140d0fa0 0x7fa4141f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa41359e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140d1bc0 0x7fa4141e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa41359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4141e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa41359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4141e7ea0 con 0x7fa41415d3b0 2023-11-24T08:06:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa41359e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4141f73b0 con 0x7fa4140d1bc0 2023-11-24T08:06:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa41359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140b89c0 con 0x7fa4140d0fa0 2023-11-24T08:06:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa412d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4141e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:02.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa412d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4141e7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52806/0 (socket says 172.21.15.64:52806) 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa412d9d640 1 -- 172.21.15.64:0/2932012317 learned_addr learned my addr 172.21.15.64:0/2932012317 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa41259c640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4140d0fa0 0x7fa4141f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.169+0000 7fa412d9d640 1 -- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4140d0fa0 msgr2=0x7fa4141f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa411d9b640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140d1bc0 0x7fa4141e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa412d9d640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4140d0fa0 0x7fa4141f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa412d9d640 1 -- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140d1bc0 msgr2=0x7fa4141e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa412d9d640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140d1bc0 0x7fa4141e6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa412d9d640 1 -- 172.21.15.64:0/2932012317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040094d0 con 0x7fa41415d3b0 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa412d9d640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4141e7510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa40c001eb0 tx=0x7fa40c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa40affd640 1 -- 172.21.15.64:0/2932012317 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0074e0 con 0x7fa41415d3b0 2023-11-24T08:06:02.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa40affd640 1 -- 172.21.15.64:0/2932012317 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c007640 con 0x7fa41415d3b0 2023-11-24T08:06:02.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa40affd640 1 -- 172.21.15.64:0/2932012317 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0049e0 con 0x7fa41415d3b0 2023-11-24T08:06:02.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4140b8410 con 0x7fa41415d3b0 2023-11-24T08:06:02.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.170+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4140b8eb0 con 0x7fa41415d3b0 2023-11-24T08:06:02.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.171+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41404e840 con 0x7fa41415d3b0 2023-11-24T08:06:02.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.173+0000 7fa40affd640 1 -- 172.21.15.64:0/2932012317 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa40c004b40 con 0x7fa41415d3b0 2023-11-24T08:06:02.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.174+0000 7fa40affd640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3f406aed0 0x7fa3f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:02.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.174+0000 7fa40affd640 1 -- 172.21.15.64:0/2932012317 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa40c088d50 con 0x7fa41415d3b0 2023-11-24T08:06:02.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.174+0000 7fa41259c640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3f406aed0 0x7fa3f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:02.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.175+0000 7fa41259c640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3f406aed0 0x7fa3f406d390 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa404036000 tx=0x7fa40403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:02.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.178+0000 7fa40affd640 1 -- 172.21.15.64:0/2932012317 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa40c054060 con 0x7fa41415d3b0 2023-11-24T08:06:02.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:02.370+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa4140d22d0 con 0x7fa41415d3b0 2023-11-24T08:06:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.042+0000 7fa40affd640 1 -- 172.21.15.64:0/2932012317 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa40c057680 con 0x7fa41415d3b0 2023-11-24T08:06:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-11-24T08:06:03.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.045+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3f406aed0 msgr2=0x7fa3f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:03.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.045+0000 7fa41359e640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3f406aed0 0x7fa3f406d390 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa404036000 tx=0x7fa40403c040 comp rx=0 tx=0).stop 2023-11-24T08:06:03.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.045+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 msgr2=0x7fa4141e7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:03.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.045+0000 7fa41359e640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4141e7510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa40c001eb0 tx=0x7fa40c00c820 comp rx=0 tx=0).stop 2023-11-24T08:06:03.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.046+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 shutdown_connections 2023-11-24T08:06:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.046+0000 7fa41359e640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3f406aed0 0x7fa3f406d390 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.046+0000 7fa41359e640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa41415d3b0 0x7fa4141e7510 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.046+0000 7fa41359e640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa4140d1bc0 0x7fa4141e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.046+0000 7fa41359e640 1 --2- 172.21.15.64:0/2932012317 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa4140d0fa0 0x7fa4141f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.046+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 >> 172.21.15.64:0/2932012317 conn(0x7fa4140d30a0 msgr2=0x7fa41415e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.046+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 shutdown_connections 2023-11-24T08:06:03.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.047+0000 7fa41359e640 1 -- 172.21.15.64:0/2932012317 wait complete. 2023-11-24T08:06:03.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:06:03.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.298+0000 7fefa1ffa640 1 Processor -- start 2023-11-24T08:06:03.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.298+0000 7fefa1ffa640 1 -- start start 2023-11-24T08:06:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa1ffa640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa1ffa640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef9c1439b0 0x7fef9c070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa1ffa640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c070d40 con 0x7fef9c1435d0 2023-11-24T08:06:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa1ffa640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c071020 con 0x7fef9c1439b0 2023-11-24T08:06:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa1ffa640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c071300 con 0x7fef9c0b4140 2023-11-24T08:06:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa17f9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef9c1439b0 0x7fef9c070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa17f9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef9c1439b0 0x7fef9c070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52808/0 (socket says 172.21.15.64:52808) 2023-11-24T08:06:03.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa0ff8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fef9c1435d0 0x7fef9c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37394/0 (socket says 172.21.15.64:37394) 2023-11-24T08:06:03.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fefa17f9640 1 -- 172.21.15.64:0/51615833 learned_addr learned my addr 172.21.15.64:0/51615833 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:03.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.300+0000 7fef9bfff640 1 --2- 172.21.15.64:0/51615833 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.301+0000 7fefa17f9640 1 -- 172.21.15.64:0/51615833 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 msgr2=0x7fef9c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.301+0000 7fefa17f9640 1 --2- 172.21.15.64:0/51615833 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.301+0000 7fefa17f9640 1 -- 172.21.15.64:0/51615833 >> v1:172.21.15.64:6789/0 conn(0x7fef9c1435d0 legacy=0x7fef9c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.301+0000 7fefa17f9640 1 -- 172.21.15.64:0/51615833 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c0715e0 con 0x7fef9c1439b0 2023-11-24T08:06:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.301+0000 7fef9bfff640 1 --2- 172.21.15.64:0/51615833 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:03.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.302+0000 7fefa17f9640 1 --2- 172.21.15.64:0/51615833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c070050 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fef94009e50 tx=0x7fef9402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e576a81f8c58751d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.302+0000 7fef9b7fe640 1 -- 172.21.15.64:0/51615833 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9402cb00 con 0x7fef9c1439b0 2023-11-24T08:06:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.302+0000 7fef9b7fe640 1 -- 172.21.15.64:0/51615833 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9402cc60 con 0x7fef9c1439b0 2023-11-24T08:06:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.302+0000 7fef9b7fe640 1 -- 172.21.15.64:0/51615833 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef940343f0 con 0x7fef9c1439b0 2023-11-24T08:06:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.303+0000 7fefa1ffa640 1 -- 172.21.15.64:0/51615833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 msgr2=0x7fef9c070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.303+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/51615833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c070050 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fef94009e50 tx=0x7fef9402bf60 comp rx=0 tx=0).stop 2023-11-24T08:06:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.303+0000 7fefa1ffa640 1 -- 172.21.15.64:0/51615833 shutdown_connections 2023-11-24T08:06:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.303+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/51615833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c070050 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.303+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/51615833 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.303+0000 7fefa1ffa640 1 -- 172.21.15.64:0/51615833 >> 172.21.15.64:0/51615833 conn(0x7fef9c0cec10 msgr2=0x7fef9c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.303+0000 7fefa1ffa640 1 -- 172.21.15.64:0/51615833 shutdown_connections 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.303+0000 7fefa1ffa640 1 -- 172.21.15.64:0/51615833 wait complete. 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.305+0000 7fefa1ffa640 1 Processor -- start 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.305+0000 7fefa1ffa640 1 -- start start 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa1ffa640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa1ffa640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef9c1435d0 0x7fef9c15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa1ffa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c15e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa1ffa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0ae230 con 0x7fef9c1439b0 2023-11-24T08:06:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa1ffa640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0ad5b0 con 0x7fef9c1435d0 2023-11-24T08:06:03.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa1ffa640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0ad890 con 0x7fef9c0b4140 2023-11-24T08:06:03.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa17f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:03.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa17f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c15e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52816/0 (socket says 172.21.15.64:52816) 2023-11-24T08:06:03.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.306+0000 7fefa17f9640 1 -- 172.21.15.64:0/2843015225 learned_addr learned my addr 172.21.15.64:0/2843015225 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:03.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fef9bfff640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef9c1435d0 0x7fef9c15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:03.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fefa0ff8640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:03.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fefa17f9640 1 -- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 msgr2=0x7fef9c15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fefa17f9640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c15a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fefa17f9640 1 -- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef9c1435d0 msgr2=0x7fef9c15ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fefa17f9640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef9c1435d0 0x7fef9c15ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fefa17f9640 1 -- 172.21.15.64:0/2843015225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94009b30 con 0x7fef9c1439b0 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fefa17f9640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c15e310 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fef94009e50 tx=0x7fef94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fef997fa640 1 -- 172.21.15.64:0/2843015225 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94009510 con 0x7fef9c1439b0 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fef997fa640 1 -- 172.21.15.64:0/2843015225 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9403e450 con 0x7fef9c1439b0 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fef997fa640 1 -- 172.21.15.64:0/2843015225 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9403e800 con 0x7fef9c1439b0 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.307+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c0adb70 con 0x7fef9c1439b0 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.308+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c15acc0 con 0x7fef9c1439b0 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.309+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c0ae370 con 0x7fef9c1439b0 2023-11-24T08:06:03.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.310+0000 7fef997fa640 1 -- 172.21.15.64:0/2843015225 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fef9403e960 con 0x7fef9c1439b0 2023-11-24T08:06:03.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.311+0000 7fef997fa640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef8006aed0 0x7fef8006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:03.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.311+0000 7fef997fa640 1 -- 172.21.15.64:0/2843015225 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fef940ac570 con 0x7fef9c1439b0 2023-11-24T08:06:03.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.312+0000 7fefa0ff8640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef8006aed0 0x7fef8006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:03.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.313+0000 7fefa0ff8640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef8006aed0 0x7fef8006d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef90007fd0 tx=0x7fef90009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:03.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.315+0000 7fef997fa640 1 -- 172.21.15.64:0/2843015225 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef94077940 con 0x7fef9c1439b0 2023-11-24T08:06:03.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:03.508+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fef9c0ae370 con 0x7fef9c1439b0 2023-11-24T08:06:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.048+0000 7fef997fa640 1 -- 172.21.15.64:0/2843015225 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7fef9c0ae370 con 0x7fef9c1439b0 2023-11-24T08:06:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-11-24T08:06:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.052+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef8006aed0 msgr2=0x7fef8006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.052+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef8006aed0 0x7fef8006d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef90007fd0 tx=0x7fef90009040 comp rx=0 tx=0).stop 2023-11-24T08:06:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.052+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 msgr2=0x7fef9c15e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.052+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c15e310 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fef94009e50 tx=0x7fef94004a40 comp rx=0 tx=0).stop 2023-11-24T08:06:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.054+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 shutdown_connections 2023-11-24T08:06:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.054+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef8006aed0 0x7fef8006d390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.054+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c1439b0 0x7fef9c15e310 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.054+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef9c1435d0 0x7fef9c15ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.054+0000 7fefa1ffa640 1 --2- 172.21.15.64:0/2843015225 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef9c0b4140 0x7fef9c15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.054+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 >> 172.21.15.64:0/2843015225 conn(0x7fef9c0cec10 msgr2=0x7fef9c0b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.055+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 shutdown_connections 2023-11-24T08:06:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.055+0000 7fefa1ffa640 1 -- 172.21.15.64:0/2843015225 wait complete. 2023-11-24T08:06:04.101 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:06:04.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.294+0000 7f069c906640 1 Processor -- start 2023-11-24T08:06:04.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.294+0000 7f069c906640 1 -- start start 2023-11-24T08:06:04.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.296+0000 7f069c906640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0698144770 0x7f069815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:04.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.296+0000 7f069c906640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f06980b35a0 0x7f069815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:04.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.296+0000 7f069c906640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069815f930 con 0x7f069806c9d0 2023-11-24T08:06:04.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.296+0000 7f069c906640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06980b43b0 con 0x7f0698144770 2023-11-24T08:06:04.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.296+0000 7f069c906640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06980b3b10 con 0x7f06980b35a0 2023-11-24T08:06:04.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.296+0000 7f068ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f06980b35a0 0x7f069815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:04.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.297+0000 7f06977fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0698144770 0x7f069815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:04.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.297+0000 7f068ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f06980b35a0 0x7f069815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:47142/0 (socket says 172.21.15.64:47142) 2023-11-24T08:06:04.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.297+0000 7f068ffff640 1 -- 172.21.15.64:0/3860706734 learned_addr learned my addr 172.21.15.64:0/3860706734 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:04.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.297+0000 7f068ffff640 1 -- 172.21.15.64:0/3860706734 >> v2:172.21.15.91:3300/0 conn(0x7f0698144770 msgr2=0x7f069815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:04.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.297+0000 7f068ffff640 1 --2- 172.21.15.64:0/3860706734 >> v2:172.21.15.91:3300/0 conn(0x7f0698144770 0x7f069815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.297+0000 7f068ffff640 1 -- 172.21.15.64:0/3860706734 >> v1:172.21.15.91:6789/0 conn(0x7f069806c9d0 legacy=0x7f069806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:04.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.297+0000 7f068ffff640 1 -- 172.21.15.64:0/3860706734 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980b3df0 con 0x7f06980b35a0 2023-11-24T08:06:04.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.297+0000 7f06977fe640 1 --2- 172.21.15.64:0/3860706734 >> v2:172.21.15.91:3300/0 conn(0x7f0698144770 0x7f069815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:06:04.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.299+0000 7f068ffff640 1 --2- 172.21.15.64:0/3860706734 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f06980b35a0 0x7f069815f0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f06800098a0 tx=0x7f068002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=576a4d252a35dd9b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:04.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.299+0000 7f0696ffd640 1 -- 172.21.15.64:0/3860706734 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680030840 con 0x7f06980b35a0 2023-11-24T08:06:04.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.299+0000 7f0696ffd640 1 -- 172.21.15.64:0/3860706734 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06800309a0 con 0x7f06980b35a0 2023-11-24T08:06:04.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.299+0000 7f0696ffd640 1 -- 172.21.15.64:0/3860706734 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680030d30 con 0x7f06980b35a0 2023-11-24T08:06:04.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.300+0000 7f069c906640 1 -- 172.21.15.64:0/3860706734 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f06980b35a0 msgr2=0x7f069815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:04.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.300+0000 7f069c906640 1 --2- 172.21.15.64:0/3860706734 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f06980b35a0 0x7f069815f0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f06800098a0 tx=0x7f068002b980 comp rx=0 tx=0).stop 2023-11-24T08:06:04.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.300+0000 7f069c906640 1 -- 172.21.15.64:0/3860706734 shutdown_connections 2023-11-24T08:06:04.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.300+0000 7f069c906640 1 --2- 172.21.15.64:0/3860706734 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f06980b35a0 0x7f069815f0a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.300+0000 7f069c906640 1 --2- 172.21.15.64:0/3860706734 >> v2:172.21.15.91:3300/0 conn(0x7f0698144770 0x7f069815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.300+0000 7f069c906640 1 -- 172.21.15.64:0/3860706734 >> 172.21.15.64:0/3860706734 conn(0x7f06980cec10 msgr2=0x7f06980cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:04.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.300+0000 7f069c906640 1 -- 172.21.15.64:0/3860706734 shutdown_connections 2023-11-24T08:06:04.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.300+0000 7f069c906640 1 -- 172.21.15.64:0/3860706734 wait complete. 2023-11-24T08:06:04.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.302+0000 7f069c906640 1 Processor -- start 2023-11-24T08:06:04.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.302+0000 7f069c906640 1 -- start start 2023-11-24T08:06:04.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.303+0000 7f069c906640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f069806c9d0 0x7f0698158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:04.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.303+0000 7f069c906640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06980b35a0 0x7f0698159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:04.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.303+0000 7f069c906640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0698144770 0x7f06981599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:04.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.303+0000 7f069c906640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069815c120 con 0x7f06980b35a0 2023-11-24T08:06:04.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.303+0000 7f069c906640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069815b510 con 0x7f069806c9d0 2023-11-24T08:06:04.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.303+0000 7f069c906640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069815b7f0 con 0x7f0698144770 2023-11-24T08:06:04.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.303+0000 7f068ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06980b35a0 0x7f0698159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:04.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f068ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06980b35a0 0x7f0698159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52828/0 (socket says 172.21.15.64:52828) 2023-11-24T08:06:04.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f068ffff640 1 -- 172.21.15.64:0/666846526 learned_addr learned my addr 172.21.15.64:0/666846526 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f06977fe640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f069806c9d0 0x7f0698158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f0697fff640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0698144770 0x7f06981599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f068ffff640 1 -- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0698144770 msgr2=0x7f06981599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f068ffff640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0698144770 0x7f06981599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f068ffff640 1 -- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f069806c9d0 msgr2=0x7f0698158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f068ffff640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f069806c9d0 0x7f0698158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f068ffff640 1 -- 172.21.15.64:0/666846526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0688009400 con 0x7f06980b35a0 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.304+0000 7f068ffff640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06980b35a0 0x7f0698159490 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0680000c00 tx=0x7f0680004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:04.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.305+0000 7f0694ff9640 1 -- 172.21.15.64:0/666846526 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680039c90 con 0x7f06980b35a0 2023-11-24T08:06:04.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.305+0000 7f0694ff9640 1 -- 172.21.15.64:0/666846526 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0680039df0 con 0x7f06980b35a0 2023-11-24T08:06:04.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.305+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06800094d0 con 0x7f06980b35a0 2023-11-24T08:06:04.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.305+0000 7f0694ff9640 1 -- 172.21.15.64:0/666846526 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680043640 con 0x7f06980b35a0 2023-11-24T08:06:04.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.305+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069815bdb0 con 0x7f06980b35a0 2023-11-24T08:06:04.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.306+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069804e840 con 0x7f06980b35a0 2023-11-24T08:06:04.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.307+0000 7f0694ff9640 1 -- 172.21.15.64:0/666846526 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f06800437a0 con 0x7f06980b35a0 2023-11-24T08:06:04.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.308+0000 7f0694ff9640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f067806af00 0x7f067806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:04.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.308+0000 7f0694ff9640 1 -- 172.21.15.64:0/666846526 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f068003e070 con 0x7f06980b35a0 2023-11-24T08:06:04.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.309+0000 7f06977fe640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f067806af00 0x7f067806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:04.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.310+0000 7f06977fe640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f067806af00 0x7f067806d3c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0688009bc0 tx=0x7f0688008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:04.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.314+0000 7f0694ff9640 1 -- 172.21.15.64:0/666846526 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0680077720 con 0x7f06980b35a0 2023-11-24T08:06:04.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:04.546+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f06980b2460 con 0x7f06980b35a0 2023-11-24T08:06:05.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.098+0000 7f0694ff9640 1 -- 172.21.15.64:0/666846526 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f068007ad40 con 0x7f06980b35a0 2023-11-24T08:06:05.101 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 6 and data pool 7 2023-11-24T08:06:05.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.103+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f067806af00 msgr2=0x7f067806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.103+0000 7f069c906640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f067806af00 0x7f067806d3c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0688009bc0 tx=0x7f0688008040 comp rx=0 tx=0).stop 2023-11-24T08:06:05.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.103+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06980b35a0 msgr2=0x7f0698159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.103+0000 7f069c906640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06980b35a0 0x7f0698159490 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0680000c00 tx=0x7f0680004730 comp rx=0 tx=0).stop 2023-11-24T08:06:05.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.105+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 shutdown_connections 2023-11-24T08:06:05.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.105+0000 7f069c906640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f067806af00 0x7f067806d3c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.105+0000 7f069c906640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0698144770 0x7f06981599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.105+0000 7f069c906640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06980b35a0 0x7f0698159490 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.105+0000 7f069c906640 1 --2- 172.21.15.64:0/666846526 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f069806c9d0 0x7f0698158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.105+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 >> 172.21.15.64:0/666846526 conn(0x7f06980cec10 msgr2=0x7f06980719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:05.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.107+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 shutdown_connections 2023-11-24T08:06:05.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.107+0000 7f069c906640 1 -- 172.21.15.64:0/666846526 wait complete. 2023-11-24T08:06:05.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.318+0000 7f6b38701640 1 Processor -- start 2023-11-24T08:06:05.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.319+0000 7f6b38701640 1 -- start start 2023-11-24T08:06:05.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.320+0000 7f6b38701640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30070780 0x7f6b3006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.320+0000 7f6b38701640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b30071cd0 con 0x7f6b30160970 2023-11-24T08:06:05.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.320+0000 7f6b38701640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b300b2120 con 0x7f6b30070cd0 2023-11-24T08:06:05.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.320+0000 7f6b38701640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300b2400 con 0x7f6b30070780 2023-11-24T08:06:05.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.321+0000 7f6b3649d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6b30160970 0x7f6b300b2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37404/0 (socket says 172.21.15.64:37404) 2023-11-24T08:06:05.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.321+0000 7f6b3649d640 1 -- 172.21.15.64:0/4210620676 learned_addr learned my addr 172.21.15.64:0/4210620676 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:05.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.321+0000 7f6b35c9c640 1 --2- 172.21.15.64:0/4210620676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30070780 0x7f6b3006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.321+0000 7f6b3549b640 1 -- 172.21.15.64:0/4210620676 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085228503 0 0) 0x7f6b30071cd0 con 0x7f6b30160970 2023-11-24T08:06:05.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.321+0000 7f6b3549b640 1 -- 172.21.15.64:0/4210620676 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b20003520 con 0x7f6b30160970 2023-11-24T08:06:05.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.322+0000 7f6b3549b640 1 -- 172.21.15.64:0/4210620676 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 952463134 0 0) 0x7f6b20003520 con 0x7f6b30160970 2023-11-24T08:06:05.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.322+0000 7f6b35c9c640 1 -- 172.21.15.64:0/4210620676 >> v1:172.21.15.91:6789/0 conn(0x7f6b30070cd0 legacy=0x7f6b300acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.322+0000 7f6b35c9c640 1 -- 172.21.15.64:0/4210620676 >> v1:172.21.15.64:6789/0 conn(0x7f6b30160970 legacy=0x7f6b300b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.322+0000 7f6b35c9c640 1 -- 172.21.15.64:0/4210620676 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300b26e0 con 0x7f6b30070780 2023-11-24T08:06:05.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.323+0000 7f6b35c9c640 1 --2- 172.21.15.64:0/4210620676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30070780 0x7f6b3006d520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0097d0 tx=0x7f6b1c02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f98a848a03f16b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:05.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.323+0000 7f6b3549b640 1 -- 172.21.15.64:0/4210620676 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240019f0 con 0x7f6b30070780 2023-11-24T08:06:05.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.323+0000 7f6b3549b640 1 -- 172.21.15.64:0/4210620676 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c02e050 con 0x7f6b30070780 2023-11-24T08:06:05.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.323+0000 7f6b3549b640 1 -- 172.21.15.64:0/4210620676 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c030710 con 0x7f6b30070780 2023-11-24T08:06:05.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.324+0000 7f6b38701640 1 -- 172.21.15.64:0/4210620676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30070780 msgr2=0x7f6b3006d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.324+0000 7f6b38701640 1 --2- 172.21.15.64:0/4210620676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30070780 0x7f6b3006d520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0097d0 tx=0x7f6b1c02b770 comp rx=0 tx=0).stop 2023-11-24T08:06:05.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.324+0000 7f6b38701640 1 -- 172.21.15.64:0/4210620676 shutdown_connections 2023-11-24T08:06:05.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.324+0000 7f6b38701640 1 --2- 172.21.15.64:0/4210620676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30070780 0x7f6b3006d520 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.324+0000 7f6b38701640 1 -- 172.21.15.64:0/4210620676 >> 172.21.15.64:0/4210620676 conn(0x7f6b300b0d80 msgr2=0x7f6b300b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:05.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.324+0000 7f6b38701640 1 -- 172.21.15.64:0/4210620676 shutdown_connections 2023-11-24T08:06:05.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.324+0000 7f6b38701640 1 -- 172.21.15.64:0/4210620676 wait complete. 2023-11-24T08:06:05.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.326+0000 7f6b38701640 1 Processor -- start 2023-11-24T08:06:05.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.326+0000 7f6b38701640 1 -- start start 2023-11-24T08:06:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b38701640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b30070780 0x7f6b30157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b38701640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b30070cd0 0x7f6b30148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b38701640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30160970 0x7f6b3015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b38701640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30155390 con 0x7f6b30070780 2023-11-24T08:06:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b38701640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b301579a0 con 0x7f6b30070cd0 2023-11-24T08:06:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b38701640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b301588a0 con 0x7f6b30160970 2023-11-24T08:06:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b3649d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b30070780 0x7f6b30157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b3649d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b30070780 0x7f6b30157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52832/0 (socket says 172.21.15.64:52832) 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.327+0000 7f6b3649d640 1 -- 172.21.15.64:0/945673329 learned_addr learned my addr 172.21.15.64:0/945673329 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.328+0000 7f6b36c9e640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30160970 0x7f6b3015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.328+0000 7f6b35c9c640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b30070cd0 0x7f6b30148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.328+0000 7f6b3649d640 1 -- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30160970 msgr2=0x7f6b3015ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.328+0000 7f6b3649d640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30160970 0x7f6b3015ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.328+0000 7f6b3649d640 1 -- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b30070cd0 msgr2=0x7f6b30148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.328+0000 7f6b3649d640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b30070cd0 0x7f6b30148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.328+0000 7f6b3649d640 1 -- 172.21.15.64:0/945673329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c009400 con 0x7f6b30070780 2023-11-24T08:06:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.328+0000 7f6b3649d640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b30070780 0x7f6b30157460 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400ccd0 tx=0x7f6b2400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.329+0000 7f6b2effd640 1 -- 172.21.15.64:0/945673329 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24012400 con 0x7f6b30070780 2023-11-24T08:06:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.329+0000 7f6b2effd640 1 -- 172.21.15.64:0/945673329 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24012560 con 0x7f6b30070780 2023-11-24T08:06:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.329+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b30157bd0 con 0x7f6b30070780 2023-11-24T08:06:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.329+0000 7f6b2effd640 1 -- 172.21.15.64:0/945673329 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240128f0 con 0x7f6b30070780 2023-11-24T08:06:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.329+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b30158190 con 0x7f6b30070780 2023-11-24T08:06:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.330+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b30067ee0 con 0x7f6b30070780 2023-11-24T08:06:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.331+0000 7f6b2effd640 1 -- 172.21.15.64:0/945673329 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6b24012a50 con 0x7f6b30070780 2023-11-24T08:06:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.332+0000 7f6b2effd640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b1406af00 0x7f6b1406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.332+0000 7f6b2effd640 1 -- 172.21.15.64:0/945673329 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b24089eb0 con 0x7f6b30070780 2023-11-24T08:06:05.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.332+0000 7f6b35c9c640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b1406af00 0x7f6b1406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.334+0000 7f6b35c9c640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b1406af00 0x7f6b1406d3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c034000 tx=0x7f6b1c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:05.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.336+0000 7f6b2effd640 1 -- 172.21.15.64:0/945673329 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b24054e80 con 0x7f6b30070780 2023-11-24T08:06:05.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.533+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b30074770 con 0x7f6b30070780 2023-11-24T08:06:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.535+0000 7f6b2effd640 1 -- 172.21.15.64:0/945673329 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16478 (secure 0 0 0) 0x7f6b240584a0 con 0x7f6b30070780 2023-11-24T08:06:05.537 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:05.537 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:04.547954+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:02.427519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:03.560696+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.537+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b1406af00 msgr2=0x7f6b1406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.537+0000 7f6b38701640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b1406af00 0x7f6b1406d3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c034000 tx=0x7f6b1c008030 comp rx=0 tx=0).stop 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.537+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b30070780 msgr2=0x7f6b30157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.537+0000 7f6b38701640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b30070780 0x7f6b30157460 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400ccd0 tx=0x7f6b2400a430 comp rx=0 tx=0).stop 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.538+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 shutdown_connections 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.538+0000 7f6b38701640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b1406af00 0x7f6b1406d3c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.538+0000 7f6b38701640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6b30160970 0x7f6b3015ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.538+0000 7f6b38701640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6b30070cd0 0x7f6b30148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.538+0000 7f6b38701640 1 --2- 172.21.15.64:0/945673329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b30070780 0x7f6b30157460 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.538+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 >> 172.21.15.64:0/945673329 conn(0x7f6b300b0d80 msgr2=0x7f6b300acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:05.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.538+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 shutdown_connections 2023-11-24T08:06:05.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.538+0000 7f6b38701640 1 -- 172.21.15.64:0/945673329 wait complete. 2023-11-24T08:06:05.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:05.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.777+0000 7f083c8d1640 1 Processor -- start 2023-11-24T08:06:05.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.778+0000 7f083c8d1640 1 -- start start 2023-11-24T08:06:05.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.779+0000 7f083c8d1640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f083815d5b0 0x7f08380d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.779+0000 7f083c8d1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f08380b1650 0x7f08380d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.779+0000 7f083c8d1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380cf0c0 con 0x7f08380d14a0 2023-11-24T08:06:05.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.779+0000 7f083c8d1640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08380b01b0 con 0x7f083815d5b0 2023-11-24T08:06:05.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.779+0000 7f083c8d1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083815d990 con 0x7f08380b1650 2023-11-24T08:06:05.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.780+0000 7f082ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f08380d14a0 0x7f08380cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37416/0 (socket says 172.21.15.64:37416) 2023-11-24T08:06:05.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.780+0000 7f082ffff640 1 -- 172.21.15.64:0/4189783830 learned_addr learned my addr 172.21.15.64:0/4189783830 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:05.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.780+0000 7f0837fff640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f08380b1650 0x7f08380d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.780+0000 7f08377fe640 1 --2- 172.21.15.64:0/4189783830 >> v2:172.21.15.91:3300/0 conn(0x7f083815d5b0 0x7f08380d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.780+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106375932 0 0) 0x7f08380cf0c0 con 0x7f08380d14a0 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.780+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0824003520 con 0x7f08380d14a0 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2538910999 0 0) 0x7f0824003520 con 0x7f08380d14a0 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f08380b1650 msgr2=0x7f08380d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f08380b1650 0x7f08380d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 >> v2:172.21.15.91:3300/0 conn(0x7f083815d5b0 msgr2=0x7f08380d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 --2- 172.21.15.64:0/4189783830 >> v2:172.21.15.91:3300/0 conn(0x7f083815d5b0 0x7f08380d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380d1e40 con 0x7f08380d14a0 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0820001670 con 0x7f08380d14a0 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 >> v1:172.21.15.64:6789/0 conn(0x7f08380d14a0 legacy=0x7f08380cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f08380b1650 0x7f08240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0824003fb0 0x7f08240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0824004d00 0x7f0824005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08380b01b0 con 0x7f0824003fb0 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083815d990 con 0x7f0824004d00 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082400ba00 con 0x7f08380b1650 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f082ffff640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0824003fb0 0x7f08240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f08377fe640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f08380b1650 0x7f08240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.781+0000 7f082ffff640 1 -- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f08380b1650 msgr2=0x7f08240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.782+0000 7f082ffff640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f08380b1650 0x7f08240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.782+0000 7f0837fff640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0824004d00 0x7f0824005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.782+0000 7f082ffff640 1 -- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0824004d00 msgr2=0x7f0824005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.782+0000 7f082ffff640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0824004d00 0x7f0824005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.782+0000 7f082ffff640 1 -- 172.21.15.64:0/4189783830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0824003520 con 0x7f0824003fb0 2023-11-24T08:06:05.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.783+0000 7f082ffff640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0824003fb0 0x7f08240047c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f082000c160 tx=0x7f08240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feafd68399b852a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.783+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820005ad0 con 0x7f0824003fb0 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.783+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0820005c30 con 0x7f0824003fb0 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.783+0000 7f0836ffd640 1 -- 172.21.15.64:0/4189783830 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082002c810 con 0x7f0824003fb0 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.784+0000 7f083c8d1640 1 -- 172.21.15.64:0/4189783830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0824003fb0 msgr2=0x7f08240047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.784+0000 7f083c8d1640 1 --2- 172.21.15.64:0/4189783830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0824003fb0 0x7f08240047c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f082000c160 tx=0x7f08240057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.784+0000 7f0837fff640 1 -- 172.21.15.64:0/4189783830 reap_dead start 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.784+0000 7f083c8d1640 1 -- 172.21.15.64:0/4189783830 shutdown_connections 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.784+0000 7f083c8d1640 1 -- 172.21.15.64:0/4189783830 >> 172.21.15.64:0/4189783830 conn(0x7f083809c3b0 msgr2=0x7f08380d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.784+0000 7f083c8d1640 1 -- 172.21.15.64:0/4189783830 shutdown_connections 2023-11-24T08:06:05.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.784+0000 7f083c8d1640 1 -- 172.21.15.64:0/4189783830 wait complete. 2023-11-24T08:06:05.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.786+0000 7f083c8d1640 1 Processor -- start 2023-11-24T08:06:05.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.787+0000 7f083c8d1640 1 -- start start 2023-11-24T08:06:05.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f083c8d1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0824003fb0 0x7f08381514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f083c8d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0838151e20 0x7f08381522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f083c8d1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f083815e7e0 0x7f083815ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f083c8d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083815f1a0 con 0x7f0838151e20 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f083c8d1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08381519f0 con 0x7f083815e7e0 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f083c8d1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08381528e0 con 0x7f0824003fb0 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f082ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0838151e20 0x7f08381522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f082ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0838151e20 0x7f08381522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52850/0 (socket says 172.21.15.64:52850) 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.788+0000 7f082ffff640 1 -- 172.21.15.64:0/3709620075 learned_addr learned my addr 172.21.15.64:0/3709620075 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f08377fe640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0824003fb0 0x7f08381514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f0837fff640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f083815e7e0 0x7f083815ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f082ffff640 1 -- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0824003fb0 msgr2=0x7f08381514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f082ffff640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0824003fb0 0x7f08381514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f082ffff640 1 -- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f083815e7e0 msgr2=0x7f083815ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:05.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f082ffff640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f083815e7e0 0x7f083815ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:05.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f082ffff640 1 -- 172.21.15.64:0/3709620075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838152bc0 con 0x7f0838151e20 2023-11-24T08:06:05.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f082ffff640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0838151e20 0x7f08381522a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0820000c00 tx=0x7f0820000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:05.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.789+0000 7f0834ff9640 1 -- 172.21.15.64:0/3709620075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082003c040 con 0x7f0838151e20 2023-11-24T08:06:05.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.790+0000 7f0834ff9640 1 -- 172.21.15.64:0/3709620075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082002cc20 con 0x7f0838151e20 2023-11-24T08:06:05.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.790+0000 7f0834ff9640 1 -- 172.21.15.64:0/3709620075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820007070 con 0x7f0838151e20 2023-11-24T08:06:05.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.790+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0838152ea0 con 0x7f0838151e20 2023-11-24T08:06:05.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.790+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08380cb5c0 con 0x7f0838151e20 2023-11-24T08:06:05.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.791+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083804e840 con 0x7f0838151e20 2023-11-24T08:06:05.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.792+0000 7f0834ff9640 1 -- 172.21.15.64:0/3709620075 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0820032070 con 0x7f0838151e20 2023-11-24T08:06:05.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.793+0000 7f0834ff9640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f081806aed0 0x7f081806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:05.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.793+0000 7f0834ff9640 1 -- 172.21.15.64:0/3709620075 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08200aa7e0 con 0x7f0838151e20 2023-11-24T08:06:05.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.793+0000 7f08377fe640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f081806aed0 0x7f081806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:05.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.795+0000 7f08377fe640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f081806aed0 0x7f081806d390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0828009a10 tx=0x7f0828008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:05.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.798+0000 7f0834ff9640 1 -- 172.21.15.64:0/3709620075 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08200757a0 con 0x7f0838151e20 2023-11-24T08:06:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.994+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08380678d0 con 0x7f0838151e20 2023-11-24T08:06:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:05.997+0000 7f0834ff9640 1 -- 172.21.15.64:0/3709620075 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16478 (secure 0 0 0) 0x7f0820078dc0 con 0x7f0838151e20 2023-11-24T08:06:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:05.999 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:04.547954+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:02.427519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:03.560696+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:06.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f081806aed0 msgr2=0x7f081806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:06.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f081806aed0 0x7f081806d390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0828009a10 tx=0x7f0828008040 comp rx=0 tx=0).stop 2023-11-24T08:06:06.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0838151e20 msgr2=0x7f08381522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:06.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0838151e20 0x7f08381522a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0820000c00 tx=0x7f0820000fe0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 shutdown_connections 2023-11-24T08:06:06.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f081806aed0 0x7f081806d390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f083815e7e0 0x7f083815ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0838151e20 0x7f08381522a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 --2- 172.21.15.64:0/3709620075 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0824003fb0 0x7f08381514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 >> 172.21.15.64:0/3709620075 conn(0x7f083809c3b0 msgr2=0x7f083806c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:06.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 shutdown_connections 2023-11-24T08:06:06.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.000+0000 7f083c8d1640 1 -- 172.21.15.64:0/3709620075 wait complete. 2023-11-24T08:06:06.023 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:06:06.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.236+0000 7fe753fff640 1 Processor -- start 2023-11-24T08:06:06.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.236+0000 7fe753fff640 1 -- start start 2023-11-24T08:06:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.238+0000 7fe753fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe7540af580 0x7fe754163220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.238+0000 7fe753fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75406fc30 con 0x7fe75406f850 2023-11-24T08:06:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.238+0000 7fe753fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe754162160 con 0x7fe75406ab50 2023-11-24T08:06:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.238+0000 7fe753fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe754162440 con 0x7fe7540af580 2023-11-24T08:06:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.238+0000 7fe752ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe75406f850 0x7fe75406a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37422/0 (socket says 172.21.15.64:37422) 2023-11-24T08:06:06.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.238+0000 7fe752ffd640 1 -- 172.21.15.64:0/2910536949 learned_addr learned my addr 172.21.15.64:0/2910536949 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:06.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe7527fc640 1 --2- 172.21.15.64:0/2910536949 >> v2:172.21.15.91:3300/0 conn(0x7fe7540af580 0x7fe754163220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409666069 0 0) 0x7fe75406fc30 con 0x7fe75406f850 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe740003520 con 0x7fe75406f850 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3543708875 0 0) 0x7fe740003520 con 0x7fe75406f850 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 >> v2:172.21.15.91:3300/0 conn(0x7fe7540af580 msgr2=0x7fe754163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 --2- 172.21.15.64:0/2910536949 >> v2:172.21.15.91:3300/0 conn(0x7fe7540af580 0x7fe754163220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 >> v1:172.21.15.91:6789/0 conn(0x7fe75406ab50 legacy=0x7fe754163760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754162720 con 0x7fe75406f850 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fe7440019f0 con 0x7fe75406f850 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 >> v1:172.21.15.64:6789/0 conn(0x7fe75406f850 legacy=0x7fe75406a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7400039e0 0x7fe740003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:06.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe7400047c0 0x7fe740004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.239+0000 7fe751ffb640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe740005180 0x7fe74000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754162440 con 0x7fe740005180 2023-11-24T08:06:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74000c0b0 con 0x7fe7400039e0 2023-11-24T08:06:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74000c270 con 0x7fe7400047c0 2023-11-24T08:06:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe7537fe640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe740005180 0x7fe74000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe7537fe640 1 -- 172.21.15.64:0/2910536949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe7400047c0 msgr2=0x7fe740004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe752ffd640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7400039e0 0x7fe740003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe7537fe640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe7400047c0 0x7fe740004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe7537fe640 1 -- 172.21.15.64:0/2910536949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7400039e0 msgr2=0x7fe740003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe7537fe640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7400039e0 0x7fe740003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe7537fe640 1 -- 172.21.15.64:0/2910536949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe740003520 con 0x7fe740005180 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.240+0000 7fe752ffd640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe7400039e0 0x7fe740003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.241+0000 7fe7537fe640 1 -- 172.21.15.64:0/2910536949 reap_dead start 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.241+0000 7fe7537fe640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe740005180 0x7fe74000b980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe74c001750 tx=0x7fe7400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36d829d278181be server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.242+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c0052d0 con 0x7fe740005180 2023-11-24T08:06:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.242+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c02e050 con 0x7fe740005180 2023-11-24T08:06:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.242+0000 7fe751ffb640 1 -- 172.21.15.64:0/2910536949 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c003830 con 0x7fe740005180 2023-11-24T08:06:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.242+0000 7fe753fff640 1 -- 172.21.15.64:0/2910536949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe740005180 msgr2=0x7fe74000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.242+0000 7fe753fff640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe740005180 0x7fe74000b980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe74c001750 tx=0x7fe7400057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.242+0000 7fe753fff640 1 -- 172.21.15.64:0/2910536949 shutdown_connections 2023-11-24T08:06:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.242+0000 7fe753fff640 1 --2- 172.21.15.64:0/2910536949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe740005180 0x7fe74000b980 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.242+0000 7fe753fff640 1 -- 172.21.15.64:0/2910536949 >> 172.21.15.64:0/2910536949 conn(0x7fe7540cfd80 msgr2=0x7fe7540d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.243+0000 7fe753fff640 1 -- 172.21.15.64:0/2910536949 shutdown_connections 2023-11-24T08:06:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.243+0000 7fe753fff640 1 -- 172.21.15.64:0/2910536949 wait complete. 2023-11-24T08:06:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.245+0000 7fe753fff640 1 Processor -- start 2023-11-24T08:06:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.245+0000 7fe753fff640 1 -- start start 2023-11-24T08:06:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.246+0000 7fe753fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe740005180 0x7fe75409d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.246+0000 7fe753fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe75409c310 0x7fe754151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.246+0000 7fe753fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75409c790 0x7fe754151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.246+0000 7fe753fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415e920 con 0x7fe75409c310 2023-11-24T08:06:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.246+0000 7fe753fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75409b270 con 0x7fe75409c790 2023-11-24T08:06:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.246+0000 7fe753fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75409b550 con 0x7fe740005180 2023-11-24T08:06:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7527fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe75409c310 0x7fe754151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7527fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe75409c310 0x7fe754151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52870/0 (socket says 172.21.15.64:52870) 2023-11-24T08:06:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7527fc640 1 -- 172.21.15.64:0/1713768081 learned_addr learned my addr 172.21.15.64:0/1713768081 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7537fe640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75409c790 0x7fe754151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe752ffd640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe740005180 0x7fe75409d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7527fc640 1 -- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe740005180 msgr2=0x7fe75409d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7527fc640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe740005180 0x7fe75409d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7527fc640 1 -- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75409c790 msgr2=0x7fe754151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7527fc640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75409c790 0x7fe754151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.247+0000 7fe7527fc640 1 -- 172.21.15.64:0/1713768081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75409b830 con 0x7fe75409c310 2023-11-24T08:06:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.248+0000 7fe7527fc640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe75409c310 0x7fe754151a80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe73c004650 tx=0x7fe73c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.248+0000 7fe74bfff640 1 -- 172.21.15.64:0/1713768081 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c00eea0 con 0x7fe75409c310 2023-11-24T08:06:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.248+0000 7fe74bfff640 1 -- 172.21.15.64:0/1713768081 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c00c8c0 con 0x7fe75409c310 2023-11-24T08:06:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.248+0000 7fe74bfff640 1 -- 172.21.15.64:0/1713768081 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c00cc50 con 0x7fe75409c310 2023-11-24T08:06:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.248+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75409bb10 con 0x7fe75409c310 2023-11-24T08:06:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.248+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75409d610 con 0x7fe75409c310 2023-11-24T08:06:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.250+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7541619e0 con 0x7fe75409c310 2023-11-24T08:06:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.251+0000 7fe74bfff640 1 -- 172.21.15.64:0/1713768081 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe73c013470 con 0x7fe75409c310 2023-11-24T08:06:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.252+0000 7fe74bfff640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe73406aed0 0x7fe73406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.252+0000 7fe74bfff640 1 -- 172.21.15.64:0/1713768081 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe73c0897a0 con 0x7fe75409c310 2023-11-24T08:06:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.253+0000 7fe752ffd640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe73406aed0 0x7fe73406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.254+0000 7fe752ffd640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe73406aed0 0x7fe73406d390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe744001b20 tx=0x7fe744009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:06.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.257+0000 7fe74bfff640 1 -- 172.21.15.64:0/1713768081 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe73c0581f0 con 0x7fe75409c310 2023-11-24T08:06:06.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:06.493+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe75404e840 con 0x7fe75409c310 2023-11-24T08:06:07.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.112+0000 7fe74bfff640 1 -- 172.21.15.64:0/1713768081 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe73c057d80 con 0x7fe75409c310 2023-11-24T08:06:07.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.116+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe73406aed0 msgr2=0x7fe73406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:07.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.116+0000 7fe753fff640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe73406aed0 0x7fe73406d390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe744001b20 tx=0x7fe744009040 comp rx=0 tx=0).stop 2023-11-24T08:06:07.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.116+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe75409c310 msgr2=0x7fe754151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:07.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.116+0000 7fe753fff640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe75409c310 0x7fe754151a80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe73c004650 tx=0x7fe73c00ec50 comp rx=0 tx=0).stop 2023-11-24T08:06:07.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.116+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 shutdown_connections 2023-11-24T08:06:07.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.116+0000 7fe753fff640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe73406aed0 0x7fe73406d390 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.116+0000 7fe753fff640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe75409c790 0x7fe754151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.117+0000 7fe753fff640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe75409c310 0x7fe754151a80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.117+0000 7fe753fff640 1 --2- 172.21.15.64:0/1713768081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe740005180 0x7fe75409d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.117+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 >> 172.21.15.64:0/1713768081 conn(0x7fe7540cfd80 msgr2=0x7fe75406d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:07.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.117+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 shutdown_connections 2023-11-24T08:06:07.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.117+0000 7fe753fff640 1 -- 172.21.15.64:0/1713768081 wait complete. 2023-11-24T08:06:07.139 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:06:07.140 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:06:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.350+0000 7fed2bbbe640 1 Processor -- start 2023-11-24T08:06:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.350+0000 7fed2bbbe640 1 -- start start 2023-11-24T08:06:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed2bbbe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fed24160b10 0x7fed2406e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed2bbbe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed24072280 0x7fed2406c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed2bbbe640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed24071780 con 0x7fed2406c900 2023-11-24T08:06:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed2bbbe640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed240b2320 con 0x7fed24160b10 2023-11-24T08:06:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed2bbbe640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2406cd90 con 0x7fed24072280 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed2a15b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fed2406c900 0x7fed24160fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51968/0 (socket says 172.21.15.64:51968) 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed2a15b640 1 -- 172.21.15.64:0/3700280884 learned_addr learned my addr 172.21.15.64:0/3700280884 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed2995a640 1 --2- 172.21.15.64:0/3700280884 >> v2:172.21.15.64:3300/0 conn(0x7fed24160b10 0x7fed2406e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.352+0000 7fed29159640 1 --2- 172.21.15.64:0/3700280884 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed24072280 0x7fed2406c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.353+0000 7fed2995a640 1 -- 172.21.15.64:0/3700280884 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed24072280 msgr2=0x7fed2406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.353+0000 7fed2995a640 1 --2- 172.21.15.64:0/3700280884 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed24072280 0x7fed2406c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.353+0000 7fed2995a640 1 -- 172.21.15.64:0/3700280884 >> v1:172.21.15.64:6789/0 conn(0x7fed2406c900 legacy=0x7fed24160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.353+0000 7fed28958640 1 -- 172.21.15.64:0/3700280884 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444484076 0 0) 0x7fed24071780 con 0x7fed2406c900 2023-11-24T08:06:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.353+0000 7fed2995a640 1 -- 172.21.15.64:0/3700280884 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed240711c0 con 0x7fed24160b10 2023-11-24T08:06:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.354+0000 7fed2995a640 1 --2- 172.21.15.64:0/3700280884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24160b10 0x7fed2406e420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fed200097a0 tx=0x7fed2002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9287322b09c2e346 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.354+0000 7fed28958640 1 -- 172.21.15.64:0/3700280884 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20031780 con 0x7fed24160b10 2023-11-24T08:06:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.354+0000 7fed28958640 1 -- 172.21.15.64:0/3700280884 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200318e0 con 0x7fed24160b10 2023-11-24T08:06:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.354+0000 7fed28958640 1 -- 172.21.15.64:0/3700280884 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20031c70 con 0x7fed24160b10 2023-11-24T08:06:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.355+0000 7fed2bbbe640 1 -- 172.21.15.64:0/3700280884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24160b10 msgr2=0x7fed2406e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.355+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/3700280884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24160b10 0x7fed2406e420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fed200097a0 tx=0x7fed2002b820 comp rx=0 tx=0).stop 2023-11-24T08:06:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.355+0000 7fed2bbbe640 1 -- 172.21.15.64:0/3700280884 shutdown_connections 2023-11-24T08:06:07.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.355+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/3700280884 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed24072280 0x7fed2406c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.355+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/3700280884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24160b10 0x7fed2406e420 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.355+0000 7fed2bbbe640 1 -- 172.21.15.64:0/3700280884 >> 172.21.15.64:0/3700280884 conn(0x7fed240b0ba0 msgr2=0x7fed240b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.355+0000 7fed2bbbe640 1 -- 172.21.15.64:0/3700280884 shutdown_connections 2023-11-24T08:06:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.355+0000 7fed2bbbe640 1 -- 172.21.15.64:0/3700280884 wait complete. 2023-11-24T08:06:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.357+0000 7fed2bbbe640 1 Processor -- start 2023-11-24T08:06:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.357+0000 7fed2bbbe640 1 -- start start 2023-11-24T08:06:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.358+0000 7fed2bbbe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed2406c900 0x7fed24154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.358+0000 7fed2bbbe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24072280 0x7fed24156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.358+0000 7fed2bbbe640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed24160b10 0x7fed240ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.358+0000 7fed2bbbe640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240cc130 con 0x7fed24072280 2023-11-24T08:06:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.358+0000 7fed2bbbe640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240ca330 con 0x7fed24160b10 2023-11-24T08:06:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.358+0000 7fed2bbbe640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed240ca050 con 0x7fed2406c900 2023-11-24T08:06:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed29159640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24072280 0x7fed24156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed29159640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24072280 0x7fed24156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42976/0 (socket says 172.21.15.64:42976) 2023-11-24T08:06:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed29159640 1 -- 172.21.15.64:0/71533625 learned_addr learned my addr 172.21.15.64:0/71533625 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed2995a640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed2406c900 0x7fed24154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed29159640 1 -- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed2406c900 msgr2=0x7fed24154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed29159640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed2406c900 0x7fed24154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed2a15b640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed24160b10 0x7fed240ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed29159640 1 -- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed24160b10 msgr2=0x7fed240ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed29159640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed24160b10 0x7fed240ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed29159640 1 -- 172.21.15.64:0/71533625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20009400 con 0x7fed24072280 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.359+0000 7fed2a15b640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed24160b10 0x7fed240ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.360+0000 7fed29159640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24072280 0x7fed24156950 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e870 tx=0x7fed1400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.360+0000 7fed1a7fc640 1 -- 172.21.15.64:0/71533625 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed140094f0 con 0x7fed24072280 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.360+0000 7fed1a7fc640 1 -- 172.21.15.64:0/71533625 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1400c940 con 0x7fed24072280 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.360+0000 7fed1a7fc640 1 -- 172.21.15.64:0/71533625 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1400ccf0 con 0x7fed24072280 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.360+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed240cbae0 con 0x7fed24072280 2023-11-24T08:06:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.360+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed24155300 con 0x7fed24072280 2023-11-24T08:06:07.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.361+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2409d200 con 0x7fed24072280 2023-11-24T08:06:07.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.362+0000 7fed1a7fc640 1 -- 172.21.15.64:0/71533625 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fed14013470 con 0x7fed24072280 2023-11-24T08:06:07.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.363+0000 7fed1a7fc640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fed0806aed0 0x7fed0806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:07.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.363+0000 7fed1a7fc640 1 -- 172.21.15.64:0/71533625 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed140897d0 con 0x7fed24072280 2023-11-24T08:06:07.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.364+0000 7fed2995a640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fed0806aed0 0x7fed0806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:07.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.365+0000 7fed2995a640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fed0806aed0 0x7fed0806d390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fed20036000 tx=0x7fed2003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:07.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.369+0000 7fed1a7fc640 1 -- 172.21.15.64:0/71533625 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed14054820 con 0x7fed24072280 2023-11-24T08:06:07.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:07.599+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fed240b3ca0 con 0x7fed24072280 2023-11-24T08:06:08.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.125+0000 7fed1a7fc640 1 -- 172.21.15.64:0/71533625 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fed14057e40 con 0x7fed24072280 2023-11-24T08:06:08.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fed0806aed0 msgr2=0x7fed0806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fed0806aed0 0x7fed0806d390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fed20036000 tx=0x7fed2003d040 comp rx=0 tx=0).stop 2023-11-24T08:06:08.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24072280 msgr2=0x7fed24156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24072280 0x7fed24156950 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e870 tx=0x7fed1400ed30 comp rx=0 tx=0).stop 2023-11-24T08:06:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 shutdown_connections 2023-11-24T08:06:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fed0806aed0 0x7fed0806d390 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fed24160b10 0x7fed240ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24072280 0x7fed24156950 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 --2- 172.21.15.64:0/71533625 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fed2406c900 0x7fed24154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 >> 172.21.15.64:0/71533625 conn(0x7fed240b0ba0 msgr2=0x7fed240abe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 shutdown_connections 2023-11-24T08:06:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.129+0000 7fed2bbbe640 1 -- 172.21.15.64:0/71533625 wait complete. 2023-11-24T08:06:08.150 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:06:08.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:06:08.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.360+0000 7f4169021640 1 Processor -- start 2023-11-24T08:06:08.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.360+0000 7f4169021640 1 -- start start 2023-11-24T08:06:08.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.361+0000 7f4169021640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 0x7f41640cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.361+0000 7f4169021640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4164140900 con 0x7f416413dfd0 2023-11-24T08:06:08.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.361+0000 7f4169021640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416415fa70 con 0x7f41640cd7b0 2023-11-24T08:06:08.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.361+0000 7f4169021640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416415c580 con 0x7f41640cf050 2023-11-24T08:06:08.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.361+0000 7f4163fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f416413dfd0 0x7f416415be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51980/0 (socket says 172.21.15.64:51980) 2023-11-24T08:06:08.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.361+0000 7f4163fff640 1 -- 172.21.15.64:0/2715301925 learned_addr learned my addr 172.21.15.64:0/2715301925 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:08.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062007396 0 0) 0x7f4164140900 con 0x7f416413dfd0 2023-11-24T08:06:08.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4150003520 con 0x7f416413dfd0 2023-11-24T08:06:08.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f415bfff640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 0x7f41640cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3053325263 0 0) 0x7f4150003520 con 0x7f416413dfd0 2023-11-24T08:06:08.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 msgr2=0x7f41640cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:08.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 0x7f41640cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 >> v1:172.21.15.91:6789/0 conn(0x7f41640cd7b0 legacy=0x7f41640cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:08.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416415ee60 con 0x7f416413dfd0 2023-11-24T08:06:08.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f4154001670 con 0x7f416413dfd0 2023-11-24T08:06:08.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 >> v1:172.21.15.64:6789/0 conn(0x7f416413dfd0 legacy=0x7f416415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 0x7f4150003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f41500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4150004d50 0x7f41500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416415c580 con 0x7f4150004070 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415000bbc0 con 0x7f4150004d50 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.362+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415000bd80 con 0x7f41640cf050 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.363+0000 7f415bfff640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f41500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.363+0000 7f4163fff640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 0x7f4150003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.363+0000 7f415bfff640 1 -- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 msgr2=0x7f4150003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.363+0000 7f415bfff640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 0x7f4150003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.363+0000 7f415bfff640 1 -- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4150004d50 msgr2=0x7f41500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.363+0000 7f415bfff640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4150004d50 0x7f41500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.363+0000 7f415bfff640 1 -- 172.21.15.64:0/2715301925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4150003520 con 0x7f4150004070 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.363+0000 7f4163fff640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640cf050 0x7f4150003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:08.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.364+0000 7f415bfff640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f41500047c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f414c00ba00 tx=0x7f41500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ebdf5d477c7947 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.364+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c032620 con 0x7f4150004070 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.364+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c032780 con 0x7f4150004070 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.364+0000 7f41637fe640 1 -- 172.21.15.64:0/2715301925 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c032b10 con 0x7f4150004070 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.365+0000 7f4169021640 1 -- 172.21.15.64:0/2715301925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 msgr2=0x7f41500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.365+0000 7f4169021640 1 --2- 172.21.15.64:0/2715301925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f41500047c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f414c00ba00 tx=0x7f41500057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.365+0000 7f4168820640 1 -- 172.21.15.64:0/2715301925 reap_dead start 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.365+0000 7f4169021640 1 -- 172.21.15.64:0/2715301925 shutdown_connections 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.365+0000 7f4169021640 1 -- 172.21.15.64:0/2715301925 >> 172.21.15.64:0/2715301925 conn(0x7f416406a860 msgr2=0x7f416406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.365+0000 7f4169021640 1 -- 172.21.15.64:0/2715301925 shutdown_connections 2023-11-24T08:06:08.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.365+0000 7f4169021640 1 -- 172.21.15.64:0/2715301925 wait complete. 2023-11-24T08:06:08.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.367+0000 7f4169021640 1 Processor -- start 2023-11-24T08:06:08.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.367+0000 7f4169021640 1 -- start start 2023-11-24T08:06:08.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4169021640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f416413b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4169021640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41640b38a0 0x7f4164155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4169021640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640ad910 0x7f41640add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4169021640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640ae2d0 con 0x7f4150004070 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4169021640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416415a7c0 con 0x7f41640b38a0 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4169021640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416415aaa0 con 0x7f41640ad910 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4163fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f416413b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4163fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f416413b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42988/0 (socket says 172.21.15.64:42988) 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.368+0000 7f4163fff640 1 -- 172.21.15.64:0/2878413483 learned_addr learned my addr 172.21.15.64:0/2878413483 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f415bfff640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41640b38a0 0x7f4164155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4168820640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640ad910 0x7f41640add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4163fff640 1 -- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640ad910 msgr2=0x7f41640add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4163fff640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640ad910 0x7f41640add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4163fff640 1 -- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41640b38a0 msgr2=0x7f4164155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4163fff640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41640b38a0 0x7f4164155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4163fff640 1 -- 172.21.15.64:0/2878413483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416415ad80 con 0x7f4150004070 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f415bfff640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41640b38a0 0x7f4164155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4163fff640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f416413b950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f415400d810 tx=0x7f415400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f41617fa640 1 -- 172.21.15.64:0/2878413483 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540074e0 con 0x7f4150004070 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f41617fa640 1 -- 172.21.15.64:0/2878413483 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4154007640 con 0x7f4150004070 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f41617fa640 1 -- 172.21.15.64:0/2878413483 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154004a00 con 0x7f4150004070 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416415b060 con 0x7f4150004070 2023-11-24T08:06:08.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.369+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41640b3e10 con 0x7f4150004070 2023-11-24T08:06:08.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.370+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416415b1f0 con 0x7f4150004070 2023-11-24T08:06:08.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.371+0000 7f41617fa640 1 -- 172.21.15.64:0/2878413483 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4154004b60 con 0x7f4150004070 2023-11-24T08:06:08.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.373+0000 7f41617fa640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f414406aed0 0x7f414406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.373+0000 7f41617fa640 1 -- 172.21.15.64:0/2878413483 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41540891a0 con 0x7f4150004070 2023-11-24T08:06:08.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.373+0000 7f415bfff640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f414406aed0 0x7f414406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.375+0000 7f415bfff640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f414406aed0 0x7f414406d390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4150004740 tx=0x7f414c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:08.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.377+0000 7f41617fa640 1 -- 172.21.15.64:0/2878413483 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41540540e0 con 0x7f4150004070 2023-11-24T08:06:08.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.605+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f416404e840 con 0x7f4150004070 2023-11-24T08:06:08.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.607+0000 7f41617fa640 1 -- 172.21.15.64:0/2878413483 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f4154057700 con 0x7f4150004070 2023-11-24T08:06:08.609 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:08.609 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/618780376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":618780376},{"type":"v1","addr":"172.21.15.64:6835","nonce":618780376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1212789084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1212789084},{"type":"v1","addr":"172.21.15.91:6836","nonce":1212789084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1895191218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1895191218},{"type":"v1","addr":"172.21.15.91:6837","nonce":1895191218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:04.548725+0000","modified":"2023-11-24T08:06:07.601727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/1271535459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1271535459},{"type":"v1","addr":"172.21.15.64:6834","nonce":1271535459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:06:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2023-11-24T08:06:08.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f414406aed0 msgr2=0x7f414406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f414406aed0 0x7f414406d390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4150004740 tx=0x7f414c030000 comp rx=0 tx=0).stop 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 msgr2=0x7f416413b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f416413b950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f415400d810 tx=0x7f415400dcd0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 shutdown_connections 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f414406aed0 0x7f414406d390 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41640ad910 0x7f41640add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f41640b38a0 0x7f4164155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 --2- 172.21.15.64:0/2878413483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4150004070 0x7f416413b950 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 >> 172.21.15.64:0/2878413483 conn(0x7f416406a860 msgr2=0x7f41640cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.609+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 shutdown_connections 2023-11-24T08:06:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.610+0000 7f4169021640 1 -- 172.21.15.64:0/2878413483 wait complete. 2023-11-24T08:06:08.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.852+0000 7f8bfecea640 1 Processor -- start 2023-11-24T08:06:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.852+0000 7f8bfecea640 1 -- start start 2023-11-24T08:06:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.854+0000 7f8bfecea640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8bf8143570 0x7f8bf80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.854+0000 7f8bfecea640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8bf8143950 0x7f8bf80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.854+0000 7f8bfecea640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf8071700 con 0x7f8bf815e6b0 2023-11-24T08:06:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.854+0000 7f8bfecea640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf815b410 con 0x7f8bf8143570 2023-11-24T08:06:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.854+0000 7f8bfecea640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf815b130 con 0x7f8bf8143950 2023-11-24T08:06:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.854+0000 7f8bfdce8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8bf8143570 0x7f8bf80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.854+0000 7f8bfdce8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8bf8143570 0x7f8bf80cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42998/0 (socket says 172.21.15.64:42998) 2023-11-24T08:06:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.854+0000 7f8bfdce8640 1 -- 172.21.15.64:0/373593951 learned_addr learned my addr 172.21.15.64:0/373593951 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:08.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.855+0000 7f8bfd4e7640 1 --2- 172.21.15.64:0/373593951 >> v2:172.21.15.91:3300/0 conn(0x7f8bf8143950 0x7f8bf80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.855+0000 7f8bfdce8640 1 -- 172.21.15.64:0/373593951 >> v2:172.21.15.91:3300/0 conn(0x7f8bf8143950 msgr2=0x7f8bf80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:08.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.855+0000 7f8bfdce8640 1 --2- 172.21.15.64:0/373593951 >> v2:172.21.15.91:3300/0 conn(0x7f8bf8143950 0x7f8bf80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.855+0000 7f8bfdce8640 1 -- 172.21.15.64:0/373593951 >> v1:172.21.15.91:6789/0 conn(0x7f8bf815e6b0 legacy=0x7f8bf815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:08.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.855+0000 7f8bfdce8640 1 -- 172.21.15.64:0/373593951 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf815dbd0 con 0x7f8bf8143570 2023-11-24T08:06:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.856+0000 7f8bfdce8640 1 --2- 172.21.15.64:0/373593951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf8143570 0x7f8bf80cf050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8070f10 tx=0x7f8bf402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cf92e1462b9ac6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.857+0000 7f8bfcce6640 1 -- 172.21.15.64:0/373593951 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf402ee00 con 0x7f8bf8143570 2023-11-24T08:06:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.857+0000 7f8bfcce6640 1 -- 172.21.15.64:0/373593951 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4036440 con 0x7f8bf8143570 2023-11-24T08:06:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.857+0000 7f8bfcce6640 1 -- 172.21.15.64:0/373593951 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40367d0 con 0x7f8bf8143570 2023-11-24T08:06:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.857+0000 7f8bfecea640 1 -- 172.21.15.64:0/373593951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf8143570 msgr2=0x7f8bf80cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.857+0000 7f8bfecea640 1 --2- 172.21.15.64:0/373593951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf8143570 0x7f8bf80cf050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8070f10 tx=0x7f8bf402e050 comp rx=0 tx=0).stop 2023-11-24T08:06:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.858+0000 7f8bfecea640 1 -- 172.21.15.64:0/373593951 shutdown_connections 2023-11-24T08:06:08.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.858+0000 7f8bfecea640 1 --2- 172.21.15.64:0/373593951 >> v2:172.21.15.91:3300/0 conn(0x7f8bf8143950 0x7f8bf80cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.858+0000 7f8bfecea640 1 --2- 172.21.15.64:0/373593951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf8143570 0x7f8bf80cf050 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.858+0000 7f8bfecea640 1 -- 172.21.15.64:0/373593951 >> 172.21.15.64:0/373593951 conn(0x7f8bf806a860 msgr2=0x7f8bf806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:08.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.858+0000 7f8bfecea640 1 -- 172.21.15.64:0/373593951 shutdown_connections 2023-11-24T08:06:08.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.858+0000 7f8bfecea640 1 -- 172.21.15.64:0/373593951 wait complete. 2023-11-24T08:06:08.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.859+0000 7f8bfecea640 1 Processor -- start 2023-11-24T08:06:08.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.860+0000 7f8bfecea640 1 -- start start 2023-11-24T08:06:08.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.860+0000 7f8bfecea640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8bf8143570 0x7f8bf813eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.860+0000 7f8bfecea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8bf8143950 0x7f8bf8139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.860+0000 7f8bfecea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf815e6b0 0x7f8bf813c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfecea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80cae70 con 0x7f8bf815e6b0 2023-11-24T08:06:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfecea640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80c83a0 con 0x7f8bf8143570 2023-11-24T08:06:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfecea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8141a50 con 0x7f8bf8143950 2023-11-24T08:06:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfe4e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf815e6b0 0x7f8bf813c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfe4e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf815e6b0 0x7f8bf813c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43012/0 (socket says 172.21.15.64:43012) 2023-11-24T08:06:08.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfe4e9640 1 -- 172.21.15.64:0/2441680209 learned_addr learned my addr 172.21.15.64:0/2441680209 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:08.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfdce8640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8bf8143570 0x7f8bf813eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfd4e7640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8bf8143950 0x7f8bf8139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfe4e9640 1 -- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8bf8143950 msgr2=0x7f8bf8139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfe4e9640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8bf8143950 0x7f8bf8139d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfe4e9640 1 -- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8bf8143570 msgr2=0x7f8bf813eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfe4e9640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8bf8143570 0x7f8bf813eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.861+0000 7f8bfe4e9640 1 -- 172.21.15.64:0/2441680209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf400b770 con 0x7f8bf815e6b0 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.862+0000 7f8bfe4e9640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf815e6b0 0x7f8bf813c9b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c780 tx=0x7f8bf000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.862+0000 7f8bee7fc640 1 -- 172.21.15.64:0/2441680209 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0004b60 con 0x7f8bf815e6b0 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.862+0000 7f8bee7fc640 1 -- 172.21.15.64:0/2441680209 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000ce90 con 0x7f8bf815e6b0 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.862+0000 7f8bee7fc640 1 -- 172.21.15.64:0/2441680209 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000a850 con 0x7f8bf815e6b0 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.862+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf8141d30 con 0x7f8bf815e6b0 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.862+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf80c8640 con 0x7f8bf815e6b0 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.863+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf80c8860 con 0x7f8bf815e6b0 2023-11-24T08:06:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.864+0000 7f8bee7fc640 1 -- 172.21.15.64:0/2441680209 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8bf000a9b0 con 0x7f8bf815e6b0 2023-11-24T08:06:08.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.865+0000 7f8bee7fc640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8bdc06aed0 0x7f8bdc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:08.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.865+0000 7f8bee7fc640 1 -- 172.21.15.64:0/2441680209 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bf0089820 con 0x7f8bf815e6b0 2023-11-24T08:06:08.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.866+0000 7f8bfdce8640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8bdc06aed0 0x7f8bdc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:08.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.867+0000 7f8bfdce8640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8bdc06aed0 0x7f8bdc06d390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4038000 tx=0x7f8bf4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:08.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:08.869+0000 7f8bee7fc640 1 -- 172.21.15.64:0/2441680209 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8bf0054870 con 0x7f8bf815e6b0 2023-11-24T08:06:09.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.064+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bf804e840 con 0x7f8bf815e6b0 2023-11-24T08:06:09.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.066+0000 7f8bee7fc640 1 -- 172.21.15.64:0/2441680209 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16465 (secure 0 0 0) 0x7f8bf0057e90 con 0x7f8bf815e6b0 2023-11-24T08:06:09.068 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:09.068 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:06.076753+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:02.427519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:03.560696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.068+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8bdc06aed0 msgr2=0x7f8bdc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.068+0000 7f8bfecea640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8bdc06aed0 0x7f8bdc06d390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4038000 tx=0x7f8bf4008040 comp rx=0 tx=0).stop 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.068+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf815e6b0 msgr2=0x7f8bf813c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.068+0000 7f8bfecea640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf815e6b0 0x7f8bf813c9b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c780 tx=0x7f8bf000cc40 comp rx=0 tx=0).stop 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.069+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 shutdown_connections 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.069+0000 7f8bfecea640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8bdc06aed0 0x7f8bdc06d390 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.069+0000 7f8bfecea640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bf815e6b0 0x7f8bf813c9b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.069+0000 7f8bfecea640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8bf8143950 0x7f8bf8139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.069+0000 7f8bfecea640 1 --2- 172.21.15.64:0/2441680209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8bf8143570 0x7f8bf813eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.069+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 >> 172.21.15.64:0/2441680209 conn(0x7f8bf806a860 msgr2=0x7f8bf806c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:09.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.069+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 shutdown_connections 2023-11-24T08:06:09.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.069+0000 7f8bfecea640 1 -- 172.21.15.64:0/2441680209 wait complete. 2023-11-24T08:06:09.092 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:06:09.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:06:09.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.304+0000 7fad2eef1640 1 Processor -- start 2023-11-24T08:06:09.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.304+0000 7fad2eef1640 1 -- start start 2023-11-24T08:06:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.305+0000 7fad2eef1640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fad28143570 0x7fad280cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.305+0000 7fad2eef1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad2815e6b0 0x7fad280cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.305+0000 7fad2eef1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad28071700 con 0x7fad28143950 2023-11-24T08:06:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.305+0000 7fad2eef1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2815b410 con 0x7fad2815e6b0 2023-11-24T08:06:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.305+0000 7fad2eef1640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2815b130 con 0x7fad28143570 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2d6ee640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fad28143950 0x7fad2815a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51984/0 (socket says 172.21.15.64:51984) 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2d6ee640 1 -- 172.21.15.64:0/185885869 learned_addr learned my addr 172.21.15.64:0/185885869 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2e6f0640 1 --2- 172.21.15.64:0/185885869 >> v2:172.21.15.64:3300/0 conn(0x7fad2815e6b0 0x7fad280cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2deef640 1 --2- 172.21.15.64:0/185885869 >> v2:172.21.15.91:3300/0 conn(0x7fad28143570 0x7fad280cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2ceed640 1 -- 172.21.15.64:0/185885869 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247044473 0 0) 0x7fad18001110 con 0x7fad28143950 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2ceed640 1 -- 172.21.15.64:0/185885869 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad14003520 con 0x7fad28143950 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2e6f0640 1 -- 172.21.15.64:0/185885869 >> v2:172.21.15.91:3300/0 conn(0x7fad28143570 msgr2=0x7fad280cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2e6f0640 1 --2- 172.21.15.64:0/185885869 >> v2:172.21.15.91:3300/0 conn(0x7fad28143570 0x7fad280cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2e6f0640 1 -- 172.21.15.64:0/185885869 >> v1:172.21.15.64:6789/0 conn(0x7fad28143950 legacy=0x7fad2815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2e6f0640 1 -- 172.21.15.64:0/185885869 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2815dbd0 con 0x7fad2815e6b0 2023-11-24T08:06:09.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.306+0000 7fad2deef640 1 --2- 172.21.15.64:0/185885869 >> v2:172.21.15.91:3300/0 conn(0x7fad28143570 0x7fad280cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:06:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.307+0000 7fad2e6f0640 1 --2- 172.21.15.64:0/185885869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad280cf590 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fad28070c70 tx=0x7fad2002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3612dbf9482fd0b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2ceed640 1 -- 172.21.15.64:0/185885869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2002cbf0 con 0x7fad2815e6b0 2023-11-24T08:06:09.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2ceed640 1 -- 172.21.15.64:0/185885869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2002cd50 con 0x7fad2815e6b0 2023-11-24T08:06:09.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2ceed640 1 -- 172.21.15.64:0/185885869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20034660 con 0x7fad2815e6b0 2023-11-24T08:06:09.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2eef1640 1 -- 172.21.15.64:0/185885869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 msgr2=0x7fad280cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2eef1640 1 --2- 172.21.15.64:0/185885869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad280cf590 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fad28070c70 tx=0x7fad2002be40 comp rx=0 tx=0).stop 2023-11-24T08:06:09.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2eef1640 1 -- 172.21.15.64:0/185885869 shutdown_connections 2023-11-24T08:06:09.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2eef1640 1 --2- 172.21.15.64:0/185885869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad280cf590 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2eef1640 1 --2- 172.21.15.64:0/185885869 >> v2:172.21.15.91:3300/0 conn(0x7fad28143570 0x7fad280cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2eef1640 1 -- 172.21.15.64:0/185885869 >> 172.21.15.64:0/185885869 conn(0x7fad2806a860 msgr2=0x7fad2806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:09.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.308+0000 7fad2eef1640 1 -- 172.21.15.64:0/185885869 shutdown_connections 2023-11-24T08:06:09.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.309+0000 7fad2eef1640 1 -- 172.21.15.64:0/185885869 wait complete. 2023-11-24T08:06:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.310+0000 7fad2eef1640 1 Processor -- start 2023-11-24T08:06:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.310+0000 7fad2eef1640 1 -- start start 2023-11-24T08:06:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.311+0000 7fad2eef1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad28143570 0x7fad280cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.311+0000 7fad2eef1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad28143950 0x7fad280cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.311+0000 7fad2eef1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad2813afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.311+0000 7fad2eef1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281f6a80 con 0x7fad2815e6b0 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.311+0000 7fad2eef1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2813da70 con 0x7fad28143950 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.311+0000 7fad2eef1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad281f6cd0 con 0x7fad28143570 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad2813afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad2813afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43028/0 (socket says 172.21.15.64:43028) 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 -- 172.21.15.64:0/2966094788 learned_addr learned my addr 172.21.15.64:0/2966094788 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2d6ee640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad28143950 0x7fad280cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2deef640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad28143570 0x7fad280cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 -- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad28143570 msgr2=0x7fad280cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad28143570 0x7fad280cad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 -- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad28143950 msgr2=0x7fad280cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad28143950 0x7fad280cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 -- 172.21.15.64:0/2966094788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20009a70 con 0x7fad2815e6b0 2023-11-24T08:06:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2d6ee640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad28143950 0x7fad280cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.312+0000 7fad2e6f0640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad2813afe0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fad20009e80 tx=0x7fad20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.313+0000 7fad1e7fc640 1 -- 172.21.15.64:0/2966094788 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20040650 con 0x7fad2815e6b0 2023-11-24T08:06:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.313+0000 7fad1e7fc640 1 -- 172.21.15.64:0/2966094788 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad200407b0 con 0x7fad2815e6b0 2023-11-24T08:06:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.313+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad281f6fb0 con 0x7fad2815e6b0 2023-11-24T08:06:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.313+0000 7fad1e7fc640 1 -- 172.21.15.64:0/2966094788 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20040b60 con 0x7fad2815e6b0 2023-11-24T08:06:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.313+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad281f7570 con 0x7fad2815e6b0 2023-11-24T08:06:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.314+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad280722c0 con 0x7fad2815e6b0 2023-11-24T08:06:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.315+0000 7fad1e7fc640 1 -- 172.21.15.64:0/2966094788 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fad20040cc0 con 0x7fad2815e6b0 2023-11-24T08:06:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.316+0000 7fad1e7fc640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad0c06aed0 0x7fad0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.316+0000 7fad1e7fc640 1 -- 172.21.15.64:0/2966094788 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad200ac4f0 con 0x7fad2815e6b0 2023-11-24T08:06:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.316+0000 7fad2deef640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad0c06aed0 0x7fad0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.318+0000 7fad2deef640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad0c06aed0 0x7fad0c06d390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad24006fd0 tx=0x7fad24008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:09.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.320+0000 7fad1e7fc640 1 -- 172.21.15.64:0/2966094788 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad200774b0 con 0x7fad2815e6b0 2023-11-24T08:06:09.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.512+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad2813dcd0 con 0x7fad0c06aed0 2023-11-24T08:06:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.537+0000 7fad1e7fc640 1 -- 172.21.15.64:0/2966094788 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279111 (secure 0 0 0) 0x7fad2813dcd0 con 0x7fad0c06aed0 2023-11-24T08:06:09.543 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:09.544 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:06:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.542+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad0c06aed0 msgr2=0x7fad0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.542+0000 7fad2eef1640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad0c06aed0 0x7fad0c06d390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad24006fd0 tx=0x7fad24008040 comp rx=0 tx=0).stop 2023-11-24T08:06:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 msgr2=0x7fad2813afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad2813afe0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fad20009e80 tx=0x7fad20004a40 comp rx=0 tx=0).stop 2023-11-24T08:06:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 shutdown_connections 2023-11-24T08:06:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad0c06aed0 0x7fad0c06d390 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2815e6b0 0x7fad2813afe0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad28143950 0x7fad280cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 --2- 172.21.15.64:0/2966094788 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad28143570 0x7fad280cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 >> 172.21.15.64:0/2966094788 conn(0x7fad2806a860 msgr2=0x7fad280b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 shutdown_connections 2023-11-24T08:06:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.543+0000 7fad2eef1640 1 -- 172.21.15.64:0/2966094788 wait complete. 2023-11-24T08:06:09.565 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-24T08:06:08.061465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":158,"acting":158,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221156,"kb_used_data":4996,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753564,"statfs":{"total":773094113280,"available":772867649536,"internally_reserved":0,"allocated":5115904,"data_stored":2747133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013715"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070617+0000","last_change":"2023-11-24T08:06:04.070617+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070617+0000","last_undegraded":"2023-11-24T08:06:04.070617+0000","last_fullsized":"2023-11-24T08:06:04.070617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.062606+0000","last_change":"2023-11-24T08:06:04.062606+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.062606+0000","last_undegraded":"2023-11-24T08:06:04.062606+0000","last_fullsized":"2023-11-24T08:06:04.062606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.069420+0000","last_change":"2023-11-24T08:06:04.069420+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.069420+0000","last_undegraded":"2023-11-24T08:06:04.069420+0000","last_fullsized":"2023-11-24T08:06:04.069420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.057506+0000","last_change":"2023-11-24T08:06:04.057506+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.057506+0000","last_undegraded":"2023-11-24T08:06:04.057506+0000","last_fullsized":"2023-11-24T08:06:04.057506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.069540+0000","last_change":"2023-11-24T08:06:04.069540+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.069540+0000","last_undegraded":"2023-11-24T08:06:04.069540+0000","last_fullsized":"2023-11-24T08:06:04.069540+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.071263+0000","last_change":"2023-11-24T08:06:04.071263+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.071263+0000","last_undegraded":"2023-11-24T08:06:04.071263+0000","last_fullsized":"2023-11-24T08:06:04.071263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065925+0000","last_change":"2023-11-24T08:06:04.065925+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065925+0000","last_undegraded":"2023-11-24T08:06:04.065925+0000","last_fullsized":"2023-11-24T08:06:04.065925+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.059611+0000","last_change":"2023-11-24T08:06:04.059611+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.059611+0000","last_undegraded":"2023-11-24T08:06:04.059611+0000","last_fullsized":"2023-11-24T08:06:04.059611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.074540+0000","last_change":"2023-11-24T08:06:04.074540+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.074540+0000","last_undegraded":"2023-11-24T08:06:04.074540+0000","last_fullsized":"2023-11-24T08:06:04.074540+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.062082+0000","last_change":"2023-11-24T08:06:04.062082+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.062082+0000","last_undegraded":"2023-11-24T08:06:04.062082+0000","last_fullsized":"2023-11-24T08:06:04.062082+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.068343+0000","last_change":"2023-11-24T08:06:04.068343+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.068343+0000","last_undegraded":"2023-11-24T08:06:04.068343+0000","last_fullsized":"2023-11-24T08:06:04.068343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.076389+0000","last_change":"2023-11-24T08:06:04.076389+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.076389+0000","last_undegraded":"2023-11-24T08:06:04.076389+0000","last_fullsized":"2023-11-24T08:06:04.076389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065382+0000","last_change":"2023-11-24T08:06:04.065382+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065382+0000","last_undegraded":"2023-11-24T08:06:04.065382+0000","last_fullsized":"2023-11-24T08:06:04.065382+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.056865+0000","last_change":"2023-11-24T08:06:04.056865+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.056865+0000","last_undegraded":"2023-11-24T08:06:04.056865+0000","last_fullsized":"2023-11-24T08:06:04.056865+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.072856+0000","last_change":"2023-11-24T08:06:04.072856+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.072856+0000","last_undegraded":"2023-11-24T08:06:04.072856+0000","last_fullsized":"2023-11-24T08:06:04.072856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065578+0000","last_change":"2023-11-24T08:06:04.065578+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065578+0000","last_undegraded":"2023-11-24T08:06:04.065578+0000","last_fullsized":"2023-11-24T08:06:04.065578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.071486+0000","last_change":"2023-11-24T08:06:04.071486+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.071486+0000","last_undegraded":"2023-11-24T08:06:04.071486+0000","last_fullsized":"2023-11-24T08:06:04.071486+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.064901+0000","last_change":"2023-11-24T08:06:04.064901+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.064901+0000","last_undegraded":"2023-11-24T08:06:04.064901+0000","last_fullsized":"2023-11-24T08:06:04.064901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.061415+0000","last_change":"2023-11-24T08:06:04.061415+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.061415+0000","last_undegraded":"2023-11-24T08:06:04.061415+0000","last_fullsized":"2023-11-24T08:06:04.061415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070541+0000","last_change":"2023-11-24T08:06:04.070541+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070541+0000","last_undegraded":"2023-11-24T08:06:04.070541+0000","last_fullsized":"2023-11-24T08:06:04.070541+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070491+0000","last_change":"2023-11-24T08:06:04.070491+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070491+0000","last_undegraded":"2023-11-24T08:06:04.070491+0000","last_fullsized":"2023-11-24T08:06:04.070491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.061606+0000","last_change":"2023-11-24T08:06:04.061606+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.061606+0000","last_undegraded":"2023-11-24T08:06:04.061606+0000","last_fullsized":"2023-11-24T08:06:04.061606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.058212+0000","last_change":"2023-11-24T08:06:04.058212+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.058212+0000","last_undegraded":"2023-11-24T08:06:04.058212+0000","last_fullsized":"2023-11-24T08:06:04.058212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.064508+0000","last_change":"2023-11-24T08:06:04.064508+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.064508+0000","last_undegraded":"2023-11-24T08:06:04.064508+0000","last_fullsized":"2023-11-24T08:06:04.064508+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.066728+0000","last_change":"2023-11-24T08:06:04.066728+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.066728+0000","last_undegraded":"2023-11-24T08:06:04.066728+0000","last_fullsized":"2023-11-24T08:06:04.066728+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.055271+0000","last_change":"2023-11-24T08:06:04.055271+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.055271+0000","last_undegraded":"2023-11-24T08:06:04.055271+0000","last_fullsized":"2023-11-24T08:06:04.055271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:35.592076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070221+0000","last_change":"2023-11-24T08:06:04.070221+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070221+0000","last_undegraded":"2023-11-24T08:06:04.070221+0000","last_fullsized":"2023-11-24T08:06:04.070221+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.063976+0000","last_change":"2023-11-24T08:06:04.063976+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.063976+0000","last_undegraded":"2023-11-24T08:06:04.063976+0000","last_fullsized":"2023-11-24T08:06:04.063976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:37.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070093+0000","last_change":"2023-11-24T08:06:04.070093+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070093+0000","last_undegraded":"2023-11-24T08:06:04.070093+0000","last_fullsized":"2023-11-24T08:06:04.070093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.058178+0000","last_change":"2023-11-24T08:06:04.058178+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.058178+0000","last_undegraded":"2023-11-24T08:06:04.058178+0000","last_fullsized":"2023-11-24T08:06:04.058178+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:38.102443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065181+0000","last_change":"2023-11-24T08:06:04.065181+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065181+0000","last_undegraded":"2023-11-24T08:06:04.065181+0000","last_fullsized":"2023-11-24T08:06:04.065181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.060251+0000","last_change":"2023-11-24T08:06:03.060251+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.060251+0000","last_undegraded":"2023-11-24T08:06:03.060251+0000","last_fullsized":"2023-11-24T08:06:03.060251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.060414+0000","last_change":"2023-11-24T08:06:04.060414+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.060414+0000","last_undegraded":"2023-11-24T08:06:04.060414+0000","last_fullsized":"2023-11-24T08:06:04.060414+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.055447+0000","last_change":"2023-11-24T08:06:04.055447+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.055447+0000","last_undegraded":"2023-11-24T08:06:04.055447+0000","last_fullsized":"2023-11-24T08:06:04.055447+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:14.570476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.059441+0000","last_change":"2023-11-24T08:06:04.059441+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.059441+0000","last_undegraded":"2023-11-24T08:06:04.059441+0000","last_fullsized":"2023-11-24T08:06:04.059441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:28.312761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.076161+0000","last_change":"2023-11-24T08:06:04.076161+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.076161+0000","last_undegraded":"2023-11-24T08:06:04.076161+0000","last_fullsized":"2023-11-24T08:06:04.076161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:13.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.073270+0000","last_change":"2023-11-24T08:06:04.073270+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.073270+0000","last_undegraded":"2023-11-24T08:06:04.073270+0000","last_fullsized":"2023-11-24T08:06:04.073270+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.059926+0000","last_change":"2023-11-24T08:06:03.059926+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.059926+0000","last_undegraded":"2023-11-24T08:06:03.059926+0000","last_fullsized":"2023-11-24T08:06:03.059926+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.066999+0000","last_change":"2023-11-24T08:06:04.066999+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.066999+0000","last_undegraded":"2023-11-24T08:06:04.066999+0000","last_fullsized":"2023-11-24T08:06:04.066999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T08:06:04.074104+0000","last_change":"2023-11-24T08:06:04.074104+0000","last_active":"2023-11-24T08:06:04.074104+0000","last_peered":"2023-11-24T08:06:04.074104+0000","last_clean":"2023-11-24T08:06:04.074104+0000","last_became_active":"2023-11-24T08:06:04.073646+0000","last_became_peered":"2023-11-24T08:06:04.073646+0000","last_unstale":"2023-11-24T08:06:04.074104+0000","last_undegraded":"2023-11-24T08:06:04.074104+0000","last_fullsized":"2023-11-24T08:06:04.074104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:57:13.378238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.059514+0000","last_change":"2023-11-24T08:06:03.059514+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.059514+0000","last_undegraded":"2023-11-24T08:06:03.059514+0000","last_fullsized":"2023-11-24T08:06:03.059514+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.056892+0000","last_change":"2023-11-24T08:06:04.056892+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.056892+0000","last_undegraded":"2023-11-24T08:06:04.056892+0000","last_fullsized":"2023-11-24T08:06:04.056892+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.061283+0000","last_change":"2023-11-24T08:06:03.061283+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.061283+0000","last_undegraded":"2023-11-24T08:06:03.061283+0000","last_fullsized":"2023-11-24T08:06:03.061283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.063076+0000","last_change":"2023-11-24T08:06:04.063076+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.063076+0000","last_undegraded":"2023-11-24T08:06:04.063076+0000","last_fullsized":"2023-11-24T08:06:04.063076+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:15.709554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.069778+0000","last_change":"2023-11-24T08:06:04.069778+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.069778+0000","last_undegraded":"2023-11-24T08:06:04.069778+0000","last_fullsized":"2023-11-24T08:06:04.069778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.055475+0000","last_change":"2023-11-24T08:06:04.055475+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.055475+0000","last_undegraded":"2023-11-24T08:06:04.055475+0000","last_fullsized":"2023-11-24T08:06:04.055475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.059770+0000","last_change":"2023-11-24T08:06:03.059770+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.059770+0000","last_undegraded":"2023-11-24T08:06:03.059770+0000","last_fullsized":"2023-11-24T08:06:03.059770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.059282+0000","last_change":"2023-11-24T08:06:04.059282+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.059282+0000","last_undegraded":"2023-11-24T08:06:04.059282+0000","last_fullsized":"2023-11-24T08:06:04.059282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.054849+0000","last_change":"2023-11-24T08:06:04.054849+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.054849+0000","last_undegraded":"2023-11-24T08:06:04.054849+0000","last_fullsized":"2023-11-24T08:06:04.054849+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:09.647409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.068746+0000","last_change":"2023-11-24T08:06:04.068746+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.068746+0000","last_undegraded":"2023-11-24T08:06:04.068746+0000","last_fullsized":"2023-11-24T08:06:04.068746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.059403+0000","last_change":"2023-11-24T08:06:03.059403+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.059403+0000","last_undegraded":"2023-11-24T08:06:03.059403+0000","last_fullsized":"2023-11-24T08:06:03.059403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.064011+0000","last_change":"2023-11-24T08:06:04.064011+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.064011+0000","last_undegraded":"2023-11-24T08:06:04.064011+0000","last_fullsized":"2023-11-24T08:06:04.064011+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T08:06:04.071810+0000","last_change":"2023-11-24T08:06:04.071810+0000","last_active":"2023-11-24T08:06:04.071810+0000","last_peered":"2023-11-24T08:06:04.071810+0000","last_clean":"2023-11-24T08:06:04.071810+0000","last_became_active":"2023-11-24T08:06:04.071196+0000","last_became_peered":"2023-11-24T08:06:04.071196+0000","last_unstale":"2023-11-24T08:06:04.071810+0000","last_undegraded":"2023-11-24T08:06:04.071810+0000","last_fullsized":"2023-11-24T08:06:04.071810+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:13.981420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.063124+0000","last_change":"2023-11-24T08:06:04.063124+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.063124+0000","last_undegraded":"2023-11-24T08:06:04.063124+0000","last_fullsized":"2023-11-24T08:06:04.063124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:38.102443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.066162+0000","last_change":"2023-11-24T08:06:04.066162+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.066162+0000","last_undegraded":"2023-11-24T08:06:04.066162+0000","last_fullsized":"2023-11-24T08:06:04.066162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.071140+0000","last_change":"2023-11-24T08:06:04.071140+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.071140+0000","last_undegraded":"2023-11-24T08:06:04.071140+0000","last_fullsized":"2023-11-24T08:06:04.071140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:29.594620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.077208+0000","last_change":"2023-11-24T08:06:04.077208+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.077208+0000","last_undegraded":"2023-11-24T08:06:04.077208+0000","last_fullsized":"2023-11-24T08:06:04.077208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.054740+0000","last_change":"2023-11-24T08:06:04.054740+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.054740+0000","last_undegraded":"2023-11-24T08:06:04.054740+0000","last_fullsized":"2023-11-24T08:06:04.054740+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:51.350188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.056822+0000","last_change":"2023-11-24T08:06:04.056822+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.056822+0000","last_undegraded":"2023-11-24T08:06:04.056822+0000","last_fullsized":"2023-11-24T08:06:04.056822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.054815+0000","last_change":"2023-11-24T08:06:04.054815+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.054815+0000","last_undegraded":"2023-11-24T08:06:04.054815+0000","last_fullsized":"2023-11-24T08:06:04.054815+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:01.889219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065179+0000","last_change":"2023-11-24T08:06:04.065179+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065179+0000","last_undegraded":"2023-11-24T08:06:04.065179+0000","last_fullsized":"2023-11-24T08:06:04.065179+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.074136+0000","last_change":"2023-11-24T08:06:04.074136+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.074136+0000","last_undegraded":"2023-11-24T08:06:04.074136+0000","last_fullsized":"2023-11-24T08:06:04.074136+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:51:33.198531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.063854+0000","last_change":"2023-11-24T08:06:04.063854+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.063854+0000","last_undegraded":"2023-11-24T08:06:04.063854+0000","last_fullsized":"2023-11-24T08:06:04.063854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.057743+0000","last_change":"2023-11-24T08:06:03.057743+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.057743+0000","last_undegraded":"2023-11-24T08:06:03.057743+0000","last_fullsized":"2023-11-24T08:06:03.057743+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.060422+0000","last_change":"2023-11-24T08:06:03.060422+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.060422+0000","last_undegraded":"2023-11-24T08:06:03.060422+0000","last_fullsized":"2023-11-24T08:06:03.060422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.068836+0000","last_change":"2023-11-24T08:06:04.068836+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.068836+0000","last_undegraded":"2023-11-24T08:06:04.068836+0000","last_fullsized":"2023-11-24T08:06:04.068836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.054696+0000","last_change":"2023-11-24T08:06:04.054696+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.054696+0000","last_undegraded":"2023-11-24T08:06:04.054696+0000","last_fullsized":"2023-11-24T08:06:04.054696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:20.739039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.060026+0000","last_change":"2023-11-24T08:06:03.060026+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.060026+0000","last_undegraded":"2023-11-24T08:06:03.060026+0000","last_fullsized":"2023-11-24T08:06:03.060026+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.058658+0000","last_change":"2023-11-24T08:06:04.058658+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.058658+0000","last_undegraded":"2023-11-24T08:06:04.058658+0000","last_fullsized":"2023-11-24T08:06:04.058658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:56.272590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.055966+0000","last_change":"2023-11-24T08:06:04.055966+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.055966+0000","last_undegraded":"2023-11-24T08:06:04.055966+0000","last_fullsized":"2023-11-24T08:06:04.055966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.060017+0000","last_change":"2023-11-24T08:06:03.060017+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.060017+0000","last_undegraded":"2023-11-24T08:06:03.060017+0000","last_fullsized":"2023-11-24T08:06:03.060017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.056194+0000","last_change":"2023-11-24T08:06:04.056194+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.056194+0000","last_undegraded":"2023-11-24T08:06:04.056194+0000","last_fullsized":"2023-11-24T08:06:04.056194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T08:06:04.070754+0000","last_change":"2023-11-24T08:06:04.070754+0000","last_active":"2023-11-24T08:06:04.070754+0000","last_peered":"2023-11-24T08:06:04.070754+0000","last_clean":"2023-11-24T08:06:04.070754+0000","last_became_active":"2023-11-24T08:06:04.070430+0000","last_became_peered":"2023-11-24T08:06:04.070430+0000","last_unstale":"2023-11-24T08:06:04.070754+0000","last_undegraded":"2023-11-24T08:06:04.070754+0000","last_fullsized":"2023-11-24T08:06:04.070754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:04.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.068386+0000","last_change":"2023-11-24T08:06:04.068386+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.068386+0000","last_undegraded":"2023-11-24T08:06:04.068386+0000","last_fullsized":"2023-11-24T08:06:04.068386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.058513+0000","last_change":"2023-11-24T08:06:04.058513+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.058513+0000","last_undegraded":"2023-11-24T08:06:04.058513+0000","last_fullsized":"2023-11-24T08:06:04.058513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:37.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.059398+0000","last_change":"2023-11-24T08:06:04.059398+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.059398+0000","last_undegraded":"2023-11-24T08:06:04.059398+0000","last_fullsized":"2023-11-24T08:06:04.059398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.057168+0000","last_change":"2023-11-24T08:06:03.057168+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.057168+0000","last_undegraded":"2023-11-24T08:06:03.057168+0000","last_fullsized":"2023-11-24T08:06:03.057168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":4,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.080644+0000","last_change":"2023-11-24T08:06:04.073839+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.080644+0000","last_undegraded":"2023-11-24T08:06:04.080644+0000","last_fullsized":"2023-11-24T08:06:04.080644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:39:00.682536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-24T08:06:00.055324+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:06:00.055324+0000","last_peered":"2023-11-24T08:06:00.055324+0000","last_clean":"2023-11-24T08:06:00.055324+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:06:00.055324+0000","last_undegraded":"2023-11-24T08:06:00.055324+0000","last_fullsized":"2023-11-24T08:06:00.055324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.65300000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.008},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.069,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3200000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7649999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.24,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4179999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1539999999999999,"15min":1.149},"min":{"1min":0.80600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.8660000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.125},{"interface":"front","average":{"1min":1.304,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.80300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8879999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.325}]},{"osd":3,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3120000000000001,"15min":1.3300000000000001},"min":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5109999999999999,"5min":2.524,"15min":2.524},"last":1.256},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.504,"15min":1.534},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7709999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.90800000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.613,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3799999999999999,"15min":1.3839999999999999},"min":{"1min":0.87,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0270000000000001,"5min":2.238,"15min":2.238},"last":1.51}]},{"osd":5,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.619,"15min":1.6319999999999999},"min":{"1min":1.1839999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.085},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.647,"15min":1.677},"min":{"1min":1.0489999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7250000000000001,"5min":2.944,"15min":2.944},"last":1.171}]},{"osd":6,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.7350000000000001,"15min":1.77},"min":{"1min":1.1699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.891,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.671,"15min":1.6870000000000001},"min":{"1min":1.179,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1589999999999998,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.0509999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.538,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.569},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.323,"15min":1.355},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5629999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.736}]},{"osd":1,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1499999999999999,"15min":1.1579999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.073},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.274,"15min":1.319},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.0029999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.282,"15min":1.3129999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0710000000000002,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.4750000000000001,"15min":1.526},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.5509999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.272}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.4299999999999999,"15min":1.458},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.554,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.131},{"interface":"front","average":{"1min":1.401,"5min":1.49,"15min":1.5049999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.7749999999999999,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.6779999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.724,"15min":1.79},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.157,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.472,"15min":1.4830000000000001},"min":{"1min":0.90400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.895,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.5089999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.7709999999999999,"15min":1.843},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3490000000000002,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.9179999999999999,"15min":1.986},"min":{"1min":0.85399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8400000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.3089999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.704},"min":{"1min":0.79400000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.2290000000000001,"5min":2.98,"15min":2.98},"last":1.621},{"interface":"front","average":{"1min":1.548,"5min":1.643,"15min":1.659},"min":{"1min":0.94199999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9379999999999999,"5min":2.29,"15min":2.29},"last":1.3720000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.042,"15min":1.0580000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.627,"5min":1.764,"15min":1.764},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.131,"5min":1.02,"15min":1.002},"min":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.101}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1739999999999999,"15min":1.1819999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6299999999999999,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.0669999999999999,"15min":1.0469999999999999},"min":{"1min":0.85699999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.046}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.206,"15min":1.212},"min":{"1min":0.67200000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7490000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.2330000000000001,"15min":1.218},"min":{"1min":0.78900000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.6259999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.99199999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.597,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.502},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":1.0369999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.667,"5min":1.889,"15min":1.889},"last":1.4359999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.288,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6419999999999999,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.413,"15min":1.413},"min":{"1min":1.206,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9339999999999999,"5min":2.27,"15min":2.27},"last":1.3859999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.464,"15min":1.4750000000000001},"min":{"1min":1.05,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.617,"5min":3.806,"15min":3.806},"last":1.292},{"interface":"front","average":{"1min":1.389,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.95199999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.726,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.339}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.52,"15min":1.528},"min":{"1min":1.1919999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8500000000000001,"5min":2.923,"15min":2.923},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.5309999999999999,"15min":1.5269999999999999},"min":{"1min":1.343,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7769999999999999,"5min":2.859,"15min":2.859},"last":1.546}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0529999999999999,"15min":1.046},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.129,"15min":1.127},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.89100000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.099,"15min":1.091},"min":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.81799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.95299999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.198,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6279999999999999,"5min":1.823,"15min":1.823},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.264,"15min":1.266},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.78600000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.298,"15min":1.302},"min":{"1min":0.90600000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.327,"5min":1.319,"15min":1.3169999999999999},"min":{"1min":0.98299999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6419999999999999,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.97199999999999998}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.304,"15min":1.304},"min":{"1min":0.86899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.702,"5min":1.944,"15min":1.944},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.337,"5min":1.361,"15min":1.365},"min":{"1min":0.96899999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.696,"5min":2.012,"15min":2.012},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3700000000000001,"15min":1.3660000000000001},"min":{"1min":0.91300000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.8560000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5129999999999999,"15min":1.5149999999999999},"min":{"1min":1.198,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0129999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.1100000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.573,"15min":1.5649999999999999},"min":{"1min":1.0880000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.5920000000000001,"5min":2.722,"15min":2.722},"last":0.87}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.82199999999999995,"15min":0.81899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.516},{"interface":"front","average":{"1min":0.874,"5min":0.94299999999999995,"15min":0.95499999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.1379999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.129,"15min":1.169},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6359999999999999,"5min":3.246,"15min":3.246},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.97299999999999998,"15min":0.96699999999999997},"min":{"1min":0.441,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.2589999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1799999999999999,"15min":1.196},"min":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.718,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.075},{"interface":"front","average":{"1min":1.038,"5min":1.147,"15min":1.165},"min":{"1min":0.54000000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5900000000000001,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.399}]},{"osd":4,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.93999999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6879999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.671},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.379,"15min":1.3979999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.73,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.6339999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.448,"15min":1.468},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9870000000000001,"5min":2.395,"15min":2.395},"last":1.9430000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.304,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2410000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.7310000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4299999999999999,"15min":1.4350000000000001},"min":{"1min":0.83199999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.1600000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4159999999999999,"15min":1.4239999999999999},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.946,"5min":1.996,"15min":1.996},"last":2.1259999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4430000000000001,"15min":1.4510000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.994},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.8340000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":1.0209999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.272,"5min":2.133,"15min":2.133},"last":1.212}]},{"osd":1,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.194,"15min":1.242},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.371,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.048,"15min":1.0840000000000001},"min":{"1min":0.5,"5min":0.438,"15min":0.438},"max":{"1min":1.208,"5min":1.998,"15min":1.998},"last":1.4550000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3089999999999999,"15min":1.333},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9259999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.175,"15min":1.204},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.552,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.5389999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3140000000000001,"15min":1.343},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.631,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.798,"5min":3.161,"15min":3.161},"last":1.089}]},{"osd":5,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3640000000000001,"15min":1.3999999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7649999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.627},{"interface":"front","average":{"1min":1.196,"5min":1.391,"15min":1.423},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7549999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.25}]},{"osd":6,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.5620000000000001,"15min":1.6080000000000001},"min":{"1min":0.73899999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.825,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.857,"5min":2.702,"15min":2.702},"last":1.786}]},{"osd":7,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.5009999999999999,"15min":1.5309999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.3980000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.6180000000000001,"15min":1.6619999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.9199999999999999,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.395}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1919999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.105,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.0069999999999999,"15min":1.0389999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2230000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.214,"15min":1.2410000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.353,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.161}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2569999999999999,"15min":1.2769999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6319999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.897},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.2110000000000001,"15min":1.2430000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.538,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.6890000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.242,"15min":1.2549999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.512,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.512},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.4390000000000001,"15min":1.4610000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6459999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.4490000000000001,"15min":1.4650000000000001},"min":{"1min":0.85699999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9590000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.468,"15min":1.484},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.746,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.655}]},{"osd":6,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.45,"15min":1.464},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9019999999999999,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4690000000000001,"15min":1.4910000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.821,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.8420000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.548,"15min":1.5700000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.744,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.738},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5800000000000001,"15min":1.599},"min":{"1min":0.94499999999999995,"5min":0.872,"15min":0.872},"max":{"1min":1.8160000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.788}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1081344,"data_stored":785276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.367,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.93000000000000005},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.05,"15min":1.054},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.706,"5min":2.29,"15min":2.29},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":0.93700000000000006},"min":{"1min":0.51000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.5049999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":0.498}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.032,"15min":1.028},"min":{"1min":0.60499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6439999999999999,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8640000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.256,"15min":1.254},"min":{"1min":0.95899999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.704,"5min":3.302,"15min":3.302},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.454,"15min":1.466},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.2389999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.119}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":0.89400000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.0590000000000002,"5min":3.101,"15min":3.101},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.4359999999999999,"15min":1.4319999999999999},"min":{"1min":0.90100000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.1779999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.2010000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3999999999999999,"15min":1.4159999999999999},"min":{"1min":0.75,"5min":0.749,"15min":0.749},"max":{"1min":2.1819999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4970000000000001,"15min":1.4930000000000001},"min":{"1min":1.0700000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1030000000000002,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.0840000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.5309999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1600000000000001,"5min":3.823,"15min":3.823},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.468,"15min":1.48},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2610000000000001,"5min":3.371,"15min":3.371},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:06:09.571 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:06:09.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.834+0000 7fbbb5947640 1 Processor -- start 2023-11-24T08:06:09.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.834+0000 7fbbb5947640 1 -- start start 2023-11-24T08:06:09.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.835+0000 7fbbb5947640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbbb013dfd0 0x7fbbb015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.835+0000 7fbbb5947640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbbb00cf500 0x7fbbb006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.835+0000 7fbbb5947640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00ce7c0 con 0x7fbbb00cf050 2023-11-24T08:06:09.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.835+0000 7fbbb5947640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb0140900 con 0x7fbbb013dfd0 2023-11-24T08:06:09.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.835+0000 7fbbb5947640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb015fa70 con 0x7fbbb00cf500 2023-11-24T08:06:09.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbb4945640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbbb013dfd0 0x7fbbb015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbaffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbbb00cf050 0x7fbbb006c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51996/0 (socket says 172.21.15.64:51996) 2023-11-24T08:06:09.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbaffff640 1 -- 172.21.15.64:0/2556453053 learned_addr learned my addr 172.21.15.64:0/2556453053 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:09.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbb4945640 1 -- 172.21.15.64:0/2556453053 >> v2:172.21.15.91:3300/0 conn(0x7fbbb00cf500 msgr2=0x7fbbb006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:09.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbb5146640 1 --2- 172.21.15.64:0/2556453053 >> v2:172.21.15.91:3300/0 conn(0x7fbbb00cf500 0x7fbbb006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbaf7fe640 1 -- 172.21.15.64:0/2556453053 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070352693 0 0) 0x7fbbb00ce7c0 con 0x7fbbb00cf050 2023-11-24T08:06:09.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbb4945640 1 --2- 172.21.15.64:0/2556453053 >> v2:172.21.15.91:3300/0 conn(0x7fbbb00cf500 0x7fbbb006c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbb4945640 1 -- 172.21.15.64:0/2556453053 >> v1:172.21.15.64:6789/0 conn(0x7fbbb00cf050 legacy=0x7fbbb006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbb4945640 1 -- 172.21.15.64:0/2556453053 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb015ee60 con 0x7fbbb013dfd0 2023-11-24T08:06:09.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.836+0000 7fbbb5146640 1 --2- 172.21.15.64:0/2556453053 >> v2:172.21.15.91:3300/0 conn(0x7fbbb00cf500 0x7fbbb006c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.838+0000 7fbbb4945640 1 --2- 172.21.15.64:0/2556453053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb015be70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbbb01416f0 tx=0x7fbba402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f0efbab6556b0c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.838+0000 7fbbaf7fe640 1 -- 172.21.15.64:0/2556453053 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4031830 con 0x7fbbb013dfd0 2023-11-24T08:06:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.838+0000 7fbbaf7fe640 1 -- 172.21.15.64:0/2556453053 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba4031990 con 0x7fbbb013dfd0 2023-11-24T08:06:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.838+0000 7fbbaf7fe640 1 -- 172.21.15.64:0/2556453053 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4031d20 con 0x7fbbb013dfd0 2023-11-24T08:06:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.839+0000 7fbbb5947640 1 -- 172.21.15.64:0/2556453053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 msgr2=0x7fbbb015be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.839+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2556453053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb015be70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbbb01416f0 tx=0x7fbba402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.839+0000 7fbbb5947640 1 -- 172.21.15.64:0/2556453053 shutdown_connections 2023-11-24T08:06:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.839+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2556453053 >> v2:172.21.15.91:3300/0 conn(0x7fbbb00cf500 0x7fbbb006c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.839+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2556453053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb015be70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.839+0000 7fbbb5947640 1 -- 172.21.15.64:0/2556453053 >> 172.21.15.64:0/2556453053 conn(0x7fbbb006a860 msgr2=0x7fbbb006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.839+0000 7fbbb5947640 1 -- 172.21.15.64:0/2556453053 shutdown_connections 2023-11-24T08:06:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.839+0000 7fbbb5947640 1 -- 172.21.15.64:0/2556453053 wait complete. 2023-11-24T08:06:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.841+0000 7fbbb5947640 1 Processor -- start 2023-11-24T08:06:09.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.841+0000 7fbbb5947640 1 -- start start 2023-11-24T08:06:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5947640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbbb00cf050 0x7fbbb0154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5947640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbbb00cf500 0x7fbbb00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5947640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb00b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5947640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00b2790 con 0x7fbbb013dfd0 2023-11-24T08:06:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5947640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00b2a70 con 0x7fbbb00cf500 2023-11-24T08:06:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5947640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00b2d50 con 0x7fbbb00cf050 2023-11-24T08:06:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5146640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb00b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5146640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb00b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43056/0 (socket says 172.21.15.64:43056) 2023-11-24T08:06:09.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.842+0000 7fbbb5146640 1 -- 172.21.15.64:0/2195617595 learned_addr learned my addr 172.21.15.64:0/2195617595 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbb5146640 1 -- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbbb00cf050 msgr2=0x7fbbb0154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbaffff640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbbb00cf500 0x7fbbb00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbb5146640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbbb00cf050 0x7fbbb0154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbb5146640 1 -- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbbb00cf500 msgr2=0x7fbbb00b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbb5146640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbbb00cf500 0x7fbbb00b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbb5146640 1 -- 172.21.15.64:0/2195617595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba4009400 con 0x7fbbb013dfd0 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbaffff640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbbb00cf500 0x7fbbb00b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbb5146640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb00b1db0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbba800cef0 tx=0x7fbba800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbad7fa640 1 -- 172.21.15.64:0/2195617595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800d840 con 0x7fbbb013dfd0 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbad7fa640 1 -- 172.21.15.64:0/2195617595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba800d9c0 con 0x7fbbb013dfd0 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb00b3030 con 0x7fbbb013dfd0 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.843+0000 7fbbad7fa640 1 -- 172.21.15.64:0/2195617595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800dd70 con 0x7fbbb013dfd0 2023-11-24T08:06:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.844+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb0154a80 con 0x7fbbb013dfd0 2023-11-24T08:06:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.845+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb00b3550 con 0x7fbbb013dfd0 2023-11-24T08:06:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.846+0000 7fbbad7fa640 1 -- 172.21.15.64:0/2195617595 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fbba801e3b0 con 0x7fbbb013dfd0 2023-11-24T08:06:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.847+0000 7fbbad7fa640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbb9c06aed0 0x7fbb9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.847+0000 7fbbad7fa640 1 -- 172.21.15.64:0/2195617595 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbba808a1e0 con 0x7fbbb013dfd0 2023-11-24T08:06:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.847+0000 7fbbb4945640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbb9c06aed0 0x7fbb9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:09.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.849+0000 7fbbb4945640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbb9c06aed0 0x7fbb9c06d390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbba4036000 tx=0x7fbba4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:09.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:09.851+0000 7fbbad7fa640 1 -- 172.21.15.64:0/2195617595 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbbb00b3550 con 0x7fbbb013dfd0 2023-11-24T08:06:10.014 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-11-24T08:06:10.012+0000 7f18bc34f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:06:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.041+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbb00b2be0 con 0x7fbb9c06aed0 2023-11-24T08:06:10.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.064+0000 7fbbad7fa640 1 -- 172.21.15.64:0/2195617595 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279111 (secure 0 0 0) 0x7fbbb00b2be0 con 0x7fbb9c06aed0 2023-11-24T08:06:10.070 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:10.071 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:06:10.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbb9c06aed0 msgr2=0x7fbb9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbb9c06aed0 0x7fbb9c06d390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbba4036000 tx=0x7fbba4030000 comp rx=0 tx=0).stop 2023-11-24T08:06:10.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 msgr2=0x7fbbb00b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb00b1db0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbba800cef0 tx=0x7fbba800a710 comp rx=0 tx=0).stop 2023-11-24T08:06:10.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 shutdown_connections 2023-11-24T08:06:10.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbb9c06aed0 0x7fbb9c06d390 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbb013dfd0 0x7fbbb00b1db0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbbb00cf500 0x7fbbb00b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 --2- 172.21.15.64:0/2195617595 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbbb00cf050 0x7fbbb0154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 >> 172.21.15.64:0/2195617595 conn(0x7fbbb006a860 msgr2=0x7fbbb0154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:10.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 shutdown_connections 2023-11-24T08:06:10.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.069+0000 7fbbb5947640 1 -- 172.21.15.64:0/2195617595 wait complete. 2023-11-24T08:06:10.091 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-24T08:06:08.061465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":158,"acting":158,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221156,"kb_used_data":4996,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753564,"statfs":{"total":773094113280,"available":772867649536,"internally_reserved":0,"allocated":5115904,"data_stored":2747133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013715"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070617+0000","last_change":"2023-11-24T08:06:04.070617+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070617+0000","last_undegraded":"2023-11-24T08:06:04.070617+0000","last_fullsized":"2023-11-24T08:06:04.070617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.062606+0000","last_change":"2023-11-24T08:06:04.062606+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.062606+0000","last_undegraded":"2023-11-24T08:06:04.062606+0000","last_fullsized":"2023-11-24T08:06:04.062606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.069420+0000","last_change":"2023-11-24T08:06:04.069420+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.069420+0000","last_undegraded":"2023-11-24T08:06:04.069420+0000","last_fullsized":"2023-11-24T08:06:04.069420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.057506+0000","last_change":"2023-11-24T08:06:04.057506+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.057506+0000","last_undegraded":"2023-11-24T08:06:04.057506+0000","last_fullsized":"2023-11-24T08:06:04.057506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.069540+0000","last_change":"2023-11-24T08:06:04.069540+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.069540+0000","last_undegraded":"2023-11-24T08:06:04.069540+0000","last_fullsized":"2023-11-24T08:06:04.069540+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.071263+0000","last_change":"2023-11-24T08:06:04.071263+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.071263+0000","last_undegraded":"2023-11-24T08:06:04.071263+0000","last_fullsized":"2023-11-24T08:06:04.071263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065925+0000","last_change":"2023-11-24T08:06:04.065925+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065925+0000","last_undegraded":"2023-11-24T08:06:04.065925+0000","last_fullsized":"2023-11-24T08:06:04.065925+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.059611+0000","last_change":"2023-11-24T08:06:04.059611+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.059611+0000","last_undegraded":"2023-11-24T08:06:04.059611+0000","last_fullsized":"2023-11-24T08:06:04.059611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.074540+0000","last_change":"2023-11-24T08:06:04.074540+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.074540+0000","last_undegraded":"2023-11-24T08:06:04.074540+0000","last_fullsized":"2023-11-24T08:06:04.074540+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.062082+0000","last_change":"2023-11-24T08:06:04.062082+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.062082+0000","last_undegraded":"2023-11-24T08:06:04.062082+0000","last_fullsized":"2023-11-24T08:06:04.062082+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.068343+0000","last_change":"2023-11-24T08:06:04.068343+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.068343+0000","last_undegraded":"2023-11-24T08:06:04.068343+0000","last_fullsized":"2023-11-24T08:06:04.068343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.076389+0000","last_change":"2023-11-24T08:06:04.076389+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.076389+0000","last_undegraded":"2023-11-24T08:06:04.076389+0000","last_fullsized":"2023-11-24T08:06:04.076389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065382+0000","last_change":"2023-11-24T08:06:04.065382+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065382+0000","last_undegraded":"2023-11-24T08:06:04.065382+0000","last_fullsized":"2023-11-24T08:06:04.065382+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.056865+0000","last_change":"2023-11-24T08:06:04.056865+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.056865+0000","last_undegraded":"2023-11-24T08:06:04.056865+0000","last_fullsized":"2023-11-24T08:06:04.056865+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.072856+0000","last_change":"2023-11-24T08:06:04.072856+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.072856+0000","last_undegraded":"2023-11-24T08:06:04.072856+0000","last_fullsized":"2023-11-24T08:06:04.072856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065578+0000","last_change":"2023-11-24T08:06:04.065578+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065578+0000","last_undegraded":"2023-11-24T08:06:04.065578+0000","last_fullsized":"2023-11-24T08:06:04.065578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.071486+0000","last_change":"2023-11-24T08:06:04.071486+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.071486+0000","last_undegraded":"2023-11-24T08:06:04.071486+0000","last_fullsized":"2023-11-24T08:06:04.071486+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.064901+0000","last_change":"2023-11-24T08:06:04.064901+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.064901+0000","last_undegraded":"2023-11-24T08:06:04.064901+0000","last_fullsized":"2023-11-24T08:06:04.064901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.061415+0000","last_change":"2023-11-24T08:06:04.061415+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.061415+0000","last_undegraded":"2023-11-24T08:06:04.061415+0000","last_fullsized":"2023-11-24T08:06:04.061415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070541+0000","last_change":"2023-11-24T08:06:04.070541+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070541+0000","last_undegraded":"2023-11-24T08:06:04.070541+0000","last_fullsized":"2023-11-24T08:06:04.070541+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070491+0000","last_change":"2023-11-24T08:06:04.070491+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070491+0000","last_undegraded":"2023-11-24T08:06:04.070491+0000","last_fullsized":"2023-11-24T08:06:04.070491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.061606+0000","last_change":"2023-11-24T08:06:04.061606+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.061606+0000","last_undegraded":"2023-11-24T08:06:04.061606+0000","last_fullsized":"2023-11-24T08:06:04.061606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.058212+0000","last_change":"2023-11-24T08:06:04.058212+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.058212+0000","last_undegraded":"2023-11-24T08:06:04.058212+0000","last_fullsized":"2023-11-24T08:06:04.058212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.064508+0000","last_change":"2023-11-24T08:06:04.064508+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.064508+0000","last_undegraded":"2023-11-24T08:06:04.064508+0000","last_fullsized":"2023-11-24T08:06:04.064508+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.066728+0000","last_change":"2023-11-24T08:06:04.066728+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.066728+0000","last_undegraded":"2023-11-24T08:06:04.066728+0000","last_fullsized":"2023-11-24T08:06:04.066728+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.055271+0000","last_change":"2023-11-24T08:06:04.055271+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.055271+0000","last_undegraded":"2023-11-24T08:06:04.055271+0000","last_fullsized":"2023-11-24T08:06:04.055271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:35.592076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070221+0000","last_change":"2023-11-24T08:06:04.070221+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070221+0000","last_undegraded":"2023-11-24T08:06:04.070221+0000","last_fullsized":"2023-11-24T08:06:04.070221+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.063976+0000","last_change":"2023-11-24T08:06:04.063976+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.063976+0000","last_undegraded":"2023-11-24T08:06:04.063976+0000","last_fullsized":"2023-11-24T08:06:04.063976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:37.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.070093+0000","last_change":"2023-11-24T08:06:04.070093+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.070093+0000","last_undegraded":"2023-11-24T08:06:04.070093+0000","last_fullsized":"2023-11-24T08:06:04.070093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.058178+0000","last_change":"2023-11-24T08:06:04.058178+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.058178+0000","last_undegraded":"2023-11-24T08:06:04.058178+0000","last_fullsized":"2023-11-24T08:06:04.058178+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:38.102443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065181+0000","last_change":"2023-11-24T08:06:04.065181+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065181+0000","last_undegraded":"2023-11-24T08:06:04.065181+0000","last_fullsized":"2023-11-24T08:06:04.065181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.060251+0000","last_change":"2023-11-24T08:06:03.060251+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.060251+0000","last_undegraded":"2023-11-24T08:06:03.060251+0000","last_fullsized":"2023-11-24T08:06:03.060251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.060414+0000","last_change":"2023-11-24T08:06:04.060414+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.060414+0000","last_undegraded":"2023-11-24T08:06:04.060414+0000","last_fullsized":"2023-11-24T08:06:04.060414+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.055447+0000","last_change":"2023-11-24T08:06:04.055447+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.055447+0000","last_undegraded":"2023-11-24T08:06:04.055447+0000","last_fullsized":"2023-11-24T08:06:04.055447+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:14.570476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.059441+0000","last_change":"2023-11-24T08:06:04.059441+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.059441+0000","last_undegraded":"2023-11-24T08:06:04.059441+0000","last_fullsized":"2023-11-24T08:06:04.059441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:28.312761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.076161+0000","last_change":"2023-11-24T08:06:04.076161+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.076161+0000","last_undegraded":"2023-11-24T08:06:04.076161+0000","last_fullsized":"2023-11-24T08:06:04.076161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:13.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.073270+0000","last_change":"2023-11-24T08:06:04.073270+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.073270+0000","last_undegraded":"2023-11-24T08:06:04.073270+0000","last_fullsized":"2023-11-24T08:06:04.073270+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.059926+0000","last_change":"2023-11-24T08:06:03.059926+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.059926+0000","last_undegraded":"2023-11-24T08:06:03.059926+0000","last_fullsized":"2023-11-24T08:06:03.059926+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.066999+0000","last_change":"2023-11-24T08:06:04.066999+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.066999+0000","last_undegraded":"2023-11-24T08:06:04.066999+0000","last_fullsized":"2023-11-24T08:06:04.066999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T08:06:04.074104+0000","last_change":"2023-11-24T08:06:04.074104+0000","last_active":"2023-11-24T08:06:04.074104+0000","last_peered":"2023-11-24T08:06:04.074104+0000","last_clean":"2023-11-24T08:06:04.074104+0000","last_became_active":"2023-11-24T08:06:04.073646+0000","last_became_peered":"2023-11-24T08:06:04.073646+0000","last_unstale":"2023-11-24T08:06:04.074104+0000","last_undegraded":"2023-11-24T08:06:04.074104+0000","last_fullsized":"2023-11-24T08:06:04.074104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:57:13.378238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.059514+0000","last_change":"2023-11-24T08:06:03.059514+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.059514+0000","last_undegraded":"2023-11-24T08:06:03.059514+0000","last_fullsized":"2023-11-24T08:06:03.059514+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.056892+0000","last_change":"2023-11-24T08:06:04.056892+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.056892+0000","last_undegraded":"2023-11-24T08:06:04.056892+0000","last_fullsized":"2023-11-24T08:06:04.056892+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.061283+0000","last_change":"2023-11-24T08:06:03.061283+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.061283+0000","last_undegraded":"2023-11-24T08:06:03.061283+0000","last_fullsized":"2023-11-24T08:06:03.061283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.063076+0000","last_change":"2023-11-24T08:06:04.063076+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.063076+0000","last_undegraded":"2023-11-24T08:06:04.063076+0000","last_fullsized":"2023-11-24T08:06:04.063076+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:15.709554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.069778+0000","last_change":"2023-11-24T08:06:04.069778+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.069778+0000","last_undegraded":"2023-11-24T08:06:04.069778+0000","last_fullsized":"2023-11-24T08:06:04.069778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.055475+0000","last_change":"2023-11-24T08:06:04.055475+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.055475+0000","last_undegraded":"2023-11-24T08:06:04.055475+0000","last_fullsized":"2023-11-24T08:06:04.055475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.059770+0000","last_change":"2023-11-24T08:06:03.059770+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.059770+0000","last_undegraded":"2023-11-24T08:06:03.059770+0000","last_fullsized":"2023-11-24T08:06:03.059770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.059282+0000","last_change":"2023-11-24T08:06:04.059282+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.059282+0000","last_undegraded":"2023-11-24T08:06:04.059282+0000","last_fullsized":"2023-11-24T08:06:04.059282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.054849+0000","last_change":"2023-11-24T08:06:04.054849+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.054849+0000","last_undegraded":"2023-11-24T08:06:04.054849+0000","last_fullsized":"2023-11-24T08:06:04.054849+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:09.647409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.068746+0000","last_change":"2023-11-24T08:06:04.068746+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.068746+0000","last_undegraded":"2023-11-24T08:06:04.068746+0000","last_fullsized":"2023-11-24T08:06:04.068746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.059403+0000","last_change":"2023-11-24T08:06:03.059403+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.059403+0000","last_undegraded":"2023-11-24T08:06:03.059403+0000","last_fullsized":"2023-11-24T08:06:03.059403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.064011+0000","last_change":"2023-11-24T08:06:04.064011+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.064011+0000","last_undegraded":"2023-11-24T08:06:04.064011+0000","last_fullsized":"2023-11-24T08:06:04.064011+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T08:06:04.071810+0000","last_change":"2023-11-24T08:06:04.071810+0000","last_active":"2023-11-24T08:06:04.071810+0000","last_peered":"2023-11-24T08:06:04.071810+0000","last_clean":"2023-11-24T08:06:04.071810+0000","last_became_active":"2023-11-24T08:06:04.071196+0000","last_became_peered":"2023-11-24T08:06:04.071196+0000","last_unstale":"2023-11-24T08:06:04.071810+0000","last_undegraded":"2023-11-24T08:06:04.071810+0000","last_fullsized":"2023-11-24T08:06:04.071810+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:13.981420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.063124+0000","last_change":"2023-11-24T08:06:04.063124+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.063124+0000","last_undegraded":"2023-11-24T08:06:04.063124+0000","last_fullsized":"2023-11-24T08:06:04.063124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:38.102443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.066162+0000","last_change":"2023-11-24T08:06:04.066162+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.066162+0000","last_undegraded":"2023-11-24T08:06:04.066162+0000","last_fullsized":"2023-11-24T08:06:04.066162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.071140+0000","last_change":"2023-11-24T08:06:04.071140+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.071140+0000","last_undegraded":"2023-11-24T08:06:04.071140+0000","last_fullsized":"2023-11-24T08:06:04.071140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:14:29.594620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.077208+0000","last_change":"2023-11-24T08:06:04.077208+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.077208+0000","last_undegraded":"2023-11-24T08:06:04.077208+0000","last_fullsized":"2023-11-24T08:06:04.077208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.054740+0000","last_change":"2023-11-24T08:06:04.054740+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.054740+0000","last_undegraded":"2023-11-24T08:06:04.054740+0000","last_fullsized":"2023-11-24T08:06:04.054740+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:51.350188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.056822+0000","last_change":"2023-11-24T08:06:04.056822+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.056822+0000","last_undegraded":"2023-11-24T08:06:04.056822+0000","last_fullsized":"2023-11-24T08:06:04.056822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.054815+0000","last_change":"2023-11-24T08:06:04.054815+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.054815+0000","last_undegraded":"2023-11-24T08:06:04.054815+0000","last_fullsized":"2023-11-24T08:06:04.054815+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:01.889219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.065179+0000","last_change":"2023-11-24T08:06:04.065179+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.065179+0000","last_undegraded":"2023-11-24T08:06:04.065179+0000","last_fullsized":"2023-11-24T08:06:04.065179+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.074136+0000","last_change":"2023-11-24T08:06:04.074136+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.074136+0000","last_undegraded":"2023-11-24T08:06:04.074136+0000","last_fullsized":"2023-11-24T08:06:04.074136+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:51:33.198531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.063854+0000","last_change":"2023-11-24T08:06:04.063854+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.063854+0000","last_undegraded":"2023-11-24T08:06:04.063854+0000","last_fullsized":"2023-11-24T08:06:04.063854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.057743+0000","last_change":"2023-11-24T08:06:03.057743+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.057743+0000","last_undegraded":"2023-11-24T08:06:03.057743+0000","last_fullsized":"2023-11-24T08:06:03.057743+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.060422+0000","last_change":"2023-11-24T08:06:03.060422+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.060422+0000","last_undegraded":"2023-11-24T08:06:03.060422+0000","last_fullsized":"2023-11-24T08:06:03.060422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.068836+0000","last_change":"2023-11-24T08:06:04.068836+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.068836+0000","last_undegraded":"2023-11-24T08:06:04.068836+0000","last_fullsized":"2023-11-24T08:06:04.068836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.054696+0000","last_change":"2023-11-24T08:06:04.054696+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.054696+0000","last_undegraded":"2023-11-24T08:06:04.054696+0000","last_fullsized":"2023-11-24T08:06:04.054696+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:20.739039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.060026+0000","last_change":"2023-11-24T08:06:03.060026+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.060026+0000","last_undegraded":"2023-11-24T08:06:03.060026+0000","last_fullsized":"2023-11-24T08:06:03.060026+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.058658+0000","last_change":"2023-11-24T08:06:04.058658+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.058658+0000","last_undegraded":"2023-11-24T08:06:04.058658+0000","last_fullsized":"2023-11-24T08:06:04.058658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:56.272590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.055966+0000","last_change":"2023-11-24T08:06:04.055966+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.055966+0000","last_undegraded":"2023-11-24T08:06:04.055966+0000","last_fullsized":"2023-11-24T08:06:04.055966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.060017+0000","last_change":"2023-11-24T08:06:03.060017+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.060017+0000","last_undegraded":"2023-11-24T08:06:03.060017+0000","last_fullsized":"2023-11-24T08:06:03.060017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.056194+0000","last_change":"2023-11-24T08:06:04.056194+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.056194+0000","last_undegraded":"2023-11-24T08:06:04.056194+0000","last_fullsized":"2023-11-24T08:06:04.056194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T08:06:04.070754+0000","last_change":"2023-11-24T08:06:04.070754+0000","last_active":"2023-11-24T08:06:04.070754+0000","last_peered":"2023-11-24T08:06:04.070754+0000","last_clean":"2023-11-24T08:06:04.070754+0000","last_became_active":"2023-11-24T08:06:04.070430+0000","last_became_peered":"2023-11-24T08:06:04.070430+0000","last_unstale":"2023-11-24T08:06:04.070754+0000","last_undegraded":"2023-11-24T08:06:04.070754+0000","last_fullsized":"2023-11-24T08:06:04.070754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:04.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.068386+0000","last_change":"2023-11-24T08:06:04.068386+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.068386+0000","last_undegraded":"2023-11-24T08:06:04.068386+0000","last_fullsized":"2023-11-24T08:06:04.068386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":3,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.058513+0000","last_change":"2023-11-24T08:06:04.058513+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.058513+0000","last_undegraded":"2023-11-24T08:06:04.058513+0000","last_fullsized":"2023-11-24T08:06:04.058513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:37.712377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T08:06:04.059398+0000","last_change":"2023-11-24T08:06:04.059398+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.059398+0000","last_undegraded":"2023-11-24T08:06:04.059398+0000","last_fullsized":"2023-11-24T08:06:04.059398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2023-11-24T08:06:03.057168+0000","last_change":"2023-11-24T08:06:03.057168+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:03.057168+0000","last_undegraded":"2023-11-24T08:06:03.057168+0000","last_fullsized":"2023-11-24T08:06:03.057168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:06:04.042801+0000","last_change":"2023-11-24T08:06:04.042801+0000","last_active":"2023-11-24T08:06:04.042801+0000","last_peered":"2023-11-24T08:06:04.042801+0000","last_clean":"2023-11-24T08:06:04.042801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.042801+0000","last_undegraded":"2023-11-24T08:06:04.042801+0000","last_fullsized":"2023-11-24T08:06:04.042801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":4,"reported_epoch":43,"state":"creating+activating","last_fresh":"2023-11-24T08:06:04.080644+0000","last_change":"2023-11-24T08:06:04.073839+0000","last_active":"2023-11-24T08:06:03.028297+0000","last_peered":"2023-11-24T08:06:03.028297+0000","last_clean":"2023-11-24T08:06:03.028297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:06:04.080644+0000","last_undegraded":"2023-11-24T08:06:04.080644+0000","last_fullsized":"2023-11-24T08:06:04.080644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:39:00.682536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-24T08:06:00.055324+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:06:00.055324+0000","last_peered":"2023-11-24T08:06:00.055324+0000","last_clean":"2023-11-24T08:06:00.055324+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:06:00.055324+0000","last_undegraded":"2023-11-24T08:06:00.055324+0000","last_fullsized":"2023-11-24T08:06:00.055324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.65300000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.008},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.069,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3200000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7649999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.24,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4179999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1539999999999999,"15min":1.149},"min":{"1min":0.80600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.8660000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.125},{"interface":"front","average":{"1min":1.304,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.80300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8879999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.325}]},{"osd":3,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3120000000000001,"15min":1.3300000000000001},"min":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5109999999999999,"5min":2.524,"15min":2.524},"last":1.256},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.504,"15min":1.534},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7709999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.90800000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.613,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3799999999999999,"15min":1.3839999999999999},"min":{"1min":0.87,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0270000000000001,"5min":2.238,"15min":2.238},"last":1.51}]},{"osd":5,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.619,"15min":1.6319999999999999},"min":{"1min":1.1839999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.085},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.647,"15min":1.677},"min":{"1min":1.0489999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7250000000000001,"5min":2.944,"15min":2.944},"last":1.171}]},{"osd":6,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.7350000000000001,"15min":1.77},"min":{"1min":1.1699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.891,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.671,"15min":1.6870000000000001},"min":{"1min":1.179,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1589999999999998,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.0509999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.538,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.569},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.323,"15min":1.355},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5629999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.736}]},{"osd":1,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1499999999999999,"15min":1.1579999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.073},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.274,"15min":1.319},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.0029999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.282,"15min":1.3129999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0710000000000002,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.4750000000000001,"15min":1.526},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.5509999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.272}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.4299999999999999,"15min":1.458},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.554,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.131},{"interface":"front","average":{"1min":1.401,"5min":1.49,"15min":1.5049999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.7749999999999999,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.6779999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.724,"15min":1.79},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.157,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.472,"15min":1.4830000000000001},"min":{"1min":0.90400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.895,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.5089999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.7709999999999999,"15min":1.843},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3490000000000002,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.9179999999999999,"15min":1.986},"min":{"1min":0.85399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8400000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.3089999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.704},"min":{"1min":0.79400000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.2290000000000001,"5min":2.98,"15min":2.98},"last":1.621},{"interface":"front","average":{"1min":1.548,"5min":1.643,"15min":1.659},"min":{"1min":0.94199999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9379999999999999,"5min":2.29,"15min":2.29},"last":1.3720000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.042,"15min":1.0580000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.627,"5min":1.764,"15min":1.764},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.131,"5min":1.02,"15min":1.002},"min":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.101}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1739999999999999,"15min":1.1819999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6299999999999999,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.0669999999999999,"15min":1.0469999999999999},"min":{"1min":0.85699999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.046}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.206,"15min":1.212},"min":{"1min":0.67200000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7490000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.2330000000000001,"15min":1.218},"min":{"1min":0.78900000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.6259999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.99199999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.597,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.502},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":1.0369999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.667,"5min":1.889,"15min":1.889},"last":1.4359999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.288,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6419999999999999,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.413,"15min":1.413},"min":{"1min":1.206,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9339999999999999,"5min":2.27,"15min":2.27},"last":1.3859999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.464,"15min":1.4750000000000001},"min":{"1min":1.05,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.617,"5min":3.806,"15min":3.806},"last":1.292},{"interface":"front","average":{"1min":1.389,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.95199999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.726,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.339}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.52,"15min":1.528},"min":{"1min":1.1919999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8500000000000001,"5min":2.923,"15min":2.923},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.5309999999999999,"15min":1.5269999999999999},"min":{"1min":1.343,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7769999999999999,"5min":2.859,"15min":2.859},"last":1.546}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0529999999999999,"15min":1.046},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.129,"15min":1.127},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.89100000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.099,"15min":1.091},"min":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.81799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.95299999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.198,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6279999999999999,"5min":1.823,"15min":1.823},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.264,"15min":1.266},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.78600000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.298,"15min":1.302},"min":{"1min":0.90600000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.327,"5min":1.319,"15min":1.3169999999999999},"min":{"1min":0.98299999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6419999999999999,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.97199999999999998}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.304,"15min":1.304},"min":{"1min":0.86899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.702,"5min":1.944,"15min":1.944},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.337,"5min":1.361,"15min":1.365},"min":{"1min":0.96899999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.696,"5min":2.012,"15min":2.012},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3700000000000001,"15min":1.3660000000000001},"min":{"1min":0.91300000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.8560000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5129999999999999,"15min":1.5149999999999999},"min":{"1min":1.198,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0129999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.1100000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.573,"15min":1.5649999999999999},"min":{"1min":1.0880000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.5920000000000001,"5min":2.722,"15min":2.722},"last":0.87}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.82199999999999995,"15min":0.81899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.516},{"interface":"front","average":{"1min":0.874,"5min":0.94299999999999995,"15min":0.95499999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.1379999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.129,"15min":1.169},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6359999999999999,"5min":3.246,"15min":3.246},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.97299999999999998,"15min":0.96699999999999997},"min":{"1min":0.441,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.2589999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1799999999999999,"15min":1.196},"min":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.718,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.075},{"interface":"front","average":{"1min":1.038,"5min":1.147,"15min":1.165},"min":{"1min":0.54000000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5900000000000001,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.399}]},{"osd":4,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.93999999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6879999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.671},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.379,"15min":1.3979999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.73,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.6339999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.448,"15min":1.468},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9870000000000001,"5min":2.395,"15min":2.395},"last":1.9430000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.304,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2410000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.7310000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4299999999999999,"15min":1.4350000000000001},"min":{"1min":0.83199999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.1600000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4159999999999999,"15min":1.4239999999999999},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.946,"5min":1.996,"15min":1.996},"last":2.1259999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4430000000000001,"15min":1.4510000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.994},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.8340000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":1.0209999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.272,"5min":2.133,"15min":2.133},"last":1.212}]},{"osd":1,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.194,"15min":1.242},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.371,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.048,"15min":1.0840000000000001},"min":{"1min":0.5,"5min":0.438,"15min":0.438},"max":{"1min":1.208,"5min":1.998,"15min":1.998},"last":1.4550000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3089999999999999,"15min":1.333},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9259999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.175,"15min":1.204},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.552,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.5389999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3140000000000001,"15min":1.343},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.631,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.798,"5min":3.161,"15min":3.161},"last":1.089}]},{"osd":5,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3640000000000001,"15min":1.3999999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7649999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.627},{"interface":"front","average":{"1min":1.196,"5min":1.391,"15min":1.423},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7549999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.25}]},{"osd":6,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.5620000000000001,"15min":1.6080000000000001},"min":{"1min":0.73899999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.825,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.857,"5min":2.702,"15min":2.702},"last":1.786}]},{"osd":7,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.5009999999999999,"15min":1.5309999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.3980000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.6180000000000001,"15min":1.6619999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.9199999999999999,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.395}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1919999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.105,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.0069999999999999,"15min":1.0389999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2230000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.214,"15min":1.2410000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.353,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.161}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2569999999999999,"15min":1.2769999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6319999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.897},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.2110000000000001,"15min":1.2430000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.538,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.6890000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.242,"15min":1.2549999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.512,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.512},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.4390000000000001,"15min":1.4610000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6459999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.4490000000000001,"15min":1.4650000000000001},"min":{"1min":0.85699999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9590000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.468,"15min":1.484},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.746,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.655}]},{"osd":6,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.45,"15min":1.464},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9019999999999999,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4690000000000001,"15min":1.4910000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.821,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.8420000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.548,"15min":1.5700000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.744,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.738},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5800000000000001,"15min":1.599},"min":{"1min":0.94499999999999995,"5min":0.872,"15min":0.872},"max":{"1min":1.8160000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.788}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1081344,"data_stored":785276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.367,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.93000000000000005},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.05,"15min":1.054},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.706,"5min":2.29,"15min":2.29},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":0.93700000000000006},"min":{"1min":0.51000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.5049999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":0.498}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.032,"15min":1.028},"min":{"1min":0.60499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6439999999999999,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8640000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.256,"15min":1.254},"min":{"1min":0.95899999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.704,"5min":3.302,"15min":3.302},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.454,"15min":1.466},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.2389999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.119}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":0.89400000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.0590000000000002,"5min":3.101,"15min":3.101},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.4359999999999999,"15min":1.4319999999999999},"min":{"1min":0.90100000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.1779999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.2010000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3999999999999999,"15min":1.4159999999999999},"min":{"1min":0.75,"5min":0.749,"15min":0.749},"max":{"1min":2.1819999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4970000000000001,"15min":1.4930000000000001},"min":{"1min":1.0700000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1030000000000002,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.0840000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.5309999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1600000000000001,"5min":3.823,"15min":3.823},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.468,"15min":1.48},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2610000000000001,"5min":3.371,"15min":3.371},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:06:10.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:06:10.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.361+0000 7f580d98a640 1 Processor -- start 2023-11-24T08:06:10.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.361+0000 7f580d98a640 1 -- start start 2023-11-24T08:06:10.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.363+0000 7f580d98a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f580815d5b0 0x7f58080d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.363+0000 7f580d98a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58080d14a0 0x7f58080d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.363+0000 7f580d98a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58080cf0c0 con 0x7f58080cf7a0 2023-11-24T08:06:10.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.363+0000 7f580d98a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58080b01b0 con 0x7f580815d5b0 2023-11-24T08:06:10.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.363+0000 7f580d98a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580815d990 con 0x7f58080d14a0 2023-11-24T08:06:10.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.363+0000 7f580d189640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f58080cf7a0 0x7f58080cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52006/0 (socket says 172.21.15.64:52006) 2023-11-24T08:06:10.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.363+0000 7f580d189640 1 -- 172.21.15.64:0/2711785578 learned_addr learned my addr 172.21.15.64:0/2711785578 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f580c988640 1 --2- 172.21.15.64:0/2711785578 >> v2:172.21.15.91:3300/0 conn(0x7f580815d5b0 0x7f58080d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f5807fff640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58080d14a0 0x7f58080d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187493145 0 0) 0x7f58080cf0c0 con 0x7f58080cf7a0 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57f4003520 con 0x7f58080cf7a0 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1709446560 0 0) 0x7f57f4003520 con 0x7f58080cf7a0 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58080d14a0 msgr2=0x7f58080d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f58077fe640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58080d14a0 0x7f58080d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 >> v2:172.21.15.91:3300/0 conn(0x7f580815d5b0 msgr2=0x7f58080d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f58077fe640 1 --2- 172.21.15.64:0/2711785578 >> v2:172.21.15.91:3300/0 conn(0x7f580815d5b0 0x7f58080d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.364+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080d1e40 con 0x7f58080cf7a0 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f5800001670 con 0x7f58080cf7a0 2023-11-24T08:06:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 >> v1:172.21.15.64:6789/0 conn(0x7f58080cf7a0 legacy=0x7f58080cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f58077fe640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f40039e0 0x7f57f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f58077fe640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58080d14a0 0x7f57f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f58077fe640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57f4004d00 0x7f57f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b01b0 con 0x7f57f4004d00 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580815d990 con 0x7f57f40039e0 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f400b980 con 0x7f58080d14a0 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f580d189640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57f4004d00 0x7f57f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f580c988640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f40039e0 0x7f57f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f5807fff640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58080d14a0 0x7f57f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.365+0000 7f580d189640 1 -- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58080d14a0 msgr2=0x7f57f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.366+0000 7f580d189640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58080d14a0 0x7f57f40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.366+0000 7f580d189640 1 -- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f40039e0 msgr2=0x7f57f4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.366+0000 7f580d189640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f40039e0 0x7f57f4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.366+0000 7f580d189640 1 -- 172.21.15.64:0/2711785578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4003520 con 0x7f57f4004d00 2023-11-24T08:06:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.367+0000 7f580d189640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57f4004d00 0x7f57f4005180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f580000c240 tx=0x7f57f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a722b4bf167469 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.367+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580002cb00 con 0x7f57f4004d00 2023-11-24T08:06:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.367+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580002cc60 con 0x7f57f4004d00 2023-11-24T08:06:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.367+0000 7f58077fe640 1 -- 172.21.15.64:0/2711785578 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800005b10 con 0x7f57f4004d00 2023-11-24T08:06:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.368+0000 7f580d98a640 1 -- 172.21.15.64:0/2711785578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57f4004d00 msgr2=0x7f57f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.368+0000 7f580d98a640 1 --2- 172.21.15.64:0/2711785578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57f4004d00 0x7f57f4005180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f580000c240 tx=0x7f57f40057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.368+0000 7f580d189640 1 -- 172.21.15.64:0/2711785578 reap_dead start 2023-11-24T08:06:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.368+0000 7f580d98a640 1 -- 172.21.15.64:0/2711785578 shutdown_connections 2023-11-24T08:06:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.368+0000 7f580d98a640 1 -- 172.21.15.64:0/2711785578 >> 172.21.15.64:0/2711785578 conn(0x7f580809c3b0 msgr2=0x7f58080d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.368+0000 7f580d98a640 1 -- 172.21.15.64:0/2711785578 shutdown_connections 2023-11-24T08:06:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.368+0000 7f580d98a640 1 -- 172.21.15.64:0/2711785578 wait complete. 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.370+0000 7f580d98a640 1 Processor -- start 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.370+0000 7f580d98a640 1 -- start start 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f580d98a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f4004d00 0x7f58080b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f580d98a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580815f370 0x7f580815f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f580d98a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58081510e0 0x7f5808151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f580d98a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081603b0 con 0x7f580815f370 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f580d98a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808151aa0 con 0x7f57f4004d00 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f580d98a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580815fdf0 con 0x7f58081510e0 2023-11-24T08:06:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f5807fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580815f370 0x7f580815f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f5807fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580815f370 0x7f580815f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43072/0 (socket says 172.21.15.64:43072) 2023-11-24T08:06:10.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.371+0000 7f5807fff640 1 -- 172.21.15.64:0/213239018 learned_addr learned my addr 172.21.15.64:0/213239018 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:10.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f580d189640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58081510e0 0x7f5808151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f580c988640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f4004d00 0x7f58080b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f5807fff640 1 -- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58081510e0 msgr2=0x7f5808151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f5807fff640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58081510e0 0x7f5808151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f5807fff640 1 -- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f4004d00 msgr2=0x7f58080b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f5807fff640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f4004d00 0x7f58080b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f5807fff640 1 -- 172.21.15.64:0/213239018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58081600d0 con 0x7f580815f370 2023-11-24T08:06:10.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f5807fff640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580815f370 0x7f580815f7f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f57f8007f30 tx=0x7f57f8011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:10.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f58057fa640 1 -- 172.21.15.64:0/213239018 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f800e030 con 0x7f580815f370 2023-11-24T08:06:10.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f58057fa640 1 -- 172.21.15.64:0/213239018 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f8016470 con 0x7f580815f370 2023-11-24T08:06:10.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58080c7c00 con 0x7f580815f370 2023-11-24T08:06:10.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f58057fa640 1 -- 172.21.15.64:0/213239018 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8016820 con 0x7f580815f370 2023-11-24T08:06:10.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.372+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080b5280 con 0x7f580815f370 2023-11-24T08:06:10.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.374+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58080c7a60 con 0x7f580815f370 2023-11-24T08:06:10.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.375+0000 7f58057fa640 1 -- 172.21.15.64:0/213239018 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f57f8016980 con 0x7f580815f370 2023-11-24T08:06:10.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.375+0000 7f58057fa640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f57e806af00 0x7f57e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.376+0000 7f58057fa640 1 -- 172.21.15.64:0/213239018 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f800f070 con 0x7f580815f370 2023-11-24T08:06:10.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.376+0000 7f580c988640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f57e806af00 0x7f57e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.377+0000 7f580c988640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f57e806af00 0x7f57e806d3c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f57f00099d0 tx=0x7f57f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.380+0000 7f58057fa640 1 -- 172.21.15.64:0/213239018 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58080c7a60 con 0x7f580815f370 2023-11-24T08:06:10.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.569+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58080afe40 con 0x7f57e806af00 2023-11-24T08:06:10.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.592+0000 7f58057fa640 1 -- 172.21.15.64:0/213239018 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286656 (secure 0 0 0) 0x7f57f0012950 con 0x7f57e806af00 2023-11-24T08:06:10.597 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:10.599 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f57e806af00 msgr2=0x7f57e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f57e806af00 0x7f57e806d3c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f57f00099d0 tx=0x7f57f0008040 comp rx=0 tx=0).stop 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580815f370 msgr2=0x7f580815f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580815f370 0x7f580815f7f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f57f8007f30 tx=0x7f57f8011660 comp rx=0 tx=0).stop 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 shutdown_connections 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f57e806af00 0x7f57e806d3c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f58081510e0 0x7f5808151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580815f370 0x7f580815f7f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 --2- 172.21.15.64:0/213239018 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57f4004d00 0x7f58080b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 >> 172.21.15.64:0/213239018 conn(0x7f580809c3b0 msgr2=0x7f580815dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 shutdown_connections 2023-11-24T08:06:10.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.598+0000 7f580d98a640 1 -- 172.21.15.64:0/213239018 wait complete. 2023-11-24T08:06:10.619 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-11-24T08:06:10.063018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221508,"kb_used_data":5348,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753212,"statfs":{"total":773094113280,"available":772867289088,"internally_reserved":0,"allocated":5476352,"data_stored":2906689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.011005"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253626+0000","last_change":"2023-11-24T08:06:05.088466+0000","last_active":"2023-11-24T08:06:06.253626+0000","last_peered":"2023-11-24T08:06:06.253626+0000","last_clean":"2023-11-24T08:06:06.253626+0000","last_became_active":"2023-11-24T08:06:05.088184+0000","last_became_peered":"2023-11-24T08:06:05.088184+0000","last_unstale":"2023-11-24T08:06:06.253626+0000","last_undegraded":"2023-11-24T08:06:06.253626+0000","last_fullsized":"2023-11-24T08:06:06.253626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:55.876487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251929+0000","last_change":"2023-11-24T08:06:05.094999+0000","last_active":"2023-11-24T08:06:06.251929+0000","last_peered":"2023-11-24T08:06:06.251929+0000","last_clean":"2023-11-24T08:06:06.251929+0000","last_became_active":"2023-11-24T08:06:05.094688+0000","last_became_peered":"2023-11-24T08:06:05.094688+0000","last_unstale":"2023-11-24T08:06:06.251929+0000","last_undegraded":"2023-11-24T08:06:06.251929+0000","last_fullsized":"2023-11-24T08:06:06.251929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088777+0000","last_change":"2023-11-24T08:06:05.088653+0000","last_active":"2023-11-24T08:06:06.088777+0000","last_peered":"2023-11-24T08:06:06.088777+0000","last_clean":"2023-11-24T08:06:06.088777+0000","last_became_active":"2023-11-24T08:06:05.087897+0000","last_became_peered":"2023-11-24T08:06:05.087897+0000","last_unstale":"2023-11-24T08:06:06.088777+0000","last_undegraded":"2023-11-24T08:06:06.088777+0000","last_fullsized":"2023-11-24T08:06:06.088777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252772+0000","last_change":"2023-11-24T08:06:05.097244+0000","last_active":"2023-11-24T08:06:06.252772+0000","last_peered":"2023-11-24T08:06:06.252772+0000","last_clean":"2023-11-24T08:06:06.252772+0000","last_became_active":"2023-11-24T08:06:05.096919+0000","last_became_peered":"2023-11-24T08:06:05.096919+0000","last_unstale":"2023-11-24T08:06:06.252772+0000","last_undegraded":"2023-11-24T08:06:06.252772+0000","last_fullsized":"2023-11-24T08:06:06.252772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:05.052808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251328+0000","last_change":"2023-11-24T08:06:05.095396+0000","last_active":"2023-11-24T08:06:06.251328+0000","last_peered":"2023-11-24T08:06:06.251328+0000","last_clean":"2023-11-24T08:06:06.251328+0000","last_became_active":"2023-11-24T08:06:05.094813+0000","last_became_peered":"2023-11-24T08:06:05.094813+0000","last_unstale":"2023-11-24T08:06:06.251328+0000","last_undegraded":"2023-11-24T08:06:06.251328+0000","last_fullsized":"2023-11-24T08:06:06.251328+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251253+0000","last_change":"2023-11-24T08:06:05.084536+0000","last_active":"2023-11-24T08:06:06.251253+0000","last_peered":"2023-11-24T08:06:06.251253+0000","last_clean":"2023-11-24T08:06:06.251253+0000","last_became_active":"2023-11-24T08:06:05.084223+0000","last_became_peered":"2023-11-24T08:06:05.084223+0000","last_unstale":"2023-11-24T08:06:06.251253+0000","last_undegraded":"2023-11-24T08:06:06.251253+0000","last_fullsized":"2023-11-24T08:06:06.251253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:03.061151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252752+0000","last_change":"2023-11-24T08:06:05.079934+0000","last_active":"2023-11-24T08:06:06.252752+0000","last_peered":"2023-11-24T08:06:06.252752+0000","last_clean":"2023-11-24T08:06:06.252752+0000","last_became_active":"2023-11-24T08:06:05.079195+0000","last_became_peered":"2023-11-24T08:06:05.079195+0000","last_unstale":"2023-11-24T08:06:06.252752+0000","last_undegraded":"2023-11-24T08:06:06.252752+0000","last_fullsized":"2023-11-24T08:06:06.252752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:28.502892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252409+0000","last_change":"2023-11-24T08:06:05.096845+0000","last_active":"2023-11-24T08:06:06.252409+0000","last_peered":"2023-11-24T08:06:06.252409+0000","last_clean":"2023-11-24T08:06:06.252409+0000","last_became_active":"2023-11-24T08:06:05.094451+0000","last_became_peered":"2023-11-24T08:06:05.094451+0000","last_unstale":"2023-11-24T08:06:06.252409+0000","last_undegraded":"2023-11-24T08:06:06.252409+0000","last_fullsized":"2023-11-24T08:06:06.252409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:45.526567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252291+0000","last_change":"2023-11-24T08:06:05.096637+0000","last_active":"2023-11-24T08:06:06.252291+0000","last_peered":"2023-11-24T08:06:06.252291+0000","last_clean":"2023-11-24T08:06:06.252291+0000","last_became_active":"2023-11-24T08:06:05.096340+0000","last_became_peered":"2023-11-24T08:06:05.096340+0000","last_unstale":"2023-11-24T08:06:06.252291+0000","last_undegraded":"2023-11-24T08:06:06.252291+0000","last_fullsized":"2023-11-24T08:06:06.252291+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:34.236261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252685+0000","last_change":"2023-11-24T08:06:05.102398+0000","last_active":"2023-11-24T08:06:06.252685+0000","last_peered":"2023-11-24T08:06:06.252685+0000","last_clean":"2023-11-24T08:06:06.252685+0000","last_became_active":"2023-11-24T08:06:05.101720+0000","last_became_peered":"2023-11-24T08:06:05.101720+0000","last_unstale":"2023-11-24T08:06:06.252685+0000","last_undegraded":"2023-11-24T08:06:06.252685+0000","last_fullsized":"2023-11-24T08:06:06.252685+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:34.594891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252477+0000","last_change":"2023-11-24T08:06:05.097295+0000","last_active":"2023-11-24T08:06:06.252477+0000","last_peered":"2023-11-24T08:06:06.252477+0000","last_clean":"2023-11-24T08:06:06.252477+0000","last_became_active":"2023-11-24T08:06:05.095953+0000","last_became_peered":"2023-11-24T08:06:05.095953+0000","last_unstale":"2023-11-24T08:06:06.252477+0000","last_undegraded":"2023-11-24T08:06:06.252477+0000","last_fullsized":"2023-11-24T08:06:06.252477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:30.368708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251480+0000","last_change":"2023-11-24T08:06:05.106405+0000","last_active":"2023-11-24T08:06:06.251480+0000","last_peered":"2023-11-24T08:06:06.251480+0000","last_clean":"2023-11-24T08:06:06.251480+0000","last_became_active":"2023-11-24T08:06:05.106098+0000","last_became_peered":"2023-11-24T08:06:05.106098+0000","last_unstale":"2023-11-24T08:06:06.251480+0000","last_undegraded":"2023-11-24T08:06:06.251480+0000","last_fullsized":"2023-11-24T08:06:06.251480+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:37.368224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248524+0000","last_change":"2023-11-24T08:06:05.087198+0000","last_active":"2023-11-24T08:06:06.248524+0000","last_peered":"2023-11-24T08:06:06.248524+0000","last_clean":"2023-11-24T08:06:06.248524+0000","last_became_active":"2023-11-24T08:06:05.086737+0000","last_became_peered":"2023-11-24T08:06:05.086737+0000","last_unstale":"2023-11-24T08:06:06.248524+0000","last_undegraded":"2023-11-24T08:06:06.248524+0000","last_fullsized":"2023-11-24T08:06:06.248524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:29.327265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252188+0000","last_change":"2023-11-24T08:06:05.100506+0000","last_active":"2023-11-24T08:06:06.252188+0000","last_peered":"2023-11-24T08:06:06.252188+0000","last_clean":"2023-11-24T08:06:06.252188+0000","last_became_active":"2023-11-24T08:06:05.094270+0000","last_became_peered":"2023-11-24T08:06:05.094270+0000","last_unstale":"2023-11-24T08:06:06.252188+0000","last_undegraded":"2023-11-24T08:06:06.252188+0000","last_fullsized":"2023-11-24T08:06:06.252188+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250948+0000","last_change":"2023-11-24T08:06:05.091977+0000","last_active":"2023-11-24T08:06:06.250948+0000","last_peered":"2023-11-24T08:06:06.250948+0000","last_clean":"2023-11-24T08:06:06.250948+0000","last_became_active":"2023-11-24T08:06:05.091622+0000","last_became_peered":"2023-11-24T08:06:05.091622+0000","last_unstale":"2023-11-24T08:06:06.250948+0000","last_undegraded":"2023-11-24T08:06:06.250948+0000","last_fullsized":"2023-11-24T08:06:06.250948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:31.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087597+0000","last_change":"2023-11-24T08:06:05.090075+0000","last_active":"2023-11-24T08:06:06.087597+0000","last_peered":"2023-11-24T08:06:06.087597+0000","last_clean":"2023-11-24T08:06:06.087597+0000","last_became_active":"2023-11-24T08:06:05.089117+0000","last_became_peered":"2023-11-24T08:06:05.089117+0000","last_unstale":"2023-11-24T08:06:06.087597+0000","last_undegraded":"2023-11-24T08:06:06.087597+0000","last_fullsized":"2023-11-24T08:06:06.087597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:30.794390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.090292+0000","last_change":"2023-11-24T08:06:05.098942+0000","last_active":"2023-11-24T08:06:06.090292+0000","last_peered":"2023-11-24T08:06:06.090292+0000","last_clean":"2023-11-24T08:06:06.090292+0000","last_became_active":"2023-11-24T08:06:05.098583+0000","last_became_peered":"2023-11-24T08:06:05.098583+0000","last_unstale":"2023-11-24T08:06:06.090292+0000","last_undegraded":"2023-11-24T08:06:06.090292+0000","last_fullsized":"2023-11-24T08:06:06.090292+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089664+0000","last_change":"2023-11-24T08:06:05.095908+0000","last_active":"2023-11-24T08:06:06.089664+0000","last_peered":"2023-11-24T08:06:06.089664+0000","last_clean":"2023-11-24T08:06:06.089664+0000","last_became_active":"2023-11-24T08:06:05.095581+0000","last_became_peered":"2023-11-24T08:06:05.095581+0000","last_unstale":"2023-11-24T08:06:06.089664+0000","last_undegraded":"2023-11-24T08:06:06.089664+0000","last_fullsized":"2023-11-24T08:06:06.089664+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089518+0000","last_change":"2023-11-24T08:06:05.093941+0000","last_active":"2023-11-24T08:06:06.089518+0000","last_peered":"2023-11-24T08:06:06.089518+0000","last_clean":"2023-11-24T08:06:06.089518+0000","last_became_active":"2023-11-24T08:06:05.092980+0000","last_became_peered":"2023-11-24T08:06:05.092980+0000","last_unstale":"2023-11-24T08:06:06.089518+0000","last_undegraded":"2023-11-24T08:06:06.089518+0000","last_fullsized":"2023-11-24T08:06:06.089518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:47.605436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248776+0000","last_change":"2023-11-24T08:06:05.096889+0000","last_active":"2023-11-24T08:06:06.248776+0000","last_peered":"2023-11-24T08:06:06.248776+0000","last_clean":"2023-11-24T08:06:06.248776+0000","last_became_active":"2023-11-24T08:06:05.096574+0000","last_became_peered":"2023-11-24T08:06:05.096574+0000","last_unstale":"2023-11-24T08:06:06.248776+0000","last_undegraded":"2023-11-24T08:06:06.248776+0000","last_fullsized":"2023-11-24T08:06:06.248776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:05.908617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251413+0000","last_change":"2023-11-24T08:06:05.092598+0000","last_active":"2023-11-24T08:06:06.251413+0000","last_peered":"2023-11-24T08:06:06.251413+0000","last_clean":"2023-11-24T08:06:06.251413+0000","last_became_active":"2023-11-24T08:06:05.088450+0000","last_became_peered":"2023-11-24T08:06:05.088450+0000","last_unstale":"2023-11-24T08:06:06.251413+0000","last_undegraded":"2023-11-24T08:06:06.251413+0000","last_fullsized":"2023-11-24T08:06:06.251413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:13.526610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252225+0000","last_change":"2023-11-24T08:06:05.098582+0000","last_active":"2023-11-24T08:06:06.252225+0000","last_peered":"2023-11-24T08:06:06.252225+0000","last_clean":"2023-11-24T08:06:06.252225+0000","last_became_active":"2023-11-24T08:06:05.094693+0000","last_became_peered":"2023-11-24T08:06:05.094693+0000","last_unstale":"2023-11-24T08:06:06.252225+0000","last_undegraded":"2023-11-24T08:06:06.252225+0000","last_fullsized":"2023-11-24T08:06:06.252225+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:29.573995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087957+0000","last_change":"2023-11-24T08:06:05.085854+0000","last_active":"2023-11-24T08:06:06.087957+0000","last_peered":"2023-11-24T08:06:06.087957+0000","last_clean":"2023-11-24T08:06:06.087957+0000","last_became_active":"2023-11-24T08:06:05.085576+0000","last_became_peered":"2023-11-24T08:06:05.085576+0000","last_unstale":"2023-11-24T08:06:06.087957+0000","last_undegraded":"2023-11-24T08:06:06.087957+0000","last_fullsized":"2023-11-24T08:06:06.087957+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:21.753543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087840+0000","last_change":"2023-11-24T08:06:05.084050+0000","last_active":"2023-11-24T08:06:06.087840+0000","last_peered":"2023-11-24T08:06:06.087840+0000","last_clean":"2023-11-24T08:06:06.087840+0000","last_became_active":"2023-11-24T08:06:05.083596+0000","last_became_peered":"2023-11-24T08:06:05.083596+0000","last_unstale":"2023-11-24T08:06:06.087840+0000","last_undegraded":"2023-11-24T08:06:06.087840+0000","last_fullsized":"2023-11-24T08:06:06.087840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:12.059275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089365+0000","last_change":"2023-11-24T08:06:05.099351+0000","last_active":"2023-11-24T08:06:06.089365+0000","last_peered":"2023-11-24T08:06:06.089365+0000","last_clean":"2023-11-24T08:06:06.089365+0000","last_became_active":"2023-11-24T08:06:05.097619+0000","last_became_peered":"2023-11-24T08:06:05.097619+0000","last_unstale":"2023-11-24T08:06:06.089365+0000","last_undegraded":"2023-11-24T08:06:06.089365+0000","last_fullsized":"2023-11-24T08:06:06.089365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:21.615036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248405+0000","last_change":"2023-11-24T08:06:05.094252+0000","last_active":"2023-11-24T08:06:06.248405+0000","last_peered":"2023-11-24T08:06:06.248405+0000","last_clean":"2023-11-24T08:06:06.248405+0000","last_became_active":"2023-11-24T08:06:05.093924+0000","last_became_peered":"2023-11-24T08:06:05.093924+0000","last_unstale":"2023-11-24T08:06:06.248405+0000","last_undegraded":"2023-11-24T08:06:06.248405+0000","last_fullsized":"2023-11-24T08:06:06.248405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:19.995373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089262+0000","last_change":"2023-11-24T08:06:05.094578+0000","last_active":"2023-11-24T08:06:06.089262+0000","last_peered":"2023-11-24T08:06:06.089262+0000","last_clean":"2023-11-24T08:06:06.089262+0000","last_became_active":"2023-11-24T08:06:05.094329+0000","last_became_peered":"2023-11-24T08:06:05.094329+0000","last_unstale":"2023-11-24T08:06:06.089262+0000","last_undegraded":"2023-11-24T08:06:06.089262+0000","last_fullsized":"2023-11-24T08:06:06.089262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:37.368224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251627+0000","last_change":"2023-11-24T08:06:05.102312+0000","last_active":"2023-11-24T08:06:06.251627+0000","last_peered":"2023-11-24T08:06:06.251627+0000","last_clean":"2023-11-24T08:06:06.251627+0000","last_became_active":"2023-11-24T08:06:05.102050+0000","last_became_peered":"2023-11-24T08:06:05.102050+0000","last_unstale":"2023-11-24T08:06:06.251627+0000","last_undegraded":"2023-11-24T08:06:06.251627+0000","last_fullsized":"2023-11-24T08:06:06.251627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088476+0000","last_change":"2023-11-24T08:06:05.093991+0000","last_active":"2023-11-24T08:06:06.088476+0000","last_peered":"2023-11-24T08:06:06.088476+0000","last_clean":"2023-11-24T08:06:06.088476+0000","last_became_active":"2023-11-24T08:06:05.093653+0000","last_became_peered":"2023-11-24T08:06:05.093653+0000","last_unstale":"2023-11-24T08:06:06.088476+0000","last_undegraded":"2023-11-24T08:06:06.088476+0000","last_fullsized":"2023-11-24T08:06:06.088476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:02.903723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087380+0000","last_change":"2023-11-24T08:06:05.095755+0000","last_active":"2023-11-24T08:06:06.087380+0000","last_peered":"2023-11-24T08:06:06.087380+0000","last_clean":"2023-11-24T08:06:06.087380+0000","last_became_active":"2023-11-24T08:06:05.095466+0000","last_became_peered":"2023-11-24T08:06:05.095466+0000","last_unstale":"2023-11-24T08:06:06.087380+0000","last_undegraded":"2023-11-24T08:06:06.087380+0000","last_fullsized":"2023-11-24T08:06:06.087380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:47.605436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251015+0000","last_change":"2023-11-24T08:06:05.087774+0000","last_active":"2023-11-24T08:06:06.251015+0000","last_peered":"2023-11-24T08:06:06.251015+0000","last_clean":"2023-11-24T08:06:06.251015+0000","last_became_active":"2023-11-24T08:06:05.087486+0000","last_became_peered":"2023-11-24T08:06:05.087486+0000","last_unstale":"2023-11-24T08:06:06.251015+0000","last_undegraded":"2023-11-24T08:06:06.251015+0000","last_fullsized":"2023-11-24T08:06:06.251015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:46.775040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248159+0000","last_change":"2023-11-24T08:06:05.093356+0000","last_active":"2023-11-24T08:06:06.248159+0000","last_peered":"2023-11-24T08:06:06.248159+0000","last_clean":"2023-11-24T08:06:06.248159+0000","last_became_active":"2023-11-24T08:06:05.093036+0000","last_became_peered":"2023-11-24T08:06:05.093036+0000","last_unstale":"2023-11-24T08:06:06.248159+0000","last_undegraded":"2023-11-24T08:06:06.248159+0000","last_fullsized":"2023-11-24T08:06:06.248159+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:44.380303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.083398+0000","last_change":"2023-11-24T08:06:05.084537+0000","last_active":"2023-11-24T08:06:06.083398+0000","last_peered":"2023-11-24T08:06:06.083398+0000","last_clean":"2023-11-24T08:06:06.083398+0000","last_became_active":"2023-11-24T08:06:05.084268+0000","last_became_peered":"2023-11-24T08:06:05.084268+0000","last_unstale":"2023-11-24T08:06:06.083398+0000","last_undegraded":"2023-11-24T08:06:06.083398+0000","last_fullsized":"2023-11-24T08:06:06.083398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:52.364691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120766+0000","last_change":"2023-11-24T08:06:04.082144+0000","last_active":"2023-11-24T08:06:05.120766+0000","last_peered":"2023-11-24T08:06:05.120766+0000","last_clean":"2023-11-24T08:06:05.120766+0000","last_became_active":"2023-11-24T08:06:04.081510+0000","last_became_peered":"2023-11-24T08:06:04.081510+0000","last_unstale":"2023-11-24T08:06:05.120766+0000","last_undegraded":"2023-11-24T08:06:05.120766+0000","last_fullsized":"2023-11-24T08:06:05.120766+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:44.512063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089609+0000","last_change":"2023-11-24T08:06:05.097573+0000","last_active":"2023-11-24T08:06:06.089609+0000","last_peered":"2023-11-24T08:06:06.089609+0000","last_clean":"2023-11-24T08:06:06.089609+0000","last_became_active":"2023-11-24T08:06:05.094728+0000","last_became_peered":"2023-11-24T08:06:05.094728+0000","last_unstale":"2023-11-24T08:06:06.089609+0000","last_undegraded":"2023-11-24T08:06:06.089609+0000","last_fullsized":"2023-11-24T08:06:06.089609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.114292+0000","last_change":"2023-11-24T08:06:04.084082+0000","last_active":"2023-11-24T08:06:05.114292+0000","last_peered":"2023-11-24T08:06:05.114292+0000","last_clean":"2023-11-24T08:06:05.114292+0000","last_became_active":"2023-11-24T08:06:04.083299+0000","last_became_peered":"2023-11-24T08:06:04.083299+0000","last_unstale":"2023-11-24T08:06:05.114292+0000","last_undegraded":"2023-11-24T08:06:05.114292+0000","last_fullsized":"2023-11-24T08:06:05.114292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251872+0000","last_change":"2023-11-24T08:06:05.096613+0000","last_active":"2023-11-24T08:06:06.251872+0000","last_peered":"2023-11-24T08:06:06.251872+0000","last_clean":"2023-11-24T08:06:06.251872+0000","last_became_active":"2023-11-24T08:06:05.096353+0000","last_became_peered":"2023-11-24T08:06:05.096353+0000","last_unstale":"2023-11-24T08:06:06.251872+0000","last_undegraded":"2023-11-24T08:06:06.251872+0000","last_fullsized":"2023-11-24T08:06:06.251872+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.115949+0000","last_change":"2023-11-24T08:06:04.087280+0000","last_active":"2023-11-24T08:06:05.115949+0000","last_peered":"2023-11-24T08:06:05.115949+0000","last_clean":"2023-11-24T08:06:05.115949+0000","last_became_active":"2023-11-24T08:06:04.086942+0000","last_became_peered":"2023-11-24T08:06:04.086942+0000","last_unstale":"2023-11-24T08:06:05.115949+0000","last_undegraded":"2023-11-24T08:06:05.115949+0000","last_fullsized":"2023-11-24T08:06:05.115949+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.083315+0000","last_change":"2023-11-24T08:06:05.086813+0000","last_active":"2023-11-24T08:06:06.083315+0000","last_peered":"2023-11-24T08:06:06.083315+0000","last_clean":"2023-11-24T08:06:06.083315+0000","last_became_active":"2023-11-24T08:06:05.086506+0000","last_became_peered":"2023-11-24T08:06:05.086506+0000","last_unstale":"2023-11-24T08:06:06.083315+0000","last_undegraded":"2023-11-24T08:06:06.083315+0000","last_fullsized":"2023-11-24T08:06:06.083315+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:21.753543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122782+0000","last_change":"2023-11-24T08:06:04.094159+0000","last_active":"2023-11-24T08:06:05.122782+0000","last_peered":"2023-11-24T08:06:05.122782+0000","last_clean":"2023-11-24T08:06:05.122782+0000","last_became_active":"2023-11-24T08:06:04.093809+0000","last_became_peered":"2023-11-24T08:06:04.093809+0000","last_unstale":"2023-11-24T08:06:05.122782+0000","last_undegraded":"2023-11-24T08:06:05.122782+0000","last_fullsized":"2023-11-24T08:06:05.122782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:54.440803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087650+0000","last_change":"2023-11-24T08:06:05.101827+0000","last_active":"2023-11-24T08:06:06.087650+0000","last_peered":"2023-11-24T08:06:06.087650+0000","last_clean":"2023-11-24T08:06:06.087650+0000","last_became_active":"2023-11-24T08:06:05.101501+0000","last_became_peered":"2023-11-24T08:06:05.101501+0000","last_unstale":"2023-11-24T08:06:06.087650+0000","last_undegraded":"2023-11-24T08:06:06.087650+0000","last_fullsized":"2023-11-24T08:06:06.087650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120725+0000","last_change":"2023-11-24T08:06:04.090249+0000","last_active":"2023-11-24T08:06:05.120725+0000","last_peered":"2023-11-24T08:06:05.120725+0000","last_clean":"2023-11-24T08:06:05.120725+0000","last_became_active":"2023-11-24T08:06:04.089893+0000","last_became_peered":"2023-11-24T08:06:04.089893+0000","last_unstale":"2023-11-24T08:06:05.120725+0000","last_undegraded":"2023-11-24T08:06:05.120725+0000","last_fullsized":"2023-11-24T08:06:05.120725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:27.488388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252382+0000","last_change":"2023-11-24T08:06:05.093052+0000","last_active":"2023-11-24T08:06:06.252382+0000","last_peered":"2023-11-24T08:06:06.252382+0000","last_clean":"2023-11-24T08:06:06.252382+0000","last_became_active":"2023-11-24T08:06:05.091759+0000","last_became_peered":"2023-11-24T08:06:05.091759+0000","last_unstale":"2023-11-24T08:06:06.252382+0000","last_undegraded":"2023-11-24T08:06:06.252382+0000","last_fullsized":"2023-11-24T08:06:06.252382+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:28.502892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.112139+0000","last_change":"2023-11-24T08:06:04.087131+0000","last_active":"2023-11-24T08:06:05.112139+0000","last_peered":"2023-11-24T08:06:05.112139+0000","last_clean":"2023-11-24T08:06:05.112139+0000","last_became_active":"2023-11-24T08:06:04.086323+0000","last_became_peered":"2023-11-24T08:06:04.086323+0000","last_unstale":"2023-11-24T08:06:05.112139+0000","last_undegraded":"2023-11-24T08:06:05.112139+0000","last_fullsized":"2023-11-24T08:06:05.112139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:01.889219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251937+0000","last_change":"2023-11-24T08:06:05.097611+0000","last_active":"2023-11-24T08:06:06.251937+0000","last_peered":"2023-11-24T08:06:06.251937+0000","last_clean":"2023-11-24T08:06:06.251937+0000","last_became_active":"2023-11-24T08:06:05.094003+0000","last_became_peered":"2023-11-24T08:06:05.094003+0000","last_unstale":"2023-11-24T08:06:06.251937+0000","last_undegraded":"2023-11-24T08:06:06.251937+0000","last_fullsized":"2023-11-24T08:06:06.251937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.090125+0000","last_change":"2023-11-24T08:06:04.085591+0000","last_active":"2023-11-24T08:06:05.090125+0000","last_peered":"2023-11-24T08:06:05.090125+0000","last_clean":"2023-11-24T08:06:05.090125+0000","last_became_active":"2023-11-24T08:06:04.085328+0000","last_became_peered":"2023-11-24T08:06:04.085328+0000","last_unstale":"2023-11-24T08:06:05.090125+0000","last_undegraded":"2023-11-24T08:06:05.090125+0000","last_fullsized":"2023-11-24T08:06:05.090125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:35.592076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087957+0000","last_change":"2023-11-24T08:06:05.080325+0000","last_active":"2023-11-24T08:06:06.087957+0000","last_peered":"2023-11-24T08:06:06.087957+0000","last_clean":"2023-11-24T08:06:06.087957+0000","last_became_active":"2023-11-24T08:06:05.079874+0000","last_became_peered":"2023-11-24T08:06:05.079874+0000","last_unstale":"2023-11-24T08:06:06.087957+0000","last_undegraded":"2023-11-24T08:06:06.087957+0000","last_fullsized":"2023-11-24T08:06:06.087957+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:52.364691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.076622+0000","last_change":"2023-11-24T08:06:04.088494+0000","last_active":"2023-11-24T08:06:05.076622+0000","last_peered":"2023-11-24T08:06:05.076622+0000","last_clean":"2023-11-24T08:06:05.076622+0000","last_became_active":"2023-11-24T08:06:04.088029+0000","last_became_peered":"2023-11-24T08:06:04.088029+0000","last_unstale":"2023-11-24T08:06:05.076622+0000","last_undegraded":"2023-11-24T08:06:05.076622+0000","last_fullsized":"2023-11-24T08:06:05.076622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:13.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089650+0000","last_change":"2023-11-24T08:06:05.095670+0000","last_active":"2023-11-24T08:06:06.089650+0000","last_peered":"2023-11-24T08:06:06.089650+0000","last_clean":"2023-11-24T08:06:06.089650+0000","last_became_active":"2023-11-24T08:06:05.095367+0000","last_became_peered":"2023-11-24T08:06:05.095367+0000","last_unstale":"2023-11-24T08:06:06.089650+0000","last_undegraded":"2023-11-24T08:06:06.089650+0000","last_fullsized":"2023-11-24T08:06:06.089650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.086734+0000","last_change":"2023-11-24T08:06:04.074104+0000","last_active":"2023-11-24T08:06:05.086734+0000","last_peered":"2023-11-24T08:06:05.086734+0000","last_clean":"2023-11-24T08:06:05.086734+0000","last_became_active":"2023-11-24T08:06:04.073646+0000","last_became_peered":"2023-11-24T08:06:04.073646+0000","last_unstale":"2023-11-24T08:06:05.086734+0000","last_undegraded":"2023-11-24T08:06:05.086734+0000","last_fullsized":"2023-11-24T08:06:05.086734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:14.570476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253768+0000","last_change":"2023-11-24T08:06:05.095886+0000","last_active":"2023-11-24T08:06:06.253768+0000","last_peered":"2023-11-24T08:06:06.253768+0000","last_clean":"2023-11-24T08:06:06.253768+0000","last_became_active":"2023-11-24T08:06:05.095590+0000","last_became_peered":"2023-11-24T08:06:05.095590+0000","last_unstale":"2023-11-24T08:06:06.253768+0000","last_undegraded":"2023-11-24T08:06:06.253768+0000","last_fullsized":"2023-11-24T08:06:06.253768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:34.594891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.082395+0000","last_change":"2023-11-24T08:06:04.086787+0000","last_active":"2023-11-24T08:06:05.082395+0000","last_peered":"2023-11-24T08:06:05.082395+0000","last_clean":"2023-11-24T08:06:05.082395+0000","last_became_active":"2023-11-24T08:06:04.086503+0000","last_became_peered":"2023-11-24T08:06:04.086503+0000","last_unstale":"2023-11-24T08:06:05.082395+0000","last_undegraded":"2023-11-24T08:06:05.082395+0000","last_fullsized":"2023-11-24T08:06:05.082395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251580+0000","last_change":"2023-11-24T08:06:05.094679+0000","last_active":"2023-11-24T08:06:06.251580+0000","last_peered":"2023-11-24T08:06:06.251580+0000","last_clean":"2023-11-24T08:06:06.251580+0000","last_became_active":"2023-11-24T08:06:05.089982+0000","last_became_peered":"2023-11-24T08:06:05.089982+0000","last_unstale":"2023-11-24T08:06:06.251580+0000","last_undegraded":"2023-11-24T08:06:06.251580+0000","last_fullsized":"2023-11-24T08:06:06.251580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:01.848501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122469+0000","last_change":"2023-11-24T08:06:04.094281+0000","last_active":"2023-11-24T08:06:05.122469+0000","last_peered":"2023-11-24T08:06:05.122469+0000","last_clean":"2023-11-24T08:06:05.122469+0000","last_became_active":"2023-11-24T08:06:04.093946+0000","last_became_peered":"2023-11-24T08:06:04.093946+0000","last_unstale":"2023-11-24T08:06:05.122469+0000","last_undegraded":"2023-11-24T08:06:05.122469+0000","last_fullsized":"2023-11-24T08:06:05.122469+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:28.559491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253369+0000","last_change":"2023-11-24T08:06:05.079345+0000","last_active":"2023-11-24T08:06:06.253369+0000","last_peered":"2023-11-24T08:06:06.253369+0000","last_clean":"2023-11-24T08:06:06.253369+0000","last_became_active":"2023-11-24T08:06:05.078273+0000","last_became_peered":"2023-11-24T08:06:05.078273+0000","last_unstale":"2023-11-24T08:06:06.253369+0000","last_undegraded":"2023-11-24T08:06:06.253369+0000","last_fullsized":"2023-11-24T08:06:06.253369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:59.520725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.125088+0000","last_change":"2023-11-24T08:06:04.080902+0000","last_active":"2023-11-24T08:06:05.125088+0000","last_peered":"2023-11-24T08:06:05.125088+0000","last_clean":"2023-11-24T08:06:05.125088+0000","last_became_active":"2023-11-24T08:06:04.080616+0000","last_became_peered":"2023-11-24T08:06:04.080616+0000","last_unstale":"2023-11-24T08:06:05.125088+0000","last_undegraded":"2023-11-24T08:06:05.125088+0000","last_fullsized":"2023-11-24T08:06:05.125088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:24.477858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088485+0000","last_change":"2023-11-24T08:06:05.093746+0000","last_active":"2023-11-24T08:06:06.088485+0000","last_peered":"2023-11-24T08:06:06.088485+0000","last_clean":"2023-11-24T08:06:06.088485+0000","last_became_active":"2023-11-24T08:06:05.093440+0000","last_became_peered":"2023-11-24T08:06:05.093440+0000","last_unstale":"2023-11-24T08:06:06.088485+0000","last_undegraded":"2023-11-24T08:06:06.088485+0000","last_fullsized":"2023-11-24T08:06:06.088485+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:10.661913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.121974+0000","last_change":"2023-11-24T08:06:04.084295+0000","last_active":"2023-11-24T08:06:05.121974+0000","last_peered":"2023-11-24T08:06:05.121974+0000","last_clean":"2023-11-24T08:06:05.121974+0000","last_became_active":"2023-11-24T08:06:04.083328+0000","last_became_peered":"2023-11-24T08:06:04.083328+0000","last_unstale":"2023-11-24T08:06:05.121974+0000","last_undegraded":"2023-11-24T08:06:05.121974+0000","last_fullsized":"2023-11-24T08:06:05.121974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:58.506222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252513+0000","last_change":"2023-11-24T08:06:05.095494+0000","last_active":"2023-11-24T08:06:06.252513+0000","last_peered":"2023-11-24T08:06:06.252513+0000","last_clean":"2023-11-24T08:06:06.252513+0000","last_became_active":"2023-11-24T08:06:05.095195+0000","last_became_peered":"2023-11-24T08:06:05.095195+0000","last_unstale":"2023-11-24T08:06:06.252513+0000","last_undegraded":"2023-11-24T08:06:06.252513+0000","last_fullsized":"2023-11-24T08:06:06.252513+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:55.876487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063734+0000","last_change":"2023-11-24T08:06:04.088130+0000","last_active":"2023-11-24T08:06:05.063734+0000","last_peered":"2023-11-24T08:06:05.063734+0000","last_clean":"2023-11-24T08:06:05.063734+0000","last_became_active":"2023-11-24T08:06:04.087804+0000","last_became_peered":"2023-11-24T08:06:04.087804+0000","last_unstale":"2023-11-24T08:06:05.063734+0000","last_undegraded":"2023-11-24T08:06:05.063734+0000","last_fullsized":"2023-11-24T08:06:05.063734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250881+0000","last_change":"2023-11-24T08:06:05.102561+0000","last_active":"2023-11-24T08:06:06.250881+0000","last_peered":"2023-11-24T08:06:06.250881+0000","last_clean":"2023-11-24T08:06:06.250881+0000","last_became_active":"2023-11-24T08:06:05.100166+0000","last_became_peered":"2023-11-24T08:06:05.100166+0000","last_unstale":"2023-11-24T08:06:06.250881+0000","last_undegraded":"2023-11-24T08:06:06.250881+0000","last_fullsized":"2023-11-24T08:06:06.250881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:32.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.068470+0000","last_change":"2023-11-24T08:06:04.090915+0000","last_active":"2023-11-24T08:06:05.068470+0000","last_peered":"2023-11-24T08:06:05.068470+0000","last_clean":"2023-11-24T08:06:05.068470+0000","last_became_active":"2023-11-24T08:06:04.090279+0000","last_became_peered":"2023-11-24T08:06:04.090279+0000","last_unstale":"2023-11-24T08:06:05.068470+0000","last_undegraded":"2023-11-24T08:06:05.068470+0000","last_fullsized":"2023-11-24T08:06:05.068470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:33.221758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252809+0000","last_change":"2023-11-24T08:06:05.091728+0000","last_active":"2023-11-24T08:06:06.252809+0000","last_peered":"2023-11-24T08:06:06.252809+0000","last_clean":"2023-11-24T08:06:06.252809+0000","last_became_active":"2023-11-24T08:06:05.090643+0000","last_became_peered":"2023-11-24T08:06:05.090643+0000","last_unstale":"2023-11-24T08:06:06.252809+0000","last_undegraded":"2023-11-24T08:06:06.252809+0000","last_fullsized":"2023-11-24T08:06:06.252809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:45.526567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122459+0000","last_change":"2023-11-24T08:06:04.088343+0000","last_active":"2023-11-24T08:06:05.122459+0000","last_peered":"2023-11-24T08:06:05.122459+0000","last_clean":"2023-11-24T08:06:05.122459+0000","last_became_active":"2023-11-24T08:06:04.087687+0000","last_became_peered":"2023-11-24T08:06:04.087687+0000","last_unstale":"2023-11-24T08:06:05.122459+0000","last_undegraded":"2023-11-24T08:06:05.122459+0000","last_fullsized":"2023-11-24T08:06:05.122459+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248883+0000","last_change":"2023-11-24T08:06:05.097524+0000","last_active":"2023-11-24T08:06:06.248883+0000","last_peered":"2023-11-24T08:06:06.248883+0000","last_clean":"2023-11-24T08:06:06.248883+0000","last_became_active":"2023-11-24T08:06:05.095942+0000","last_became_peered":"2023-11-24T08:06:05.095942+0000","last_unstale":"2023-11-24T08:06:06.248883+0000","last_undegraded":"2023-11-24T08:06:06.248883+0000","last_fullsized":"2023-11-24T08:06:06.248883+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:57:14.392741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.121968+0000","last_change":"2023-11-24T08:06:04.071810+0000","last_active":"2023-11-24T08:06:05.121968+0000","last_peered":"2023-11-24T08:06:05.121968+0000","last_clean":"2023-11-24T08:06:05.121968+0000","last_became_active":"2023-11-24T08:06:04.071196+0000","last_became_peered":"2023-11-24T08:06:04.071196+0000","last_unstale":"2023-11-24T08:06:05.121968+0000","last_undegraded":"2023-11-24T08:06:05.121968+0000","last_fullsized":"2023-11-24T08:06:05.121968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:13:43.496491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252462+0000","last_change":"2023-11-24T08:06:05.093265+0000","last_active":"2023-11-24T08:06:06.252462+0000","last_peered":"2023-11-24T08:06:06.252462+0000","last_clean":"2023-11-24T08:06:06.252462+0000","last_became_active":"2023-11-24T08:06:05.092904+0000","last_became_peered":"2023-11-24T08:06:05.092904+0000","last_unstale":"2023-11-24T08:06:06.252462+0000","last_undegraded":"2023-11-24T08:06:06.252462+0000","last_fullsized":"2023-11-24T08:06:06.252462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:59.520725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.113304+0000","last_change":"2023-11-24T08:06:04.081460+0000","last_active":"2023-11-24T08:06:05.113304+0000","last_peered":"2023-11-24T08:06:05.113304+0000","last_clean":"2023-11-24T08:06:05.113304+0000","last_became_active":"2023-11-24T08:06:04.081176+0000","last_became_peered":"2023-11-24T08:06:04.081176+0000","last_unstale":"2023-11-24T08:06:05.113304+0000","last_undegraded":"2023-11-24T08:06:05.113304+0000","last_fullsized":"2023-11-24T08:06:05.113304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:11.044772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089003+0000","last_change":"2023-11-24T08:06:05.086676+0000","last_active":"2023-11-24T08:06:06.089003+0000","last_peered":"2023-11-24T08:06:06.089003+0000","last_clean":"2023-11-24T08:06:06.089003+0000","last_became_active":"2023-11-24T08:06:05.086342+0000","last_became_peered":"2023-11-24T08:06:05.086342+0000","last_unstale":"2023-11-24T08:06:06.089003+0000","last_undegraded":"2023-11-24T08:06:06.089003+0000","last_fullsized":"2023-11-24T08:06:06.089003+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:36.606580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120951+0000","last_change":"2023-11-24T08:06:04.085453+0000","last_active":"2023-11-24T08:06:05.120951+0000","last_peered":"2023-11-24T08:06:05.120951+0000","last_clean":"2023-11-24T08:06:05.120951+0000","last_became_active":"2023-11-24T08:06:04.085126+0000","last_became_peered":"2023-11-24T08:06:04.085126+0000","last_unstale":"2023-11-24T08:06:05.120951+0000","last_undegraded":"2023-11-24T08:06:05.120951+0000","last_fullsized":"2023-11-24T08:06:05.120951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089297+0000","last_change":"2023-11-24T08:06:05.092694+0000","last_active":"2023-11-24T08:06:06.089297+0000","last_peered":"2023-11-24T08:06:06.089297+0000","last_clean":"2023-11-24T08:06:06.089297+0000","last_became_active":"2023-11-24T08:06:05.092393+0000","last_became_peered":"2023-11-24T08:06:05.092393+0000","last_unstale":"2023-11-24T08:06:06.089297+0000","last_undegraded":"2023-11-24T08:06:06.089297+0000","last_fullsized":"2023-11-24T08:06:06.089297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:15.584979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.071093+0000","last_change":"2023-11-24T08:06:04.085155+0000","last_active":"2023-11-24T08:06:05.071093+0000","last_peered":"2023-11-24T08:06:05.071093+0000","last_clean":"2023-11-24T08:06:05.071093+0000","last_became_active":"2023-11-24T08:06:04.084831+0000","last_became_peered":"2023-11-24T08:06:04.084831+0000","last_unstale":"2023-11-24T08:06:05.071093+0000","last_undegraded":"2023-11-24T08:06:05.071093+0000","last_fullsized":"2023-11-24T08:06:05.071093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:33.580388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251386+0000","last_change":"2023-11-24T08:06:05.101508+0000","last_active":"2023-11-24T08:06:06.251386+0000","last_peered":"2023-11-24T08:06:06.251386+0000","last_clean":"2023-11-24T08:06:06.251386+0000","last_became_active":"2023-11-24T08:06:05.099363+0000","last_became_peered":"2023-11-24T08:06:05.099363+0000","last_unstale":"2023-11-24T08:06:06.251386+0000","last_undegraded":"2023-11-24T08:06:06.251386+0000","last_fullsized":"2023-11-24T08:06:06.251386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:42.213975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120853+0000","last_change":"2023-11-24T08:06:04.087536+0000","last_active":"2023-11-24T08:06:05.120853+0000","last_peered":"2023-11-24T08:06:05.120853+0000","last_clean":"2023-11-24T08:06:05.120853+0000","last_became_active":"2023-11-24T08:06:04.087269+0000","last_became_peered":"2023-11-24T08:06:04.087269+0000","last_unstale":"2023-11-24T08:06:05.120853+0000","last_undegraded":"2023-11-24T08:06:05.120853+0000","last_fullsized":"2023-11-24T08:06:05.120853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:29.354205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.082906+0000","last_change":"2023-11-24T08:06:05.093409+0000","last_active":"2023-11-24T08:06:06.082906+0000","last_peered":"2023-11-24T08:06:06.082906+0000","last_clean":"2023-11-24T08:06:06.082906+0000","last_became_active":"2023-11-24T08:06:05.090895+0000","last_became_peered":"2023-11-24T08:06:05.090895+0000","last_unstale":"2023-11-24T08:06:06.082906+0000","last_undegraded":"2023-11-24T08:06:06.082906+0000","last_fullsized":"2023-11-24T08:06:06.082906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:12.059275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.090278+0000","last_change":"2023-11-24T08:06:04.085441+0000","last_active":"2023-11-24T08:06:05.090278+0000","last_peered":"2023-11-24T08:06:05.090278+0000","last_clean":"2023-11-24T08:06:05.090278+0000","last_became_active":"2023-11-24T08:06:04.085170+0000","last_became_peered":"2023-11-24T08:06:04.085170+0000","last_unstale":"2023-11-24T08:06:05.090278+0000","last_undegraded":"2023-11-24T08:06:05.090278+0000","last_fullsized":"2023-11-24T08:06:05.090278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:14.570476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248353+0000","last_change":"2023-11-24T08:06:05.095928+0000","last_active":"2023-11-24T08:06:06.248353+0000","last_peered":"2023-11-24T08:06:06.248353+0000","last_clean":"2023-11-24T08:06:06.248353+0000","last_became_active":"2023-11-24T08:06:05.095601+0000","last_became_peered":"2023-11-24T08:06:05.095601+0000","last_unstale":"2023-11-24T08:06:06.248353+0000","last_undegraded":"2023-11-24T08:06:06.248353+0000","last_fullsized":"2023-11-24T08:06:06.248353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:33.727928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.083543+0000","last_change":"2023-11-24T08:06:04.086214+0000","last_active":"2023-11-24T08:06:05.083543+0000","last_peered":"2023-11-24T08:06:05.083543+0000","last_clean":"2023-11-24T08:06:05.083543+0000","last_became_active":"2023-11-24T08:06:04.085918+0000","last_became_peered":"2023-11-24T08:06:04.085918+0000","last_unstale":"2023-11-24T08:06:05.083543+0000","last_undegraded":"2023-11-24T08:06:05.083543+0000","last_fullsized":"2023-11-24T08:06:05.083543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:54.440803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251927+0000","last_change":"2023-11-24T08:06:05.095494+0000","last_active":"2023-11-24T08:06:06.251927+0000","last_peered":"2023-11-24T08:06:06.251927+0000","last_clean":"2023-11-24T08:06:06.251927+0000","last_became_active":"2023-11-24T08:06:05.094309+0000","last_became_peered":"2023-11-24T08:06:05.094309+0000","last_unstale":"2023-11-24T08:06:06.251927+0000","last_undegraded":"2023-11-24T08:06:06.251927+0000","last_fullsized":"2023-11-24T08:06:06.251927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063326+0000","last_change":"2023-11-24T08:06:04.093116+0000","last_active":"2023-11-24T08:06:05.063326+0000","last_peered":"2023-11-24T08:06:05.063326+0000","last_clean":"2023-11-24T08:06:05.063326+0000","last_became_active":"2023-11-24T08:06:04.092446+0000","last_became_peered":"2023-11-24T08:06:04.092446+0000","last_unstale":"2023-11-24T08:06:05.063326+0000","last_undegraded":"2023-11-24T08:06:05.063326+0000","last_fullsized":"2023-11-24T08:06:05.063326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:35.592076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089777+0000","last_change":"2023-11-24T08:06:05.093077+0000","last_active":"2023-11-24T08:06:06.089777+0000","last_peered":"2023-11-24T08:06:06.089777+0000","last_clean":"2023-11-24T08:06:06.089777+0000","last_became_active":"2023-11-24T08:06:05.092754+0000","last_became_peered":"2023-11-24T08:06:05.092754+0000","last_unstale":"2023-11-24T08:06:06.089777+0000","last_undegraded":"2023-11-24T08:06:06.089777+0000","last_fullsized":"2023-11-24T08:06:06.089777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.074657+0000","last_change":"2023-11-24T08:06:04.079922+0000","last_active":"2023-11-24T08:06:05.074657+0000","last_peered":"2023-11-24T08:06:05.074657+0000","last_clean":"2023-11-24T08:06:05.074657+0000","last_became_active":"2023-11-24T08:06:04.079489+0000","last_became_peered":"2023-11-24T08:06:04.079489+0000","last_unstale":"2023-11-24T08:06:05.074657+0000","last_undegraded":"2023-11-24T08:06:05.074657+0000","last_fullsized":"2023-11-24T08:06:05.074657+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:04.038305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252779+0000","last_change":"2023-11-24T08:06:05.105800+0000","last_active":"2023-11-24T08:06:06.252779+0000","last_peered":"2023-11-24T08:06:06.252779+0000","last_clean":"2023-11-24T08:06:06.252779+0000","last_became_active":"2023-11-24T08:06:05.105521+0000","last_became_peered":"2023-11-24T08:06:05.105521+0000","last_unstale":"2023-11-24T08:06:06.252779+0000","last_undegraded":"2023-11-24T08:06:06.252779+0000","last_fullsized":"2023-11-24T08:06:06.252779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:32.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.067003+0000","last_change":"2023-11-24T08:06:04.094274+0000","last_active":"2023-11-24T08:06:05.067003+0000","last_peered":"2023-11-24T08:06:05.067003+0000","last_clean":"2023-11-24T08:06:05.067003+0000","last_became_active":"2023-11-24T08:06:04.093959+0000","last_became_peered":"2023-11-24T08:06:04.093959+0000","last_unstale":"2023-11-24T08:06:05.067003+0000","last_undegraded":"2023-11-24T08:06:05.067003+0000","last_fullsized":"2023-11-24T08:06:05.067003+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:33.221758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253430+0000","last_change":"2023-11-24T08:06:05.082611+0000","last_active":"2023-11-24T08:06:06.253430+0000","last_peered":"2023-11-24T08:06:06.253430+0000","last_clean":"2023-11-24T08:06:06.253430+0000","last_became_active":"2023-11-24T08:06:05.082314+0000","last_became_peered":"2023-11-24T08:06:05.082314+0000","last_unstale":"2023-11-24T08:06:06.253430+0000","last_undegraded":"2023-11-24T08:06:06.253430+0000","last_fullsized":"2023-11-24T08:06:06.253430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:30.368708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.060457+0000","last_change":"2023-11-24T08:06:04.087054+0000","last_active":"2023-11-24T08:06:05.060457+0000","last_peered":"2023-11-24T08:06:05.060457+0000","last_clean":"2023-11-24T08:06:05.060457+0000","last_became_active":"2023-11-24T08:06:04.086627+0000","last_became_peered":"2023-11-24T08:06:04.086627+0000","last_unstale":"2023-11-24T08:06:05.060457+0000","last_undegraded":"2023-11-24T08:06:05.060457+0000","last_fullsized":"2023-11-24T08:06:05.060457+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:01.324343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251497+0000","last_change":"2023-11-24T08:06:05.090923+0000","last_active":"2023-11-24T08:06:06.251497+0000","last_peered":"2023-11-24T08:06:06.251497+0000","last_clean":"2023-11-24T08:06:06.251497+0000","last_became_active":"2023-11-24T08:06:05.089665+0000","last_became_peered":"2023-11-24T08:06:05.089665+0000","last_unstale":"2023-11-24T08:06:06.251497+0000","last_undegraded":"2023-11-24T08:06:06.251497+0000","last_fullsized":"2023-11-24T08:06:06.251497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:11.077626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.064897+0000","last_change":"2023-11-24T08:06:04.094285+0000","last_active":"2023-11-24T08:06:05.064897+0000","last_peered":"2023-11-24T08:06:05.064897+0000","last_clean":"2023-11-24T08:06:05.064897+0000","last_became_active":"2023-11-24T08:06:04.093933+0000","last_became_peered":"2023-11-24T08:06:04.093933+0000","last_unstale":"2023-11-24T08:06:05.064897+0000","last_undegraded":"2023-11-24T08:06:05.064897+0000","last_fullsized":"2023-11-24T08:06:05.064897+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:36.353720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251553+0000","last_change":"2023-11-24T08:06:05.086959+0000","last_active":"2023-11-24T08:06:06.251553+0000","last_peered":"2023-11-24T08:06:06.251553+0000","last_clean":"2023-11-24T08:06:06.251553+0000","last_became_active":"2023-11-24T08:06:05.085952+0000","last_became_peered":"2023-11-24T08:06:05.085952+0000","last_unstale":"2023-11-24T08:06:06.251553+0000","last_undegraded":"2023-11-24T08:06:06.251553+0000","last_fullsized":"2023-11-24T08:06:06.251553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:59.924717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.081417+0000","last_change":"2023-11-24T08:06:04.070754+0000","last_active":"2023-11-24T08:06:05.081417+0000","last_peered":"2023-11-24T08:06:05.081417+0000","last_clean":"2023-11-24T08:06:05.081417+0000","last_became_active":"2023-11-24T08:06:04.070430+0000","last_became_peered":"2023-11-24T08:06:04.070430+0000","last_unstale":"2023-11-24T08:06:05.081417+0000","last_undegraded":"2023-11-24T08:06:05.081417+0000","last_fullsized":"2023-11-24T08:06:05.081417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:09.647409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250913+0000","last_change":"2023-11-24T08:06:05.093665+0000","last_active":"2023-11-24T08:06:06.250913+0000","last_peered":"2023-11-24T08:06:06.250913+0000","last_clean":"2023-11-24T08:06:06.250913+0000","last_became_active":"2023-11-24T08:06:05.093393+0000","last_became_peered":"2023-11-24T08:06:05.093393+0000","last_unstale":"2023-11-24T08:06:06.250913+0000","last_undegraded":"2023-11-24T08:06:06.250913+0000","last_fullsized":"2023-11-24T08:06:06.250913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.061092+0000","last_change":"2023-11-24T08:06:04.084482+0000","last_active":"2023-11-24T08:06:05.061092+0000","last_peered":"2023-11-24T08:06:05.061092+0000","last_clean":"2023-11-24T08:06:05.061092+0000","last_became_active":"2023-11-24T08:06:04.084022+0000","last_became_peered":"2023-11-24T08:06:04.084022+0000","last_unstale":"2023-11-24T08:06:05.061092+0000","last_undegraded":"2023-11-24T08:06:05.061092+0000","last_fullsized":"2023-11-24T08:06:05.061092+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:51:33.198531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087356+0000","last_change":"2023-11-24T08:06:05.093359+0000","last_active":"2023-11-24T08:06:06.087356+0000","last_peered":"2023-11-24T08:06:06.087356+0000","last_clean":"2023-11-24T08:06:06.087356+0000","last_became_active":"2023-11-24T08:06:05.093091+0000","last_became_peered":"2023-11-24T08:06:05.093091+0000","last_unstale":"2023-11-24T08:06:06.087356+0000","last_undegraded":"2023-11-24T08:06:06.087356+0000","last_fullsized":"2023-11-24T08:06:06.087356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:21.615036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.077077+0000","last_change":"2023-11-24T08:06:04.081603+0000","last_active":"2023-11-24T08:06:05.077077+0000","last_peered":"2023-11-24T08:06:05.077077+0000","last_clean":"2023-11-24T08:06:05.077077+0000","last_became_active":"2023-11-24T08:06:04.081376+0000","last_became_peered":"2023-11-24T08:06:04.081376+0000","last_unstale":"2023-11-24T08:06:05.077077+0000","last_undegraded":"2023-11-24T08:06:05.077077+0000","last_fullsized":"2023-11-24T08:06:05.077077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:11.044772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252755+0000","last_change":"2023-11-24T08:06:05.095041+0000","last_active":"2023-11-24T08:06:06.252755+0000","last_peered":"2023-11-24T08:06:06.252755+0000","last_clean":"2023-11-24T08:06:06.252755+0000","last_became_active":"2023-11-24T08:06:05.094688+0000","last_became_peered":"2023-11-24T08:06:05.094688+0000","last_unstale":"2023-11-24T08:06:06.252755+0000","last_undegraded":"2023-11-24T08:06:06.252755+0000","last_fullsized":"2023-11-24T08:06:06.252755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:49.120781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122403+0000","last_change":"2023-11-24T08:06:04.088807+0000","last_active":"2023-11-24T08:06:05.122403+0000","last_peered":"2023-11-24T08:06:05.122403+0000","last_clean":"2023-11-24T08:06:05.122403+0000","last_became_active":"2023-11-24T08:06:04.087267+0000","last_became_peered":"2023-11-24T08:06:04.087267+0000","last_unstale":"2023-11-24T08:06:05.122403+0000","last_undegraded":"2023-11-24T08:06:05.122403+0000","last_fullsized":"2023-11-24T08:06:05.122403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:46.590933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063198+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:06:05.063198+0000","last_peered":"2023-11-24T08:06:05.063198+0000","last_clean":"2023-11-24T08:06:05.063198+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:06:05.063198+0000","last_undegraded":"2023-11-24T08:06:05.063198+0000","last_fullsized":"2023-11-24T08:06:05.063198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.65300000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.431},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.069,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3200000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7649999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4179999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.5569999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1539999999999999,"15min":1.149},"min":{"1min":0.80600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.8660000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.80300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8879999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.2270000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3120000000000001,"15min":1.3300000000000001},"min":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5109999999999999,"5min":2.524,"15min":2.524},"last":1.498},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.504,"15min":1.534},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7709999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3169999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.90800000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.613,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3799999999999999,"15min":1.3839999999999999},"min":{"1min":0.87,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0270000000000001,"5min":2.238,"15min":2.238},"last":1.4510000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.619,"15min":1.6319999999999999},"min":{"1min":1.1839999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.383},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.647,"15min":1.677},"min":{"1min":1.0489999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7250000000000001,"5min":2.944,"15min":2.944},"last":1.633}]},{"osd":6,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.7350000000000001,"15min":1.77},"min":{"1min":1.1699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.891,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.671,"15min":1.6870000000000001},"min":{"1min":1.179,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1589999999999998,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.4059999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":215941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.538,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.323,"15min":1.355},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5629999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.4610000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1499999999999999,"15min":1.1579999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.304},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.274,"15min":1.319},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.6830000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.282,"15min":1.3129999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0710000000000002,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.4750000000000001,"15min":1.526},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.5509999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.5860000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.4299999999999999,"15min":1.458},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.554,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.081},{"interface":"front","average":{"1min":1.401,"5min":1.49,"15min":1.5049999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.7749999999999999,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.645}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.724,"15min":1.79},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.157,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.21},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.472,"15min":1.4830000000000001},"min":{"1min":0.90400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.895,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.76}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.7709999999999999,"15min":1.843},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3490000000000002,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.71},{"interface":"front","average":{"1min":1.508,"5min":1.9179999999999999,"15min":1.986},"min":{"1min":0.85399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8400000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.704},"min":{"1min":0.79400000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.2290000000000001,"5min":2.98,"15min":2.98},"last":1.859},{"interface":"front","average":{"1min":1.548,"5min":1.643,"15min":1.659},"min":{"1min":0.94199999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9379999999999999,"5min":2.29,"15min":2.29},"last":1.829}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.042,"15min":1.0580000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.627,"5min":1.764,"15min":1.764},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.02,"15min":1.002},"min":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.268}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1739999999999999,"15min":1.1819999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6299999999999999,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.0669999999999999,"15min":1.0469999999999999},"min":{"1min":0.85699999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.327}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.206,"15min":1.212},"min":{"1min":0.67200000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7490000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.2330000000000001,"15min":1.218},"min":{"1min":0.78900000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.4830000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.99199999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.597,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.419},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":1.0369999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.667,"5min":1.889,"15min":1.889},"last":1.1200000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.288,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6419999999999999,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.413,"15min":1.413},"min":{"1min":1.206,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9339999999999999,"5min":2.27,"15min":2.27},"last":1.365}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.464,"15min":1.4750000000000001},"min":{"1min":1.05,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.617,"5min":3.806,"15min":3.806},"last":1.232},{"interface":"front","average":{"1min":1.389,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.95199999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.726,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.522}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.52,"15min":1.528},"min":{"1min":1.1919999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8500000000000001,"5min":2.923,"15min":2.923},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.5309999999999999,"15min":1.5269999999999999},"min":{"1min":1.343,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7769999999999999,"5min":2.859,"15min":2.859},"last":1.3919999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0529999999999999,"15min":1.046},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.129,"15min":1.127},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.097}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.099,"15min":1.091},"min":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.81799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.3520000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.198,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6279999999999999,"5min":1.823,"15min":1.823},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.264,"15min":1.266},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.1519999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.298,"15min":1.302},"min":{"1min":0.90600000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.597},{"interface":"front","average":{"1min":1.327,"5min":1.319,"15min":1.3169999999999999},"min":{"1min":0.98299999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6419999999999999,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.9530000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.304,"15min":1.304},"min":{"1min":0.86899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.702,"5min":1.944,"15min":1.944},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.361,"15min":1.365},"min":{"1min":0.96899999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.696,"5min":2.012,"15min":2.012},"last":1.7809999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3700000000000001,"15min":1.3660000000000001},"min":{"1min":0.91300000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.8560000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":2.105},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5129999999999999,"15min":1.5149999999999999},"min":{"1min":1.198,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0129999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.927}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.1100000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.431},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.573,"15min":1.5649999999999999},"min":{"1min":1.0880000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.5920000000000001,"5min":2.722,"15min":2.722},"last":2.411}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":215907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.82199999999999995,"15min":0.81899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.94299999999999995,"15min":0.95499999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.4399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.129,"15min":1.169},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6359999999999999,"5min":3.246,"15min":3.246},"last":1.835},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.97299999999999998,"15min":0.96699999999999997},"min":{"1min":0.441,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.94499999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1799999999999999,"15min":1.196},"min":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.718,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.147,"15min":1.165},"min":{"1min":0.54000000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5900000000000001,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.236}]},{"osd":4,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.93999999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6879999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.379,"15min":1.3979999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.73,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.379}]},{"osd":5,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.448,"15min":1.468},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9870000000000001,"5min":2.395,"15min":2.395},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.304,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2410000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4930000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4299999999999999,"15min":1.4350000000000001},"min":{"1min":0.83199999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.1600000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4159999999999999,"15min":1.4239999999999999},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.946,"5min":1.996,"15min":1.996},"last":1.321}]},{"osd":7,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4430000000000001,"15min":1.4510000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.645},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.895}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.436},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":1.0209999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.272,"5min":2.133,"15min":2.133},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.194,"15min":1.242},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.371,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.048,"15min":1.0840000000000001},"min":{"1min":0.5,"5min":0.438,"15min":0.438},"max":{"1min":1.208,"5min":1.998,"15min":1.998},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3089999999999999,"15min":1.333},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9259999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.175,"15min":1.204},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.552,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3140000000000001,"15min":1.343},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.631,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.754},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.798,"5min":3.161,"15min":3.161},"last":1.123}]},{"osd":5,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3640000000000001,"15min":1.3999999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7649999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.391,"15min":1.423},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7549999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.5620000000000001,"15min":1.6080000000000001},"min":{"1min":0.73899999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.825,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.159},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.857,"5min":2.702,"15min":2.702},"last":0.94199999999999995}]},{"osd":7,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.5009999999999999,"15min":1.5309999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.3980000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.6180000000000001,"15min":1.6619999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.9199999999999999,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.085}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1919999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.105,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.0069999999999999,"15min":1.0389999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2230000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.214,"15min":1.2410000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.353,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.161}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2569999999999999,"15min":1.2769999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6319999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.897},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.2110000000000001,"15min":1.2430000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.538,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.6890000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.242,"15min":1.2549999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.512,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.512},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.4390000000000001,"15min":1.4610000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6459999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.4490000000000001,"15min":1.4650000000000001},"min":{"1min":0.85699999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9590000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.468,"15min":1.484},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.746,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.655}]},{"osd":6,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.45,"15min":1.464},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9019999999999999,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4690000000000001,"15min":1.4910000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.821,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.8420000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.548,"15min":1.5700000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.744,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.738},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5800000000000001,"15min":1.599},"min":{"1min":0.94499999999999995,"5min":0.872,"15min":0.872},"max":{"1min":1.8160000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.788}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":804652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.367,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":3.423},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.93000000000000005},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":3.3180000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.05,"15min":1.054},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.706,"5min":2.29,"15min":2.29},"last":3.899},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":0.93700000000000006},"min":{"1min":0.51000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.5049999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":4.0140000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.032,"15min":1.028},"min":{"1min":0.60499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6439999999999999,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":3.4830000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8640000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":6.3570000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.256,"15min":1.254},"min":{"1min":0.95899999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.704,"5min":3.302,"15min":3.302},"last":3.8479999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.454,"15min":1.466},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.2389999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":3.399}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":0.89400000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.0590000000000002,"5min":3.101,"15min":3.101},"last":5.9939999999999998},{"interface":"front","average":{"1min":1.462,"5min":1.4359999999999999,"15min":1.4319999999999999},"min":{"1min":0.90100000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.1779999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.391}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3999999999999999,"15min":1.4159999999999999},"min":{"1min":0.75,"5min":0.749,"15min":0.749},"max":{"1min":2.1819999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":3.9729999999999999},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4970000000000001,"15min":1.4930000000000001},"min":{"1min":1.0700000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1030000000000002,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":3.7320000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.5309999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1600000000000001,"5min":3.823,"15min":3.823},"last":3.4540000000000002},{"interface":"front","average":{"1min":1.397,"5min":1.468,"15min":1.48},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2610000000000001,"5min":3.371,"15min":3.371},"last":3.9169999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:06:10.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T08:06:10.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.887+0000 7f001d4ca640 1 Processor -- start 2023-11-24T08:06:10.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.887+0000 7f001d4ca640 1 -- start start 2023-11-24T08:06:10.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.888+0000 7f001d4ca640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815d5b0 0x7f00180d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.889+0000 7f001d4ca640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00180cf0c0 con 0x7f00180d14a0 2023-11-24T08:06:10.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.889+0000 7f001d4ca640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00180b01b0 con 0x7f00180b1650 2023-11-24T08:06:10.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.889+0000 7f001d4ca640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180d05c0 con 0x7f001815d5b0 2023-11-24T08:06:10.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.889+0000 7f000ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f00180d14a0 0x7f00180cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52012/0 (socket says 172.21.15.64:52012) 2023-11-24T08:06:10.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.889+0000 7f000ffff640 1 -- 172.21.15.64:0/3582015495 learned_addr learned my addr 172.21.15.64:0/3582015495 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.889+0000 7f0017fff640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815d5b0 0x7f00180d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.889+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184725946 0 0) 0x7f00180cf0c0 con 0x7f00180d14a0 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.889+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0004003520 con 0x7f00180d14a0 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 187411415 0 0) 0x7f0004003520 con 0x7f00180d14a0 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815d5b0 msgr2=0x7f00180d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815d5b0 0x7f00180d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 >> v1:172.21.15.91:6789/0 conn(0x7f00180b1650 legacy=0x7f00180cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001815d990 con 0x7f00180d14a0 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0000001670 con 0x7f00180d14a0 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 >> v1:172.21.15.64:6789/0 conn(0x7f00180d14a0 legacy=0x7f00180cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815d5b0 0x7f0004003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0004004d00 0x7f0004005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180d05c0 con 0x7f0004003f90 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000400bae0 con 0x7f0004004d00 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000400bca0 con 0x7f001815d5b0 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f000ffff640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f0017fff640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815d5b0 0x7f0004003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.890+0000 7f000ffff640 1 -- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815d5b0 msgr2=0x7f0004003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.891+0000 7f000ffff640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815d5b0 0x7f0004003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.891+0000 7f000ffff640 1 -- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0004004d00 msgr2=0x7f0004005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.891+0000 7f001ccc9640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0004004d00 0x7f0004005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.891+0000 7f000ffff640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0004004d00 0x7f0004005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.891+0000 7f000ffff640 1 -- 172.21.15.64:0/3582015495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004003520 con 0x7f0004003f90 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.892+0000 7f000ffff640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00040047c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f000000c240 tx=0x7f00040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b584b4df635f67 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.892+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000005ad0 con 0x7f0004003f90 2023-11-24T08:06:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.892+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0000005c30 con 0x7f0004003f90 2023-11-24T08:06:10.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.892+0000 7f00177fe640 1 -- 172.21.15.64:0/3582015495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000002ca50 con 0x7f0004003f90 2023-11-24T08:06:10.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.893+0000 7f001d4ca640 1 -- 172.21.15.64:0/3582015495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 msgr2=0x7f00040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.893+0000 7f001d4ca640 1 --2- 172.21.15.64:0/3582015495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00040047c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f000000c240 tx=0x7f00040057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.893+0000 7f001ccc9640 1 -- 172.21.15.64:0/3582015495 reap_dead start 2023-11-24T08:06:10.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.893+0000 7f001d4ca640 1 -- 172.21.15.64:0/3582015495 shutdown_connections 2023-11-24T08:06:10.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.893+0000 7f001d4ca640 1 -- 172.21.15.64:0/3582015495 >> 172.21.15.64:0/3582015495 conn(0x7f001809c3b0 msgr2=0x7f00180d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:10.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.893+0000 7f001d4ca640 1 -- 172.21.15.64:0/3582015495 shutdown_connections 2023-11-24T08:06:10.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.893+0000 7f001d4ca640 1 -- 172.21.15.64:0/3582015495 wait complete. 2023-11-24T08:06:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.894+0000 7f001d4ca640 1 Processor -- start 2023-11-24T08:06:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.895+0000 7f001d4ca640 1 -- start start 2023-11-24T08:06:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.895+0000 7f001d4ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00180b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.895+0000 7f001d4ca640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00180c6ba0 0x7f00180b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f001d4ca640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815b690 0x7f00180d3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f001d4ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b6b10 con 0x7f0004003f90 2023-11-24T08:06:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f001d4ca640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180c7020 con 0x7f00180c6ba0 2023-11-24T08:06:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f001d4ca640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b7600 con 0x7f001815b690 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00180b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00180b6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43090/0 (socket says 172.21.15.64:43090) 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 -- 172.21.15.64:0/3497396004 learned_addr learned my addr 172.21.15.64:0/3497396004 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f000ffff640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00180c6ba0 0x7f00180b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 -- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815b690 msgr2=0x7f00180d3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815b690 0x7f00180d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 -- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00180c6ba0 msgr2=0x7f00180b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00180c6ba0 0x7f00180b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 -- 172.21.15.64:0/3497396004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180b6d60 con 0x7f0004003f90 2023-11-24T08:06:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.896+0000 7f0017fff640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00180b6370 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0008011600 tx=0x7f0008011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.897+0000 7f00157fa640 1 -- 172.21.15.64:0/3497396004 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800f920 con 0x7f0004003f90 2023-11-24T08:06:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.897+0000 7f00157fa640 1 -- 172.21.15.64:0/3497396004 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000800fa80 con 0x7f0004003f90 2023-11-24T08:06:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.897+0000 7f00157fa640 1 -- 172.21.15.64:0/3497396004 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800fdf0 con 0x7f0004003f90 2023-11-24T08:06:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.897+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00180b7040 con 0x7f0004003f90 2023-11-24T08:06:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.897+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001815bb10 con 0x7f0004003f90 2023-11-24T08:06:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.898+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001804e840 con 0x7f0004003f90 2023-11-24T08:06:10.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.899+0000 7f00157fa640 1 -- 172.21.15.64:0/3497396004 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0008015460 con 0x7f0004003f90 2023-11-24T08:06:10.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.900+0000 7f00157fa640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efff806aed0 0x7efff806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:10.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.900+0000 7f00157fa640 1 -- 172.21.15.64:0/3497396004 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0008089b00 con 0x7f0004003f90 2023-11-24T08:06:10.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.901+0000 7f000ffff640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efff806aed0 0x7efff806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:10.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.902+0000 7f000ffff640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efff806aed0 0x7efff806d390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0000034000 tx=0x7f0000009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:10.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:10.904+0000 7f00157fa640 1 -- 172.21.15.64:0/3497396004 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0008054c50 con 0x7f0004003f90 2023-11-24T08:06:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.171+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f00180678d0 con 0x7f0004003f90 2023-11-24T08:06:11.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.172+0000 7f00157fa640 1 -- 172.21.15.64:0/3497396004 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1431 (secure 0 0 0) 0x7f00080225b0 con 0x7f0004003f90 2023-11-24T08:06:11.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:11.174 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":160,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1700813023,"num_in_osds":8,"osd_in_since":1700813014,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":226824192,"bytes_avail":772867289088,"bytes_total":773094113280,"write_bytes_sec":2384,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5724}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-24T08:05:52.056257+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efff806aed0 msgr2=0x7efff806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efff806aed0 0x7efff806d390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0000034000 tx=0x7f0000009030 comp rx=0 tx=0).stop 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 msgr2=0x7f00180b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00180b6370 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0008011600 tx=0x7f0008011ac0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 shutdown_connections 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efff806aed0 0x7efff806d390 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f001815b690 0x7f00180d3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f00180c6ba0 0x7f00180b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 --2- 172.21.15.64:0/3497396004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0004003f90 0x7f00180b6370 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 >> 172.21.15.64:0/3497396004 conn(0x7f001809c3b0 msgr2=0x7f00180d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:11.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 shutdown_connections 2023-11-24T08:06:11.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.175+0000 7f001d4ca640 1 -- 172.21.15.64:0/3497396004 wait complete. 2023-11-24T08:06:11.195 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T08:06:11.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:06:11.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.424+0000 7efc8b027640 1 Processor -- start 2023-11-24T08:06:11.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.425+0000 7efc8b027640 1 -- start start 2023-11-24T08:06:11.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.426+0000 7efc8b027640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc8415d5b0 0x7efc840d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:11.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.426+0000 7efc8b027640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efc840b1650 0x7efc840d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:11.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.426+0000 7efc8b027640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc840cf0c0 con 0x7efc840d14a0 2023-11-24T08:06:11.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.426+0000 7efc8b027640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc840b01b0 con 0x7efc8415d5b0 2023-11-24T08:06:11.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.426+0000 7efc8b027640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8415d990 con 0x7efc840b1650 2023-11-24T08:06:11.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc8a025640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc8415d5b0 0x7efc840d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:11.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc8a025640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc8415d5b0 0x7efc840d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43106/0 (socket says 172.21.15.64:43106) 2023-11-24T08:06:11.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc8a025640 1 -- 172.21.15.64:0/1448428736 learned_addr learned my addr 172.21.15.64:0/1448428736 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:11.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc89824640 1 --1- 172.21.15.64:0/1448428736 >> v1:172.21.15.64:6789/0 conn(0x7efc840d14a0 0x7efc840cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52020/0 (socket says 172.21.15.64:52020) 2023-11-24T08:06:11.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc8a826640 1 --2- 172.21.15.64:0/1448428736 >> v2:172.21.15.91:3300/0 conn(0x7efc840b1650 0x7efc840d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc8a025640 1 -- 172.21.15.64:0/1448428736 >> v2:172.21.15.91:3300/0 conn(0x7efc840b1650 msgr2=0x7efc840d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc8a025640 1 --2- 172.21.15.64:0/1448428736 >> v2:172.21.15.91:3300/0 conn(0x7efc840b1650 0x7efc840d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc8a025640 1 -- 172.21.15.64:0/1448428736 >> v1:172.21.15.64:6789/0 conn(0x7efc840d14a0 legacy=0x7efc840cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.427+0000 7efc8a025640 1 -- 172.21.15.64:0/1448428736 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840d1e40 con 0x7efc8415d5b0 2023-11-24T08:06:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.428+0000 7efc8a025640 1 --2- 172.21.15.64:0/1448428736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840d3990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efc8408bc30 tx=0x7efc7402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4e27ef0b077b7f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.429+0000 7efc89023640 1 -- 172.21.15.64:0/1448428736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7402cb10 con 0x7efc8415d5b0 2023-11-24T08:06:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.429+0000 7efc89023640 1 -- 172.21.15.64:0/1448428736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7402cc70 con 0x7efc8415d5b0 2023-11-24T08:06:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.429+0000 7efc89023640 1 -- 172.21.15.64:0/1448428736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc740343f0 con 0x7efc8415d5b0 2023-11-24T08:06:11.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.429+0000 7efc8b027640 1 -- 172.21.15.64:0/1448428736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 msgr2=0x7efc840d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:11.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.429+0000 7efc8b027640 1 --2- 172.21.15.64:0/1448428736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840d3990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efc8408bc30 tx=0x7efc7402bf70 comp rx=0 tx=0).stop 2023-11-24T08:06:11.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.430+0000 7efc8b027640 1 -- 172.21.15.64:0/1448428736 shutdown_connections 2023-11-24T08:06:11.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.430+0000 7efc8b027640 1 --2- 172.21.15.64:0/1448428736 >> v2:172.21.15.91:3300/0 conn(0x7efc840b1650 0x7efc840d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.430+0000 7efc8b027640 1 --2- 172.21.15.64:0/1448428736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840d3990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.430+0000 7efc8b027640 1 -- 172.21.15.64:0/1448428736 >> 172.21.15.64:0/1448428736 conn(0x7efc8409c3b0 msgr2=0x7efc840d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.430+0000 7efc8b027640 1 -- 172.21.15.64:0/1448428736 shutdown_connections 2023-11-24T08:06:11.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.430+0000 7efc8b027640 1 -- 172.21.15.64:0/1448428736 wait complete. 2023-11-24T08:06:11.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.431+0000 7efc8b027640 1 Processor -- start 2023-11-24T08:06:11.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.431+0000 7efc8b027640 1 -- start start 2023-11-24T08:06:11.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.432+0000 7efc8b027640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc840b1650 0x7efc8415b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:11.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.432+0000 7efc8b027640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc840d14a0 0x7efc840c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:11.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.432+0000 7efc8b027640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:11.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.432+0000 7efc8b027640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c7950 con 0x7efc8415d5b0 2023-11-24T08:06:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.432+0000 7efc8b027640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8415be60 con 0x7efc840b1650 2023-11-24T08:06:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.432+0000 7efc8b027640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c6cd0 con 0x7efc840d14a0 2023-11-24T08:06:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a826640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a826640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43120/0 (socket says 172.21.15.64:43120) 2023-11-24T08:06:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a826640 1 -- 172.21.15.64:0/2594574274 learned_addr learned my addr 172.21.15.64:0/2594574274 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a025640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc840b1650 0x7efc8415b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc89824640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc840d14a0 0x7efc840c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a826640 1 -- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc840d14a0 msgr2=0x7efc840c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a826640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc840d14a0 0x7efc840c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a826640 1 -- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc840b1650 msgr2=0x7efc8415b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a826640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc840b1650 0x7efc8415b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a826640 1 -- 172.21.15.64:0/2594574274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc74009b30 con 0x7efc8415d5b0 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.433+0000 7efc8a025640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc840b1650 0x7efc8415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.434+0000 7efc8a826640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840b2d40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c540 tx=0x7efc7c00ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.434+0000 7efc82ffd640 1 -- 172.21.15.64:0/2594574274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c004220 con 0x7efc8415d5b0 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.434+0000 7efc82ffd640 1 -- 172.21.15.64:0/2594574274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c004380 con 0x7efc8415d5b0 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.434+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc840c6fb0 con 0x7efc8415d5b0 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.434+0000 7efc82ffd640 1 -- 172.21.15.64:0/2594574274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c003850 con 0x7efc8415d5b0 2023-11-24T08:06:11.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.434+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc840c7570 con 0x7efc8415d5b0 2023-11-24T08:06:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.435+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc840c7a90 con 0x7efc8415d5b0 2023-11-24T08:06:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.436+0000 7efc82ffd640 1 -- 172.21.15.64:0/2594574274 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7efc7c0039b0 con 0x7efc8415d5b0 2023-11-24T08:06:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.437+0000 7efc82ffd640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc7006aed0 0x7efc7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.437+0000 7efc82ffd640 1 -- 172.21.15.64:0/2594574274 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc7c089390 con 0x7efc8415d5b0 2023-11-24T08:06:11.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.438+0000 7efc8a025640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc7006aed0 0x7efc7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:11.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.439+0000 7efc8a025640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc7006aed0 0x7efc7006d390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc74035000 tx=0x7efc74030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:11.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.442+0000 7efc82ffd640 1 -- 172.21.15.64:0/2594574274 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc840c7a90 con 0x7efc8415d5b0 2023-11-24T08:06:11.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.636+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc840afe40 con 0x7efc7006aed0 2023-11-24T08:06:11.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.659+0000 7efc82ffd640 1 -- 172.21.15.64:0/2594574274 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286656 (secure 0 0 0) 0x7efc740343f0 con 0x7efc7006aed0 2023-11-24T08:06:11.663 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:06:11.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc7006aed0 msgr2=0x7efc7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:11.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc7006aed0 0x7efc7006d390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc74035000 tx=0x7efc74030000 comp rx=0 tx=0).stop 2023-11-24T08:06:11.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 msgr2=0x7efc840b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:11.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840b2d40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c540 tx=0x7efc7c00ca00 comp rx=0 tx=0).stop 2023-11-24T08:06:11.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 shutdown_connections 2023-11-24T08:06:11.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc7006aed0 0x7efc7006d390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8415d5b0 0x7efc840b2d40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc840d14a0 0x7efc840c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 --2- 172.21.15.64:0/2594574274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc840b1650 0x7efc8415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:11.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 >> 172.21.15.64:0/2594574274 conn(0x7efc8409c3b0 msgr2=0x7efc840d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:11.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.666+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 shutdown_connections 2023-11-24T08:06:11.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:11.667+0000 7efc8b027640 1 -- 172.21.15.64:0/2594574274 wait complete. 2023-11-24T08:06:11.689 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-11-24T08:06:10.063018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221508,"kb_used_data":5348,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753212,"statfs":{"total":773094113280,"available":772867289088,"internally_reserved":0,"allocated":5476352,"data_stored":2906689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.011005"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253626+0000","last_change":"2023-11-24T08:06:05.088466+0000","last_active":"2023-11-24T08:06:06.253626+0000","last_peered":"2023-11-24T08:06:06.253626+0000","last_clean":"2023-11-24T08:06:06.253626+0000","last_became_active":"2023-11-24T08:06:05.088184+0000","last_became_peered":"2023-11-24T08:06:05.088184+0000","last_unstale":"2023-11-24T08:06:06.253626+0000","last_undegraded":"2023-11-24T08:06:06.253626+0000","last_fullsized":"2023-11-24T08:06:06.253626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:55.876487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251929+0000","last_change":"2023-11-24T08:06:05.094999+0000","last_active":"2023-11-24T08:06:06.251929+0000","last_peered":"2023-11-24T08:06:06.251929+0000","last_clean":"2023-11-24T08:06:06.251929+0000","last_became_active":"2023-11-24T08:06:05.094688+0000","last_became_peered":"2023-11-24T08:06:05.094688+0000","last_unstale":"2023-11-24T08:06:06.251929+0000","last_undegraded":"2023-11-24T08:06:06.251929+0000","last_fullsized":"2023-11-24T08:06:06.251929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088777+0000","last_change":"2023-11-24T08:06:05.088653+0000","last_active":"2023-11-24T08:06:06.088777+0000","last_peered":"2023-11-24T08:06:06.088777+0000","last_clean":"2023-11-24T08:06:06.088777+0000","last_became_active":"2023-11-24T08:06:05.087897+0000","last_became_peered":"2023-11-24T08:06:05.087897+0000","last_unstale":"2023-11-24T08:06:06.088777+0000","last_undegraded":"2023-11-24T08:06:06.088777+0000","last_fullsized":"2023-11-24T08:06:06.088777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252772+0000","last_change":"2023-11-24T08:06:05.097244+0000","last_active":"2023-11-24T08:06:06.252772+0000","last_peered":"2023-11-24T08:06:06.252772+0000","last_clean":"2023-11-24T08:06:06.252772+0000","last_became_active":"2023-11-24T08:06:05.096919+0000","last_became_peered":"2023-11-24T08:06:05.096919+0000","last_unstale":"2023-11-24T08:06:06.252772+0000","last_undegraded":"2023-11-24T08:06:06.252772+0000","last_fullsized":"2023-11-24T08:06:06.252772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:05.052808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251328+0000","last_change":"2023-11-24T08:06:05.095396+0000","last_active":"2023-11-24T08:06:06.251328+0000","last_peered":"2023-11-24T08:06:06.251328+0000","last_clean":"2023-11-24T08:06:06.251328+0000","last_became_active":"2023-11-24T08:06:05.094813+0000","last_became_peered":"2023-11-24T08:06:05.094813+0000","last_unstale":"2023-11-24T08:06:06.251328+0000","last_undegraded":"2023-11-24T08:06:06.251328+0000","last_fullsized":"2023-11-24T08:06:06.251328+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251253+0000","last_change":"2023-11-24T08:06:05.084536+0000","last_active":"2023-11-24T08:06:06.251253+0000","last_peered":"2023-11-24T08:06:06.251253+0000","last_clean":"2023-11-24T08:06:06.251253+0000","last_became_active":"2023-11-24T08:06:05.084223+0000","last_became_peered":"2023-11-24T08:06:05.084223+0000","last_unstale":"2023-11-24T08:06:06.251253+0000","last_undegraded":"2023-11-24T08:06:06.251253+0000","last_fullsized":"2023-11-24T08:06:06.251253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:03.061151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252752+0000","last_change":"2023-11-24T08:06:05.079934+0000","last_active":"2023-11-24T08:06:06.252752+0000","last_peered":"2023-11-24T08:06:06.252752+0000","last_clean":"2023-11-24T08:06:06.252752+0000","last_became_active":"2023-11-24T08:06:05.079195+0000","last_became_peered":"2023-11-24T08:06:05.079195+0000","last_unstale":"2023-11-24T08:06:06.252752+0000","last_undegraded":"2023-11-24T08:06:06.252752+0000","last_fullsized":"2023-11-24T08:06:06.252752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:28.502892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252409+0000","last_change":"2023-11-24T08:06:05.096845+0000","last_active":"2023-11-24T08:06:06.252409+0000","last_peered":"2023-11-24T08:06:06.252409+0000","last_clean":"2023-11-24T08:06:06.252409+0000","last_became_active":"2023-11-24T08:06:05.094451+0000","last_became_peered":"2023-11-24T08:06:05.094451+0000","last_unstale":"2023-11-24T08:06:06.252409+0000","last_undegraded":"2023-11-24T08:06:06.252409+0000","last_fullsized":"2023-11-24T08:06:06.252409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:45.526567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252291+0000","last_change":"2023-11-24T08:06:05.096637+0000","last_active":"2023-11-24T08:06:06.252291+0000","last_peered":"2023-11-24T08:06:06.252291+0000","last_clean":"2023-11-24T08:06:06.252291+0000","last_became_active":"2023-11-24T08:06:05.096340+0000","last_became_peered":"2023-11-24T08:06:05.096340+0000","last_unstale":"2023-11-24T08:06:06.252291+0000","last_undegraded":"2023-11-24T08:06:06.252291+0000","last_fullsized":"2023-11-24T08:06:06.252291+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:34.236261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252685+0000","last_change":"2023-11-24T08:06:05.102398+0000","last_active":"2023-11-24T08:06:06.252685+0000","last_peered":"2023-11-24T08:06:06.252685+0000","last_clean":"2023-11-24T08:06:06.252685+0000","last_became_active":"2023-11-24T08:06:05.101720+0000","last_became_peered":"2023-11-24T08:06:05.101720+0000","last_unstale":"2023-11-24T08:06:06.252685+0000","last_undegraded":"2023-11-24T08:06:06.252685+0000","last_fullsized":"2023-11-24T08:06:06.252685+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:34.594891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252477+0000","last_change":"2023-11-24T08:06:05.097295+0000","last_active":"2023-11-24T08:06:06.252477+0000","last_peered":"2023-11-24T08:06:06.252477+0000","last_clean":"2023-11-24T08:06:06.252477+0000","last_became_active":"2023-11-24T08:06:05.095953+0000","last_became_peered":"2023-11-24T08:06:05.095953+0000","last_unstale":"2023-11-24T08:06:06.252477+0000","last_undegraded":"2023-11-24T08:06:06.252477+0000","last_fullsized":"2023-11-24T08:06:06.252477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:30.368708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251480+0000","last_change":"2023-11-24T08:06:05.106405+0000","last_active":"2023-11-24T08:06:06.251480+0000","last_peered":"2023-11-24T08:06:06.251480+0000","last_clean":"2023-11-24T08:06:06.251480+0000","last_became_active":"2023-11-24T08:06:05.106098+0000","last_became_peered":"2023-11-24T08:06:05.106098+0000","last_unstale":"2023-11-24T08:06:06.251480+0000","last_undegraded":"2023-11-24T08:06:06.251480+0000","last_fullsized":"2023-11-24T08:06:06.251480+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:37.368224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248524+0000","last_change":"2023-11-24T08:06:05.087198+0000","last_active":"2023-11-24T08:06:06.248524+0000","last_peered":"2023-11-24T08:06:06.248524+0000","last_clean":"2023-11-24T08:06:06.248524+0000","last_became_active":"2023-11-24T08:06:05.086737+0000","last_became_peered":"2023-11-24T08:06:05.086737+0000","last_unstale":"2023-11-24T08:06:06.248524+0000","last_undegraded":"2023-11-24T08:06:06.248524+0000","last_fullsized":"2023-11-24T08:06:06.248524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:29.327265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252188+0000","last_change":"2023-11-24T08:06:05.100506+0000","last_active":"2023-11-24T08:06:06.252188+0000","last_peered":"2023-11-24T08:06:06.252188+0000","last_clean":"2023-11-24T08:06:06.252188+0000","last_became_active":"2023-11-24T08:06:05.094270+0000","last_became_peered":"2023-11-24T08:06:05.094270+0000","last_unstale":"2023-11-24T08:06:06.252188+0000","last_undegraded":"2023-11-24T08:06:06.252188+0000","last_fullsized":"2023-11-24T08:06:06.252188+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250948+0000","last_change":"2023-11-24T08:06:05.091977+0000","last_active":"2023-11-24T08:06:06.250948+0000","last_peered":"2023-11-24T08:06:06.250948+0000","last_clean":"2023-11-24T08:06:06.250948+0000","last_became_active":"2023-11-24T08:06:05.091622+0000","last_became_peered":"2023-11-24T08:06:05.091622+0000","last_unstale":"2023-11-24T08:06:06.250948+0000","last_undegraded":"2023-11-24T08:06:06.250948+0000","last_fullsized":"2023-11-24T08:06:06.250948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:31.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087597+0000","last_change":"2023-11-24T08:06:05.090075+0000","last_active":"2023-11-24T08:06:06.087597+0000","last_peered":"2023-11-24T08:06:06.087597+0000","last_clean":"2023-11-24T08:06:06.087597+0000","last_became_active":"2023-11-24T08:06:05.089117+0000","last_became_peered":"2023-11-24T08:06:05.089117+0000","last_unstale":"2023-11-24T08:06:06.087597+0000","last_undegraded":"2023-11-24T08:06:06.087597+0000","last_fullsized":"2023-11-24T08:06:06.087597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:30.794390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.090292+0000","last_change":"2023-11-24T08:06:05.098942+0000","last_active":"2023-11-24T08:06:06.090292+0000","last_peered":"2023-11-24T08:06:06.090292+0000","last_clean":"2023-11-24T08:06:06.090292+0000","last_became_active":"2023-11-24T08:06:05.098583+0000","last_became_peered":"2023-11-24T08:06:05.098583+0000","last_unstale":"2023-11-24T08:06:06.090292+0000","last_undegraded":"2023-11-24T08:06:06.090292+0000","last_fullsized":"2023-11-24T08:06:06.090292+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089664+0000","last_change":"2023-11-24T08:06:05.095908+0000","last_active":"2023-11-24T08:06:06.089664+0000","last_peered":"2023-11-24T08:06:06.089664+0000","last_clean":"2023-11-24T08:06:06.089664+0000","last_became_active":"2023-11-24T08:06:05.095581+0000","last_became_peered":"2023-11-24T08:06:05.095581+0000","last_unstale":"2023-11-24T08:06:06.089664+0000","last_undegraded":"2023-11-24T08:06:06.089664+0000","last_fullsized":"2023-11-24T08:06:06.089664+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089518+0000","last_change":"2023-11-24T08:06:05.093941+0000","last_active":"2023-11-24T08:06:06.089518+0000","last_peered":"2023-11-24T08:06:06.089518+0000","last_clean":"2023-11-24T08:06:06.089518+0000","last_became_active":"2023-11-24T08:06:05.092980+0000","last_became_peered":"2023-11-24T08:06:05.092980+0000","last_unstale":"2023-11-24T08:06:06.089518+0000","last_undegraded":"2023-11-24T08:06:06.089518+0000","last_fullsized":"2023-11-24T08:06:06.089518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:47.605436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248776+0000","last_change":"2023-11-24T08:06:05.096889+0000","last_active":"2023-11-24T08:06:06.248776+0000","last_peered":"2023-11-24T08:06:06.248776+0000","last_clean":"2023-11-24T08:06:06.248776+0000","last_became_active":"2023-11-24T08:06:05.096574+0000","last_became_peered":"2023-11-24T08:06:05.096574+0000","last_unstale":"2023-11-24T08:06:06.248776+0000","last_undegraded":"2023-11-24T08:06:06.248776+0000","last_fullsized":"2023-11-24T08:06:06.248776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:05.908617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251413+0000","last_change":"2023-11-24T08:06:05.092598+0000","last_active":"2023-11-24T08:06:06.251413+0000","last_peered":"2023-11-24T08:06:06.251413+0000","last_clean":"2023-11-24T08:06:06.251413+0000","last_became_active":"2023-11-24T08:06:05.088450+0000","last_became_peered":"2023-11-24T08:06:05.088450+0000","last_unstale":"2023-11-24T08:06:06.251413+0000","last_undegraded":"2023-11-24T08:06:06.251413+0000","last_fullsized":"2023-11-24T08:06:06.251413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:13.526610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252225+0000","last_change":"2023-11-24T08:06:05.098582+0000","last_active":"2023-11-24T08:06:06.252225+0000","last_peered":"2023-11-24T08:06:06.252225+0000","last_clean":"2023-11-24T08:06:06.252225+0000","last_became_active":"2023-11-24T08:06:05.094693+0000","last_became_peered":"2023-11-24T08:06:05.094693+0000","last_unstale":"2023-11-24T08:06:06.252225+0000","last_undegraded":"2023-11-24T08:06:06.252225+0000","last_fullsized":"2023-11-24T08:06:06.252225+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:29.573995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087957+0000","last_change":"2023-11-24T08:06:05.085854+0000","last_active":"2023-11-24T08:06:06.087957+0000","last_peered":"2023-11-24T08:06:06.087957+0000","last_clean":"2023-11-24T08:06:06.087957+0000","last_became_active":"2023-11-24T08:06:05.085576+0000","last_became_peered":"2023-11-24T08:06:05.085576+0000","last_unstale":"2023-11-24T08:06:06.087957+0000","last_undegraded":"2023-11-24T08:06:06.087957+0000","last_fullsized":"2023-11-24T08:06:06.087957+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:21.753543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087840+0000","last_change":"2023-11-24T08:06:05.084050+0000","last_active":"2023-11-24T08:06:06.087840+0000","last_peered":"2023-11-24T08:06:06.087840+0000","last_clean":"2023-11-24T08:06:06.087840+0000","last_became_active":"2023-11-24T08:06:05.083596+0000","last_became_peered":"2023-11-24T08:06:05.083596+0000","last_unstale":"2023-11-24T08:06:06.087840+0000","last_undegraded":"2023-11-24T08:06:06.087840+0000","last_fullsized":"2023-11-24T08:06:06.087840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:12.059275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089365+0000","last_change":"2023-11-24T08:06:05.099351+0000","last_active":"2023-11-24T08:06:06.089365+0000","last_peered":"2023-11-24T08:06:06.089365+0000","last_clean":"2023-11-24T08:06:06.089365+0000","last_became_active":"2023-11-24T08:06:05.097619+0000","last_became_peered":"2023-11-24T08:06:05.097619+0000","last_unstale":"2023-11-24T08:06:06.089365+0000","last_undegraded":"2023-11-24T08:06:06.089365+0000","last_fullsized":"2023-11-24T08:06:06.089365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:21.615036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248405+0000","last_change":"2023-11-24T08:06:05.094252+0000","last_active":"2023-11-24T08:06:06.248405+0000","last_peered":"2023-11-24T08:06:06.248405+0000","last_clean":"2023-11-24T08:06:06.248405+0000","last_became_active":"2023-11-24T08:06:05.093924+0000","last_became_peered":"2023-11-24T08:06:05.093924+0000","last_unstale":"2023-11-24T08:06:06.248405+0000","last_undegraded":"2023-11-24T08:06:06.248405+0000","last_fullsized":"2023-11-24T08:06:06.248405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:19.995373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089262+0000","last_change":"2023-11-24T08:06:05.094578+0000","last_active":"2023-11-24T08:06:06.089262+0000","last_peered":"2023-11-24T08:06:06.089262+0000","last_clean":"2023-11-24T08:06:06.089262+0000","last_became_active":"2023-11-24T08:06:05.094329+0000","last_became_peered":"2023-11-24T08:06:05.094329+0000","last_unstale":"2023-11-24T08:06:06.089262+0000","last_undegraded":"2023-11-24T08:06:06.089262+0000","last_fullsized":"2023-11-24T08:06:06.089262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:37.368224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251627+0000","last_change":"2023-11-24T08:06:05.102312+0000","last_active":"2023-11-24T08:06:06.251627+0000","last_peered":"2023-11-24T08:06:06.251627+0000","last_clean":"2023-11-24T08:06:06.251627+0000","last_became_active":"2023-11-24T08:06:05.102050+0000","last_became_peered":"2023-11-24T08:06:05.102050+0000","last_unstale":"2023-11-24T08:06:06.251627+0000","last_undegraded":"2023-11-24T08:06:06.251627+0000","last_fullsized":"2023-11-24T08:06:06.251627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088476+0000","last_change":"2023-11-24T08:06:05.093991+0000","last_active":"2023-11-24T08:06:06.088476+0000","last_peered":"2023-11-24T08:06:06.088476+0000","last_clean":"2023-11-24T08:06:06.088476+0000","last_became_active":"2023-11-24T08:06:05.093653+0000","last_became_peered":"2023-11-24T08:06:05.093653+0000","last_unstale":"2023-11-24T08:06:06.088476+0000","last_undegraded":"2023-11-24T08:06:06.088476+0000","last_fullsized":"2023-11-24T08:06:06.088476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:02.903723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087380+0000","last_change":"2023-11-24T08:06:05.095755+0000","last_active":"2023-11-24T08:06:06.087380+0000","last_peered":"2023-11-24T08:06:06.087380+0000","last_clean":"2023-11-24T08:06:06.087380+0000","last_became_active":"2023-11-24T08:06:05.095466+0000","last_became_peered":"2023-11-24T08:06:05.095466+0000","last_unstale":"2023-11-24T08:06:06.087380+0000","last_undegraded":"2023-11-24T08:06:06.087380+0000","last_fullsized":"2023-11-24T08:06:06.087380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:47.605436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251015+0000","last_change":"2023-11-24T08:06:05.087774+0000","last_active":"2023-11-24T08:06:06.251015+0000","last_peered":"2023-11-24T08:06:06.251015+0000","last_clean":"2023-11-24T08:06:06.251015+0000","last_became_active":"2023-11-24T08:06:05.087486+0000","last_became_peered":"2023-11-24T08:06:05.087486+0000","last_unstale":"2023-11-24T08:06:06.251015+0000","last_undegraded":"2023-11-24T08:06:06.251015+0000","last_fullsized":"2023-11-24T08:06:06.251015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:46.775040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248159+0000","last_change":"2023-11-24T08:06:05.093356+0000","last_active":"2023-11-24T08:06:06.248159+0000","last_peered":"2023-11-24T08:06:06.248159+0000","last_clean":"2023-11-24T08:06:06.248159+0000","last_became_active":"2023-11-24T08:06:05.093036+0000","last_became_peered":"2023-11-24T08:06:05.093036+0000","last_unstale":"2023-11-24T08:06:06.248159+0000","last_undegraded":"2023-11-24T08:06:06.248159+0000","last_fullsized":"2023-11-24T08:06:06.248159+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:44.380303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.083398+0000","last_change":"2023-11-24T08:06:05.084537+0000","last_active":"2023-11-24T08:06:06.083398+0000","last_peered":"2023-11-24T08:06:06.083398+0000","last_clean":"2023-11-24T08:06:06.083398+0000","last_became_active":"2023-11-24T08:06:05.084268+0000","last_became_peered":"2023-11-24T08:06:05.084268+0000","last_unstale":"2023-11-24T08:06:06.083398+0000","last_undegraded":"2023-11-24T08:06:06.083398+0000","last_fullsized":"2023-11-24T08:06:06.083398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:52.364691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120766+0000","last_change":"2023-11-24T08:06:04.082144+0000","last_active":"2023-11-24T08:06:05.120766+0000","last_peered":"2023-11-24T08:06:05.120766+0000","last_clean":"2023-11-24T08:06:05.120766+0000","last_became_active":"2023-11-24T08:06:04.081510+0000","last_became_peered":"2023-11-24T08:06:04.081510+0000","last_unstale":"2023-11-24T08:06:05.120766+0000","last_undegraded":"2023-11-24T08:06:05.120766+0000","last_fullsized":"2023-11-24T08:06:05.120766+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:44.512063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089609+0000","last_change":"2023-11-24T08:06:05.097573+0000","last_active":"2023-11-24T08:06:06.089609+0000","last_peered":"2023-11-24T08:06:06.089609+0000","last_clean":"2023-11-24T08:06:06.089609+0000","last_became_active":"2023-11-24T08:06:05.094728+0000","last_became_peered":"2023-11-24T08:06:05.094728+0000","last_unstale":"2023-11-24T08:06:06.089609+0000","last_undegraded":"2023-11-24T08:06:06.089609+0000","last_fullsized":"2023-11-24T08:06:06.089609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.114292+0000","last_change":"2023-11-24T08:06:04.084082+0000","last_active":"2023-11-24T08:06:05.114292+0000","last_peered":"2023-11-24T08:06:05.114292+0000","last_clean":"2023-11-24T08:06:05.114292+0000","last_became_active":"2023-11-24T08:06:04.083299+0000","last_became_peered":"2023-11-24T08:06:04.083299+0000","last_unstale":"2023-11-24T08:06:05.114292+0000","last_undegraded":"2023-11-24T08:06:05.114292+0000","last_fullsized":"2023-11-24T08:06:05.114292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251872+0000","last_change":"2023-11-24T08:06:05.096613+0000","last_active":"2023-11-24T08:06:06.251872+0000","last_peered":"2023-11-24T08:06:06.251872+0000","last_clean":"2023-11-24T08:06:06.251872+0000","last_became_active":"2023-11-24T08:06:05.096353+0000","last_became_peered":"2023-11-24T08:06:05.096353+0000","last_unstale":"2023-11-24T08:06:06.251872+0000","last_undegraded":"2023-11-24T08:06:06.251872+0000","last_fullsized":"2023-11-24T08:06:06.251872+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.115949+0000","last_change":"2023-11-24T08:06:04.087280+0000","last_active":"2023-11-24T08:06:05.115949+0000","last_peered":"2023-11-24T08:06:05.115949+0000","last_clean":"2023-11-24T08:06:05.115949+0000","last_became_active":"2023-11-24T08:06:04.086942+0000","last_became_peered":"2023-11-24T08:06:04.086942+0000","last_unstale":"2023-11-24T08:06:05.115949+0000","last_undegraded":"2023-11-24T08:06:05.115949+0000","last_fullsized":"2023-11-24T08:06:05.115949+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.083315+0000","last_change":"2023-11-24T08:06:05.086813+0000","last_active":"2023-11-24T08:06:06.083315+0000","last_peered":"2023-11-24T08:06:06.083315+0000","last_clean":"2023-11-24T08:06:06.083315+0000","last_became_active":"2023-11-24T08:06:05.086506+0000","last_became_peered":"2023-11-24T08:06:05.086506+0000","last_unstale":"2023-11-24T08:06:06.083315+0000","last_undegraded":"2023-11-24T08:06:06.083315+0000","last_fullsized":"2023-11-24T08:06:06.083315+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:21.753543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122782+0000","last_change":"2023-11-24T08:06:04.094159+0000","last_active":"2023-11-24T08:06:05.122782+0000","last_peered":"2023-11-24T08:06:05.122782+0000","last_clean":"2023-11-24T08:06:05.122782+0000","last_became_active":"2023-11-24T08:06:04.093809+0000","last_became_peered":"2023-11-24T08:06:04.093809+0000","last_unstale":"2023-11-24T08:06:05.122782+0000","last_undegraded":"2023-11-24T08:06:05.122782+0000","last_fullsized":"2023-11-24T08:06:05.122782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:54.440803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087650+0000","last_change":"2023-11-24T08:06:05.101827+0000","last_active":"2023-11-24T08:06:06.087650+0000","last_peered":"2023-11-24T08:06:06.087650+0000","last_clean":"2023-11-24T08:06:06.087650+0000","last_became_active":"2023-11-24T08:06:05.101501+0000","last_became_peered":"2023-11-24T08:06:05.101501+0000","last_unstale":"2023-11-24T08:06:06.087650+0000","last_undegraded":"2023-11-24T08:06:06.087650+0000","last_fullsized":"2023-11-24T08:06:06.087650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120725+0000","last_change":"2023-11-24T08:06:04.090249+0000","last_active":"2023-11-24T08:06:05.120725+0000","last_peered":"2023-11-24T08:06:05.120725+0000","last_clean":"2023-11-24T08:06:05.120725+0000","last_became_active":"2023-11-24T08:06:04.089893+0000","last_became_peered":"2023-11-24T08:06:04.089893+0000","last_unstale":"2023-11-24T08:06:05.120725+0000","last_undegraded":"2023-11-24T08:06:05.120725+0000","last_fullsized":"2023-11-24T08:06:05.120725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:27.488388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252382+0000","last_change":"2023-11-24T08:06:05.093052+0000","last_active":"2023-11-24T08:06:06.252382+0000","last_peered":"2023-11-24T08:06:06.252382+0000","last_clean":"2023-11-24T08:06:06.252382+0000","last_became_active":"2023-11-24T08:06:05.091759+0000","last_became_peered":"2023-11-24T08:06:05.091759+0000","last_unstale":"2023-11-24T08:06:06.252382+0000","last_undegraded":"2023-11-24T08:06:06.252382+0000","last_fullsized":"2023-11-24T08:06:06.252382+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:28.502892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.112139+0000","last_change":"2023-11-24T08:06:04.087131+0000","last_active":"2023-11-24T08:06:05.112139+0000","last_peered":"2023-11-24T08:06:05.112139+0000","last_clean":"2023-11-24T08:06:05.112139+0000","last_became_active":"2023-11-24T08:06:04.086323+0000","last_became_peered":"2023-11-24T08:06:04.086323+0000","last_unstale":"2023-11-24T08:06:05.112139+0000","last_undegraded":"2023-11-24T08:06:05.112139+0000","last_fullsized":"2023-11-24T08:06:05.112139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:01.889219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251937+0000","last_change":"2023-11-24T08:06:05.097611+0000","last_active":"2023-11-24T08:06:06.251937+0000","last_peered":"2023-11-24T08:06:06.251937+0000","last_clean":"2023-11-24T08:06:06.251937+0000","last_became_active":"2023-11-24T08:06:05.094003+0000","last_became_peered":"2023-11-24T08:06:05.094003+0000","last_unstale":"2023-11-24T08:06:06.251937+0000","last_undegraded":"2023-11-24T08:06:06.251937+0000","last_fullsized":"2023-11-24T08:06:06.251937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.090125+0000","last_change":"2023-11-24T08:06:04.085591+0000","last_active":"2023-11-24T08:06:05.090125+0000","last_peered":"2023-11-24T08:06:05.090125+0000","last_clean":"2023-11-24T08:06:05.090125+0000","last_became_active":"2023-11-24T08:06:04.085328+0000","last_became_peered":"2023-11-24T08:06:04.085328+0000","last_unstale":"2023-11-24T08:06:05.090125+0000","last_undegraded":"2023-11-24T08:06:05.090125+0000","last_fullsized":"2023-11-24T08:06:05.090125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:35.592076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087957+0000","last_change":"2023-11-24T08:06:05.080325+0000","last_active":"2023-11-24T08:06:06.087957+0000","last_peered":"2023-11-24T08:06:06.087957+0000","last_clean":"2023-11-24T08:06:06.087957+0000","last_became_active":"2023-11-24T08:06:05.079874+0000","last_became_peered":"2023-11-24T08:06:05.079874+0000","last_unstale":"2023-11-24T08:06:06.087957+0000","last_undegraded":"2023-11-24T08:06:06.087957+0000","last_fullsized":"2023-11-24T08:06:06.087957+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:52.364691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.076622+0000","last_change":"2023-11-24T08:06:04.088494+0000","last_active":"2023-11-24T08:06:05.076622+0000","last_peered":"2023-11-24T08:06:05.076622+0000","last_clean":"2023-11-24T08:06:05.076622+0000","last_became_active":"2023-11-24T08:06:04.088029+0000","last_became_peered":"2023-11-24T08:06:04.088029+0000","last_unstale":"2023-11-24T08:06:05.076622+0000","last_undegraded":"2023-11-24T08:06:05.076622+0000","last_fullsized":"2023-11-24T08:06:05.076622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:13.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089650+0000","last_change":"2023-11-24T08:06:05.095670+0000","last_active":"2023-11-24T08:06:06.089650+0000","last_peered":"2023-11-24T08:06:06.089650+0000","last_clean":"2023-11-24T08:06:06.089650+0000","last_became_active":"2023-11-24T08:06:05.095367+0000","last_became_peered":"2023-11-24T08:06:05.095367+0000","last_unstale":"2023-11-24T08:06:06.089650+0000","last_undegraded":"2023-11-24T08:06:06.089650+0000","last_fullsized":"2023-11-24T08:06:06.089650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.086734+0000","last_change":"2023-11-24T08:06:04.074104+0000","last_active":"2023-11-24T08:06:05.086734+0000","last_peered":"2023-11-24T08:06:05.086734+0000","last_clean":"2023-11-24T08:06:05.086734+0000","last_became_active":"2023-11-24T08:06:04.073646+0000","last_became_peered":"2023-11-24T08:06:04.073646+0000","last_unstale":"2023-11-24T08:06:05.086734+0000","last_undegraded":"2023-11-24T08:06:05.086734+0000","last_fullsized":"2023-11-24T08:06:05.086734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:14.570476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253768+0000","last_change":"2023-11-24T08:06:05.095886+0000","last_active":"2023-11-24T08:06:06.253768+0000","last_peered":"2023-11-24T08:06:06.253768+0000","last_clean":"2023-11-24T08:06:06.253768+0000","last_became_active":"2023-11-24T08:06:05.095590+0000","last_became_peered":"2023-11-24T08:06:05.095590+0000","last_unstale":"2023-11-24T08:06:06.253768+0000","last_undegraded":"2023-11-24T08:06:06.253768+0000","last_fullsized":"2023-11-24T08:06:06.253768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:34.594891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.082395+0000","last_change":"2023-11-24T08:06:04.086787+0000","last_active":"2023-11-24T08:06:05.082395+0000","last_peered":"2023-11-24T08:06:05.082395+0000","last_clean":"2023-11-24T08:06:05.082395+0000","last_became_active":"2023-11-24T08:06:04.086503+0000","last_became_peered":"2023-11-24T08:06:04.086503+0000","last_unstale":"2023-11-24T08:06:05.082395+0000","last_undegraded":"2023-11-24T08:06:05.082395+0000","last_fullsized":"2023-11-24T08:06:05.082395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251580+0000","last_change":"2023-11-24T08:06:05.094679+0000","last_active":"2023-11-24T08:06:06.251580+0000","last_peered":"2023-11-24T08:06:06.251580+0000","last_clean":"2023-11-24T08:06:06.251580+0000","last_became_active":"2023-11-24T08:06:05.089982+0000","last_became_peered":"2023-11-24T08:06:05.089982+0000","last_unstale":"2023-11-24T08:06:06.251580+0000","last_undegraded":"2023-11-24T08:06:06.251580+0000","last_fullsized":"2023-11-24T08:06:06.251580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:01.848501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122469+0000","last_change":"2023-11-24T08:06:04.094281+0000","last_active":"2023-11-24T08:06:05.122469+0000","last_peered":"2023-11-24T08:06:05.122469+0000","last_clean":"2023-11-24T08:06:05.122469+0000","last_became_active":"2023-11-24T08:06:04.093946+0000","last_became_peered":"2023-11-24T08:06:04.093946+0000","last_unstale":"2023-11-24T08:06:05.122469+0000","last_undegraded":"2023-11-24T08:06:05.122469+0000","last_fullsized":"2023-11-24T08:06:05.122469+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:28.559491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253369+0000","last_change":"2023-11-24T08:06:05.079345+0000","last_active":"2023-11-24T08:06:06.253369+0000","last_peered":"2023-11-24T08:06:06.253369+0000","last_clean":"2023-11-24T08:06:06.253369+0000","last_became_active":"2023-11-24T08:06:05.078273+0000","last_became_peered":"2023-11-24T08:06:05.078273+0000","last_unstale":"2023-11-24T08:06:06.253369+0000","last_undegraded":"2023-11-24T08:06:06.253369+0000","last_fullsized":"2023-11-24T08:06:06.253369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:59.520725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.125088+0000","last_change":"2023-11-24T08:06:04.080902+0000","last_active":"2023-11-24T08:06:05.125088+0000","last_peered":"2023-11-24T08:06:05.125088+0000","last_clean":"2023-11-24T08:06:05.125088+0000","last_became_active":"2023-11-24T08:06:04.080616+0000","last_became_peered":"2023-11-24T08:06:04.080616+0000","last_unstale":"2023-11-24T08:06:05.125088+0000","last_undegraded":"2023-11-24T08:06:05.125088+0000","last_fullsized":"2023-11-24T08:06:05.125088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:24.477858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088485+0000","last_change":"2023-11-24T08:06:05.093746+0000","last_active":"2023-11-24T08:06:06.088485+0000","last_peered":"2023-11-24T08:06:06.088485+0000","last_clean":"2023-11-24T08:06:06.088485+0000","last_became_active":"2023-11-24T08:06:05.093440+0000","last_became_peered":"2023-11-24T08:06:05.093440+0000","last_unstale":"2023-11-24T08:06:06.088485+0000","last_undegraded":"2023-11-24T08:06:06.088485+0000","last_fullsized":"2023-11-24T08:06:06.088485+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:10.661913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.121974+0000","last_change":"2023-11-24T08:06:04.084295+0000","last_active":"2023-11-24T08:06:05.121974+0000","last_peered":"2023-11-24T08:06:05.121974+0000","last_clean":"2023-11-24T08:06:05.121974+0000","last_became_active":"2023-11-24T08:06:04.083328+0000","last_became_peered":"2023-11-24T08:06:04.083328+0000","last_unstale":"2023-11-24T08:06:05.121974+0000","last_undegraded":"2023-11-24T08:06:05.121974+0000","last_fullsized":"2023-11-24T08:06:05.121974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:58.506222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252513+0000","last_change":"2023-11-24T08:06:05.095494+0000","last_active":"2023-11-24T08:06:06.252513+0000","last_peered":"2023-11-24T08:06:06.252513+0000","last_clean":"2023-11-24T08:06:06.252513+0000","last_became_active":"2023-11-24T08:06:05.095195+0000","last_became_peered":"2023-11-24T08:06:05.095195+0000","last_unstale":"2023-11-24T08:06:06.252513+0000","last_undegraded":"2023-11-24T08:06:06.252513+0000","last_fullsized":"2023-11-24T08:06:06.252513+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:55.876487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063734+0000","last_change":"2023-11-24T08:06:04.088130+0000","last_active":"2023-11-24T08:06:05.063734+0000","last_peered":"2023-11-24T08:06:05.063734+0000","last_clean":"2023-11-24T08:06:05.063734+0000","last_became_active":"2023-11-24T08:06:04.087804+0000","last_became_peered":"2023-11-24T08:06:04.087804+0000","last_unstale":"2023-11-24T08:06:05.063734+0000","last_undegraded":"2023-11-24T08:06:05.063734+0000","last_fullsized":"2023-11-24T08:06:05.063734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250881+0000","last_change":"2023-11-24T08:06:05.102561+0000","last_active":"2023-11-24T08:06:06.250881+0000","last_peered":"2023-11-24T08:06:06.250881+0000","last_clean":"2023-11-24T08:06:06.250881+0000","last_became_active":"2023-11-24T08:06:05.100166+0000","last_became_peered":"2023-11-24T08:06:05.100166+0000","last_unstale":"2023-11-24T08:06:06.250881+0000","last_undegraded":"2023-11-24T08:06:06.250881+0000","last_fullsized":"2023-11-24T08:06:06.250881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:32.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.068470+0000","last_change":"2023-11-24T08:06:04.090915+0000","last_active":"2023-11-24T08:06:05.068470+0000","last_peered":"2023-11-24T08:06:05.068470+0000","last_clean":"2023-11-24T08:06:05.068470+0000","last_became_active":"2023-11-24T08:06:04.090279+0000","last_became_peered":"2023-11-24T08:06:04.090279+0000","last_unstale":"2023-11-24T08:06:05.068470+0000","last_undegraded":"2023-11-24T08:06:05.068470+0000","last_fullsized":"2023-11-24T08:06:05.068470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:33.221758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252809+0000","last_change":"2023-11-24T08:06:05.091728+0000","last_active":"2023-11-24T08:06:06.252809+0000","last_peered":"2023-11-24T08:06:06.252809+0000","last_clean":"2023-11-24T08:06:06.252809+0000","last_became_active":"2023-11-24T08:06:05.090643+0000","last_became_peered":"2023-11-24T08:06:05.090643+0000","last_unstale":"2023-11-24T08:06:06.252809+0000","last_undegraded":"2023-11-24T08:06:06.252809+0000","last_fullsized":"2023-11-24T08:06:06.252809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:45.526567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122459+0000","last_change":"2023-11-24T08:06:04.088343+0000","last_active":"2023-11-24T08:06:05.122459+0000","last_peered":"2023-11-24T08:06:05.122459+0000","last_clean":"2023-11-24T08:06:05.122459+0000","last_became_active":"2023-11-24T08:06:04.087687+0000","last_became_peered":"2023-11-24T08:06:04.087687+0000","last_unstale":"2023-11-24T08:06:05.122459+0000","last_undegraded":"2023-11-24T08:06:05.122459+0000","last_fullsized":"2023-11-24T08:06:05.122459+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248883+0000","last_change":"2023-11-24T08:06:05.097524+0000","last_active":"2023-11-24T08:06:06.248883+0000","last_peered":"2023-11-24T08:06:06.248883+0000","last_clean":"2023-11-24T08:06:06.248883+0000","last_became_active":"2023-11-24T08:06:05.095942+0000","last_became_peered":"2023-11-24T08:06:05.095942+0000","last_unstale":"2023-11-24T08:06:06.248883+0000","last_undegraded":"2023-11-24T08:06:06.248883+0000","last_fullsized":"2023-11-24T08:06:06.248883+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:57:14.392741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.121968+0000","last_change":"2023-11-24T08:06:04.071810+0000","last_active":"2023-11-24T08:06:05.121968+0000","last_peered":"2023-11-24T08:06:05.121968+0000","last_clean":"2023-11-24T08:06:05.121968+0000","last_became_active":"2023-11-24T08:06:04.071196+0000","last_became_peered":"2023-11-24T08:06:04.071196+0000","last_unstale":"2023-11-24T08:06:05.121968+0000","last_undegraded":"2023-11-24T08:06:05.121968+0000","last_fullsized":"2023-11-24T08:06:05.121968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:13:43.496491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252462+0000","last_change":"2023-11-24T08:06:05.093265+0000","last_active":"2023-11-24T08:06:06.252462+0000","last_peered":"2023-11-24T08:06:06.252462+0000","last_clean":"2023-11-24T08:06:06.252462+0000","last_became_active":"2023-11-24T08:06:05.092904+0000","last_became_peered":"2023-11-24T08:06:05.092904+0000","last_unstale":"2023-11-24T08:06:06.252462+0000","last_undegraded":"2023-11-24T08:06:06.252462+0000","last_fullsized":"2023-11-24T08:06:06.252462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:59.520725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.113304+0000","last_change":"2023-11-24T08:06:04.081460+0000","last_active":"2023-11-24T08:06:05.113304+0000","last_peered":"2023-11-24T08:06:05.113304+0000","last_clean":"2023-11-24T08:06:05.113304+0000","last_became_active":"2023-11-24T08:06:04.081176+0000","last_became_peered":"2023-11-24T08:06:04.081176+0000","last_unstale":"2023-11-24T08:06:05.113304+0000","last_undegraded":"2023-11-24T08:06:05.113304+0000","last_fullsized":"2023-11-24T08:06:05.113304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:11.044772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089003+0000","last_change":"2023-11-24T08:06:05.086676+0000","last_active":"2023-11-24T08:06:06.089003+0000","last_peered":"2023-11-24T08:06:06.089003+0000","last_clean":"2023-11-24T08:06:06.089003+0000","last_became_active":"2023-11-24T08:06:05.086342+0000","last_became_peered":"2023-11-24T08:06:05.086342+0000","last_unstale":"2023-11-24T08:06:06.089003+0000","last_undegraded":"2023-11-24T08:06:06.089003+0000","last_fullsized":"2023-11-24T08:06:06.089003+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:36.606580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120951+0000","last_change":"2023-11-24T08:06:04.085453+0000","last_active":"2023-11-24T08:06:05.120951+0000","last_peered":"2023-11-24T08:06:05.120951+0000","last_clean":"2023-11-24T08:06:05.120951+0000","last_became_active":"2023-11-24T08:06:04.085126+0000","last_became_peered":"2023-11-24T08:06:04.085126+0000","last_unstale":"2023-11-24T08:06:05.120951+0000","last_undegraded":"2023-11-24T08:06:05.120951+0000","last_fullsized":"2023-11-24T08:06:05.120951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089297+0000","last_change":"2023-11-24T08:06:05.092694+0000","last_active":"2023-11-24T08:06:06.089297+0000","last_peered":"2023-11-24T08:06:06.089297+0000","last_clean":"2023-11-24T08:06:06.089297+0000","last_became_active":"2023-11-24T08:06:05.092393+0000","last_became_peered":"2023-11-24T08:06:05.092393+0000","last_unstale":"2023-11-24T08:06:06.089297+0000","last_undegraded":"2023-11-24T08:06:06.089297+0000","last_fullsized":"2023-11-24T08:06:06.089297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:15.584979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.071093+0000","last_change":"2023-11-24T08:06:04.085155+0000","last_active":"2023-11-24T08:06:05.071093+0000","last_peered":"2023-11-24T08:06:05.071093+0000","last_clean":"2023-11-24T08:06:05.071093+0000","last_became_active":"2023-11-24T08:06:04.084831+0000","last_became_peered":"2023-11-24T08:06:04.084831+0000","last_unstale":"2023-11-24T08:06:05.071093+0000","last_undegraded":"2023-11-24T08:06:05.071093+0000","last_fullsized":"2023-11-24T08:06:05.071093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:33.580388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251386+0000","last_change":"2023-11-24T08:06:05.101508+0000","last_active":"2023-11-24T08:06:06.251386+0000","last_peered":"2023-11-24T08:06:06.251386+0000","last_clean":"2023-11-24T08:06:06.251386+0000","last_became_active":"2023-11-24T08:06:05.099363+0000","last_became_peered":"2023-11-24T08:06:05.099363+0000","last_unstale":"2023-11-24T08:06:06.251386+0000","last_undegraded":"2023-11-24T08:06:06.251386+0000","last_fullsized":"2023-11-24T08:06:06.251386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:42.213975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120853+0000","last_change":"2023-11-24T08:06:04.087536+0000","last_active":"2023-11-24T08:06:05.120853+0000","last_peered":"2023-11-24T08:06:05.120853+0000","last_clean":"2023-11-24T08:06:05.120853+0000","last_became_active":"2023-11-24T08:06:04.087269+0000","last_became_peered":"2023-11-24T08:06:04.087269+0000","last_unstale":"2023-11-24T08:06:05.120853+0000","last_undegraded":"2023-11-24T08:06:05.120853+0000","last_fullsized":"2023-11-24T08:06:05.120853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:29.354205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.082906+0000","last_change":"2023-11-24T08:06:05.093409+0000","last_active":"2023-11-24T08:06:06.082906+0000","last_peered":"2023-11-24T08:06:06.082906+0000","last_clean":"2023-11-24T08:06:06.082906+0000","last_became_active":"2023-11-24T08:06:05.090895+0000","last_became_peered":"2023-11-24T08:06:05.090895+0000","last_unstale":"2023-11-24T08:06:06.082906+0000","last_undegraded":"2023-11-24T08:06:06.082906+0000","last_fullsized":"2023-11-24T08:06:06.082906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:12.059275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.090278+0000","last_change":"2023-11-24T08:06:04.085441+0000","last_active":"2023-11-24T08:06:05.090278+0000","last_peered":"2023-11-24T08:06:05.090278+0000","last_clean":"2023-11-24T08:06:05.090278+0000","last_became_active":"2023-11-24T08:06:04.085170+0000","last_became_peered":"2023-11-24T08:06:04.085170+0000","last_unstale":"2023-11-24T08:06:05.090278+0000","last_undegraded":"2023-11-24T08:06:05.090278+0000","last_fullsized":"2023-11-24T08:06:05.090278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:14.570476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248353+0000","last_change":"2023-11-24T08:06:05.095928+0000","last_active":"2023-11-24T08:06:06.248353+0000","last_peered":"2023-11-24T08:06:06.248353+0000","last_clean":"2023-11-24T08:06:06.248353+0000","last_became_active":"2023-11-24T08:06:05.095601+0000","last_became_peered":"2023-11-24T08:06:05.095601+0000","last_unstale":"2023-11-24T08:06:06.248353+0000","last_undegraded":"2023-11-24T08:06:06.248353+0000","last_fullsized":"2023-11-24T08:06:06.248353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:33.727928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.083543+0000","last_change":"2023-11-24T08:06:04.086214+0000","last_active":"2023-11-24T08:06:05.083543+0000","last_peered":"2023-11-24T08:06:05.083543+0000","last_clean":"2023-11-24T08:06:05.083543+0000","last_became_active":"2023-11-24T08:06:04.085918+0000","last_became_peered":"2023-11-24T08:06:04.085918+0000","last_unstale":"2023-11-24T08:06:05.083543+0000","last_undegraded":"2023-11-24T08:06:05.083543+0000","last_fullsized":"2023-11-24T08:06:05.083543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:54.440803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251927+0000","last_change":"2023-11-24T08:06:05.095494+0000","last_active":"2023-11-24T08:06:06.251927+0000","last_peered":"2023-11-24T08:06:06.251927+0000","last_clean":"2023-11-24T08:06:06.251927+0000","last_became_active":"2023-11-24T08:06:05.094309+0000","last_became_peered":"2023-11-24T08:06:05.094309+0000","last_unstale":"2023-11-24T08:06:06.251927+0000","last_undegraded":"2023-11-24T08:06:06.251927+0000","last_fullsized":"2023-11-24T08:06:06.251927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063326+0000","last_change":"2023-11-24T08:06:04.093116+0000","last_active":"2023-11-24T08:06:05.063326+0000","last_peered":"2023-11-24T08:06:05.063326+0000","last_clean":"2023-11-24T08:06:05.063326+0000","last_became_active":"2023-11-24T08:06:04.092446+0000","last_became_peered":"2023-11-24T08:06:04.092446+0000","last_unstale":"2023-11-24T08:06:05.063326+0000","last_undegraded":"2023-11-24T08:06:05.063326+0000","last_fullsized":"2023-11-24T08:06:05.063326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:35.592076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089777+0000","last_change":"2023-11-24T08:06:05.093077+0000","last_active":"2023-11-24T08:06:06.089777+0000","last_peered":"2023-11-24T08:06:06.089777+0000","last_clean":"2023-11-24T08:06:06.089777+0000","last_became_active":"2023-11-24T08:06:05.092754+0000","last_became_peered":"2023-11-24T08:06:05.092754+0000","last_unstale":"2023-11-24T08:06:06.089777+0000","last_undegraded":"2023-11-24T08:06:06.089777+0000","last_fullsized":"2023-11-24T08:06:06.089777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.074657+0000","last_change":"2023-11-24T08:06:04.079922+0000","last_active":"2023-11-24T08:06:05.074657+0000","last_peered":"2023-11-24T08:06:05.074657+0000","last_clean":"2023-11-24T08:06:05.074657+0000","last_became_active":"2023-11-24T08:06:04.079489+0000","last_became_peered":"2023-11-24T08:06:04.079489+0000","last_unstale":"2023-11-24T08:06:05.074657+0000","last_undegraded":"2023-11-24T08:06:05.074657+0000","last_fullsized":"2023-11-24T08:06:05.074657+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:04.038305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252779+0000","last_change":"2023-11-24T08:06:05.105800+0000","last_active":"2023-11-24T08:06:06.252779+0000","last_peered":"2023-11-24T08:06:06.252779+0000","last_clean":"2023-11-24T08:06:06.252779+0000","last_became_active":"2023-11-24T08:06:05.105521+0000","last_became_peered":"2023-11-24T08:06:05.105521+0000","last_unstale":"2023-11-24T08:06:06.252779+0000","last_undegraded":"2023-11-24T08:06:06.252779+0000","last_fullsized":"2023-11-24T08:06:06.252779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:32.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.067003+0000","last_change":"2023-11-24T08:06:04.094274+0000","last_active":"2023-11-24T08:06:05.067003+0000","last_peered":"2023-11-24T08:06:05.067003+0000","last_clean":"2023-11-24T08:06:05.067003+0000","last_became_active":"2023-11-24T08:06:04.093959+0000","last_became_peered":"2023-11-24T08:06:04.093959+0000","last_unstale":"2023-11-24T08:06:05.067003+0000","last_undegraded":"2023-11-24T08:06:05.067003+0000","last_fullsized":"2023-11-24T08:06:05.067003+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:33.221758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253430+0000","last_change":"2023-11-24T08:06:05.082611+0000","last_active":"2023-11-24T08:06:06.253430+0000","last_peered":"2023-11-24T08:06:06.253430+0000","last_clean":"2023-11-24T08:06:06.253430+0000","last_became_active":"2023-11-24T08:06:05.082314+0000","last_became_peered":"2023-11-24T08:06:05.082314+0000","last_unstale":"2023-11-24T08:06:06.253430+0000","last_undegraded":"2023-11-24T08:06:06.253430+0000","last_fullsized":"2023-11-24T08:06:06.253430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:30.368708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.060457+0000","last_change":"2023-11-24T08:06:04.087054+0000","last_active":"2023-11-24T08:06:05.060457+0000","last_peered":"2023-11-24T08:06:05.060457+0000","last_clean":"2023-11-24T08:06:05.060457+0000","last_became_active":"2023-11-24T08:06:04.086627+0000","last_became_peered":"2023-11-24T08:06:04.086627+0000","last_unstale":"2023-11-24T08:06:05.060457+0000","last_undegraded":"2023-11-24T08:06:05.060457+0000","last_fullsized":"2023-11-24T08:06:05.060457+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:01.324343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251497+0000","last_change":"2023-11-24T08:06:05.090923+0000","last_active":"2023-11-24T08:06:06.251497+0000","last_peered":"2023-11-24T08:06:06.251497+0000","last_clean":"2023-11-24T08:06:06.251497+0000","last_became_active":"2023-11-24T08:06:05.089665+0000","last_became_peered":"2023-11-24T08:06:05.089665+0000","last_unstale":"2023-11-24T08:06:06.251497+0000","last_undegraded":"2023-11-24T08:06:06.251497+0000","last_fullsized":"2023-11-24T08:06:06.251497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:11.077626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.064897+0000","last_change":"2023-11-24T08:06:04.094285+0000","last_active":"2023-11-24T08:06:05.064897+0000","last_peered":"2023-11-24T08:06:05.064897+0000","last_clean":"2023-11-24T08:06:05.064897+0000","last_became_active":"2023-11-24T08:06:04.093933+0000","last_became_peered":"2023-11-24T08:06:04.093933+0000","last_unstale":"2023-11-24T08:06:05.064897+0000","last_undegraded":"2023-11-24T08:06:05.064897+0000","last_fullsized":"2023-11-24T08:06:05.064897+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:36.353720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251553+0000","last_change":"2023-11-24T08:06:05.086959+0000","last_active":"2023-11-24T08:06:06.251553+0000","last_peered":"2023-11-24T08:06:06.251553+0000","last_clean":"2023-11-24T08:06:06.251553+0000","last_became_active":"2023-11-24T08:06:05.085952+0000","last_became_peered":"2023-11-24T08:06:05.085952+0000","last_unstale":"2023-11-24T08:06:06.251553+0000","last_undegraded":"2023-11-24T08:06:06.251553+0000","last_fullsized":"2023-11-24T08:06:06.251553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:59.924717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.081417+0000","last_change":"2023-11-24T08:06:04.070754+0000","last_active":"2023-11-24T08:06:05.081417+0000","last_peered":"2023-11-24T08:06:05.081417+0000","last_clean":"2023-11-24T08:06:05.081417+0000","last_became_active":"2023-11-24T08:06:04.070430+0000","last_became_peered":"2023-11-24T08:06:04.070430+0000","last_unstale":"2023-11-24T08:06:05.081417+0000","last_undegraded":"2023-11-24T08:06:05.081417+0000","last_fullsized":"2023-11-24T08:06:05.081417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:09.647409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250913+0000","last_change":"2023-11-24T08:06:05.093665+0000","last_active":"2023-11-24T08:06:06.250913+0000","last_peered":"2023-11-24T08:06:06.250913+0000","last_clean":"2023-11-24T08:06:06.250913+0000","last_became_active":"2023-11-24T08:06:05.093393+0000","last_became_peered":"2023-11-24T08:06:05.093393+0000","last_unstale":"2023-11-24T08:06:06.250913+0000","last_undegraded":"2023-11-24T08:06:06.250913+0000","last_fullsized":"2023-11-24T08:06:06.250913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.061092+0000","last_change":"2023-11-24T08:06:04.084482+0000","last_active":"2023-11-24T08:06:05.061092+0000","last_peered":"2023-11-24T08:06:05.061092+0000","last_clean":"2023-11-24T08:06:05.061092+0000","last_became_active":"2023-11-24T08:06:04.084022+0000","last_became_peered":"2023-11-24T08:06:04.084022+0000","last_unstale":"2023-11-24T08:06:05.061092+0000","last_undegraded":"2023-11-24T08:06:05.061092+0000","last_fullsized":"2023-11-24T08:06:05.061092+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:51:33.198531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087356+0000","last_change":"2023-11-24T08:06:05.093359+0000","last_active":"2023-11-24T08:06:06.087356+0000","last_peered":"2023-11-24T08:06:06.087356+0000","last_clean":"2023-11-24T08:06:06.087356+0000","last_became_active":"2023-11-24T08:06:05.093091+0000","last_became_peered":"2023-11-24T08:06:05.093091+0000","last_unstale":"2023-11-24T08:06:06.087356+0000","last_undegraded":"2023-11-24T08:06:06.087356+0000","last_fullsized":"2023-11-24T08:06:06.087356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:21.615036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.077077+0000","last_change":"2023-11-24T08:06:04.081603+0000","last_active":"2023-11-24T08:06:05.077077+0000","last_peered":"2023-11-24T08:06:05.077077+0000","last_clean":"2023-11-24T08:06:05.077077+0000","last_became_active":"2023-11-24T08:06:04.081376+0000","last_became_peered":"2023-11-24T08:06:04.081376+0000","last_unstale":"2023-11-24T08:06:05.077077+0000","last_undegraded":"2023-11-24T08:06:05.077077+0000","last_fullsized":"2023-11-24T08:06:05.077077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:11.044772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252755+0000","last_change":"2023-11-24T08:06:05.095041+0000","last_active":"2023-11-24T08:06:06.252755+0000","last_peered":"2023-11-24T08:06:06.252755+0000","last_clean":"2023-11-24T08:06:06.252755+0000","last_became_active":"2023-11-24T08:06:05.094688+0000","last_became_peered":"2023-11-24T08:06:05.094688+0000","last_unstale":"2023-11-24T08:06:06.252755+0000","last_undegraded":"2023-11-24T08:06:06.252755+0000","last_fullsized":"2023-11-24T08:06:06.252755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:49.120781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122403+0000","last_change":"2023-11-24T08:06:04.088807+0000","last_active":"2023-11-24T08:06:05.122403+0000","last_peered":"2023-11-24T08:06:05.122403+0000","last_clean":"2023-11-24T08:06:05.122403+0000","last_became_active":"2023-11-24T08:06:04.087267+0000","last_became_peered":"2023-11-24T08:06:04.087267+0000","last_unstale":"2023-11-24T08:06:05.122403+0000","last_undegraded":"2023-11-24T08:06:05.122403+0000","last_fullsized":"2023-11-24T08:06:05.122403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:46.590933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063198+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:06:05.063198+0000","last_peered":"2023-11-24T08:06:05.063198+0000","last_clean":"2023-11-24T08:06:05.063198+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:06:05.063198+0000","last_undegraded":"2023-11-24T08:06:05.063198+0000","last_fullsized":"2023-11-24T08:06:05.063198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.65300000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.431},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.069,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3200000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7649999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4179999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.5569999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1539999999999999,"15min":1.149},"min":{"1min":0.80600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.8660000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.80300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8879999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.2270000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3120000000000001,"15min":1.3300000000000001},"min":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5109999999999999,"5min":2.524,"15min":2.524},"last":1.498},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.504,"15min":1.534},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7709999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3169999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.90800000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.613,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3799999999999999,"15min":1.3839999999999999},"min":{"1min":0.87,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0270000000000001,"5min":2.238,"15min":2.238},"last":1.4510000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.619,"15min":1.6319999999999999},"min":{"1min":1.1839999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.383},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.647,"15min":1.677},"min":{"1min":1.0489999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7250000000000001,"5min":2.944,"15min":2.944},"last":1.633}]},{"osd":6,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.7350000000000001,"15min":1.77},"min":{"1min":1.1699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.891,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.671,"15min":1.6870000000000001},"min":{"1min":1.179,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1589999999999998,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.4059999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":215941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.538,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.323,"15min":1.355},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5629999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.4610000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1499999999999999,"15min":1.1579999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.304},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.274,"15min":1.319},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.6830000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.282,"15min":1.3129999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0710000000000002,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.4750000000000001,"15min":1.526},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.5509999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.5860000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.4299999999999999,"15min":1.458},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.554,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.081},{"interface":"front","average":{"1min":1.401,"5min":1.49,"15min":1.5049999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.7749999999999999,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.645}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.724,"15min":1.79},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.157,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.21},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.472,"15min":1.4830000000000001},"min":{"1min":0.90400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.895,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.76}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.7709999999999999,"15min":1.843},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3490000000000002,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.71},{"interface":"front","average":{"1min":1.508,"5min":1.9179999999999999,"15min":1.986},"min":{"1min":0.85399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8400000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.704},"min":{"1min":0.79400000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.2290000000000001,"5min":2.98,"15min":2.98},"last":1.859},{"interface":"front","average":{"1min":1.548,"5min":1.643,"15min":1.659},"min":{"1min":0.94199999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9379999999999999,"5min":2.29,"15min":2.29},"last":1.829}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.042,"15min":1.0580000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.627,"5min":1.764,"15min":1.764},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.02,"15min":1.002},"min":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.268}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1739999999999999,"15min":1.1819999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6299999999999999,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.0669999999999999,"15min":1.0469999999999999},"min":{"1min":0.85699999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.327}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.206,"15min":1.212},"min":{"1min":0.67200000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7490000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.2330000000000001,"15min":1.218},"min":{"1min":0.78900000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.4830000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.99199999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.597,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.419},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":1.0369999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.667,"5min":1.889,"15min":1.889},"last":1.1200000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.288,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6419999999999999,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.413,"15min":1.413},"min":{"1min":1.206,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9339999999999999,"5min":2.27,"15min":2.27},"last":1.365}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.464,"15min":1.4750000000000001},"min":{"1min":1.05,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.617,"5min":3.806,"15min":3.806},"last":1.232},{"interface":"front","average":{"1min":1.389,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.95199999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.726,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.522}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.52,"15min":1.528},"min":{"1min":1.1919999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8500000000000001,"5min":2.923,"15min":2.923},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.5309999999999999,"15min":1.5269999999999999},"min":{"1min":1.343,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7769999999999999,"5min":2.859,"15min":2.859},"last":1.3919999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0529999999999999,"15min":1.046},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.129,"15min":1.127},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.097}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.099,"15min":1.091},"min":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.81799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.3520000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.198,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6279999999999999,"5min":1.823,"15min":1.823},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.264,"15min":1.266},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.1519999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.298,"15min":1.302},"min":{"1min":0.90600000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.597},{"interface":"front","average":{"1min":1.327,"5min":1.319,"15min":1.3169999999999999},"min":{"1min":0.98299999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6419999999999999,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.9530000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.304,"15min":1.304},"min":{"1min":0.86899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.702,"5min":1.944,"15min":1.944},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.361,"15min":1.365},"min":{"1min":0.96899999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.696,"5min":2.012,"15min":2.012},"last":1.7809999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3700000000000001,"15min":1.3660000000000001},"min":{"1min":0.91300000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.8560000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":2.105},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5129999999999999,"15min":1.5149999999999999},"min":{"1min":1.198,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0129999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.927}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.1100000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.431},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.573,"15min":1.5649999999999999},"min":{"1min":1.0880000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.5920000000000001,"5min":2.722,"15min":2.722},"last":2.411}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":215907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.82199999999999995,"15min":0.81899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.94299999999999995,"15min":0.95499999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.4399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.129,"15min":1.169},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6359999999999999,"5min":3.246,"15min":3.246},"last":1.835},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.97299999999999998,"15min":0.96699999999999997},"min":{"1min":0.441,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.94499999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1799999999999999,"15min":1.196},"min":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.718,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.147,"15min":1.165},"min":{"1min":0.54000000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5900000000000001,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.236}]},{"osd":4,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.93999999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6879999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.379,"15min":1.3979999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.73,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.379}]},{"osd":5,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.448,"15min":1.468},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9870000000000001,"5min":2.395,"15min":2.395},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.304,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2410000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4930000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4299999999999999,"15min":1.4350000000000001},"min":{"1min":0.83199999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.1600000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4159999999999999,"15min":1.4239999999999999},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.946,"5min":1.996,"15min":1.996},"last":1.321}]},{"osd":7,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4430000000000001,"15min":1.4510000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.645},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.895}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.436},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":1.0209999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.272,"5min":2.133,"15min":2.133},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.194,"15min":1.242},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.371,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.048,"15min":1.0840000000000001},"min":{"1min":0.5,"5min":0.438,"15min":0.438},"max":{"1min":1.208,"5min":1.998,"15min":1.998},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3089999999999999,"15min":1.333},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9259999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.175,"15min":1.204},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.552,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3140000000000001,"15min":1.343},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.631,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.754},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.798,"5min":3.161,"15min":3.161},"last":1.123}]},{"osd":5,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3640000000000001,"15min":1.3999999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7649999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.391,"15min":1.423},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7549999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.5620000000000001,"15min":1.6080000000000001},"min":{"1min":0.73899999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.825,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.159},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.857,"5min":2.702,"15min":2.702},"last":0.94199999999999995}]},{"osd":7,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.5009999999999999,"15min":1.5309999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.3980000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.6180000000000001,"15min":1.6619999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.9199999999999999,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.085}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1919999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.105,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.0069999999999999,"15min":1.0389999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2230000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.214,"15min":1.2410000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.353,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.161}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2569999999999999,"15min":1.2769999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6319999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.897},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.2110000000000001,"15min":1.2430000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.538,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.6890000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.242,"15min":1.2549999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.512,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.512},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.4390000000000001,"15min":1.4610000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6459999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.4490000000000001,"15min":1.4650000000000001},"min":{"1min":0.85699999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9590000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.468,"15min":1.484},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.746,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.655}]},{"osd":6,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.45,"15min":1.464},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9019999999999999,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4690000000000001,"15min":1.4910000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.821,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.8420000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.548,"15min":1.5700000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.744,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.738},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5800000000000001,"15min":1.599},"min":{"1min":0.94499999999999995,"5min":0.872,"15min":0.872},"max":{"1min":1.8160000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.788}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":804652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.367,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":3.423},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.93000000000000005},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":3.3180000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.05,"15min":1.054},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.706,"5min":2.29,"15min":2.29},"last":3.899},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":0.93700000000000006},"min":{"1min":0.51000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.5049999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":4.0140000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.032,"15min":1.028},"min":{"1min":0.60499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6439999999999999,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":3.4830000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8640000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":6.3570000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.256,"15min":1.254},"min":{"1min":0.95899999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.704,"5min":3.302,"15min":3.302},"last":3.8479999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.454,"15min":1.466},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.2389999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":3.399}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":0.89400000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.0590000000000002,"5min":3.101,"15min":3.101},"last":5.9939999999999998},{"interface":"front","average":{"1min":1.462,"5min":1.4359999999999999,"15min":1.4319999999999999},"min":{"1min":0.90100000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.1779999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.391}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3999999999999999,"15min":1.4159999999999999},"min":{"1min":0.75,"5min":0.749,"15min":0.749},"max":{"1min":2.1819999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":3.9729999999999999},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4970000000000001,"15min":1.4930000000000001},"min":{"1min":1.0700000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1030000000000002,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":3.7320000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.5309999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1600000000000001,"5min":3.823,"15min":3.823},"last":3.4540000000000002},{"interface":"front","average":{"1min":1.397,"5min":1.468,"15min":1.48},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2610000000000001,"5min":3.371,"15min":3.371},"last":3.9169999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:06:14.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:06:14.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.930+0000 7fc8fb4ca640 1 Processor -- start 2023-11-24T08:06:14.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.930+0000 7fc8fb4ca640 1 -- start start 2023-11-24T08:06:14.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.931+0000 7fc8fb4ca640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc8f40d14a0 0x7fc8f40d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:14.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.931+0000 7fc8fb4ca640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f40b0640 0x7fc8f40b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:14.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.931+0000 7fc8fb4ca640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f40cf0c0 con 0x7fc8f415d5b0 2023-11-24T08:06:14.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.931+0000 7fc8fb4ca640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f40b01b0 con 0x7fc8f40d14a0 2023-11-24T08:06:14.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.931+0000 7fc8fb4ca640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f415d990 con 0x7fc8f40b0640 2023-11-24T08:06:14.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.932+0000 7fc8f9cc7640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc8f40d14a0 0x7fc8f40d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:14.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.932+0000 7fc8f9cc7640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc8f40d14a0 0x7fc8f40d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:41204/0 (socket says 172.21.15.64:41204) 2023-11-24T08:06:14.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.932+0000 7fc8f9cc7640 1 -- 172.21.15.64:0/2071783594 learned_addr learned my addr 172.21.15.64:0/2071783594 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:14.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.932+0000 7fc8facc9640 1 --2- 172.21.15.64:0/2071783594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f40b0640 0x7fc8f40b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:14.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.933+0000 7fc8f9cc7640 1 -- 172.21.15.64:0/2071783594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f40b0640 msgr2=0x7fc8f40b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:14.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.933+0000 7fc8f9cc7640 1 --2- 172.21.15.64:0/2071783594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f40b0640 0x7fc8f40b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:14.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.933+0000 7fc8f9cc7640 1 -- 172.21.15.64:0/2071783594 >> v1:172.21.15.91:6789/0 conn(0x7fc8f415d5b0 legacy=0x7fc8f40d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:14.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.933+0000 7fc8f9cc7640 1 -- 172.21.15.64:0/2071783594 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f40d1e40 con 0x7fc8f40d14a0 2023-11-24T08:06:14.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.934+0000 7fc8f9cc7640 1 --2- 172.21.15.64:0/2071783594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 0x7fc8f40d1900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009870 tx=0x7fc8e402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42bfe7297ec45990 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:14.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.935+0000 7fc8f94c6640 1 -- 172.21.15.64:0/2071783594 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4030840 con 0x7fc8f40d14a0 2023-11-24T08:06:14.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.935+0000 7fc8f94c6640 1 -- 172.21.15.64:0/2071783594 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e40309a0 con 0x7fc8f40d14a0 2023-11-24T08:06:14.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.935+0000 7fc8f94c6640 1 -- 172.21.15.64:0/2071783594 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4030d30 con 0x7fc8f40d14a0 2023-11-24T08:06:14.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.936+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/2071783594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 msgr2=0x7fc8f40d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:14.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.936+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/2071783594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 0x7fc8f40d1900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009870 tx=0x7fc8e402b980 comp rx=0 tx=0).stop 2023-11-24T08:06:14.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.936+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/2071783594 shutdown_connections 2023-11-24T08:06:14.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.936+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/2071783594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f40b0640 0x7fc8f40b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:14.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.936+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/2071783594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 0x7fc8f40d1900 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:14.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.936+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/2071783594 >> 172.21.15.64:0/2071783594 conn(0x7fc8f409c3b0 msgr2=0x7fc8f40d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:14.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.936+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/2071783594 shutdown_connections 2023-11-24T08:06:14.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.936+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/2071783594 wait complete. 2023-11-24T08:06:14.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.938+0000 7fc8fb4ca640 1 Processor -- start 2023-11-24T08:06:14.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.939+0000 7fc8fb4ca640 1 -- start start 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fb4ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f40b0640 0x7fc8f40b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fb4ca640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 0x7fc8f4160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fb4ca640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f415d5b0 0x7fc8f4152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fb4ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40b84e0 con 0x7fc8f40b0640 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fb4ca640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f41f6230 con 0x7fc8f40d14a0 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fb4ca640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f41f6510 con 0x7fc8f415d5b0 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fa4c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f40b0640 0x7fc8f40b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fa4c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f40b0640 0x7fc8f40b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43136/0 (socket says 172.21.15.64:43136) 2023-11-24T08:06:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.940+0000 7fc8fa4c8640 1 -- 172.21.15.64:0/3808598013 learned_addr learned my addr 172.21.15.64:0/3808598013 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8f9cc7640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 0x7fc8f4160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8facc9640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f415d5b0 0x7fc8f4152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8fa4c8640 1 -- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f415d5b0 msgr2=0x7fc8f4152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8fa4c8640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f415d5b0 0x7fc8f4152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8fa4c8640 1 -- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 msgr2=0x7fc8f4160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8fa4c8640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 0x7fc8f4160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8fa4c8640 1 -- 172.21.15.64:0/3808598013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40094d0 con 0x7fc8f40b0640 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8fa4c8640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f40b0640 0x7fc8f40b7fa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c790 tx=0x7fc8f000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.941+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3808598013 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000cea0 con 0x7fc8f40b0640 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.942+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3808598013 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0006d50 con 0x7fc8f40b0640 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.942+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3808598013 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0003bb0 con 0x7fc8f40b0640 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.942+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f41f67f0 con 0x7fc8f40b0640 2023-11-24T08:06:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.942+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f41f6db0 con 0x7fc8f40b0640 2023-11-24T08:06:14.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.943+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f40b8f60 con 0x7fc8f40b0640 2023-11-24T08:06:14.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.944+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3808598013 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc8f0003d10 con 0x7fc8f40b0640 2023-11-24T08:06:14.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.945+0000 7fc8eaffd640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc8d806aed0 0x7fc8d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:14.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.945+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3808598013 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8f0089200 con 0x7fc8f40b0640 2023-11-24T08:06:14.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.945+0000 7fc8f9cc7640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc8d806aed0 0x7fc8d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:14.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.947+0000 7fc8f9cc7640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc8d806aed0 0x7fc8d806d390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4036000 tx=0x7fc8e403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:14.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:14.949+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3808598013 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8f0057de0 con 0x7fc8f40b0640 2023-11-24T08:06:15.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.145+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8f40afe40 con 0x7fc8d806aed0 2023-11-24T08:06:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.168+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3808598013 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286656 (secure 0 0 0) 0x7fc8e4030b10 con 0x7fc8d806aed0 2023-11-24T08:06:15.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:15.176 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:06:15.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.175+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc8d806aed0 msgr2=0x7fc8d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:15.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.175+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc8d806aed0 0x7fc8d806d390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4036000 tx=0x7fc8e403b040 comp rx=0 tx=0).stop 2023-11-24T08:06:15.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.175+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f40b0640 msgr2=0x7fc8f40b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:15.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.175+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f40b0640 0x7fc8f40b7fa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c790 tx=0x7fc8f000cc50 comp rx=0 tx=0).stop 2023-11-24T08:06:15.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.176+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 shutdown_connections 2023-11-24T08:06:15.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.176+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc8d806aed0 0x7fc8d806d390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.176+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8f415d5b0 0x7fc8f4152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.176+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc8f40d14a0 0x7fc8f4160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.176+0000 7fc8fb4ca640 1 --2- 172.21.15.64:0/3808598013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f40b0640 0x7fc8f40b7fa0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.176+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 >> 172.21.15.64:0/3808598013 conn(0x7fc8f409c3b0 msgr2=0x7fc8f40cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:15.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.176+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 shutdown_connections 2023-11-24T08:06:15.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.176+0000 7fc8fb4ca640 1 -- 172.21.15.64:0/3808598013 wait complete. 2023-11-24T08:06:15.197 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-11-24T08:06:14.064730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221508,"kb_used_data":5348,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753212,"statfs":{"total":773094113280,"available":772867289088,"internally_reserved":0,"allocated":5476352,"data_stored":2906689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.006203"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253626+0000","last_change":"2023-11-24T08:06:05.088466+0000","last_active":"2023-11-24T08:06:06.253626+0000","last_peered":"2023-11-24T08:06:06.253626+0000","last_clean":"2023-11-24T08:06:06.253626+0000","last_became_active":"2023-11-24T08:06:05.088184+0000","last_became_peered":"2023-11-24T08:06:05.088184+0000","last_unstale":"2023-11-24T08:06:06.253626+0000","last_undegraded":"2023-11-24T08:06:06.253626+0000","last_fullsized":"2023-11-24T08:06:06.253626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:55.876487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251929+0000","last_change":"2023-11-24T08:06:05.094999+0000","last_active":"2023-11-24T08:06:06.251929+0000","last_peered":"2023-11-24T08:06:06.251929+0000","last_clean":"2023-11-24T08:06:06.251929+0000","last_became_active":"2023-11-24T08:06:05.094688+0000","last_became_peered":"2023-11-24T08:06:05.094688+0000","last_unstale":"2023-11-24T08:06:06.251929+0000","last_undegraded":"2023-11-24T08:06:06.251929+0000","last_fullsized":"2023-11-24T08:06:06.251929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088777+0000","last_change":"2023-11-24T08:06:05.088653+0000","last_active":"2023-11-24T08:06:06.088777+0000","last_peered":"2023-11-24T08:06:06.088777+0000","last_clean":"2023-11-24T08:06:06.088777+0000","last_became_active":"2023-11-24T08:06:05.087897+0000","last_became_peered":"2023-11-24T08:06:05.087897+0000","last_unstale":"2023-11-24T08:06:06.088777+0000","last_undegraded":"2023-11-24T08:06:06.088777+0000","last_fullsized":"2023-11-24T08:06:06.088777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252772+0000","last_change":"2023-11-24T08:06:05.097244+0000","last_active":"2023-11-24T08:06:06.252772+0000","last_peered":"2023-11-24T08:06:06.252772+0000","last_clean":"2023-11-24T08:06:06.252772+0000","last_became_active":"2023-11-24T08:06:05.096919+0000","last_became_peered":"2023-11-24T08:06:05.096919+0000","last_unstale":"2023-11-24T08:06:06.252772+0000","last_undegraded":"2023-11-24T08:06:06.252772+0000","last_fullsized":"2023-11-24T08:06:06.252772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:05.052808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251328+0000","last_change":"2023-11-24T08:06:05.095396+0000","last_active":"2023-11-24T08:06:06.251328+0000","last_peered":"2023-11-24T08:06:06.251328+0000","last_clean":"2023-11-24T08:06:06.251328+0000","last_became_active":"2023-11-24T08:06:05.094813+0000","last_became_peered":"2023-11-24T08:06:05.094813+0000","last_unstale":"2023-11-24T08:06:06.251328+0000","last_undegraded":"2023-11-24T08:06:06.251328+0000","last_fullsized":"2023-11-24T08:06:06.251328+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251253+0000","last_change":"2023-11-24T08:06:05.084536+0000","last_active":"2023-11-24T08:06:06.251253+0000","last_peered":"2023-11-24T08:06:06.251253+0000","last_clean":"2023-11-24T08:06:06.251253+0000","last_became_active":"2023-11-24T08:06:05.084223+0000","last_became_peered":"2023-11-24T08:06:05.084223+0000","last_unstale":"2023-11-24T08:06:06.251253+0000","last_undegraded":"2023-11-24T08:06:06.251253+0000","last_fullsized":"2023-11-24T08:06:06.251253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:03.061151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252752+0000","last_change":"2023-11-24T08:06:05.079934+0000","last_active":"2023-11-24T08:06:06.252752+0000","last_peered":"2023-11-24T08:06:06.252752+0000","last_clean":"2023-11-24T08:06:06.252752+0000","last_became_active":"2023-11-24T08:06:05.079195+0000","last_became_peered":"2023-11-24T08:06:05.079195+0000","last_unstale":"2023-11-24T08:06:06.252752+0000","last_undegraded":"2023-11-24T08:06:06.252752+0000","last_fullsized":"2023-11-24T08:06:06.252752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:28.502892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252409+0000","last_change":"2023-11-24T08:06:05.096845+0000","last_active":"2023-11-24T08:06:06.252409+0000","last_peered":"2023-11-24T08:06:06.252409+0000","last_clean":"2023-11-24T08:06:06.252409+0000","last_became_active":"2023-11-24T08:06:05.094451+0000","last_became_peered":"2023-11-24T08:06:05.094451+0000","last_unstale":"2023-11-24T08:06:06.252409+0000","last_undegraded":"2023-11-24T08:06:06.252409+0000","last_fullsized":"2023-11-24T08:06:06.252409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:45.526567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252291+0000","last_change":"2023-11-24T08:06:05.096637+0000","last_active":"2023-11-24T08:06:06.252291+0000","last_peered":"2023-11-24T08:06:06.252291+0000","last_clean":"2023-11-24T08:06:06.252291+0000","last_became_active":"2023-11-24T08:06:05.096340+0000","last_became_peered":"2023-11-24T08:06:05.096340+0000","last_unstale":"2023-11-24T08:06:06.252291+0000","last_undegraded":"2023-11-24T08:06:06.252291+0000","last_fullsized":"2023-11-24T08:06:06.252291+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:34.236261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252685+0000","last_change":"2023-11-24T08:06:05.102398+0000","last_active":"2023-11-24T08:06:06.252685+0000","last_peered":"2023-11-24T08:06:06.252685+0000","last_clean":"2023-11-24T08:06:06.252685+0000","last_became_active":"2023-11-24T08:06:05.101720+0000","last_became_peered":"2023-11-24T08:06:05.101720+0000","last_unstale":"2023-11-24T08:06:06.252685+0000","last_undegraded":"2023-11-24T08:06:06.252685+0000","last_fullsized":"2023-11-24T08:06:06.252685+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:34.594891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252477+0000","last_change":"2023-11-24T08:06:05.097295+0000","last_active":"2023-11-24T08:06:06.252477+0000","last_peered":"2023-11-24T08:06:06.252477+0000","last_clean":"2023-11-24T08:06:06.252477+0000","last_became_active":"2023-11-24T08:06:05.095953+0000","last_became_peered":"2023-11-24T08:06:05.095953+0000","last_unstale":"2023-11-24T08:06:06.252477+0000","last_undegraded":"2023-11-24T08:06:06.252477+0000","last_fullsized":"2023-11-24T08:06:06.252477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:30.368708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251480+0000","last_change":"2023-11-24T08:06:05.106405+0000","last_active":"2023-11-24T08:06:06.251480+0000","last_peered":"2023-11-24T08:06:06.251480+0000","last_clean":"2023-11-24T08:06:06.251480+0000","last_became_active":"2023-11-24T08:06:05.106098+0000","last_became_peered":"2023-11-24T08:06:05.106098+0000","last_unstale":"2023-11-24T08:06:06.251480+0000","last_undegraded":"2023-11-24T08:06:06.251480+0000","last_fullsized":"2023-11-24T08:06:06.251480+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:37.368224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248524+0000","last_change":"2023-11-24T08:06:05.087198+0000","last_active":"2023-11-24T08:06:06.248524+0000","last_peered":"2023-11-24T08:06:06.248524+0000","last_clean":"2023-11-24T08:06:06.248524+0000","last_became_active":"2023-11-24T08:06:05.086737+0000","last_became_peered":"2023-11-24T08:06:05.086737+0000","last_unstale":"2023-11-24T08:06:06.248524+0000","last_undegraded":"2023-11-24T08:06:06.248524+0000","last_fullsized":"2023-11-24T08:06:06.248524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:29.327265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252188+0000","last_change":"2023-11-24T08:06:05.100506+0000","last_active":"2023-11-24T08:06:06.252188+0000","last_peered":"2023-11-24T08:06:06.252188+0000","last_clean":"2023-11-24T08:06:06.252188+0000","last_became_active":"2023-11-24T08:06:05.094270+0000","last_became_peered":"2023-11-24T08:06:05.094270+0000","last_unstale":"2023-11-24T08:06:06.252188+0000","last_undegraded":"2023-11-24T08:06:06.252188+0000","last_fullsized":"2023-11-24T08:06:06.252188+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250948+0000","last_change":"2023-11-24T08:06:05.091977+0000","last_active":"2023-11-24T08:06:06.250948+0000","last_peered":"2023-11-24T08:06:06.250948+0000","last_clean":"2023-11-24T08:06:06.250948+0000","last_became_active":"2023-11-24T08:06:05.091622+0000","last_became_peered":"2023-11-24T08:06:05.091622+0000","last_unstale":"2023-11-24T08:06:06.250948+0000","last_undegraded":"2023-11-24T08:06:06.250948+0000","last_fullsized":"2023-11-24T08:06:06.250948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:31.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087597+0000","last_change":"2023-11-24T08:06:05.090075+0000","last_active":"2023-11-24T08:06:06.087597+0000","last_peered":"2023-11-24T08:06:06.087597+0000","last_clean":"2023-11-24T08:06:06.087597+0000","last_became_active":"2023-11-24T08:06:05.089117+0000","last_became_peered":"2023-11-24T08:06:05.089117+0000","last_unstale":"2023-11-24T08:06:06.087597+0000","last_undegraded":"2023-11-24T08:06:06.087597+0000","last_fullsized":"2023-11-24T08:06:06.087597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:30.794390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.090292+0000","last_change":"2023-11-24T08:06:05.098942+0000","last_active":"2023-11-24T08:06:06.090292+0000","last_peered":"2023-11-24T08:06:06.090292+0000","last_clean":"2023-11-24T08:06:06.090292+0000","last_became_active":"2023-11-24T08:06:05.098583+0000","last_became_peered":"2023-11-24T08:06:05.098583+0000","last_unstale":"2023-11-24T08:06:06.090292+0000","last_undegraded":"2023-11-24T08:06:06.090292+0000","last_fullsized":"2023-11-24T08:06:06.090292+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:58.817784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089664+0000","last_change":"2023-11-24T08:06:05.095908+0000","last_active":"2023-11-24T08:06:06.089664+0000","last_peered":"2023-11-24T08:06:06.089664+0000","last_clean":"2023-11-24T08:06:06.089664+0000","last_became_active":"2023-11-24T08:06:05.095581+0000","last_became_peered":"2023-11-24T08:06:05.095581+0000","last_unstale":"2023-11-24T08:06:06.089664+0000","last_undegraded":"2023-11-24T08:06:06.089664+0000","last_fullsized":"2023-11-24T08:06:06.089664+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089518+0000","last_change":"2023-11-24T08:06:05.093941+0000","last_active":"2023-11-24T08:06:06.089518+0000","last_peered":"2023-11-24T08:06:06.089518+0000","last_clean":"2023-11-24T08:06:06.089518+0000","last_became_active":"2023-11-24T08:06:05.092980+0000","last_became_peered":"2023-11-24T08:06:05.092980+0000","last_unstale":"2023-11-24T08:06:06.089518+0000","last_undegraded":"2023-11-24T08:06:06.089518+0000","last_fullsized":"2023-11-24T08:06:06.089518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:47.605436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248776+0000","last_change":"2023-11-24T08:06:05.096889+0000","last_active":"2023-11-24T08:06:06.248776+0000","last_peered":"2023-11-24T08:06:06.248776+0000","last_clean":"2023-11-24T08:06:06.248776+0000","last_became_active":"2023-11-24T08:06:05.096574+0000","last_became_peered":"2023-11-24T08:06:05.096574+0000","last_unstale":"2023-11-24T08:06:06.248776+0000","last_undegraded":"2023-11-24T08:06:06.248776+0000","last_fullsized":"2023-11-24T08:06:06.248776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:05.908617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251413+0000","last_change":"2023-11-24T08:06:05.092598+0000","last_active":"2023-11-24T08:06:06.251413+0000","last_peered":"2023-11-24T08:06:06.251413+0000","last_clean":"2023-11-24T08:06:06.251413+0000","last_became_active":"2023-11-24T08:06:05.088450+0000","last_became_peered":"2023-11-24T08:06:05.088450+0000","last_unstale":"2023-11-24T08:06:06.251413+0000","last_undegraded":"2023-11-24T08:06:06.251413+0000","last_fullsized":"2023-11-24T08:06:06.251413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:13.526610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252225+0000","last_change":"2023-11-24T08:06:05.098582+0000","last_active":"2023-11-24T08:06:06.252225+0000","last_peered":"2023-11-24T08:06:06.252225+0000","last_clean":"2023-11-24T08:06:06.252225+0000","last_became_active":"2023-11-24T08:06:05.094693+0000","last_became_peered":"2023-11-24T08:06:05.094693+0000","last_unstale":"2023-11-24T08:06:06.252225+0000","last_undegraded":"2023-11-24T08:06:06.252225+0000","last_fullsized":"2023-11-24T08:06:06.252225+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:29.573995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087957+0000","last_change":"2023-11-24T08:06:05.085854+0000","last_active":"2023-11-24T08:06:06.087957+0000","last_peered":"2023-11-24T08:06:06.087957+0000","last_clean":"2023-11-24T08:06:06.087957+0000","last_became_active":"2023-11-24T08:06:05.085576+0000","last_became_peered":"2023-11-24T08:06:05.085576+0000","last_unstale":"2023-11-24T08:06:06.087957+0000","last_undegraded":"2023-11-24T08:06:06.087957+0000","last_fullsized":"2023-11-24T08:06:06.087957+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:21.753543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087840+0000","last_change":"2023-11-24T08:06:05.084050+0000","last_active":"2023-11-24T08:06:06.087840+0000","last_peered":"2023-11-24T08:06:06.087840+0000","last_clean":"2023-11-24T08:06:06.087840+0000","last_became_active":"2023-11-24T08:06:05.083596+0000","last_became_peered":"2023-11-24T08:06:05.083596+0000","last_unstale":"2023-11-24T08:06:06.087840+0000","last_undegraded":"2023-11-24T08:06:06.087840+0000","last_fullsized":"2023-11-24T08:06:06.087840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:12.059275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089365+0000","last_change":"2023-11-24T08:06:05.099351+0000","last_active":"2023-11-24T08:06:06.089365+0000","last_peered":"2023-11-24T08:06:06.089365+0000","last_clean":"2023-11-24T08:06:06.089365+0000","last_became_active":"2023-11-24T08:06:05.097619+0000","last_became_peered":"2023-11-24T08:06:05.097619+0000","last_unstale":"2023-11-24T08:06:06.089365+0000","last_undegraded":"2023-11-24T08:06:06.089365+0000","last_fullsized":"2023-11-24T08:06:06.089365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:21.615036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248405+0000","last_change":"2023-11-24T08:06:05.094252+0000","last_active":"2023-11-24T08:06:06.248405+0000","last_peered":"2023-11-24T08:06:06.248405+0000","last_clean":"2023-11-24T08:06:06.248405+0000","last_became_active":"2023-11-24T08:06:05.093924+0000","last_became_peered":"2023-11-24T08:06:05.093924+0000","last_unstale":"2023-11-24T08:06:06.248405+0000","last_undegraded":"2023-11-24T08:06:06.248405+0000","last_fullsized":"2023-11-24T08:06:06.248405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:19.995373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089262+0000","last_change":"2023-11-24T08:06:05.094578+0000","last_active":"2023-11-24T08:06:06.089262+0000","last_peered":"2023-11-24T08:06:06.089262+0000","last_clean":"2023-11-24T08:06:06.089262+0000","last_became_active":"2023-11-24T08:06:05.094329+0000","last_became_peered":"2023-11-24T08:06:05.094329+0000","last_unstale":"2023-11-24T08:06:06.089262+0000","last_undegraded":"2023-11-24T08:06:06.089262+0000","last_fullsized":"2023-11-24T08:06:06.089262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:37.368224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251627+0000","last_change":"2023-11-24T08:06:05.102312+0000","last_active":"2023-11-24T08:06:06.251627+0000","last_peered":"2023-11-24T08:06:06.251627+0000","last_clean":"2023-11-24T08:06:06.251627+0000","last_became_active":"2023-11-24T08:06:05.102050+0000","last_became_peered":"2023-11-24T08:06:05.102050+0000","last_unstale":"2023-11-24T08:06:06.251627+0000","last_undegraded":"2023-11-24T08:06:06.251627+0000","last_fullsized":"2023-11-24T08:06:06.251627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088476+0000","last_change":"2023-11-24T08:06:05.093991+0000","last_active":"2023-11-24T08:06:06.088476+0000","last_peered":"2023-11-24T08:06:06.088476+0000","last_clean":"2023-11-24T08:06:06.088476+0000","last_became_active":"2023-11-24T08:06:05.093653+0000","last_became_peered":"2023-11-24T08:06:05.093653+0000","last_unstale":"2023-11-24T08:06:06.088476+0000","last_undegraded":"2023-11-24T08:06:06.088476+0000","last_fullsized":"2023-11-24T08:06:06.088476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:02.903723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087380+0000","last_change":"2023-11-24T08:06:05.095755+0000","last_active":"2023-11-24T08:06:06.087380+0000","last_peered":"2023-11-24T08:06:06.087380+0000","last_clean":"2023-11-24T08:06:06.087380+0000","last_became_active":"2023-11-24T08:06:05.095466+0000","last_became_peered":"2023-11-24T08:06:05.095466+0000","last_unstale":"2023-11-24T08:06:06.087380+0000","last_undegraded":"2023-11-24T08:06:06.087380+0000","last_fullsized":"2023-11-24T08:06:06.087380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:47.605436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251015+0000","last_change":"2023-11-24T08:06:05.087774+0000","last_active":"2023-11-24T08:06:06.251015+0000","last_peered":"2023-11-24T08:06:06.251015+0000","last_clean":"2023-11-24T08:06:06.251015+0000","last_became_active":"2023-11-24T08:06:05.087486+0000","last_became_peered":"2023-11-24T08:06:05.087486+0000","last_unstale":"2023-11-24T08:06:06.251015+0000","last_undegraded":"2023-11-24T08:06:06.251015+0000","last_fullsized":"2023-11-24T08:06:06.251015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:46.775040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248159+0000","last_change":"2023-11-24T08:06:05.093356+0000","last_active":"2023-11-24T08:06:06.248159+0000","last_peered":"2023-11-24T08:06:06.248159+0000","last_clean":"2023-11-24T08:06:06.248159+0000","last_became_active":"2023-11-24T08:06:05.093036+0000","last_became_peered":"2023-11-24T08:06:05.093036+0000","last_unstale":"2023-11-24T08:06:06.248159+0000","last_undegraded":"2023-11-24T08:06:06.248159+0000","last_fullsized":"2023-11-24T08:06:06.248159+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:44.380303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.083398+0000","last_change":"2023-11-24T08:06:05.084537+0000","last_active":"2023-11-24T08:06:06.083398+0000","last_peered":"2023-11-24T08:06:06.083398+0000","last_clean":"2023-11-24T08:06:06.083398+0000","last_became_active":"2023-11-24T08:06:05.084268+0000","last_became_peered":"2023-11-24T08:06:05.084268+0000","last_unstale":"2023-11-24T08:06:06.083398+0000","last_undegraded":"2023-11-24T08:06:06.083398+0000","last_fullsized":"2023-11-24T08:06:06.083398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:52.364691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120766+0000","last_change":"2023-11-24T08:06:04.082144+0000","last_active":"2023-11-24T08:06:05.120766+0000","last_peered":"2023-11-24T08:06:05.120766+0000","last_clean":"2023-11-24T08:06:05.120766+0000","last_became_active":"2023-11-24T08:06:04.081510+0000","last_became_peered":"2023-11-24T08:06:04.081510+0000","last_unstale":"2023-11-24T08:06:05.120766+0000","last_undegraded":"2023-11-24T08:06:05.120766+0000","last_fullsized":"2023-11-24T08:06:05.120766+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:44.512063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089609+0000","last_change":"2023-11-24T08:06:05.097573+0000","last_active":"2023-11-24T08:06:06.089609+0000","last_peered":"2023-11-24T08:06:06.089609+0000","last_clean":"2023-11-24T08:06:06.089609+0000","last_became_active":"2023-11-24T08:06:05.094728+0000","last_became_peered":"2023-11-24T08:06:05.094728+0000","last_unstale":"2023-11-24T08:06:06.089609+0000","last_undegraded":"2023-11-24T08:06:06.089609+0000","last_fullsized":"2023-11-24T08:06:06.089609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.114292+0000","last_change":"2023-11-24T08:06:04.084082+0000","last_active":"2023-11-24T08:06:05.114292+0000","last_peered":"2023-11-24T08:06:05.114292+0000","last_clean":"2023-11-24T08:06:05.114292+0000","last_became_active":"2023-11-24T08:06:04.083299+0000","last_became_peered":"2023-11-24T08:06:04.083299+0000","last_unstale":"2023-11-24T08:06:05.114292+0000","last_undegraded":"2023-11-24T08:06:05.114292+0000","last_fullsized":"2023-11-24T08:06:05.114292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251872+0000","last_change":"2023-11-24T08:06:05.096613+0000","last_active":"2023-11-24T08:06:06.251872+0000","last_peered":"2023-11-24T08:06:06.251872+0000","last_clean":"2023-11-24T08:06:06.251872+0000","last_became_active":"2023-11-24T08:06:05.096353+0000","last_became_peered":"2023-11-24T08:06:05.096353+0000","last_unstale":"2023-11-24T08:06:06.251872+0000","last_undegraded":"2023-11-24T08:06:06.251872+0000","last_fullsized":"2023-11-24T08:06:06.251872+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.115949+0000","last_change":"2023-11-24T08:06:04.087280+0000","last_active":"2023-11-24T08:06:05.115949+0000","last_peered":"2023-11-24T08:06:05.115949+0000","last_clean":"2023-11-24T08:06:05.115949+0000","last_became_active":"2023-11-24T08:06:04.086942+0000","last_became_peered":"2023-11-24T08:06:04.086942+0000","last_unstale":"2023-11-24T08:06:05.115949+0000","last_undegraded":"2023-11-24T08:06:05.115949+0000","last_fullsized":"2023-11-24T08:06:05.115949+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.083315+0000","last_change":"2023-11-24T08:06:05.086813+0000","last_active":"2023-11-24T08:06:06.083315+0000","last_peered":"2023-11-24T08:06:06.083315+0000","last_clean":"2023-11-24T08:06:06.083315+0000","last_became_active":"2023-11-24T08:06:05.086506+0000","last_became_peered":"2023-11-24T08:06:05.086506+0000","last_unstale":"2023-11-24T08:06:06.083315+0000","last_undegraded":"2023-11-24T08:06:06.083315+0000","last_fullsized":"2023-11-24T08:06:06.083315+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:21.753543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122782+0000","last_change":"2023-11-24T08:06:04.094159+0000","last_active":"2023-11-24T08:06:05.122782+0000","last_peered":"2023-11-24T08:06:05.122782+0000","last_clean":"2023-11-24T08:06:05.122782+0000","last_became_active":"2023-11-24T08:06:04.093809+0000","last_became_peered":"2023-11-24T08:06:04.093809+0000","last_unstale":"2023-11-24T08:06:05.122782+0000","last_undegraded":"2023-11-24T08:06:05.122782+0000","last_fullsized":"2023-11-24T08:06:05.122782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:54.440803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087650+0000","last_change":"2023-11-24T08:06:05.101827+0000","last_active":"2023-11-24T08:06:06.087650+0000","last_peered":"2023-11-24T08:06:06.087650+0000","last_clean":"2023-11-24T08:06:06.087650+0000","last_became_active":"2023-11-24T08:06:05.101501+0000","last_became_peered":"2023-11-24T08:06:05.101501+0000","last_unstale":"2023-11-24T08:06:06.087650+0000","last_undegraded":"2023-11-24T08:06:06.087650+0000","last_fullsized":"2023-11-24T08:06:06.087650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120725+0000","last_change":"2023-11-24T08:06:04.090249+0000","last_active":"2023-11-24T08:06:05.120725+0000","last_peered":"2023-11-24T08:06:05.120725+0000","last_clean":"2023-11-24T08:06:05.120725+0000","last_became_active":"2023-11-24T08:06:04.089893+0000","last_became_peered":"2023-11-24T08:06:04.089893+0000","last_unstale":"2023-11-24T08:06:05.120725+0000","last_undegraded":"2023-11-24T08:06:05.120725+0000","last_fullsized":"2023-11-24T08:06:05.120725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:27.488388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252382+0000","last_change":"2023-11-24T08:06:05.093052+0000","last_active":"2023-11-24T08:06:06.252382+0000","last_peered":"2023-11-24T08:06:06.252382+0000","last_clean":"2023-11-24T08:06:06.252382+0000","last_became_active":"2023-11-24T08:06:05.091759+0000","last_became_peered":"2023-11-24T08:06:05.091759+0000","last_unstale":"2023-11-24T08:06:06.252382+0000","last_undegraded":"2023-11-24T08:06:06.252382+0000","last_fullsized":"2023-11-24T08:06:06.252382+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:28.502892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.112139+0000","last_change":"2023-11-24T08:06:04.087131+0000","last_active":"2023-11-24T08:06:05.112139+0000","last_peered":"2023-11-24T08:06:05.112139+0000","last_clean":"2023-11-24T08:06:05.112139+0000","last_became_active":"2023-11-24T08:06:04.086323+0000","last_became_peered":"2023-11-24T08:06:04.086323+0000","last_unstale":"2023-11-24T08:06:05.112139+0000","last_undegraded":"2023-11-24T08:06:05.112139+0000","last_fullsized":"2023-11-24T08:06:05.112139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:01.889219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251937+0000","last_change":"2023-11-24T08:06:05.097611+0000","last_active":"2023-11-24T08:06:06.251937+0000","last_peered":"2023-11-24T08:06:06.251937+0000","last_clean":"2023-11-24T08:06:06.251937+0000","last_became_active":"2023-11-24T08:06:05.094003+0000","last_became_peered":"2023-11-24T08:06:05.094003+0000","last_unstale":"2023-11-24T08:06:06.251937+0000","last_undegraded":"2023-11-24T08:06:06.251937+0000","last_fullsized":"2023-11-24T08:06:06.251937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:53.898779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.090125+0000","last_change":"2023-11-24T08:06:04.085591+0000","last_active":"2023-11-24T08:06:05.090125+0000","last_peered":"2023-11-24T08:06:05.090125+0000","last_clean":"2023-11-24T08:06:05.090125+0000","last_became_active":"2023-11-24T08:06:04.085328+0000","last_became_peered":"2023-11-24T08:06:04.085328+0000","last_unstale":"2023-11-24T08:06:05.090125+0000","last_undegraded":"2023-11-24T08:06:05.090125+0000","last_fullsized":"2023-11-24T08:06:05.090125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:35.592076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087957+0000","last_change":"2023-11-24T08:06:05.080325+0000","last_active":"2023-11-24T08:06:06.087957+0000","last_peered":"2023-11-24T08:06:06.087957+0000","last_clean":"2023-11-24T08:06:06.087957+0000","last_became_active":"2023-11-24T08:06:05.079874+0000","last_became_peered":"2023-11-24T08:06:05.079874+0000","last_unstale":"2023-11-24T08:06:06.087957+0000","last_undegraded":"2023-11-24T08:06:06.087957+0000","last_fullsized":"2023-11-24T08:06:06.087957+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:52.364691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.076622+0000","last_change":"2023-11-24T08:06:04.088494+0000","last_active":"2023-11-24T08:06:05.076622+0000","last_peered":"2023-11-24T08:06:05.076622+0000","last_clean":"2023-11-24T08:06:05.076622+0000","last_became_active":"2023-11-24T08:06:04.088029+0000","last_became_peered":"2023-11-24T08:06:04.088029+0000","last_unstale":"2023-11-24T08:06:05.076622+0000","last_undegraded":"2023-11-24T08:06:05.076622+0000","last_fullsized":"2023-11-24T08:06:05.076622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:13.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089650+0000","last_change":"2023-11-24T08:06:05.095670+0000","last_active":"2023-11-24T08:06:06.089650+0000","last_peered":"2023-11-24T08:06:06.089650+0000","last_clean":"2023-11-24T08:06:06.089650+0000","last_became_active":"2023-11-24T08:06:05.095367+0000","last_became_peered":"2023-11-24T08:06:05.095367+0000","last_unstale":"2023-11-24T08:06:06.089650+0000","last_undegraded":"2023-11-24T08:06:06.089650+0000","last_fullsized":"2023-11-24T08:06:06.089650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.086734+0000","last_change":"2023-11-24T08:06:04.074104+0000","last_active":"2023-11-24T08:06:05.086734+0000","last_peered":"2023-11-24T08:06:05.086734+0000","last_clean":"2023-11-24T08:06:05.086734+0000","last_became_active":"2023-11-24T08:06:04.073646+0000","last_became_peered":"2023-11-24T08:06:04.073646+0000","last_unstale":"2023-11-24T08:06:05.086734+0000","last_undegraded":"2023-11-24T08:06:05.086734+0000","last_fullsized":"2023-11-24T08:06:05.086734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:14.570476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253768+0000","last_change":"2023-11-24T08:06:05.095886+0000","last_active":"2023-11-24T08:06:06.253768+0000","last_peered":"2023-11-24T08:06:06.253768+0000","last_clean":"2023-11-24T08:06:06.253768+0000","last_became_active":"2023-11-24T08:06:05.095590+0000","last_became_peered":"2023-11-24T08:06:05.095590+0000","last_unstale":"2023-11-24T08:06:06.253768+0000","last_undegraded":"2023-11-24T08:06:06.253768+0000","last_fullsized":"2023-11-24T08:06:06.253768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:34.594891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.082395+0000","last_change":"2023-11-24T08:06:04.086787+0000","last_active":"2023-11-24T08:06:05.082395+0000","last_peered":"2023-11-24T08:06:05.082395+0000","last_clean":"2023-11-24T08:06:05.082395+0000","last_became_active":"2023-11-24T08:06:04.086503+0000","last_became_peered":"2023-11-24T08:06:04.086503+0000","last_unstale":"2023-11-24T08:06:05.082395+0000","last_undegraded":"2023-11-24T08:06:05.082395+0000","last_fullsized":"2023-11-24T08:06:05.082395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251580+0000","last_change":"2023-11-24T08:06:05.094679+0000","last_active":"2023-11-24T08:06:06.251580+0000","last_peered":"2023-11-24T08:06:06.251580+0000","last_clean":"2023-11-24T08:06:06.251580+0000","last_became_active":"2023-11-24T08:06:05.089982+0000","last_became_peered":"2023-11-24T08:06:05.089982+0000","last_unstale":"2023-11-24T08:06:06.251580+0000","last_undegraded":"2023-11-24T08:06:06.251580+0000","last_fullsized":"2023-11-24T08:06:06.251580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:01.848501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122469+0000","last_change":"2023-11-24T08:06:04.094281+0000","last_active":"2023-11-24T08:06:05.122469+0000","last_peered":"2023-11-24T08:06:05.122469+0000","last_clean":"2023-11-24T08:06:05.122469+0000","last_became_active":"2023-11-24T08:06:04.093946+0000","last_became_peered":"2023-11-24T08:06:04.093946+0000","last_unstale":"2023-11-24T08:06:05.122469+0000","last_undegraded":"2023-11-24T08:06:05.122469+0000","last_fullsized":"2023-11-24T08:06:05.122469+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:28.559491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253369+0000","last_change":"2023-11-24T08:06:05.079345+0000","last_active":"2023-11-24T08:06:06.253369+0000","last_peered":"2023-11-24T08:06:06.253369+0000","last_clean":"2023-11-24T08:06:06.253369+0000","last_became_active":"2023-11-24T08:06:05.078273+0000","last_became_peered":"2023-11-24T08:06:05.078273+0000","last_unstale":"2023-11-24T08:06:06.253369+0000","last_undegraded":"2023-11-24T08:06:06.253369+0000","last_fullsized":"2023-11-24T08:06:06.253369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:59.520725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.125088+0000","last_change":"2023-11-24T08:06:04.080902+0000","last_active":"2023-11-24T08:06:05.125088+0000","last_peered":"2023-11-24T08:06:05.125088+0000","last_clean":"2023-11-24T08:06:05.125088+0000","last_became_active":"2023-11-24T08:06:04.080616+0000","last_became_peered":"2023-11-24T08:06:04.080616+0000","last_unstale":"2023-11-24T08:06:05.125088+0000","last_undegraded":"2023-11-24T08:06:05.125088+0000","last_fullsized":"2023-11-24T08:06:05.125088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:24.477858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.088485+0000","last_change":"2023-11-24T08:06:05.093746+0000","last_active":"2023-11-24T08:06:06.088485+0000","last_peered":"2023-11-24T08:06:06.088485+0000","last_clean":"2023-11-24T08:06:06.088485+0000","last_became_active":"2023-11-24T08:06:05.093440+0000","last_became_peered":"2023-11-24T08:06:05.093440+0000","last_unstale":"2023-11-24T08:06:06.088485+0000","last_undegraded":"2023-11-24T08:06:06.088485+0000","last_fullsized":"2023-11-24T08:06:06.088485+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:10.661913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.121974+0000","last_change":"2023-11-24T08:06:04.084295+0000","last_active":"2023-11-24T08:06:05.121974+0000","last_peered":"2023-11-24T08:06:05.121974+0000","last_clean":"2023-11-24T08:06:05.121974+0000","last_became_active":"2023-11-24T08:06:04.083328+0000","last_became_peered":"2023-11-24T08:06:04.083328+0000","last_unstale":"2023-11-24T08:06:05.121974+0000","last_undegraded":"2023-11-24T08:06:05.121974+0000","last_fullsized":"2023-11-24T08:06:05.121974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:58.506222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252513+0000","last_change":"2023-11-24T08:06:05.095494+0000","last_active":"2023-11-24T08:06:06.252513+0000","last_peered":"2023-11-24T08:06:06.252513+0000","last_clean":"2023-11-24T08:06:06.252513+0000","last_became_active":"2023-11-24T08:06:05.095195+0000","last_became_peered":"2023-11-24T08:06:05.095195+0000","last_unstale":"2023-11-24T08:06:06.252513+0000","last_undegraded":"2023-11-24T08:06:06.252513+0000","last_fullsized":"2023-11-24T08:06:06.252513+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:55.876487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063734+0000","last_change":"2023-11-24T08:06:04.088130+0000","last_active":"2023-11-24T08:06:05.063734+0000","last_peered":"2023-11-24T08:06:05.063734+0000","last_clean":"2023-11-24T08:06:05.063734+0000","last_became_active":"2023-11-24T08:06:04.087804+0000","last_became_peered":"2023-11-24T08:06:04.087804+0000","last_unstale":"2023-11-24T08:06:05.063734+0000","last_undegraded":"2023-11-24T08:06:05.063734+0000","last_fullsized":"2023-11-24T08:06:05.063734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250881+0000","last_change":"2023-11-24T08:06:05.102561+0000","last_active":"2023-11-24T08:06:06.250881+0000","last_peered":"2023-11-24T08:06:06.250881+0000","last_clean":"2023-11-24T08:06:06.250881+0000","last_became_active":"2023-11-24T08:06:05.100166+0000","last_became_peered":"2023-11-24T08:06:05.100166+0000","last_unstale":"2023-11-24T08:06:06.250881+0000","last_undegraded":"2023-11-24T08:06:06.250881+0000","last_fullsized":"2023-11-24T08:06:06.250881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:32.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.068470+0000","last_change":"2023-11-24T08:06:04.090915+0000","last_active":"2023-11-24T08:06:05.068470+0000","last_peered":"2023-11-24T08:06:05.068470+0000","last_clean":"2023-11-24T08:06:05.068470+0000","last_became_active":"2023-11-24T08:06:04.090279+0000","last_became_peered":"2023-11-24T08:06:04.090279+0000","last_unstale":"2023-11-24T08:06:05.068470+0000","last_undegraded":"2023-11-24T08:06:05.068470+0000","last_fullsized":"2023-11-24T08:06:05.068470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:33.221758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252809+0000","last_change":"2023-11-24T08:06:05.091728+0000","last_active":"2023-11-24T08:06:06.252809+0000","last_peered":"2023-11-24T08:06:06.252809+0000","last_clean":"2023-11-24T08:06:06.252809+0000","last_became_active":"2023-11-24T08:06:05.090643+0000","last_became_peered":"2023-11-24T08:06:05.090643+0000","last_unstale":"2023-11-24T08:06:06.252809+0000","last_undegraded":"2023-11-24T08:06:06.252809+0000","last_fullsized":"2023-11-24T08:06:06.252809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:45.526567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122459+0000","last_change":"2023-11-24T08:06:04.088343+0000","last_active":"2023-11-24T08:06:05.122459+0000","last_peered":"2023-11-24T08:06:05.122459+0000","last_clean":"2023-11-24T08:06:05.122459+0000","last_became_active":"2023-11-24T08:06:04.087687+0000","last_became_peered":"2023-11-24T08:06:04.087687+0000","last_unstale":"2023-11-24T08:06:05.122459+0000","last_undegraded":"2023-11-24T08:06:05.122459+0000","last_fullsized":"2023-11-24T08:06:05.122459+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:29.779887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248883+0000","last_change":"2023-11-24T08:06:05.097524+0000","last_active":"2023-11-24T08:06:06.248883+0000","last_peered":"2023-11-24T08:06:06.248883+0000","last_clean":"2023-11-24T08:06:06.248883+0000","last_became_active":"2023-11-24T08:06:05.095942+0000","last_became_peered":"2023-11-24T08:06:05.095942+0000","last_unstale":"2023-11-24T08:06:06.248883+0000","last_undegraded":"2023-11-24T08:06:06.248883+0000","last_fullsized":"2023-11-24T08:06:06.248883+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:57:14.392741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.121968+0000","last_change":"2023-11-24T08:06:04.071810+0000","last_active":"2023-11-24T08:06:05.121968+0000","last_peered":"2023-11-24T08:06:05.121968+0000","last_clean":"2023-11-24T08:06:05.121968+0000","last_became_active":"2023-11-24T08:06:04.071196+0000","last_became_peered":"2023-11-24T08:06:04.071196+0000","last_unstale":"2023-11-24T08:06:05.121968+0000","last_undegraded":"2023-11-24T08:06:05.121968+0000","last_fullsized":"2023-11-24T08:06:05.121968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:13:43.496491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252462+0000","last_change":"2023-11-24T08:06:05.093265+0000","last_active":"2023-11-24T08:06:06.252462+0000","last_peered":"2023-11-24T08:06:06.252462+0000","last_clean":"2023-11-24T08:06:06.252462+0000","last_became_active":"2023-11-24T08:06:05.092904+0000","last_became_peered":"2023-11-24T08:06:05.092904+0000","last_unstale":"2023-11-24T08:06:06.252462+0000","last_undegraded":"2023-11-24T08:06:06.252462+0000","last_fullsized":"2023-11-24T08:06:06.252462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:59.520725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.113304+0000","last_change":"2023-11-24T08:06:04.081460+0000","last_active":"2023-11-24T08:06:05.113304+0000","last_peered":"2023-11-24T08:06:05.113304+0000","last_clean":"2023-11-24T08:06:05.113304+0000","last_became_active":"2023-11-24T08:06:04.081176+0000","last_became_peered":"2023-11-24T08:06:04.081176+0000","last_unstale":"2023-11-24T08:06:05.113304+0000","last_undegraded":"2023-11-24T08:06:05.113304+0000","last_fullsized":"2023-11-24T08:06:05.113304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:11.044772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089003+0000","last_change":"2023-11-24T08:06:05.086676+0000","last_active":"2023-11-24T08:06:06.089003+0000","last_peered":"2023-11-24T08:06:06.089003+0000","last_clean":"2023-11-24T08:06:06.089003+0000","last_became_active":"2023-11-24T08:06:05.086342+0000","last_became_peered":"2023-11-24T08:06:05.086342+0000","last_unstale":"2023-11-24T08:06:06.089003+0000","last_undegraded":"2023-11-24T08:06:06.089003+0000","last_fullsized":"2023-11-24T08:06:06.089003+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:36.606580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120951+0000","last_change":"2023-11-24T08:06:04.085453+0000","last_active":"2023-11-24T08:06:05.120951+0000","last_peered":"2023-11-24T08:06:05.120951+0000","last_clean":"2023-11-24T08:06:05.120951+0000","last_became_active":"2023-11-24T08:06:04.085126+0000","last_became_peered":"2023-11-24T08:06:04.085126+0000","last_unstale":"2023-11-24T08:06:05.120951+0000","last_undegraded":"2023-11-24T08:06:05.120951+0000","last_fullsized":"2023-11-24T08:06:05.120951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:20.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089297+0000","last_change":"2023-11-24T08:06:05.092694+0000","last_active":"2023-11-24T08:06:06.089297+0000","last_peered":"2023-11-24T08:06:06.089297+0000","last_clean":"2023-11-24T08:06:06.089297+0000","last_became_active":"2023-11-24T08:06:05.092393+0000","last_became_peered":"2023-11-24T08:06:05.092393+0000","last_unstale":"2023-11-24T08:06:06.089297+0000","last_undegraded":"2023-11-24T08:06:06.089297+0000","last_fullsized":"2023-11-24T08:06:06.089297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:15.584979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.071093+0000","last_change":"2023-11-24T08:06:04.085155+0000","last_active":"2023-11-24T08:06:05.071093+0000","last_peered":"2023-11-24T08:06:05.071093+0000","last_clean":"2023-11-24T08:06:05.071093+0000","last_became_active":"2023-11-24T08:06:04.084831+0000","last_became_peered":"2023-11-24T08:06:04.084831+0000","last_unstale":"2023-11-24T08:06:05.071093+0000","last_undegraded":"2023-11-24T08:06:05.071093+0000","last_fullsized":"2023-11-24T08:06:05.071093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:33.580388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251386+0000","last_change":"2023-11-24T08:06:05.101508+0000","last_active":"2023-11-24T08:06:06.251386+0000","last_peered":"2023-11-24T08:06:06.251386+0000","last_clean":"2023-11-24T08:06:06.251386+0000","last_became_active":"2023-11-24T08:06:05.099363+0000","last_became_peered":"2023-11-24T08:06:05.099363+0000","last_unstale":"2023-11-24T08:06:06.251386+0000","last_undegraded":"2023-11-24T08:06:06.251386+0000","last_fullsized":"2023-11-24T08:06:06.251386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:42.213975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.120853+0000","last_change":"2023-11-24T08:06:04.087536+0000","last_active":"2023-11-24T08:06:05.120853+0000","last_peered":"2023-11-24T08:06:05.120853+0000","last_clean":"2023-11-24T08:06:05.120853+0000","last_became_active":"2023-11-24T08:06:04.087269+0000","last_became_peered":"2023-11-24T08:06:04.087269+0000","last_unstale":"2023-11-24T08:06:05.120853+0000","last_undegraded":"2023-11-24T08:06:05.120853+0000","last_fullsized":"2023-11-24T08:06:05.120853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:29.354205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.082906+0000","last_change":"2023-11-24T08:06:05.093409+0000","last_active":"2023-11-24T08:06:06.082906+0000","last_peered":"2023-11-24T08:06:06.082906+0000","last_clean":"2023-11-24T08:06:06.082906+0000","last_became_active":"2023-11-24T08:06:05.090895+0000","last_became_peered":"2023-11-24T08:06:05.090895+0000","last_unstale":"2023-11-24T08:06:06.082906+0000","last_undegraded":"2023-11-24T08:06:06.082906+0000","last_fullsized":"2023-11-24T08:06:06.082906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:12.059275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.090278+0000","last_change":"2023-11-24T08:06:04.085441+0000","last_active":"2023-11-24T08:06:05.090278+0000","last_peered":"2023-11-24T08:06:05.090278+0000","last_clean":"2023-11-24T08:06:05.090278+0000","last_became_active":"2023-11-24T08:06:04.085170+0000","last_became_peered":"2023-11-24T08:06:04.085170+0000","last_unstale":"2023-11-24T08:06:05.090278+0000","last_undegraded":"2023-11-24T08:06:05.090278+0000","last_fullsized":"2023-11-24T08:06:05.090278+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:14.570476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.248353+0000","last_change":"2023-11-24T08:06:05.095928+0000","last_active":"2023-11-24T08:06:06.248353+0000","last_peered":"2023-11-24T08:06:06.248353+0000","last_clean":"2023-11-24T08:06:06.248353+0000","last_became_active":"2023-11-24T08:06:05.095601+0000","last_became_peered":"2023-11-24T08:06:05.095601+0000","last_unstale":"2023-11-24T08:06:06.248353+0000","last_undegraded":"2023-11-24T08:06:06.248353+0000","last_fullsized":"2023-11-24T08:06:06.248353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:33.727928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.083543+0000","last_change":"2023-11-24T08:06:04.086214+0000","last_active":"2023-11-24T08:06:05.083543+0000","last_peered":"2023-11-24T08:06:05.083543+0000","last_clean":"2023-11-24T08:06:05.083543+0000","last_became_active":"2023-11-24T08:06:04.085918+0000","last_became_peered":"2023-11-24T08:06:04.085918+0000","last_unstale":"2023-11-24T08:06:05.083543+0000","last_undegraded":"2023-11-24T08:06:05.083543+0000","last_fullsized":"2023-11-24T08:06:05.083543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:54.440803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251927+0000","last_change":"2023-11-24T08:06:05.095494+0000","last_active":"2023-11-24T08:06:06.251927+0000","last_peered":"2023-11-24T08:06:06.251927+0000","last_clean":"2023-11-24T08:06:06.251927+0000","last_became_active":"2023-11-24T08:06:05.094309+0000","last_became_peered":"2023-11-24T08:06:05.094309+0000","last_unstale":"2023-11-24T08:06:06.251927+0000","last_undegraded":"2023-11-24T08:06:06.251927+0000","last_fullsized":"2023-11-24T08:06:06.251927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:49.105781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063326+0000","last_change":"2023-11-24T08:06:04.093116+0000","last_active":"2023-11-24T08:06:05.063326+0000","last_peered":"2023-11-24T08:06:05.063326+0000","last_clean":"2023-11-24T08:06:05.063326+0000","last_became_active":"2023-11-24T08:06:04.092446+0000","last_became_peered":"2023-11-24T08:06:04.092446+0000","last_unstale":"2023-11-24T08:06:05.063326+0000","last_undegraded":"2023-11-24T08:06:05.063326+0000","last_fullsized":"2023-11-24T08:06:05.063326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:35.592076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.089777+0000","last_change":"2023-11-24T08:06:05.093077+0000","last_active":"2023-11-24T08:06:06.089777+0000","last_peered":"2023-11-24T08:06:06.089777+0000","last_clean":"2023-11-24T08:06:06.089777+0000","last_became_active":"2023-11-24T08:06:05.092754+0000","last_became_peered":"2023-11-24T08:06:05.092754+0000","last_unstale":"2023-11-24T08:06:06.089777+0000","last_undegraded":"2023-11-24T08:06:06.089777+0000","last_fullsized":"2023-11-24T08:06:06.089777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:13.879787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.074657+0000","last_change":"2023-11-24T08:06:04.079922+0000","last_active":"2023-11-24T08:06:05.074657+0000","last_peered":"2023-11-24T08:06:05.074657+0000","last_clean":"2023-11-24T08:06:05.074657+0000","last_became_active":"2023-11-24T08:06:04.079489+0000","last_became_peered":"2023-11-24T08:06:04.079489+0000","last_unstale":"2023-11-24T08:06:05.074657+0000","last_undegraded":"2023-11-24T08:06:05.074657+0000","last_fullsized":"2023-11-24T08:06:05.074657+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:04.038305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252779+0000","last_change":"2023-11-24T08:06:05.105800+0000","last_active":"2023-11-24T08:06:06.252779+0000","last_peered":"2023-11-24T08:06:06.252779+0000","last_clean":"2023-11-24T08:06:06.252779+0000","last_became_active":"2023-11-24T08:06:05.105521+0000","last_became_peered":"2023-11-24T08:06:05.105521+0000","last_unstale":"2023-11-24T08:06:06.252779+0000","last_undegraded":"2023-11-24T08:06:06.252779+0000","last_fullsized":"2023-11-24T08:06:06.252779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:32.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.067003+0000","last_change":"2023-11-24T08:06:04.094274+0000","last_active":"2023-11-24T08:06:05.067003+0000","last_peered":"2023-11-24T08:06:05.067003+0000","last_clean":"2023-11-24T08:06:05.067003+0000","last_became_active":"2023-11-24T08:06:04.093959+0000","last_became_peered":"2023-11-24T08:06:04.093959+0000","last_unstale":"2023-11-24T08:06:05.067003+0000","last_undegraded":"2023-11-24T08:06:05.067003+0000","last_fullsized":"2023-11-24T08:06:05.067003+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:33.221758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.253430+0000","last_change":"2023-11-24T08:06:05.082611+0000","last_active":"2023-11-24T08:06:06.253430+0000","last_peered":"2023-11-24T08:06:06.253430+0000","last_clean":"2023-11-24T08:06:06.253430+0000","last_became_active":"2023-11-24T08:06:05.082314+0000","last_became_peered":"2023-11-24T08:06:05.082314+0000","last_unstale":"2023-11-24T08:06:06.253430+0000","last_undegraded":"2023-11-24T08:06:06.253430+0000","last_fullsized":"2023-11-24T08:06:06.253430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:30.368708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.060457+0000","last_change":"2023-11-24T08:06:04.087054+0000","last_active":"2023-11-24T08:06:05.060457+0000","last_peered":"2023-11-24T08:06:05.060457+0000","last_clean":"2023-11-24T08:06:05.060457+0000","last_became_active":"2023-11-24T08:06:04.086627+0000","last_became_peered":"2023-11-24T08:06:04.086627+0000","last_unstale":"2023-11-24T08:06:05.060457+0000","last_undegraded":"2023-11-24T08:06:05.060457+0000","last_fullsized":"2023-11-24T08:06:05.060457+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:01.324343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251497+0000","last_change":"2023-11-24T08:06:05.090923+0000","last_active":"2023-11-24T08:06:06.251497+0000","last_peered":"2023-11-24T08:06:06.251497+0000","last_clean":"2023-11-24T08:06:06.251497+0000","last_became_active":"2023-11-24T08:06:05.089665+0000","last_became_peered":"2023-11-24T08:06:05.089665+0000","last_unstale":"2023-11-24T08:06:06.251497+0000","last_undegraded":"2023-11-24T08:06:06.251497+0000","last_fullsized":"2023-11-24T08:06:06.251497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:11.077626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.064897+0000","last_change":"2023-11-24T08:06:04.094285+0000","last_active":"2023-11-24T08:06:05.064897+0000","last_peered":"2023-11-24T08:06:05.064897+0000","last_clean":"2023-11-24T08:06:05.064897+0000","last_became_active":"2023-11-24T08:06:04.093933+0000","last_became_peered":"2023-11-24T08:06:04.093933+0000","last_unstale":"2023-11-24T08:06:05.064897+0000","last_undegraded":"2023-11-24T08:06:05.064897+0000","last_fullsized":"2023-11-24T08:06:05.064897+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:36.353720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.251553+0000","last_change":"2023-11-24T08:06:05.086959+0000","last_active":"2023-11-24T08:06:06.251553+0000","last_peered":"2023-11-24T08:06:06.251553+0000","last_clean":"2023-11-24T08:06:06.251553+0000","last_became_active":"2023-11-24T08:06:05.085952+0000","last_became_peered":"2023-11-24T08:06:05.085952+0000","last_unstale":"2023-11-24T08:06:06.251553+0000","last_undegraded":"2023-11-24T08:06:06.251553+0000","last_fullsized":"2023-11-24T08:06:06.251553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:59.924717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.081417+0000","last_change":"2023-11-24T08:06:04.070754+0000","last_active":"2023-11-24T08:06:05.081417+0000","last_peered":"2023-11-24T08:06:05.081417+0000","last_clean":"2023-11-24T08:06:05.081417+0000","last_became_active":"2023-11-24T08:06:04.070430+0000","last_became_peered":"2023-11-24T08:06:04.070430+0000","last_unstale":"2023-11-24T08:06:05.081417+0000","last_undegraded":"2023-11-24T08:06:05.081417+0000","last_fullsized":"2023-11-24T08:06:05.081417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:09.647409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.250913+0000","last_change":"2023-11-24T08:06:05.093665+0000","last_active":"2023-11-24T08:06:06.250913+0000","last_peered":"2023-11-24T08:06:06.250913+0000","last_clean":"2023-11-24T08:06:06.250913+0000","last_became_active":"2023-11-24T08:06:05.093393+0000","last_became_peered":"2023-11-24T08:06:05.093393+0000","last_unstale":"2023-11-24T08:06:06.250913+0000","last_undegraded":"2023-11-24T08:06:06.250913+0000","last_fullsized":"2023-11-24T08:06:06.250913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:55.455306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.061092+0000","last_change":"2023-11-24T08:06:04.084482+0000","last_active":"2023-11-24T08:06:05.061092+0000","last_peered":"2023-11-24T08:06:05.061092+0000","last_clean":"2023-11-24T08:06:05.061092+0000","last_became_active":"2023-11-24T08:06:04.084022+0000","last_became_peered":"2023-11-24T08:06:04.084022+0000","last_unstale":"2023-11-24T08:06:05.061092+0000","last_undegraded":"2023-11-24T08:06:05.061092+0000","last_fullsized":"2023-11-24T08:06:05.061092+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:51:33.198531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.087356+0000","last_change":"2023-11-24T08:06:05.093359+0000","last_active":"2023-11-24T08:06:06.087356+0000","last_peered":"2023-11-24T08:06:06.087356+0000","last_clean":"2023-11-24T08:06:06.087356+0000","last_became_active":"2023-11-24T08:06:05.093091+0000","last_became_peered":"2023-11-24T08:06:05.093091+0000","last_unstale":"2023-11-24T08:06:06.087356+0000","last_undegraded":"2023-11-24T08:06:06.087356+0000","last_fullsized":"2023-11-24T08:06:06.087356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:21.615036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.077077+0000","last_change":"2023-11-24T08:06:04.081603+0000","last_active":"2023-11-24T08:06:05.077077+0000","last_peered":"2023-11-24T08:06:05.077077+0000","last_clean":"2023-11-24T08:06:05.077077+0000","last_became_active":"2023-11-24T08:06:04.081376+0000","last_became_peered":"2023-11-24T08:06:04.081376+0000","last_unstale":"2023-11-24T08:06:05.077077+0000","last_undegraded":"2023-11-24T08:06:05.077077+0000","last_fullsized":"2023-11-24T08:06:05.077077+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:11.044772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T08:06:06.252755+0000","last_change":"2023-11-24T08:06:05.095041+0000","last_active":"2023-11-24T08:06:06.252755+0000","last_peered":"2023-11-24T08:06:06.252755+0000","last_clean":"2023-11-24T08:06:06.252755+0000","last_became_active":"2023-11-24T08:06:05.094688+0000","last_became_peered":"2023-11-24T08:06:05.094688+0000","last_unstale":"2023-11-24T08:06:06.252755+0000","last_undegraded":"2023-11-24T08:06:06.252755+0000","last_fullsized":"2023-11-24T08:06:06.252755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:04.042801+0000","last_clean_scrub_stamp":"2023-11-24T08:06:04.042801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:49.120781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.122403+0000","last_change":"2023-11-24T08:06:04.088807+0000","last_active":"2023-11-24T08:06:05.122403+0000","last_peered":"2023-11-24T08:06:05.122403+0000","last_clean":"2023-11-24T08:06:05.122403+0000","last_became_active":"2023-11-24T08:06:04.087267+0000","last_became_peered":"2023-11-24T08:06:04.087267+0000","last_unstale":"2023-11-24T08:06:05.122403+0000","last_undegraded":"2023-11-24T08:06:05.122403+0000","last_fullsized":"2023-11-24T08:06:05.122403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:03.028297+0000","last_clean_scrub_stamp":"2023-11-24T08:06:03.028297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:46.590933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:06:05.063198+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:06:05.063198+0000","last_peered":"2023-11-24T08:06:05.063198+0000","last_clean":"2023-11-24T08:06:05.063198+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:06:05.063198+0000","last_undegraded":"2023-11-24T08:06:05.063198+0000","last_fullsized":"2023-11-24T08:06:05.063198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.65300000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.431},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.069,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3200000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7649999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4179999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.5569999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1539999999999999,"15min":1.149},"min":{"1min":0.80600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.8660000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.80300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8879999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.2270000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3120000000000001,"15min":1.3300000000000001},"min":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5109999999999999,"5min":2.524,"15min":2.524},"last":1.498},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.504,"15min":1.534},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7709999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3169999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.90800000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.613,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3799999999999999,"15min":1.3839999999999999},"min":{"1min":0.87,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0270000000000001,"5min":2.238,"15min":2.238},"last":1.4510000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.619,"15min":1.6319999999999999},"min":{"1min":1.1839999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.9399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.383},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.647,"15min":1.677},"min":{"1min":1.0489999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7250000000000001,"5min":2.944,"15min":2.944},"last":1.633}]},{"osd":6,"last update":"Fri Nov 24 08:05:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.7350000000000001,"15min":1.77},"min":{"1min":1.1699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.891,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.671,"15min":1.6870000000000001},"min":{"1min":1.179,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1589999999999998,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.4059999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":215941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.538,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.323,"15min":1.355},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5629999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.4610000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1499999999999999,"15min":1.1579999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.304},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.274,"15min":1.319},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.6830000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.282,"15min":1.3129999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0710000000000002,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.4750000000000001,"15min":1.526},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.5509999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.5860000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.4299999999999999,"15min":1.458},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.554,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.081},{"interface":"front","average":{"1min":1.401,"5min":1.49,"15min":1.5049999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.7749999999999999,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.645}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.724,"15min":1.79},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.157,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.21},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.472,"15min":1.4830000000000001},"min":{"1min":0.90400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.895,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.76}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.7709999999999999,"15min":1.843},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3490000000000002,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.71},{"interface":"front","average":{"1min":1.508,"5min":1.9179999999999999,"15min":1.986},"min":{"1min":0.85399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8400000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.704},"min":{"1min":0.79400000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.2290000000000001,"5min":2.98,"15min":2.98},"last":1.859},{"interface":"front","average":{"1min":1.548,"5min":1.643,"15min":1.659},"min":{"1min":0.94199999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9379999999999999,"5min":2.29,"15min":2.29},"last":1.829}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.042,"15min":1.0580000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.627,"5min":1.764,"15min":1.764},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.02,"15min":1.002},"min":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.268}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1739999999999999,"15min":1.1819999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6299999999999999,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.0669999999999999,"15min":1.0469999999999999},"min":{"1min":0.85699999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.327}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.206,"15min":1.212},"min":{"1min":0.67200000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7490000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.2330000000000001,"15min":1.218},"min":{"1min":0.78900000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.4830000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.99199999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.597,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.419},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":1.0369999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.667,"5min":1.889,"15min":1.889},"last":1.1200000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.288,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6419999999999999,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.413,"15min":1.413},"min":{"1min":1.206,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9339999999999999,"5min":2.27,"15min":2.27},"last":1.365}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.464,"15min":1.4750000000000001},"min":{"1min":1.05,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.617,"5min":3.806,"15min":3.806},"last":1.232},{"interface":"front","average":{"1min":1.389,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.95199999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.726,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.522}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.52,"15min":1.528},"min":{"1min":1.1919999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8500000000000001,"5min":2.923,"15min":2.923},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.5309999999999999,"15min":1.5269999999999999},"min":{"1min":1.343,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7769999999999999,"5min":2.859,"15min":2.859},"last":1.3919999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0529999999999999,"15min":1.046},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.129,"15min":1.127},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.097}]},{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.099,"15min":1.091},"min":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.81799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.3520000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.198,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6279999999999999,"5min":1.823,"15min":1.823},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.264,"15min":1.266},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.1519999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.298,"15min":1.302},"min":{"1min":0.90600000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.597},{"interface":"front","average":{"1min":1.327,"5min":1.319,"15min":1.3169999999999999},"min":{"1min":0.98299999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6419999999999999,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.9530000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.304,"15min":1.304},"min":{"1min":0.86899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.702,"5min":1.944,"15min":1.944},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.361,"15min":1.365},"min":{"1min":0.96899999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.696,"5min":2.012,"15min":2.012},"last":1.7809999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3700000000000001,"15min":1.3660000000000001},"min":{"1min":0.91300000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.8560000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":2.105},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5129999999999999,"15min":1.5149999999999999},"min":{"1min":1.198,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0129999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.927}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.1100000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.431},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.573,"15min":1.5649999999999999},"min":{"1min":1.0880000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.5920000000000001,"5min":2.722,"15min":2.722},"last":2.411}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":215907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.82199999999999995,"15min":0.81899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.94299999999999995,"15min":0.95499999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.4399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.129,"15min":1.169},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6359999999999999,"5min":3.246,"15min":3.246},"last":1.835},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.97299999999999998,"15min":0.96699999999999997},"min":{"1min":0.441,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.94499999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1799999999999999,"15min":1.196},"min":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.718,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.147,"15min":1.165},"min":{"1min":0.54000000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5900000000000001,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.236}]},{"osd":4,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.93999999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6879999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.379,"15min":1.3979999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.73,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.379}]},{"osd":5,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.448,"15min":1.468},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9870000000000001,"5min":2.395,"15min":2.395},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.304,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.2410000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4930000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4299999999999999,"15min":1.4350000000000001},"min":{"1min":0.83199999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.1600000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4159999999999999,"15min":1.4239999999999999},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.946,"5min":1.996,"15min":1.996},"last":1.321}]},{"osd":7,"last update":"Fri Nov 24 08:05:53 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4430000000000001,"15min":1.4510000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.645},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.895}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.436},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":1.0209999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.272,"5min":2.133,"15min":2.133},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.194,"15min":1.242},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.371,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.048,"15min":1.0840000000000001},"min":{"1min":0.5,"5min":0.438,"15min":0.438},"max":{"1min":1.208,"5min":1.998,"15min":1.998},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3089999999999999,"15min":1.333},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9259999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.175,"15min":1.204},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.552,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3140000000000001,"15min":1.343},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.631,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.754},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.798,"5min":3.161,"15min":3.161},"last":1.123}]},{"osd":5,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3640000000000001,"15min":1.3999999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7649999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.391,"15min":1.423},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7549999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.5620000000000001,"15min":1.6080000000000001},"min":{"1min":0.73899999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.825,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.159},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.857,"5min":2.702,"15min":2.702},"last":0.94199999999999995}]},{"osd":7,"last update":"Fri Nov 24 08:05:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.5009999999999999,"15min":1.5309999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.3980000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.6180000000000001,"15min":1.6619999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.9199999999999999,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.085}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1919999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.105,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.0069999999999999,"15min":1.0389999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2230000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.214,"15min":1.2410000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.353,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.161}]},{"osd":3,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2569999999999999,"15min":1.2769999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6319999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.897},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.2110000000000001,"15min":1.2430000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.538,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.6890000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.242,"15min":1.2549999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.512,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.512},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.4390000000000001,"15min":1.4610000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6459999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.4490000000000001,"15min":1.4650000000000001},"min":{"1min":0.85699999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9590000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.468,"15min":1.484},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.746,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.655}]},{"osd":6,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.45,"15min":1.464},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9019999999999999,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4690000000000001,"15min":1.4910000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.821,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.8420000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.548,"15min":1.5700000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.744,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.738},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5800000000000001,"15min":1.599},"min":{"1min":0.94499999999999995,"5min":0.872,"15min":0.872},"max":{"1min":1.8160000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.788}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":804652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.367,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":3.423},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.93000000000000005},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":3.3180000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.05,"15min":1.054},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.706,"5min":2.29,"15min":2.29},"last":3.899},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":0.93700000000000006},"min":{"1min":0.51000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.5049999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":4.0140000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.032,"15min":1.028},"min":{"1min":0.60499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6439999999999999,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":3.4830000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8640000000000001,"5min":3.6320000000000001,"15min":3.6320000000000001},"last":6.3570000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.256,"15min":1.254},"min":{"1min":0.95899999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.704,"5min":3.302,"15min":3.302},"last":3.8479999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.454,"15min":1.466},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.2389999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":3.399}]},{"osd":5,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":0.89400000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.0590000000000002,"5min":3.101,"15min":3.101},"last":5.9939999999999998},{"interface":"front","average":{"1min":1.462,"5min":1.4359999999999999,"15min":1.4319999999999999},"min":{"1min":0.90100000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.1779999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.391}]},{"osd":6,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3999999999999999,"15min":1.4159999999999999},"min":{"1min":0.75,"5min":0.749,"15min":0.749},"max":{"1min":2.1819999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":3.9729999999999999},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4970000000000001,"15min":1.4930000000000001},"min":{"1min":1.0700000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1030000000000002,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":3.7320000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:05:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.5309999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1600000000000001,"5min":3.823,"15min":3.823},"last":3.4540000000000002},{"interface":"front","average":{"1min":1.397,"5min":1.468,"15min":1.48},"min":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2610000000000001,"5min":3.371,"15min":3.371},"last":3.9169999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:06:15.202 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:06:15.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:06:15.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:06:15.293 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:06:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.465+0000 7f9015999640 1 Processor -- start 2023-11-24T08:06:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.465+0000 7f9015999640 1 -- start start 2023-11-24T08:06:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.467+0000 7f9015999640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90101448c0 0x7f901006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.467+0000 7f9015999640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f901006d910 0x7f901015b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.467+0000 7f9015999640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f901015bbd0 0x7f901015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.467+0000 7f9015999640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901006dd50 con 0x7f901006d910 2023-11-24T08:06:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.467+0000 7f9015999640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9010144ca0 con 0x7f901015bbd0 2023-11-24T08:06:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.467+0000 7f9015999640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901015c450 con 0x7f90101448c0 2023-11-24T08:06:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f900ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f901006d910 0x7f901015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f900ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f901006d910 0x7f901015b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43138/0 (socket says 172.21.15.64:43138) 2023-11-24T08:06:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f900ffff640 1 -- 172.21.15.64:0/2025620226 learned_addr learned my addr 172.21.15.64:0/2025620226 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f9014997640 1 --2- 172.21.15.64:0/2025620226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90101448c0 0x7f901006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f9015198640 1 --2- 172.21.15.64:0/2025620226 >> v2:172.21.15.91:3300/0 conn(0x7f901015bbd0 0x7f901015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f900ffff640 1 -- 172.21.15.64:0/2025620226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90101448c0 msgr2=0x7f901006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f900ffff640 1 --2- 172.21.15.64:0/2025620226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90101448c0 0x7f901006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f900ffff640 1 -- 172.21.15.64:0/2025620226 >> v2:172.21.15.91:3300/0 conn(0x7f901015bbd0 msgr2=0x7f901015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f900ffff640 1 --2- 172.21.15.64:0/2025620226 >> v2:172.21.15.91:3300/0 conn(0x7f901015bbd0 0x7f901015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.468+0000 7f900ffff640 1 -- 172.21.15.64:0/2025620226 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90100715b0 con 0x7f901006d910 2023-11-24T08:06:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.469+0000 7f900ffff640 1 --2- 172.21.15.64:0/2025620226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f901015b690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f90100cc090 tx=0x7f8ffc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e69b556225a71d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f900f7fe640 1 -- 172.21.15.64:0/2025620226 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc004230 con 0x7f901006d910 2023-11-24T08:06:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f900f7fe640 1 -- 172.21.15.64:0/2025620226 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc004390 con 0x7f901006d910 2023-11-24T08:06:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f900f7fe640 1 -- 172.21.15.64:0/2025620226 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc003810 con 0x7f901006d910 2023-11-24T08:06:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 -- 172.21.15.64:0/2025620226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 msgr2=0x7f901015b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 --2- 172.21.15.64:0/2025620226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f901015b690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f90100cc090 tx=0x7f8ffc02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 -- 172.21.15.64:0/2025620226 shutdown_connections 2023-11-24T08:06:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 --2- 172.21.15.64:0/2025620226 >> v2:172.21.15.91:3300/0 conn(0x7f901015bbd0 0x7f901015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 --2- 172.21.15.64:0/2025620226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f901015b690 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 --2- 172.21.15.64:0/2025620226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90101448c0 0x7f901006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 -- 172.21.15.64:0/2025620226 >> 172.21.15.64:0/2025620226 conn(0x7f90100b0aa0 msgr2=0x7f90100b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 -- 172.21.15.64:0/2025620226 shutdown_connections 2023-11-24T08:06:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.470+0000 7f9015999640 1 -- 172.21.15.64:0/2025620226 wait complete. 2023-11-24T08:06:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.472+0000 7f9015999640 1 Processor -- start 2023-11-24T08:06:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.472+0000 7f9015999640 1 -- start start 2023-11-24T08:06:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.473+0000 7f9015999640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f90101f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.473+0000 7f9015999640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90101448c0 0x7f90101f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.473+0000 7f9015999640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f901015bbd0 0x7f9010143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.473+0000 7f9015999640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90100a8460 con 0x7f901006d910 2023-11-24T08:06:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.473+0000 7f9015999640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90101f1980 con 0x7f90101448c0 2023-11-24T08:06:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.473+0000 7f9015999640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90100a8700 con 0x7f901015bbd0 2023-11-24T08:06:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.473+0000 7f9014997640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f90101f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9014997640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f90101f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43154/0 (socket says 172.21.15.64:43154) 2023-11-24T08:06:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9014997640 1 -- 172.21.15.64:0/3273970888 learned_addr learned my addr 172.21.15.64:0/3273970888 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9015198640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f901015bbd0 0x7f9010143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f900ffff640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90101448c0 0x7f90101f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9014997640 1 -- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f901015bbd0 msgr2=0x7f9010143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9014997640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f901015bbd0 0x7f9010143170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9014997640 1 -- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90101448c0 msgr2=0x7f90101f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9014997640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90101448c0 0x7f90101f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9014997640 1 -- 172.21.15.64:0/3273970888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc009460 con 0x7f901006d910 2023-11-24T08:06:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.474+0000 7f9014997640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f90101f1440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f900400d800 tx=0x7f900400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.475+0000 7f900d7fa640 1 -- 172.21.15.64:0/3273970888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400b6c0 con 0x7f901006d910 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.475+0000 7f900d7fa640 1 -- 172.21.15.64:0/3273970888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400b820 con 0x7f901006d910 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.475+0000 7f900d7fa640 1 -- 172.21.15.64:0/3273970888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400bbb0 con 0x7f901006d910 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.475+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90100a89e0 con 0x7f901006d910 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.475+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90100a8fa0 con 0x7f901006d910 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.476+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901004e840 con 0x7f901006d910 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.477+0000 7f900d7fa640 1 -- 172.21.15.64:0/3273970888 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f900400bd10 con 0x7f901006d910 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.478+0000 7f900d7fa640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ff406aed0 0x7f8ff406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.478+0000 7f900d7fa640 1 -- 172.21.15.64:0/3273970888 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9004089ce0 con 0x7f901006d910 2023-11-24T08:06:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.478+0000 7f900ffff640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ff406aed0 0x7f8ff406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.480+0000 7f900ffff640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ff406aed0 0x7f8ff406d390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc035000 tx=0x7f8ffc008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:15.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.482+0000 7f900d7fa640 1 -- 172.21.15.64:0/3273970888 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9004054ca0 con 0x7f901006d910 2023-11-24T08:06:15.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.746+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f901006d5d0 con 0x7f901006d910 2023-11-24T08:06:15.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.750+0000 7f900d7fa640 1 -- 172.21.15.64:0/3273970888 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f90040582c0 con 0x7f901006d910 2023-11-24T08:06:15.753 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-11-24T08:06:15.753 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA== 2023-11-24T08:06:15.753 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow" 2023-11-24T08:06:15.753 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow r" 2023-11-24T08:06:15.753 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:06:15.753 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-11-24T08:06:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ff406aed0 msgr2=0x7f8ff406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ff406aed0 0x7f8ff406d390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc035000 tx=0x7f8ffc008030 comp rx=0 tx=0).stop 2023-11-24T08:06:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 msgr2=0x7f90101f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f90101f1440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f900400d800 tx=0x7f900400dcc0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 shutdown_connections 2023-11-24T08:06:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8ff406aed0 0x7f8ff406d390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f901015bbd0 0x7f9010143170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90101448c0 0x7f90101f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 --2- 172.21.15.64:0/3273970888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f901006d910 0x7f90101f1440 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:15.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 >> 172.21.15.64:0/3273970888 conn(0x7f90100b0aa0 msgr2=0x7f901015a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:15.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 shutdown_connections 2023-11-24T08:06:15.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:15.753+0000 7f9015999640 1 -- 172.21.15.64:0/3273970888 wait complete. 2023-11-24T08:06:15.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:06:16.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.000+0000 7f7d8ac94640 1 Processor -- start 2023-11-24T08:06:16.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.001+0000 7f7d8ac94640 1 -- start start 2023-11-24T08:06:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.002+0000 7f7d8ac94640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d840719c0 0x7f7d840ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.003+0000 7f7d8ac94640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d840ad510 con 0x7f7d8406d520 2023-11-24T08:06:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.003+0000 7f7d8ac94640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d840acc50 con 0x7f7d84161930 2023-11-24T08:06:16.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.003+0000 7f7d8ac94640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d84071e60 con 0x7f7d840719c0 2023-11-24T08:06:16.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.003+0000 7f7d89231640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7d8406d520 0x7f7d840b2080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52022/0 (socket says 172.21.15.64:52022) 2023-11-24T08:06:16.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.003+0000 7f7d89231640 1 -- 172.21.15.64:0/935682242 learned_addr learned my addr 172.21.15.64:0/935682242 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:16.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.003+0000 7f7d83fff640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d840719c0 0x7f7d840ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462955650 0 0) 0x7f7d840ad510 con 0x7f7d8406d520 2023-11-24T08:06:16.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d6c003520 con 0x7f7d8406d520 2023-11-24T08:06:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2727218222 0 0) 0x7f7d6c003520 con 0x7f7d8406d520 2023-11-24T08:06:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d840719c0 msgr2=0x7f7d840ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d840719c0 0x7f7d840ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 >> v1:172.21.15.91:6789/0 conn(0x7f7d84161930 legacy=0x7f7d840ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d840acec0 con 0x7f7d8406d520 2023-11-24T08:06:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f7d7c001670 con 0x7f7d8406d520 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 >> v1:172.21.15.64:6789/0 conn(0x7f7d8406d520 legacy=0x7f7d840b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d6c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d6c0047c0 0x7f7d6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d840719c0 0x7f7d6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d84071e60 con 0x7f7d6c0039a0 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c00ba30 con 0x7f7d6c0047c0 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.004+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c00bbf0 con 0x7f7d840719c0 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.005+0000 7f7d88a30640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d6c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.005+0000 7f7d83fff640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d6c0047c0 0x7f7d6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.005+0000 7f7d89231640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d840719c0 0x7f7d6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.005+0000 7f7d88a30640 1 -- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d840719c0 msgr2=0x7f7d6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.005+0000 7f7d88a30640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d840719c0 0x7f7d6c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.005+0000 7f7d88a30640 1 -- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d6c0047c0 msgr2=0x7f7d6c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.005+0000 7f7d88a30640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d6c0047c0 0x7f7d6c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.005+0000 7f7d88a30640 1 -- 172.21.15.64:0/935682242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c003520 con 0x7f7d6c0039a0 2023-11-24T08:06:16.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.006+0000 7f7d88a30640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d6c003dc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7d74001e20 tx=0x7f7d6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b580ae22d051067c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.008+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74003b70 con 0x7f7d6c0039a0 2023-11-24T08:06:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.008+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74003cd0 con 0x7f7d6c0039a0 2023-11-24T08:06:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.008+0000 7f7d837fe640 1 -- 172.21.15.64:0/935682242 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74005310 con 0x7f7d6c0039a0 2023-11-24T08:06:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.009+0000 7f7d8ac94640 1 -- 172.21.15.64:0/935682242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 msgr2=0x7f7d6c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.009+0000 7f7d8ac94640 1 --2- 172.21.15.64:0/935682242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d6c003dc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7d74001e20 tx=0x7f7d6c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.009+0000 7f7d89231640 1 -- 172.21.15.64:0/935682242 reap_dead start 2023-11-24T08:06:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.009+0000 7f7d8ac94640 1 -- 172.21.15.64:0/935682242 shutdown_connections 2023-11-24T08:06:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.009+0000 7f7d8ac94640 1 -- 172.21.15.64:0/935682242 >> 172.21.15.64:0/935682242 conn(0x7f7d840b0d80 msgr2=0x7f7d840b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.009+0000 7f7d8ac94640 1 -- 172.21.15.64:0/935682242 shutdown_connections 2023-11-24T08:06:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.009+0000 7f7d8ac94640 1 -- 172.21.15.64:0/935682242 wait complete. 2023-11-24T08:06:16.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.011+0000 7f7d8ac94640 1 Processor -- start 2023-11-24T08:06:16.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.011+0000 7f7d8ac94640 1 -- start start 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d8ac94640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d841f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d8ac94640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d84159680 0x7f7d8409bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d8ac94640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d84159ae0 0x7f7d8409c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d8ac94640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8409b200 con 0x7f7d6c0039a0 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d8ac94640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840cd380 con 0x7f7d84159680 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d8ac94640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840cca50 con 0x7f7d84159ae0 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d88a30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d841f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d88a30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d841f7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43164/0 (socket says 172.21.15.64:43164) 2023-11-24T08:06:16.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.012+0000 7f7d88a30640 1 -- 172.21.15.64:0/869621622 learned_addr learned my addr 172.21.15.64:0/869621622 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d89231640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d84159ae0 0x7f7d8409c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d83fff640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d84159680 0x7f7d8409bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d88a30640 1 -- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d84159ae0 msgr2=0x7f7d8409c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d88a30640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d84159ae0 0x7f7d8409c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d88a30640 1 -- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d84159680 msgr2=0x7f7d8409bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d88a30640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d84159680 0x7f7d8409bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d88a30640 1 -- 172.21.15.64:0/869621622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d840ccd30 con 0x7f7d6c0039a0 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d83fff640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d84159680 0x7f7d8409bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:16.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d88a30640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d841f7520 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7d7402bf50 tx=0x7f7d740014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.013+0000 7f7d817fa640 1 -- 172.21.15.64:0/869621622 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7403e070 con 0x7f7d6c0039a0 2023-11-24T08:06:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.014+0000 7f7d817fa640 1 -- 172.21.15.64:0/869621622 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74006b50 con 0x7f7d6c0039a0 2023-11-24T08:06:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.014+0000 7f7d817fa640 1 -- 172.21.15.64:0/869621622 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74007030 con 0x7f7d6c0039a0 2023-11-24T08:06:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.014+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d840cd010 con 0x7f7d6c0039a0 2023-11-24T08:06:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.014+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8409a5b0 con 0x7f7d6c0039a0 2023-11-24T08:06:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.015+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d84074770 con 0x7f7d6c0039a0 2023-11-24T08:06:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.016+0000 7f7d817fa640 1 -- 172.21.15.64:0/869621622 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7d74007190 con 0x7f7d6c0039a0 2023-11-24T08:06:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.017+0000 7f7d817fa640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d7006aed0 0x7f7d7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.017+0000 7f7d817fa640 1 -- 172.21.15.64:0/869621622 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d740ab190 con 0x7f7d6c0039a0 2023-11-24T08:06:16.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.017+0000 7f7d83fff640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d7006aed0 0x7f7d7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.019+0000 7f7d83fff640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d7006aed0 0x7f7d7006d390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d8415d440 tx=0x7f7d78008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.021+0000 7f7d817fa640 1 -- 172.21.15.64:0/869621622 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d74076140 con 0x7f7d6c0039a0 2023-11-24T08:06:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.255+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d840717a0 con 0x7f7d6c0039a0 2023-11-24T08:06:16.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.256+0000 7f7d817fa640 1 -- 172.21.15.64:0/869621622 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f7d74079760 con 0x7f7d6c0039a0 2023-11-24T08:06:16.258 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:16.258 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/618780376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":618780376},{"type":"v1","addr":"172.21.15.64:6835","nonce":618780376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1212789084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1212789084},{"type":"v1","addr":"172.21.15.91:6836","nonce":1212789084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1895191218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1895191218},{"type":"v1","addr":"172.21.15.91:6837","nonce":1895191218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:04.548725+0000","modified":"2023-11-24T08:06:07.601727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/1271535459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1271535459},{"type":"v1","addr":"172.21.15.64:6834","nonce":1271535459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:06:16.258 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2023-11-24T08:06:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d7006aed0 msgr2=0x7f7d7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d7006aed0 0x7f7d7006d390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d8415d440 tx=0x7f7d78008040 comp rx=0 tx=0).stop 2023-11-24T08:06:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 msgr2=0x7f7d841f7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d841f7520 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7d7402bf50 tx=0x7f7d740014a0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 shutdown_connections 2023-11-24T08:06:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7d7006aed0 0x7f7d7006d390 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7d84159ae0 0x7f7d8409c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7d84159680 0x7f7d8409bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 --2- 172.21.15.64:0/869621622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0039a0 0x7f7d841f7520 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 >> 172.21.15.64:0/869621622 conn(0x7f7d840b0d80 msgr2=0x7f7d840af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.258+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 shutdown_connections 2023-11-24T08:06:16.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.259+0000 7f7d8ac94640 1 -- 172.21.15.64:0/869621622 wait complete. 2023-11-24T08:06:16.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:06:04.548725+0000', 'modified': '2023-11-24T08:06:07.601727+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5724}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5724': {'gid': 5724, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6834/1271535459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 1271535459}, {'type': 'v1', 'addr': '172.21.15.64:6834', 'nonce': 1271535459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:06:16.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:06:16.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T08:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.913+0000 7f937306e640 1 Processor -- start 2023-11-24T08:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.914+0000 7f937306e640 1 -- start start 2023-11-24T08:06:16.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.915+0000 7f937306e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f936c071620 0x7f936c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.916+0000 7f937306e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c15b600 con 0x7f936c071a00 2023-11-24T08:06:16.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.916+0000 7f937306e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c0ce8d0 con 0x7f936c070e90 2023-11-24T08:06:16.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.916+0000 7f937306e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c0700d0 con 0x7f936c071620 2023-11-24T08:06:16.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.916+0000 7f937186b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f936c071a00 0x7f936c0cc630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52030/0 (socket says 172.21.15.64:52030) 2023-11-24T08:06:16.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.916+0000 7f937186b640 1 -- 172.21.15.64:0/1526846657 learned_addr learned my addr 172.21.15.64:0/1526846657 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.916+0000 7f937206c640 1 --2- 172.21.15.64:0/1526846657 >> v2:172.21.15.91:3300/0 conn(0x7f936c071620 0x7f936c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.916+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546505856 0 0) 0x7f936c15b600 con 0x7f936c071a00 2023-11-24T08:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.916+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9354003520 con 0x7f936c071a00 2023-11-24T08:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633332028 0 0) 0x7f9354003520 con 0x7f936c071a00 2023-11-24T08:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 >> v2:172.21.15.91:3300/0 conn(0x7f936c071620 msgr2=0x7f936c15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 --2- 172.21.15.64:0/1526846657 >> v2:172.21.15.91:3300/0 conn(0x7f936c071620 0x7f936c15e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 >> v1:172.21.15.91:6789/0 conn(0x7f936c070e90 legacy=0x7f936c15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0703b0 con 0x7f936c071a00 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f9360001670 con 0x7f936c071a00 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 >> v1:172.21.15.64:6789/0 conn(0x7f936c071a00 legacy=0x7f936c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9354003a30 0x7f9354003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93540047c0 0x7f9354004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93540051d0 0x7f935400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0700d0 con 0x7f9354003a30 2023-11-24T08:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935400c100 con 0x7f93540051d0 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935400c2c0 con 0x7f93540047c0 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.917+0000 7f937206c640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9354003a30 0x7f9354003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.918+0000 7f937286d640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93540051d0 0x7f935400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.918+0000 7f937206c640 1 -- 172.21.15.64:0/1526846657 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93540047c0 msgr2=0x7f9354004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.918+0000 7f937206c640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93540047c0 0x7f9354004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.918+0000 7f937206c640 1 -- 172.21.15.64:0/1526846657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93540051d0 msgr2=0x7f935400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.918+0000 7f937206c640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93540051d0 0x7f935400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.918+0000 7f937206c640 1 -- 172.21.15.64:0/1526846657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9354003520 con 0x7f9354003a30 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.918+0000 7f937286d640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93540051d0 0x7f935400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.918+0000 7f937286d640 1 -- 172.21.15.64:0/1526846657 reap_dead start 2023-11-24T08:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.919+0000 7f937206c640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9354003a30 0x7f9354003e50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f935c009fc0 tx=0x7f93540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a490122b4e9af2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.919+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c02cc20 con 0x7f9354003a30 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.919+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c00f040 con 0x7f9354003a30 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.919+0000 7f937106a640 1 -- 172.21.15.64:0/1526846657 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c034440 con 0x7f9354003a30 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.920+0000 7f937306e640 1 -- 172.21.15.64:0/1526846657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9354003a30 msgr2=0x7f9354003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.920+0000 7f937306e640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9354003a30 0x7f9354003e50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f935c009fc0 tx=0x7f93540057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.920+0000 7f937306e640 1 -- 172.21.15.64:0/1526846657 shutdown_connections 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.920+0000 7f937306e640 1 --2- 172.21.15.64:0/1526846657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9354003a30 0x7f9354003e50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.920+0000 7f937306e640 1 -- 172.21.15.64:0/1526846657 >> 172.21.15.64:0/1526846657 conn(0x7f936c0b0840 msgr2=0x7f936c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.920+0000 7f937306e640 1 -- 172.21.15.64:0/1526846657 shutdown_connections 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.920+0000 7f937306e640 1 -- 172.21.15.64:0/1526846657 wait complete. 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.922+0000 7f937306e640 1 Processor -- start 2023-11-24T08:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.922+0000 7f937306e640 1 -- start start 2023-11-24T08:06:16.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937306e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9354003a30 0x7f936c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937306e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f936c146d30 0x7f936c1f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937306e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936c0b2300 0x7f936c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937306e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0b35e0 con 0x7f936c0b2300 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937306e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c153290 con 0x7f936c146d30 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937306e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c1f3570 con 0x7f9354003a30 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937286d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936c0b2300 0x7f936c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937286d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936c0b2300 0x7f936c15a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43184/0 (socket says 172.21.15.64:43184) 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937286d640 1 -- 172.21.15.64:0/4017543668 learned_addr learned my addr 172.21.15.64:0/4017543668 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.923+0000 7f937206c640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9354003a30 0x7f936c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937286d640 1 -- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9354003a30 msgr2=0x7f936c151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937286d640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9354003a30 0x7f936c151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937186b640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f936c146d30 0x7f936c1f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937286d640 1 -- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f936c146d30 msgr2=0x7f936c1f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937286d640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f936c146d30 0x7f936c1f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937286d640 1 -- 172.21.15.64:0/4017543668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c1534e0 con 0x7f936c0b2300 2023-11-24T08:06:16.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937286d640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936c0b2300 0x7f936c15a0a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f936c0cfd10 tx=0x7f936400d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f936affd640 1 -- 172.21.15.64:0/4017543668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400b070 con 0x7f936c0b2300 2023-11-24T08:06:16.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f936affd640 1 -- 172.21.15.64:0/4017543668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93640052f0 con 0x7f936c0b2300 2023-11-24T08:06:16.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f936affd640 1 -- 172.21.15.64:0/4017543668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364003830 con 0x7f936c0b2300 2023-11-24T08:06:16.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937306e640 1 -- 172.21.15.64:0/4017543668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c1537c0 con 0x7f936c0b2300 2023-11-24T08:06:16.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.924+0000 7f937306e640 1 -- 172.21.15.64:0/4017543668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c0b3270 con 0x7f936c0b2300 2023-11-24T08:06:16.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.927+0000 7f936affd640 1 -- 172.21.15.64:0/4017543668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9364003990 con 0x7f936c0b2300 2023-11-24T08:06:16.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.928+0000 7f936affd640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f935806af00 0x7f935806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.928+0000 7f936affd640 1 -- 172.21.15.64:0/4017543668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9364088c70 con 0x7f936c0b2300 2023-11-24T08:06:16.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.929+0000 7f937206c640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f935806af00 0x7f935806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.929+0000 7f9376e5f740 1 Processor -- start 2023-11-24T08:06:16.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.929+0000 7f9376e5f740 1 -- start start 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa3070 0x564172aa3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172aa2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa27b0 0x564172ae6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f9376e5f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564172a01880 con 0x564172a9dd70 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f9376e5f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564172a10ae0 con 0x564172aa27b0 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f9376e5f740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x564172a0e1d0 con 0x564172aa3070 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f937206c640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f935806af00 0x7f935806d3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f935c036000 tx=0x7f935c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f937186b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172aa2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f937186b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172aa2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43190/0 (socket says 172.21.15.64:43190) 2023-11-24T08:06:16.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.930+0000 7f937186b640 1 -- 172.21.15.64:0/3007868814 learned_addr learned my addr 172.21.15.64:0/3007868814 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:16.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f937206c640 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa3070 0x564172aa3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f937186b640 1 -- 172.21.15.64:0/3007868814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa3070 msgr2=0x564172aa3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f937186b640 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa3070 0x564172aa3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f937286d640 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa27b0 0x564172ae6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f937186b640 1 -- 172.21.15.64:0/3007868814 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa27b0 msgr2=0x564172ae6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f937186b640 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa27b0 0x564172ae6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f937186b640 1 -- 172.21.15.64:0/3007868814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641729d1860 con 0x564172a9dd70 2023-11-24T08:06:16.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f937186b640 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172aa2270 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9360007870 tx=0x7f9360007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb5edfd1b62f5a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f9353fff640 1 -- 172.21.15.64:0/3007868814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93600074e0 con 0x564172a9dd70 2023-11-24T08:06:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f9353fff640 1 -- 172.21.15.64:0/3007868814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9360007660 con 0x564172a9dd70 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.931+0000 7f9353fff640 1 -- 172.21.15.64:0/3007868814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360004a20 con 0x564172a9dd70 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 -- 172.21.15.64:0/3007868814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 msgr2=0x564172aa2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172aa2270 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9360007870 tx=0x7f9360007d30 comp rx=0 tx=0).stop 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 -- 172.21.15.64:0/3007868814 shutdown_connections 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa27b0 0x564172ae6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172aa2270 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3007868814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa3070 0x564172aa3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 -- 172.21.15.64:0/3007868814 >> 172.21.15.64:0/3007868814 conn(0x564172a64c70 msgr2=0x564172ae8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 -- 172.21.15.64:0/3007868814 shutdown_connections 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.932+0000 7f9376e5f740 1 -- 172.21.15.64:0/3007868814 wait complete. 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.933+0000 7f9376e5f740 1 -- start start 2023-11-24T08:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.933+0000 7f9376e5f740 1 Processor -- start 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.934+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172ac7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.934+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172ad1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.934+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa3070 0x564172af0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.934+0000 7f9376e5f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564172a09ac0 con 0x564172a9dd70 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.934+0000 7f9376e5f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641729a62f0 con 0x564172aa3070 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.934+0000 7f9376e5f740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641729c3c70 con 0x564172aa27b0 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.934+0000 7f9376e5f740 10 client.0 _ll_register_callbacks cb 0x5641729c83b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172ac7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172ac7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43192/0 (socket says 172.21.15.64:43192) 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 -- 172.21.15.64:0/3591791509 learned_addr learned my addr 172.21.15.64:0/3591791509 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 -- 172.21.15.64:0/3591791509 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 msgr2=0x564172ad1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172ad1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 -- 172.21.15.64:0/3591791509 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa3070 msgr2=0x564172af0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa3070 0x564172af0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 -- 172.21.15.64:0/3591791509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c0703b0 con 0x564172a9dd70 2023-11-24T08:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f937206c640 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172ac7f20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f935c00e200 tx=0x7f935c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4089fc84441bd2f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.935+0000 7f93517fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f93517fa640 1 -- 172.21.15.64:0/3591791509 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c03e820 con 0x564172a9dd70 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 1 -- 172.21.15.64:0/3591791509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564172a02aa0 con 0x564172a9dd70 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f93517fa640 1 -- 172.21.15.64:0/3591791509 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f935c03e980 con 0x564172a9dd70 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f93517fa640 1 -- 172.21.15.64:0/3591791509 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f935c03eb40 con 0x564172a9dd70 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 10 client.5901 fetch_fsmap learned FSMap version 24 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 1 -- 172.21.15.64:0/3591791509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x564172a08140 con 0x564172a9dd70 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f93517fa640 1 -- 172.21.15.64:0/3591791509 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564172a02aa0 con 0x564172a9dd70 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f93517fa640 1 -- 172.21.15.64:0/3591791509 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f935c03eca0 con 0x564172a9dd70 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 10 client.5901 fetch_fsmap finished waiting for FSMap version 24 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 10 client.5901 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 20 client.5901 populate_metadata read hostname 'smithi064' 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] conn(0x564172afcc20 0x564172aedbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 4 client.5901 mds_command: new command op to 5724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:06:16.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.936+0000 7f9376e5f740 1 -- 172.21.15.64:0/3591791509 --> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564172a027c0 con 0x564172afcc20 2023-11-24T08:06:16.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.937+0000 7f937186b640 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] conn(0x564172afcc20 0x564172aedbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.937+0000 7f937186b640 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] conn(0x564172afcc20 0x564172aedbd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32eda76b555f1f1d server_cookie=2272df08bd567897 in_seq=0 out_seq=0 2023-11-24T08:06:16.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.937+0000 7f93517fa640 10 client.5901 ms_handle_connect on v2:172.21.15.64:6832/1271535459 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f93517fa640 1 -- 172.21.15.64:0/3591791509 <== mds.0 v2:172.21.15.64:6832/1271535459 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564172a027c0 con 0x564172afcc20 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f93517fa640 10 client.5901 handle_command_reply: tid=0 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 client.5901 shutdown 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 -- 172.21.15.64:0/3591791509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 msgr2=0x564172ac7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172ac7f20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f935c00e200 tx=0x7f935c004ab0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 -- 172.21.15.64:0/3591791509 shutdown_connections 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172aa3070 0x564172af0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172ad1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] conn(0x564172afcc20 0x564172aedbd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 --2- 172.21.15.64:0/3591791509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172a9dd70 0x564172ac7f20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f9376e5f740 1 -- 172.21.15.64:0/3591791509 >> 172.21.15.64:0/3591791509 conn(0x564172a64c70 msgr2=0x564172ae5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.938+0000 7f93517fa640 0 client.5901 ms_handle_reset on v2:172.21.15.64:6832/1271535459 2023-11-24T08:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.939+0000 7f9376e5f740 1 -- 172.21.15.64:0/3591791509 shutdown_connections 2023-11-24T08:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.939+0000 7f9376e5f740 1 -- 172.21.15.64:0/3591791509 wait complete. 2023-11-24T08:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.939+0000 7f9376e5f740 20 client.5901 trim_cache size 0 max 16384 2023-11-24T08:06:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.956+0000 7f9376e5f740 1 Processor -- start 2023-11-24T08:06:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.956+0000 7f9376e5f740 1 -- start start 2023-11-24T08:06:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 0x564172a997b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172ac8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ac8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f9376e5f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417276cf20 con 0x564172aa3070 2023-11-24T08:06:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f9376e5f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564172ad0ef0 con 0x564172a9dd70 2023-11-24T08:06:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f9376e5f740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641729bde20 con 0x564172aa27b0 2023-11-24T08:06:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f937286d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ac8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f937286d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ac8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43202/0 (socket says 172.21.15.64:43202) 2023-11-24T08:06:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f937286d640 1 -- 172.21.15.64:0/699316100 learned_addr learned my addr 172.21.15.64:0/699316100 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f937206c640 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 0x564172a997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.957+0000 7f937186b640 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172ac8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f937286d640 1 -- 172.21.15.64:0/699316100 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 msgr2=0x564172ac8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f937286d640 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172ac8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f937286d640 1 -- 172.21.15.64:0/699316100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 msgr2=0x564172a997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f937286d640 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 0x564172a997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f937286d640 1 -- 172.21.15.64:0/699316100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641729a5b80 con 0x564172aa3070 2023-11-24T08:06:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f937286d640 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ac8a90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f93640050a0 tx=0x7f9364057ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bc02713b9be60e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f9368dff640 1 -- 172.21.15.64:0/699316100 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936405fae0 con 0x564172aa3070 2023-11-24T08:06:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f9368dff640 1 -- 172.21.15.64:0/699316100 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364087200 con 0x564172aa3070 2023-11-24T08:06:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.958+0000 7f9368dff640 1 -- 172.21.15.64:0/699316100 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364003c40 con 0x564172aa3070 2023-11-24T08:06:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 -- 172.21.15.64:0/699316100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 msgr2=0x564172ac8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ac8a90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f93640050a0 tx=0x7f9364057ee0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 -- 172.21.15.64:0/699316100 shutdown_connections 2023-11-24T08:06:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ac8a90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172ac8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 --2- 172.21.15.64:0/699316100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 0x564172a997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 -- 172.21.15.64:0/699316100 >> 172.21.15.64:0/699316100 conn(0x564172a64c70 msgr2=0x564172ac7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:16.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 -- 172.21.15.64:0/699316100 shutdown_connections 2023-11-24T08:06:16.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.959+0000 7f9376e5f740 1 -- 172.21.15.64:0/699316100 wait complete. 2023-11-24T08:06:16.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.960+0000 7f9376e5f740 1 -- start start 2023-11-24T08:06:16.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.960+0000 7f9376e5f740 1 Processor -- start 2023-11-24T08:06:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 0x564172b02a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172b52b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f9376e5f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ae6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f9376e5f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564172ac7f60 con 0x564172aa3070 2023-11-24T08:06:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f9376e5f740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x564172af8220 con 0x564172a9dd70 2023-11-24T08:06:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f9376e5f740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x564172b53580 con 0x564172aa27b0 2023-11-24T08:06:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f9376e5f740 10 client.0 _ll_register_callbacks cb 0x7f935c009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:06:16.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f937286d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ae6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f937286d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ae6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43216/0 (socket says 172.21.15.64:43216) 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f937286d640 1 -- 172.21.15.64:0/1648058036 learned_addr learned my addr 172.21.15.64:0/1648058036 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f937286d640 1 -- 172.21.15.64:0/1648058036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 msgr2=0x564172b52b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f937286d640 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172b52b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f937286d640 1 -- 172.21.15.64:0/1648058036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 msgr2=0x564172b02a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f937286d640 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 0x564172b02a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.961+0000 7f937286d640 1 -- 172.21.15.64:0/1648058036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c0b3270 con 0x564172aa3070 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f937286d640 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ae6fd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9364058090 tx=0x7f9364015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb248a144b4fb88 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f93517fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f93517fa640 1 -- 172.21.15.64:0/1648058036 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364087370 con 0x564172aa3070 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f93517fa640 1 -- 172.21.15.64:0/1648058036 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f936405fae0 con 0x564172aa3070 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f9376e5f740 1 -- 172.21.15.64:0/1648058036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9360007660 con 0x564172aa3070 2023-11-24T08:06:16.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f93517fa640 1 -- 172.21.15.64:0/1648058036 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9364003e50 con 0x564172aa3070 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f9376e5f740 10 client.5907 fetch_fsmap learned FSMap version 24 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f9376e5f740 1 -- 172.21.15.64:0/1648058036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5641729cb170 con 0x564172aa3070 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.962+0000 7f93517fa640 1 -- 172.21.15.64:0/1648058036 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936407a080 con 0x564172aa3070 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f93517fa640 1 -- 172.21.15.64:0/1648058036 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f936405f650 con 0x564172aa3070 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f9376e5f740 10 client.5907 fetch_fsmap finished waiting for FSMap version 24 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f9376e5f740 10 client.5907 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f9376e5f740 20 client.5907 populate_metadata read hostname 'smithi064' 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f9376e5f740 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] conn(0x564172a9d140 0x564172a9d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f9376e5f740 4 client.5907 mds_command: new command op to 5724 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f9376e5f740 1 -- 172.21.15.64:0/1648058036 --> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56417270a6c0 con 0x564172a9d140 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f937206c640 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] conn(0x564172a9d140 0x564172a9d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f93517fa640 10 client.5907 ms_handle_connect on v2:172.21.15.64:6832/1271535459 2023-11-24T08:06:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.963+0000 7f937206c640 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] conn(0x564172a9d140 0x564172a9d520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2f2f67a8e64db4 server_cookie=3b0b8712f774845b in_seq=0 out_seq=0 2023-11-24T08:06:16.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f93517fa640 1 -- 172.21.15.64:0/1648058036 <== mds.0 v2:172.21.15.64:6832/1271535459 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56417270a6c0 con 0x564172a9d140 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f93517fa640 10 client.5907 handle_command_reply: tid=0 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 client.5907 shutdown 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 -- 172.21.15.64:0/1648058036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 msgr2=0x564172ae6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ae6fd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9364058090 tx=0x7f9364015070 comp rx=0 tx=0).stop 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 -- 172.21.15.64:0/1648058036 shutdown_connections 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.64:6832/1271535459,v1:172.21.15.64:6834/1271535459] conn(0x564172a9d140 0x564172a9d520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564172aa3070 0x564172ae6fd0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x564172aa27b0 0x564172b52b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 --2- 172.21.15.64:0/1648058036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x564172a9dd70 0x564172b02a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f93517fa640 0 client.5907 ms_handle_reset on v2:172.21.15.64:6832/1271535459 2023-11-24T08:06:19.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 -- 172.21.15.64:0/1648058036 >> 172.21.15.64:0/1648058036 conn(0x564172a64c70 msgr2=0x564172ac8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:19.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 -- 172.21.15.64:0/1648058036 shutdown_connections 2023-11-24T08:06:19.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 1 -- 172.21.15.64:0/1648058036 wait complete. 2023-11-24T08:06:19.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.964+0000 7f9376e5f740 20 client.5907 trim_cache size 0 max 16384 2023-11-24T08:06:19.885 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:06:19.886 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918", 2023-11-24T08:06:19.886 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-11-24T08:06:19.886 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5724, 2023-11-24T08:06:19.886 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-11-24T08:06:19.886 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-11-24T08:06:19.886 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-11-24T08:06:19.887 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 11.864628714, 2023-11-24T08:06:19.887 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2023-11-24T08:06:19.887 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 45, 2023-11-24T08:06:19.887 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 45, 2023-11-24T08:06:19.887 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 17.266269101999999 2023-11-24T08:06:19.887 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:06:19.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 -- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f935806af00 msgr2=0x7f935806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:19.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f935806af00 0x7f935806d3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f935c036000 tx=0x7f935c02e000 comp rx=0 tx=0).stop 2023-11-24T08:06:19.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 -- 172.21.15.64:0/4017543668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936c0b2300 msgr2=0x7f936c15a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:19.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936c0b2300 0x7f936c15a0a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f936c0cfd10 tx=0x7f936400d5e0 comp rx=0 tx=0).stop 2023-11-24T08:06:19.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 -- 172.21.15.64:0/4017543668 shutdown_connections 2023-11-24T08:06:19.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f935806af00 0x7f935806d3c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:19.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f936c0b2300 0x7f936c15a0a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:19.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f936c146d30 0x7f936c1f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:19.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 --2- 172.21.15.64:0/4017543668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9354003a30 0x7f936c151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:19.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 -- 172.21.15.64:0/4017543668 >> 172.21.15.64:0/4017543668 conn(0x7f936c0b0840 msgr2=0x7f936c154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:19.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 -- 172.21.15.64:0/4017543668 shutdown_connections 2023-11-24T08:06:19.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:16.965+0000 7f937306e640 1 -- 172.21.15.64:0/4017543668 wait complete. 2023-11-24T08:06:19.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:06:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.133+0000 7f4e52fba640 1 Processor -- start 2023-11-24T08:06:20.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.134+0000 7f4e52fba640 1 -- start start 2023-11-24T08:06:20.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.135+0000 7f4e52fba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4e4c0cf050 0x7f4e4c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:20.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.135+0000 7f4e52fba640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c140900 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.135+0000 7f4e52fba640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c15fa70 con 0x7f4e4c0cd7b0 2023-11-24T08:06:20.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.135+0000 7f4e52fba640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0cfa40 con 0x7f4e4c0cf050 2023-11-24T08:06:20.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.135+0000 7f4e51fb8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4e4c13dfd0 0x7f4e4c15be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52442/0 (socket says 172.21.15.64:52442) 2023-11-24T08:06:20.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.135+0000 7f4e51fb8640 1 -- 172.21.15.64:0/3513140179 learned_addr learned my addr 172.21.15.64:0/3513140179 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:20.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.135+0000 7f4e517b7640 1 --2- 172.21.15.64:0/3513140179 >> v2:172.21.15.64:3300/0 conn(0x7f4e4c0cf050 0x7f4e4c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:20.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.136+0000 7f4e50fb6640 1 -- 172.21.15.64:0/3513140179 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456483030 0 0) 0x7f4e4c140900 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.136+0000 7f4e50fb6640 1 -- 172.21.15.64:0/3513140179 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e38003520 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.136+0000 7f4e517b7640 1 -- 172.21.15.64:0/3513140179 >> v1:172.21.15.91:6789/0 conn(0x7f4e4c0cd7b0 legacy=0x7f4e4c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:20.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.136+0000 7f4e517b7640 1 -- 172.21.15.64:0/3513140179 >> v1:172.21.15.64:6789/0 conn(0x7f4e4c13dfd0 legacy=0x7f4e4c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:20.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.136+0000 7f4e517b7640 1 -- 172.21.15.64:0/3513140179 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c15ee60 con 0x7f4e4c0cf050 2023-11-24T08:06:20.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.137+0000 7f4e517b7640 1 --2- 172.21.15.64:0/3513140179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c0cf050 0x7f4e4c0cf500 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0097a0 tx=0x7f4e3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560270dd9c775759 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:20.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.137+0000 7f4e50fb6640 1 -- 172.21.15.64:0/3513140179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c031830 con 0x7f4e4c0cf050 2023-11-24T08:06:20.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.137+0000 7f4e50fb6640 1 -- 172.21.15.64:0/3513140179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c031990 con 0x7f4e4c0cf050 2023-11-24T08:06:20.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.137+0000 7f4e50fb6640 1 -- 172.21.15.64:0/3513140179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c031d20 con 0x7f4e4c0cf050 2023-11-24T08:06:20.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.138+0000 7f4e52fba640 1 -- 172.21.15.64:0/3513140179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c0cf050 msgr2=0x7f4e4c0cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:20.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.138+0000 7f4e52fba640 1 --2- 172.21.15.64:0/3513140179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c0cf050 0x7f4e4c0cf500 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0097a0 tx=0x7f4e3c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:06:20.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.138+0000 7f4e52fba640 1 -- 172.21.15.64:0/3513140179 shutdown_connections 2023-11-24T08:06:20.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.138+0000 7f4e52fba640 1 --2- 172.21.15.64:0/3513140179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c0cf050 0x7f4e4c0cf500 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:20.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.138+0000 7f4e52fba640 1 -- 172.21.15.64:0/3513140179 >> 172.21.15.64:0/3513140179 conn(0x7f4e4c06a860 msgr2=0x7f4e4c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:20.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.138+0000 7f4e52fba640 1 -- 172.21.15.64:0/3513140179 shutdown_connections 2023-11-24T08:06:20.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.138+0000 7f4e52fba640 1 -- 172.21.15.64:0/3513140179 wait complete. 2023-11-24T08:06:20.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.140+0000 7f4e52fba640 1 Processor -- start 2023-11-24T08:06:20.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.140+0000 7f4e52fba640 1 -- start start 2023-11-24T08:06:20.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e52fba640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e4c0cd7b0 0x7f4e4c156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:20.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e52fba640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4e4c0cf050 0x7f4e4c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e52fba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c13dfd0 0x7f4e4c12f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e52fba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c157f20 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e52fba640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c157310 con 0x7f4e4c0cd7b0 2023-11-24T08:06:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e52fba640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c1575f0 con 0x7f4e4c0cf050 2023-11-24T08:06:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e527b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c13dfd0 0x7f4e4c12f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e527b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c13dfd0 0x7f4e4c12f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47642/0 (socket says 172.21.15.64:47642) 2023-11-24T08:06:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.141+0000 7f4e527b9640 1 -- 172.21.15.64:0/1246344649 learned_addr learned my addr 172.21.15.64:0/1246344649 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e51fb8640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e4c0cd7b0 0x7f4e4c156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e517b7640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4e4c0cf050 0x7f4e4c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e527b9640 1 -- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4e4c0cf050 msgr2=0x7f4e4c0c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e527b9640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4e4c0cf050 0x7f4e4c0c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e527b9640 1 -- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e4c0cd7b0 msgr2=0x7f4e4c156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e527b9640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e4c0cd7b0 0x7f4e4c156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e527b9640 1 -- 172.21.15.64:0/1246344649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c009400 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e527b9640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c13dfd0 0x7f4e4c12f560 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c780 tx=0x7f4e4400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e427fc640 1 -- 172.21.15.64:0/1246344649 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44004b60 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e427fc640 1 -- 172.21.15.64:0/1246344649 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4400ce90 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e427fc640 1 -- 172.21.15.64:0/1246344649 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400a850 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.142+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c1578d0 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.143+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c1583b0 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.144+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c158090 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.145+0000 7f4e427fc640 1 -- 172.21.15.64:0/1246344649 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4e4400a9b0 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.146+0000 7f4e427fc640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4e3006aed0 0x7f4e3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.146+0000 7f4e427fc640 1 -- 172.21.15.64:0/1246344649 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e440898e0 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.147+0000 7f4e51fb8640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4e3006aed0 0x7f4e3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:20.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.148+0000 7f4e51fb8640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4e3006aed0 0x7f4e3006d390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4e480011e0 tx=0x7f4e48009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.151+0000 7f4e427fc640 1 -- 172.21.15.64:0/1246344649 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e440548a0 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.378+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4e4c04e840 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.379+0000 7f4e427fc640 1 -- 172.21.15.64:0/1246344649 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4e44057ec0 con 0x7f4e4c13dfd0 2023-11-24T08:06:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:06:20.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4e3006aed0 msgr2=0x7f4e3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:20.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4e3006aed0 0x7f4e3006d390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4e480011e0 tx=0x7f4e48009040 comp rx=0 tx=0).stop 2023-11-24T08:06:20.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c13dfd0 msgr2=0x7f4e4c12f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:20.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c13dfd0 0x7f4e4c12f560 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c780 tx=0x7f4e4400cc40 comp rx=0 tx=0).stop 2023-11-24T08:06:20.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 shutdown_connections 2023-11-24T08:06:20.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4e3006aed0 0x7f4e3006d390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:20.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e4c13dfd0 0x7f4e4c12f560 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:20.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4e4c0cf050 0x7f4e4c0c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 --2- 172.21.15.64:0/1246344649 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4e4c0cd7b0 0x7f4e4c156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 >> 172.21.15.64:0/1246344649 conn(0x7f4e4c06a860 msgr2=0x7f4e4c15a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 shutdown_connections 2023-11-24T08:06:20.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:20.382+0000 7f4e52fba640 1 -- 172.21.15.64:0/1246344649 wait complete. 2023-11-24T08:06:20.403 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:06:20.403 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:06:20.403 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T08:06:20.404 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:06:20.404 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:06:20.404 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:06:20.404 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:06:20.404 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:06:20.404 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:06:20.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:20.404 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T08:06:20.458 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:06:20.458 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:06:20.458 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:06:20.458 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:20.458 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:06:20.458 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:20.458 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:06:20.458 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42202sec preferred_lft 42202sec 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:20.459 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3c83:ffff:fee3:836f/64 scope link 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::54d4:76ff:fe60:984a/64 scope link 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether ae:e6:20:dc:18:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ace6:20ff:fedc:18d9/64 scope link 2023-11-24T08:06:20.460 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:20.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:20.462 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:06:20.551 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T08:06:20.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:20.556 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T08:06:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T08:06:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:06:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:06:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:06:20.617 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:06:20.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:20.618 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:06:20.618 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:06:20.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:06:20.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:06:20.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:06:20.618 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:06:20.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:20.823 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:06:20.882 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:06:20.882 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:06:20.883 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:06:20.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:20.883 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-11-24T08:06:20.986 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T08:06:20.986 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:06:20.986 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:06:21.042 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:06:21.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:06:21.043 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:06:21.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:06:21.196 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:06:21.254 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:06:21.254 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:06:21.255 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:06:21.256 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:06:21.257 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:06:21.257 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:06:21.257 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:06:21.257 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:06:21.257 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:06:21.257 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:06:21.257 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:06:21.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:21.258 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:06:21.312 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:06:21.312 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T08:06:21.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:06:21.395 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:06:21.440 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:06:21.440 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:06:21.440 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:06:21.440 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:06:21.440 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:06:21.440 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:06:21.441 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:06:21.442 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:06:21.443 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:06:21.443 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:06:21.443 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:06:21.443 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:06:21.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:21.443 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:06:21.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-11-24T08:06:21.506+0000 7fc6e343a480 -1 init, newargv = 0x55f0be3b4c90 newargc=15 2023-11-24T08:06:21.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[105745]: starting ceph client 2023-11-24T08:06:21.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[105745]: starting fuse 2023-11-24T08:06:21.531 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T08:06:21.531 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T08:06:21.531 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:06:21.531 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T08:06:21.531 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T08:06:21.531 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T08:06:21.532 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T08:06:21.532 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:06:21.532 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T08:06:21.532 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T08:06:21.532 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T08:06:21.533 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T08:06:21.533 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:06:21.533 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T08:06:21.533 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T08:06:21.533 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T08:06:21.533 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T08:06:21.533 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-11-24T08:06:21.534 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T08:06:21.744 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.105745.asok 2023-11-24T08:06:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:06:21.762 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.105745.asok 2023-11-24T08:06:21.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:21.762 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.105745.asok status 2023-11-24T08:06:22.013 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:06:22.013 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T08:06:22.014 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:06:22.014 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:06:22.014 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-11-24T08:06:22.014 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T08:06:22.014 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T08:06:22.014 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "105745", 2023-11-24T08:06:22.014 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T08:06:22.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:22.015 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T08:06:22.015 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T08:06:22.015 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5925, 2023-11-24T08:06:22.015 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T08:06:22.015 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T08:06:22.015 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T08:06:22.016 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5925 2023-11-24T08:06:22.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:22.016 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:06:22.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:06:22.016 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:06:22.016 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3762053457 2023-11-24T08:06:22.017 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:22.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:22.017 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:06:22.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:06:22.017 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:06:22.017 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3762053457 2023-11-24T08:06:22.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:22.018 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5925 192.168.0.1:0/3762053457", 2023-11-24T08:06:22.018 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3762053457", 2023-11-24T08:06:22.018 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T08:06:22.018 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 24, 2023-11-24T08:06:22.018 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 45, 2023-11-24T08:06:22.019 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:06:22.019 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T08:06:22.019 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T08:06:22.019 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:06:22.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:22.029 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T08:06:22.029 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T08:06:22.089 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T08:06:22.090 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T08:06:22.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:22.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:06:22.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:06:22.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.425+0000 7f52a288c640 1 Processor -- start 2023-11-24T08:06:22.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.425+0000 7f52a288c640 1 -- start start 2023-11-24T08:06:22.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.426+0000 7f52a288c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f529c157770 0x7f529c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.426+0000 7f52a288c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f529c15df80 0x7f529c070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.426+0000 7f52a288c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c161670 con 0x7f529c06d080 2023-11-24T08:06:22.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.426+0000 7f52a288c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c15e410 con 0x7f529c157770 2023-11-24T08:06:22.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.426+0000 7f52a288c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c06d500 con 0x7f529c15df80 2023-11-24T08:06:22.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.427+0000 7f52a1089640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f529c06d080 0x7f529c15d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52448/0 (socket says 172.21.15.64:52448) 2023-11-24T08:06:22.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.427+0000 7f52a1089640 1 -- 172.21.15.64:0/408395128 learned_addr learned my addr 172.21.15.64:0/408395128 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:22.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.427+0000 7f52a188a640 1 --2- 172.21.15.64:0/408395128 >> v2:172.21.15.91:3300/0 conn(0x7f529c157770 0x7f529c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.427+0000 7f52a208b640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f529c15df80 0x7f529c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.427+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773231816 0 0) 0x7f529c161670 con 0x7f529c06d080 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.427+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5288003520 con 0x7f529c06d080 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 196870202 0 0) 0x7f5288003520 con 0x7f529c06d080 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f529c15df80 msgr2=0x7f529c070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f529c15df80 0x7f529c070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 >> v2:172.21.15.91:3300/0 conn(0x7f529c157770 msgr2=0x7f529c06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 --2- 172.21.15.64:0/408395128 >> v2:172.21.15.91:3300/0 conn(0x7f529c157770 0x7f529c06cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c15e610 con 0x7f529c06d080 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f5290001670 con 0x7f529c06d080 2023-11-24T08:06:22.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 >> v1:172.21.15.64:6789/0 conn(0x7f529c06d080 legacy=0x7f529c15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:22.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880039e0 0x7f5288003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f529c15df80 0x7f52880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5288004d00 0x7f5288005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c15e410 con 0x7f52880039e0 2023-11-24T08:06:22.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c06d500 con 0x7f5288004d00 2023-11-24T08:06:22.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528800b9b0 con 0x7f529c15df80 2023-11-24T08:06:22.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.428+0000 7f52a188a640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880039e0 0x7f5288003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:22.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.429+0000 7f52a1089640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f529c15df80 0x7f52880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:22.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.429+0000 7f52a188a640 1 -- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f529c15df80 msgr2=0x7f52880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:22.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.429+0000 7f52a188a640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f529c15df80 0x7f52880047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.429+0000 7f52a188a640 1 -- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5288004d00 msgr2=0x7f5288005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:22.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.429+0000 7f52a188a640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5288004d00 0x7f5288005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.429+0000 7f52a188a640 1 -- 172.21.15.64:0/408395128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288003520 con 0x7f52880039e0 2023-11-24T08:06:22.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.430+0000 7f52a188a640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880039e0 0x7f5288003e00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5298009ac0 tx=0x7f52880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef7929c8399db86 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:22.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.430+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529802c710 con 0x7f52880039e0 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.430+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529802c870 con 0x7f52880039e0 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.430+0000 7f52a0888640 1 -- 172.21.15.64:0/408395128 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529802cc00 con 0x7f52880039e0 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.431+0000 7f52a288c640 1 -- 172.21.15.64:0/408395128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880039e0 msgr2=0x7f5288003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.431+0000 7f52a288c640 1 --2- 172.21.15.64:0/408395128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52880039e0 0x7f5288003e00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5298009ac0 tx=0x7f52880057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.431+0000 7f52a208b640 1 -- 172.21.15.64:0/408395128 reap_dead start 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.431+0000 7f52a288c640 1 -- 172.21.15.64:0/408395128 shutdown_connections 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.431+0000 7f52a288c640 1 -- 172.21.15.64:0/408395128 >> 172.21.15.64:0/408395128 conn(0x7f529c06a860 msgr2=0x7f529c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.431+0000 7f52a288c640 1 -- 172.21.15.64:0/408395128 shutdown_connections 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.431+0000 7f52a288c640 1 -- 172.21.15.64:0/408395128 wait complete. 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.433+0000 7f52a288c640 1 Processor -- start 2023-11-24T08:06:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.433+0000 7f52a288c640 1 -- start start 2023-11-24T08:06:22.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a288c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f52880039e0 0x7f529c14c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a288c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f529c145a50 0x7f529c1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a288c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f529c15f560 0x7f529c155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a288c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0cbf50 con 0x7f529c15f560 2023-11-24T08:06:22.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a288c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1f3340 con 0x7f529c145a50 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a288c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1f3620 con 0x7f52880039e0 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a208b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f529c15f560 0x7f529c155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a208b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f529c15f560 0x7f529c155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47664/0 (socket says 172.21.15.64:47664) 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a208b640 1 -- 172.21.15.64:0/973482364 learned_addr learned my addr 172.21.15.64:0/973482364 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.434+0000 7f52a1089640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f529c145a50 0x7f529c1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a188a640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f52880039e0 0x7f529c14c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a208b640 1 -- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f52880039e0 msgr2=0x7f529c14c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a208b640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f52880039e0 0x7f529c14c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a208b640 1 -- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f529c145a50 msgr2=0x7f529c1f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a208b640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f529c145a50 0x7f529c1f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a208b640 1 -- 172.21.15.64:0/973482364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c1f3900 con 0x7f529c15f560 2023-11-24T08:06:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a1089640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f529c145a50 0x7f529c1f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a208b640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f529c15f560 0x7f529c155210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f529c15e360 tx=0x7f529400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f528e7fc640 1 -- 172.21.15.64:0/973482364 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294010040 con 0x7f529c15f560 2023-11-24T08:06:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f528e7fc640 1 -- 172.21.15.64:0/973482364 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529400bce0 con 0x7f529c15f560 2023-11-24T08:06:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f528e7fc640 1 -- 172.21.15.64:0/973482364 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294014640 con 0x7f529c15f560 2023-11-24T08:06:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c15fa10 con 0x7f529c15f560 2023-11-24T08:06:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.435+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c155750 con 0x7f529c15f560 2023-11-24T08:06:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.436+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c0cbdb0 con 0x7f529c15f560 2023-11-24T08:06:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.438+0000 7f528e7fc640 1 -- 172.21.15.64:0/973482364 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f52940147a0 con 0x7f529c15f560 2023-11-24T08:06:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.438+0000 7f528e7fc640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f528006aed0 0x7f528006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.439+0000 7f528e7fc640 1 -- 172.21.15.64:0/973482364 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5294089fc0 con 0x7f529c15f560 2023-11-24T08:06:22.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.439+0000 7f52a188a640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f528006aed0 0x7f528006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:22.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.440+0000 7f52a188a640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f528006aed0 0x7f528006d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5298036000 tx=0x7f5298030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:22.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.443+0000 7f528e7fc640 1 -- 172.21.15.64:0/973482364 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f529c0cbdb0 con 0x7f529c15f560 2023-11-24T08:06:22.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.677+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f529c0cbdb0 con 0x7f529c15f560 2023-11-24T08:06:22.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.678+0000 7f528e7fc640 1 -- 172.21.15.64:0/973482364 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f529c0cbdb0 con 0x7f529c15f560 2023-11-24T08:06:22.680 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:06:22.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f528006aed0 msgr2=0x7f528006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:22.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f528006aed0 0x7f528006d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5298036000 tx=0x7f5298030000 comp rx=0 tx=0).stop 2023-11-24T08:06:22.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f529c15f560 msgr2=0x7f529c155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f529c15f560 0x7f529c155210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f529c15e360 tx=0x7f529400b480 comp rx=0 tx=0).stop 2023-11-24T08:06:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 shutdown_connections 2023-11-24T08:06:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f528006aed0 0x7f528006d390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f529c15f560 0x7f529c155210 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f529c145a50 0x7f529c1f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 --2- 172.21.15.64:0/973482364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f52880039e0 0x7f529c14c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.681+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 >> 172.21.15.64:0/973482364 conn(0x7f529c06a860 msgr2=0x7f529c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.682+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 shutdown_connections 2023-11-24T08:06:22.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:22.682+0000 7f52a288c640 1 -- 172.21.15.64:0/973482364 wait complete. 2023-11-24T08:06:22.704 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:06:22.704 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:06:22.704 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2023-11-24T08:06:22.704 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:06:22.704 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:06:22.704 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:06:22.704 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:06:22.705 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:06:22.705 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:06:22.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:22.705 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2023-11-24T08:06:22.764 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:22.765 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42190sec preferred_lft 42190sec 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::5073:3fff:fef1:71bb/64 scope link 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:06:22.766 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:06:22.767 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::2497:37ff:fe0c:852a/64 scope link 2023-11-24T08:06:22.767 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:22.767 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:06:22.767 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 32:d9:7a:07:6c:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:06:22.767 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::30d9:7aff:fe07:6ce0/64 scope link 2023-11-24T08:06:22.767 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:06:22.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:22.768 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:06:22.858 INFO:teuthology.orchestra.run.smithi091.stdout:1 2023-11-24T08:06:22.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:22.862 DEBUG:teuthology.orchestra.run.smithi091:> route 2023-11-24T08:06:22.912 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2023-11-24T08:06:22.912 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:06:22.912 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:06:22.912 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:06:22.912 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:06:22.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:22.913 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:06:22.913 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:06:22.913 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:06:22.913 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:06:22.914 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:06:22.914 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:06:23.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:23.098 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:06:23.149 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:06:23.149 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:06:23.149 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:06:23.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:23.149 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2023-11-24T08:06:23.241 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T08:06:23.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:06:23.242 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:06:23.298 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:06:23.298 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:06:23.299 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:06:23.349 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:06:23.445 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:06:23.503 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:06:23.503 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:06:23.503 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:06:23.503 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:06:23.503 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:06:23.504 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:06:23.504 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:06:23.504 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:06:23.504 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:06:23.504 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:06:23.504 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:06:23.504 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:06:23.505 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:06:23.506 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:06:23.506 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:06:23.506 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:06:23.506 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:06:23.507 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:06:23.507 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:06:23.507 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:06:23.507 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:06:23.507 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:06:23.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:23.508 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:06:23.562 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:06:23.562 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T08:06:23.568 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:06:23.650 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:06:23.692 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:06:23.693 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:06:23.693 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:06:23.693 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:06:23.693 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:06:23.693 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:06:23.694 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:06:23.694 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:06:23.694 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:06:23.694 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:06:23.694 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:06:23.695 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:06:23.695 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:06:23.695 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:06:23.695 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:06:23.696 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:06:23.696 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:06:23.696 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:06:23.696 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:06:23.696 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:06:23.697 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:06:23.697 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:06:23.697 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:06:23.697 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:06:23.697 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:06:23.697 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:06:23.698 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:06:23.698 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:06:23.698 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:06:23.698 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:06:23.699 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:06:23.699 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:06:23.699 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:06:23.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:23.701 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:06:23.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:2023-11-24T08:06:23.752+0000 7ff0dea0d480 -1 init, newargv = 0x5651c8327880 newargc=15 2023-11-24T08:06:23.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[96513]: starting ceph client 2023-11-24T08:06:23.765 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[96513]: starting fuse 2023-11-24T08:06:23.784 INFO:teuthology.orchestra.run.smithi091.stdout:41 2023-11-24T08:06:23.784 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T08:06:23.784 DEBUG:teuthology.orchestra.run.smithi091:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:06:23.784 DEBUG:teuthology.orchestra.run.smithi091:> import glob 2023-11-24T08:06:23.784 DEBUG:teuthology.orchestra.run.smithi091:> import re 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> import os 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> def _find_admin_socket(client_name): 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> files = glob.glob(asok_path) 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:06:23.785 DEBUG:teuthology.orchestra.run.smithi091:> # Given a non-glob path, it better be there 2023-11-24T08:06:23.786 DEBUG:teuthology.orchestra.run.smithi091:> if "*" not in asok_path: 2023-11-24T08:06:23.786 DEBUG:teuthology.orchestra.run.smithi091:> assert(len(files) == 1) 2023-11-24T08:06:23.786 DEBUG:teuthology.orchestra.run.smithi091:> return files[0] 2023-11-24T08:06:23.786 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:06:23.786 DEBUG:teuthology.orchestra.run.smithi091:> for f in files: 2023-11-24T08:06:23.787 DEBUG:teuthology.orchestra.run.smithi091:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:06:23.787 DEBUG:teuthology.orchestra.run.smithi091:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:06:23.787 DEBUG:teuthology.orchestra.run.smithi091:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:06:23.787 DEBUG:teuthology.orchestra.run.smithi091:> contents = proc_f.read() 2023-11-24T08:06:23.788 DEBUG:teuthology.orchestra.run.smithi091:> if mountpoint in contents: 2023-11-24T08:06:23.788 DEBUG:teuthology.orchestra.run.smithi091:> return f 2023-11-24T08:06:23.788 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:06:23.788 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:06:23.788 DEBUG:teuthology.orchestra.run.smithi091:> print(_find_admin_socket("client.1")) 2023-11-24T08:06:23.789 DEBUG:teuthology.orchestra.run.smithi091:> ' 2023-11-24T08:06:23.980 INFO:teuthology.orchestra.run.smithi091.stdout:/var/run/ceph/ceph-client.1.96513.asok 2023-11-24T08:06:23.984 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:06:23.995 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.96513.asok 2023-11-24T08:06:23.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:23.996 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.96513.asok status 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": { 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout: "entity_id": "1", 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout: "hostname": "smithi091", 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout: "pid": "96513", 2023-11-24T08:06:24.234 INFO:teuthology.orchestra.run.smithi091.stdout: "root": "/" 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_count": 0, 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_pinned_count": 0, 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 5272, 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "inst": { 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "name": { 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "client", 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "num": 5272 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:06:24.235 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 1674623743 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: } 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 1674623743 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "inst_str": "client.5272 192.168.0.1:0/1674623743", 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "addr_str": "192.168.0.1:0/1674623743", 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "inode_count": 1, 2023-11-24T08:06:24.236 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_epoch": 24, 2023-11-24T08:06:24.237 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch": 45, 2023-11-24T08:06:24.237 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:06:24.237 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklisted": false, 2023-11-24T08:06:24.237 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs" 2023-11-24T08:06:24.237 INFO:teuthology.orchestra.run.smithi091.stdout:} 2023-11-24T08:06:24.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:24.248 DEBUG:teuthology.orchestra.run.smithi091:> stat --file-system '--printf=%T 2023-11-24T08:06:24.249 DEBUG:teuthology.orchestra.run.smithi091:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T08:06:24.302 INFO:teuthology.orchestra.run.smithi091.stdout:fuseblk 2023-11-24T08:06:24.303 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T08:06:24.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:24.303 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:06:24.396 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:06:24.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.629+0000 7f69245f5640 1 Processor -- start 2023-11-24T08:06:24.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.629+0000 7f69245f5640 1 -- start start 2023-11-24T08:06:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f69245f5640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f691c09cb30 0x7f691c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f69245f5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f691c0adb60 0x7f691c0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f69245f5640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c15b730 con 0x7f691c15b350 2023-11-24T08:06:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f69245f5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691c0ac740 con 0x7f691c0adb60 2023-11-24T08:06:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f69245f5640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691c0ac460 con 0x7f691c09cb30 2023-11-24T08:06:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f6922b92640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f691c0adb60 0x7f691c0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f6922b92640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f691c0adb60 0x7f691c0adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47674/0 (socket says 172.21.15.64:47674) 2023-11-24T08:06:24.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f6922b92640 1 -- 172.21.15.64:0/3260202412 learned_addr learned my addr 172.21.15.64:0/3260202412 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:24.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.631+0000 7f6921b90640 1 --2- 172.21.15.64:0/3260202412 >> v2:172.21.15.91:3300/0 conn(0x7f691c09cb30 0x7f691c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:24.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.632+0000 7f6922b92640 1 -- 172.21.15.64:0/3260202412 >> v2:172.21.15.91:3300/0 conn(0x7f691c09cb30 msgr2=0x7f691c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:24.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.632+0000 7f6922b92640 1 --2- 172.21.15.64:0/3260202412 >> v2:172.21.15.91:3300/0 conn(0x7f691c09cb30 0x7f691c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:24.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.632+0000 7f6922b92640 1 -- 172.21.15.64:0/3260202412 >> v1:172.21.15.91:6789/0 conn(0x7f691c15b350 legacy=0x7f691c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:24.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.632+0000 7f6922b92640 1 -- 172.21.15.64:0/3260202412 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c0b2a50 con 0x7f691c0adb60 2023-11-24T08:06:24.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.633+0000 7f6922b92640 1 --2- 172.21.15.64:0/3260202412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c0adb60 0x7f691c0adf70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f691c0d0060 tx=0x7f691402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9604a389bd40ef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:24.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.633+0000 7f692138f640 1 -- 172.21.15.64:0/3260202412 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691402ca80 con 0x7f691c0adb60 2023-11-24T08:06:24.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.633+0000 7f692138f640 1 -- 172.21.15.64:0/3260202412 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691402cbe0 con 0x7f691c0adb60 2023-11-24T08:06:24.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.633+0000 7f692138f640 1 -- 172.21.15.64:0/3260202412 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69140343f0 con 0x7f691c0adb60 2023-11-24T08:06:24.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.634+0000 7f69245f5640 1 -- 172.21.15.64:0/3260202412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c0adb60 msgr2=0x7f691c0adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:24.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.634+0000 7f69245f5640 1 --2- 172.21.15.64:0/3260202412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c0adb60 0x7f691c0adf70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f691c0d0060 tx=0x7f691402bee0 comp rx=0 tx=0).stop 2023-11-24T08:06:24.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.634+0000 7f69245f5640 1 -- 172.21.15.64:0/3260202412 shutdown_connections 2023-11-24T08:06:24.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.634+0000 7f69245f5640 1 --2- 172.21.15.64:0/3260202412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c0adb60 0x7f691c0adf70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:24.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.634+0000 7f69245f5640 1 --2- 172.21.15.64:0/3260202412 >> v2:172.21.15.91:3300/0 conn(0x7f691c09cb30 0x7f691c0abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:24.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.634+0000 7f69245f5640 1 -- 172.21.15.64:0/3260202412 >> 172.21.15.64:0/3260202412 conn(0x7f691c0b0ba0 msgr2=0x7f691c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:24.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.634+0000 7f69245f5640 1 -- 172.21.15.64:0/3260202412 shutdown_connections 2023-11-24T08:06:24.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.634+0000 7f69245f5640 1 -- 172.21.15.64:0/3260202412 wait complete. 2023-11-24T08:06:24.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.636+0000 7f69245f5640 1 Processor -- start 2023-11-24T08:06:24.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.636+0000 7f69245f5640 1 -- start start 2023-11-24T08:06:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.637+0000 7f69245f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c09cb30 0x7f691c070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.637+0000 7f69245f5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f691c0adb60 0x7f691c148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:24.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.637+0000 7f69245f5640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c15b350 0x7f691c155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.637+0000 7f69245f5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0724c0 con 0x7f691c09cb30 2023-11-24T08:06:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.637+0000 7f69245f5640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c071f00 con 0x7f691c15b350 2023-11-24T08:06:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.637+0000 7f69245f5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0721e0 con 0x7f691c0adb60 2023-11-24T08:06:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c09cb30 0x7f691c070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c09cb30 0x7f691c070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47678/0 (socket says 172.21.15.64:47678) 2023-11-24T08:06:24.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 -- 172.21.15.64:0/3449298190 learned_addr learned my addr 172.21.15.64:0/3449298190 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 -- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f691c0adb60 msgr2=0x7f691c148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f691c0adb60 0x7f691c148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 -- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c15b350 msgr2=0x7f691c155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c15b350 0x7f691c155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:24.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 -- 172.21.15.64:0/3449298190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6914009b30 con 0x7f691c09cb30 2023-11-24T08:06:24.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.638+0000 7f6922391640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c09cb30 0x7f691c070bd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f691800c430 tx=0x7f691800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:24.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.639+0000 7f6912ffd640 1 -- 172.21.15.64:0/3449298190 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918011790 con 0x7f691c09cb30 2023-11-24T08:06:24.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.639+0000 7f6912ffd640 1 -- 172.21.15.64:0/3449298190 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69180118f0 con 0x7f691c09cb30 2023-11-24T08:06:24.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.639+0000 7f6912ffd640 1 -- 172.21.15.64:0/3449298190 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918011ce0 con 0x7f691c09cb30 2023-11-24T08:06:24.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.639+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c154950 con 0x7f691c09cb30 2023-11-24T08:06:24.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.639+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c155690 con 0x7f691c09cb30 2023-11-24T08:06:24.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.640+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691c154e70 con 0x7f691c09cb30 2023-11-24T08:06:24.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.641+0000 7f6912ffd640 1 -- 172.21.15.64:0/3449298190 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f691801c460 con 0x7f691c09cb30 2023-11-24T08:06:24.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.642+0000 7f6912ffd640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f690006aed0 0x7f690006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:24.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.642+0000 7f6912ffd640 1 -- 172.21.15.64:0/3449298190 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6918089570 con 0x7f691c09cb30 2023-11-24T08:06:24.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.643+0000 7f6921b90640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f690006aed0 0x7f690006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:24.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.644+0000 7f6921b90640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f690006aed0 0x7f690006d390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f690c006fd0 tx=0x7f690c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:24.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.647+0000 7f6912ffd640 1 -- 172.21.15.64:0/3449298190 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f691c154e70 con 0x7f691c09cb30 2023-11-24T08:06:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:24.879+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f691c154e70 con 0x7f691c09cb30 2023-11-24T08:06:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.151+0000 7f6912ffd640 1 -- 172.21.15.64:0/3449298190 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 136+0+0 (secure 0 0 0) 0x7f691c154e70 con 0x7f691c09cb30 2023-11-24T08:06:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.154+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f690006aed0 msgr2=0x7f690006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.154+0000 7f69245f5640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f690006aed0 0x7f690006d390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f690c006fd0 tx=0x7f690c008040 comp rx=0 tx=0).stop 2023-11-24T08:06:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.154+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c09cb30 msgr2=0x7f691c070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.154+0000 7f69245f5640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c09cb30 0x7f691c070bd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f691800c430 tx=0x7f691800c8f0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.155+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 shutdown_connections 2023-11-24T08:06:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.155+0000 7f69245f5640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f690006aed0 0x7f690006d390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.155+0000 7f69245f5640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691c15b350 0x7f691c155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.155+0000 7f69245f5640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f691c0adb60 0x7f691c148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.155+0000 7f69245f5640 1 --2- 172.21.15.64:0/3449298190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691c09cb30 0x7f691c070bd0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.155+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 >> 172.21.15.64:0/3449298190 conn(0x7f691c0b0ba0 msgr2=0x7f691c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.155+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 shutdown_connections 2023-11-24T08:06:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.155+0000 7f69245f5640 1 -- 172.21.15.64:0/3449298190 wait complete. 2023-11-24T08:06:25.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:06:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.402+0000 7f6babfff640 1 Processor -- start 2023-11-24T08:06:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.403+0000 7f6babfff640 1 -- start start 2023-11-24T08:06:25.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.404+0000 7f6babfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6bac160bf0 0x7f6bac0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.404+0000 7f6babfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6bac0d0060 0x7f6bac0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6babfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0b2350 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6babfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac0d1e70 con 0x7f6bac160bf0 2023-11-24T08:06:25.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6babfff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac160fd0 con 0x7f6bac0d0060 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6baaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6bac160bf0 0x7f6bac0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6baaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6bac160bf0 0x7f6bac0cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47694/0 (socket says 172.21.15.64:47694) 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6baaffd640 1 -- 172.21.15.64:0/1556178382 learned_addr learned my addr 172.21.15.64:0/1556178382 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6baa7fc640 1 --2- 172.21.15.64:0/1556178382 >> v2:172.21.15.91:3300/0 conn(0x7f6bac0d0060 0x7f6bac0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6baaffd640 1 -- 172.21.15.64:0/1556178382 >> v2:172.21.15.91:3300/0 conn(0x7f6bac0d0060 msgr2=0x7f6bac0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6baaffd640 1 --2- 172.21.15.64:0/1556178382 >> v2:172.21.15.91:3300/0 conn(0x7f6bac0d0060 0x7f6bac0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6baaffd640 1 -- 172.21.15.64:0/1556178382 >> v1:172.21.15.91:6789/0 conn(0x7f6bac0b2fd0 legacy=0x7f6bac0b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.405+0000 7f6baaffd640 1 -- 172.21.15.64:0/1556178382 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0d2290 con 0x7f6bac160bf0 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.407+0000 7f6baaffd640 1 --2- 172.21.15.64:0/1556178382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac160bf0 0x7f6bac0cfb20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6bac067dd0 tx=0x7f6b9c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c8af1e759f9e9f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:25.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.407+0000 7f6ba9ffb640 1 -- 172.21.15.64:0/1556178382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c031870 con 0x7f6bac160bf0 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.407+0000 7f6ba9ffb640 1 -- 172.21.15.64:0/1556178382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0319d0 con 0x7f6bac160bf0 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.407+0000 7f6ba9ffb640 1 -- 172.21.15.64:0/1556178382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c031d60 con 0x7f6bac160bf0 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.408+0000 7f6babfff640 1 -- 172.21.15.64:0/1556178382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac160bf0 msgr2=0x7f6bac0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.408+0000 7f6babfff640 1 --2- 172.21.15.64:0/1556178382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac160bf0 0x7f6bac0cfb20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6bac067dd0 tx=0x7f6b9c02b910 comp rx=0 tx=0).stop 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.408+0000 7f6babfff640 1 -- 172.21.15.64:0/1556178382 shutdown_connections 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.408+0000 7f6babfff640 1 --2- 172.21.15.64:0/1556178382 >> v2:172.21.15.91:3300/0 conn(0x7f6bac0d0060 0x7f6bac0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.408+0000 7f6babfff640 1 --2- 172.21.15.64:0/1556178382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac160bf0 0x7f6bac0cfb20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.408+0000 7f6babfff640 1 -- 172.21.15.64:0/1556178382 >> 172.21.15.64:0/1556178382 conn(0x7f6bac09c460 msgr2=0x7f6bac0d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.408+0000 7f6babfff640 1 -- 172.21.15.64:0/1556178382 shutdown_connections 2023-11-24T08:06:25.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.408+0000 7f6babfff640 1 -- 172.21.15.64:0/1556178382 wait complete. 2023-11-24T08:06:25.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.410+0000 7f6babfff640 1 Processor -- start 2023-11-24T08:06:25.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.410+0000 7f6babfff640 1 -- start start 2023-11-24T08:06:25.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6babfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac0b2fd0 0x7f6bac1f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6babfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6bac0d0060 0x7f6bac0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6babfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6bac160bf0 0x7f6bac1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6babfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0b72a0 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6babfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1f6f10 con 0x7f6bac160bf0 2023-11-24T08:06:25.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6babfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1f71f0 con 0x7f6bac0d0060 2023-11-24T08:06:25.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6baaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac0b2fd0 0x7f6bac1f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6baaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac0b2fd0 0x7f6bac1f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47704/0 (socket says 172.21.15.64:47704) 2023-11-24T08:06:25.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6baaffd640 1 -- 172.21.15.64:0/3033099359 learned_addr learned my addr 172.21.15.64:0/3033099359 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:25.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6baaffd640 1 -- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6bac0d0060 msgr2=0x7f6bac0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6baaffd640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6bac0d0060 0x7f6bac0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6baaffd640 1 -- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6bac160bf0 msgr2=0x7f6bac1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.411+0000 7f6bab7fe640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6bac160bf0 0x7f6bac1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6baaffd640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6bac160bf0 0x7f6bac1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6baaffd640 1 -- 172.21.15.64:0/3033099359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0094d0 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6bab7fe640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6bac160bf0 0x7f6bac1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6baaffd640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac0b2fd0 0x7f6bac1f6650 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c000c00 tx=0x7f6b9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6ba37fe640 1 -- 172.21.15.64:0/3033099359 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c031840 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6ba37fe640 1 -- 172.21.15.64:0/3033099359 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0443f0 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6ba37fe640 1 -- 172.21.15.64:0/3033099359 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0447a0 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac0b7510 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.412+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac1f6b90 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.413+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac067960 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.414+0000 7f6ba37fe640 1 -- 172.21.15.64:0/3033099359 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6b9c04d450 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.415+0000 7f6ba37fe640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b8c06aed0 0x7f6b8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.415+0000 7f6ba37fe640 1 -- 172.21.15.64:0/3033099359 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b9c02d040 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.416+0000 7f6baa7fc640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b8c06aed0 0x7f6b8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.417+0000 7f6baa7fc640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b8c06aed0 0x7f6b8c06d390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0cd6e0 tx=0x7f6b94007390 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:25.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.420+0000 7f6ba37fe640 1 -- 172.21.15.64:0/3033099359 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b9c076ce0 con 0x7f6bac0b2fd0 2023-11-24T08:06:25.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.637+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bac0d14d0 con 0x7f6b8c06aed0 2023-11-24T08:06:25.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.642+0000 7f6ba37fe640 1 -- 172.21.15.64:0/3033099359 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6b94007a50 con 0x7f6b8c06aed0 2023-11-24T08:06:25.643 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:06:25.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:25.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-11-24T08:06:25.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:25.644 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:06:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b8c06aed0 msgr2=0x7f6b8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b8c06aed0 0x7f6b8c06d390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0cd6e0 tx=0x7f6b94007390 comp rx=0 tx=0).stop 2023-11-24T08:06:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac0b2fd0 msgr2=0x7f6bac1f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac0b2fd0 0x7f6bac1f6650 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c000c00 tx=0x7f6b9c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 shutdown_connections 2023-11-24T08:06:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6b8c06aed0 0x7f6b8c06d390 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6bac160bf0 0x7f6bac1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6bac0d0060 0x7f6bac0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 --2- 172.21.15.64:0/3033099359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bac0b2fd0 0x7f6bac1f6650 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 >> 172.21.15.64:0/3033099359 conn(0x7f6bac09c460 msgr2=0x7f6bac0b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.643+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 shutdown_connections 2023-11-24T08:06:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.644+0000 7f6babfff640 1 -- 172.21.15.64:0/3033099359 wait complete. 2023-11-24T08:06:25.665 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:06:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.891+0000 7f131d0ff640 1 Processor -- start 2023-11-24T08:06:25.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.891+0000 7f131d0ff640 1 -- start start 2023-11-24T08:06:25.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.892+0000 7f131d0ff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13181448c0 0x7f131806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.892+0000 7f131d0ff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f131806d910 0x7f131815b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f131d0ff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131806dd50 con 0x7f131815bbd0 2023-11-24T08:06:25.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f131d0ff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1318071080 con 0x7f13181448c0 2023-11-24T08:06:25.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f131d0ff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1318144ca0 con 0x7f131806d910 2023-11-24T08:06:25.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f1317fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13181448c0 0x7f131806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f1317fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13181448c0 0x7f131806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47712/0 (socket says 172.21.15.64:47712) 2023-11-24T08:06:25.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f1317fff640 1 -- 172.21.15.64:0/2025152448 learned_addr learned my addr 172.21.15.64:0/2025152448 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:25.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f130f7fe640 1 --2- 172.21.15.64:0/2025152448 >> v2:172.21.15.91:3300/0 conn(0x7f131806d910 0x7f131815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f1317fff640 1 -- 172.21.15.64:0/2025152448 >> v2:172.21.15.91:3300/0 conn(0x7f131806d910 msgr2=0x7f131815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f1317fff640 1 --2- 172.21.15.64:0/2025152448 >> v2:172.21.15.91:3300/0 conn(0x7f131806d910 0x7f131815b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f1317fff640 1 -- 172.21.15.64:0/2025152448 >> v1:172.21.15.91:6789/0 conn(0x7f131815bbd0 legacy=0x7f131815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.893+0000 7f1317fff640 1 -- 172.21.15.64:0/2025152448 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131815c450 con 0x7f13181448c0 2023-11-24T08:06:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.895+0000 7f1317fff640 1 --2- 172.21.15.64:0/2025152448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13181448c0 0x7f131806d3d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f13180b1a90 tx=0x7f130802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ad401881e25da0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.895+0000 7f13177fe640 1 -- 172.21.15.64:0/2025152448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130802ca80 con 0x7f13181448c0 2023-11-24T08:06:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.895+0000 7f13177fe640 1 -- 172.21.15.64:0/2025152448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130802cbe0 con 0x7f13181448c0 2023-11-24T08:06:25.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.895+0000 7f13177fe640 1 -- 172.21.15.64:0/2025152448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13080343f0 con 0x7f13181448c0 2023-11-24T08:06:25.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.896+0000 7f131d0ff640 1 -- 172.21.15.64:0/2025152448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13181448c0 msgr2=0x7f131806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.896+0000 7f131d0ff640 1 --2- 172.21.15.64:0/2025152448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13181448c0 0x7f131806d3d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f13180b1a90 tx=0x7f130802bee0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.896+0000 7f131d0ff640 1 -- 172.21.15.64:0/2025152448 shutdown_connections 2023-11-24T08:06:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.896+0000 7f131d0ff640 1 --2- 172.21.15.64:0/2025152448 >> v2:172.21.15.91:3300/0 conn(0x7f131806d910 0x7f131815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.896+0000 7f131d0ff640 1 --2- 172.21.15.64:0/2025152448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13181448c0 0x7f131806d3d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.896+0000 7f131d0ff640 1 -- 172.21.15.64:0/2025152448 >> 172.21.15.64:0/2025152448 conn(0x7f13180b0aa0 msgr2=0x7f13180b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.896+0000 7f131d0ff640 1 -- 172.21.15.64:0/2025152448 shutdown_connections 2023-11-24T08:06:25.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.896+0000 7f131d0ff640 1 -- 172.21.15.64:0/2025152448 wait complete. 2023-11-24T08:06:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.898+0000 7f131d0ff640 1 Processor -- start 2023-11-24T08:06:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.898+0000 7f131d0ff640 1 -- start start 2023-11-24T08:06:25.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131d0ff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f131806d910 0x7f1318142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131d0ff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13181448c0 0x7f13181f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131d0ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f131815bbd0 0x7f131812bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131d0ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318143540 con 0x7f131815bbd0 2023-11-24T08:06:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131d0ff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318143820 con 0x7f131806d910 2023-11-24T08:06:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131d0ff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1318143b00 con 0x7f13181448c0 2023-11-24T08:06:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131c8fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f131815bbd0 0x7f131812bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131c8fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f131815bbd0 0x7f131812bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47726/0 (socket says 172.21.15.64:47726) 2023-11-24T08:06:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f131c8fe640 1 -- 172.21.15.64:0/926571219 learned_addr learned my addr 172.21.15.64:0/926571219 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f1317fff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f131806d910 0x7f1318142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.899+0000 7f130f7fe640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13181448c0 0x7f13181f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f131c8fe640 1 -- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13181448c0 msgr2=0x7f13181f0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f131c8fe640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13181448c0 0x7f13181f0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f131c8fe640 1 -- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f131806d910 msgr2=0x7f1318142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f131c8fe640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f131806d910 0x7f1318142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:25.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f131c8fe640 1 -- 172.21.15.64:0/926571219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308009b30 con 0x7f131815bbd0 2023-11-24T08:06:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f131c8fe640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f131815bbd0 0x7f131812bfa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f131000b410 tx=0x7f131000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:25.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f13157fa640 1 -- 172.21.15.64:0/926571219 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310004020 con 0x7f131815bbd0 2023-11-24T08:06:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f13157fa640 1 -- 172.21.15.64:0/926571219 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1310004180 con 0x7f131815bbd0 2023-11-24T08:06:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f13157fa640 1 -- 172.21.15.64:0/926571219 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310011630 con 0x7f131815bbd0 2023-11-24T08:06:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1318129f60 con 0x7f131815bbd0 2023-11-24T08:06:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.900+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1318143150 con 0x7f131815bbd0 2023-11-24T08:06:25.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.902+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131804e840 con 0x7f131815bbd0 2023-11-24T08:06:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.903+0000 7f13157fa640 1 -- 172.21.15.64:0/926571219 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1310011790 con 0x7f131815bbd0 2023-11-24T08:06:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.904+0000 7f13157fa640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12f806aed0 0x7f12f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.904+0000 7f13157fa640 1 -- 172.21.15.64:0/926571219 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1310089410 con 0x7f131815bbd0 2023-11-24T08:06:25.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.905+0000 7f1317fff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12f806aed0 0x7f12f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:25.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.906+0000 7f1317fff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12f806aed0 0x7f12f806d390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1308035000 tx=0x7f1308030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:25.908+0000 7f13157fa640 1 -- 172.21.15.64:0/926571219 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13100543d0 con 0x7f131815bbd0 2023-11-24T08:06:26.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.102+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f13180b4f40 con 0x7f131815bbd0 2023-11-24T08:06:26.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.103+0000 7f13157fa640 1 -- 172.21.15.64:0/926571219 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f13100579f0 con 0x7f131815bbd0 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.106+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12f806aed0 msgr2=0x7f12f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.106+0000 7f131d0ff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12f806aed0 0x7f12f806d390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1308035000 tx=0x7f1308030000 comp rx=0 tx=0).stop 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.106+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f131815bbd0 msgr2=0x7f131812bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.106+0000 7f131d0ff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f131815bbd0 0x7f131812bfa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f131000b410 tx=0x7f131000b8d0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.107+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 shutdown_connections 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.107+0000 7f131d0ff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12f806aed0 0x7f12f806d390 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.107+0000 7f131d0ff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f131815bbd0 0x7f131812bfa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.107+0000 7f131d0ff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13181448c0 0x7f13181f0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.107+0000 7f131d0ff640 1 --2- 172.21.15.64:0/926571219 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f131806d910 0x7f1318142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.107+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 >> 172.21.15.64:0/926571219 conn(0x7f13180b0aa0 msgr2=0x7f1318159380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:26.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.107+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 shutdown_connections 2023-11-24T08:06:26.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.107+0000 7f131d0ff640 1 -- 172.21.15.64:0/926571219 wait complete. 2023-11-24T08:06:26.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp934 2023-11-24T08:06:26.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.359+0000 7f0326cf8640 1 Processor -- start 2023-11-24T08:06:26.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.360+0000 7f0326cf8640 1 -- start start 2023-11-24T08:06:26.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.361+0000 7f0326cf8640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200d01c0 0x7f03200d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.361+0000 7f0326cf8640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03200d19b0 con 0x7f032015d550 2023-11-24T08:06:26.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.362+0000 7f0326cf8640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03200d0da0 con 0x7f03200ceaa0 2023-11-24T08:06:26.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.362+0000 7f0326cf8640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03200d2380 con 0x7f03200d01c0 2023-11-24T08:06:26.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.362+0000 7f0325cf6640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f032015d550 0x7f03200cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52450/0 (socket says 172.21.15.64:52450) 2023-11-24T08:06:26.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.362+0000 7f0325cf6640 1 -- 172.21.15.64:0/249475786 learned_addr learned my addr 172.21.15.64:0/249475786 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:26.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.362+0000 7f03254f5640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200d01c0 0x7f03200d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.362+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051527776 0 0) 0x7f03200d19b0 con 0x7f032015d550 2023-11-24T08:06:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.362+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f030c003520 con 0x7f032015d550 2023-11-24T08:06:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3409277315 0 0) 0x7f030c003520 con 0x7f032015d550 2023-11-24T08:06:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200d01c0 msgr2=0x7f03200d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200d01c0 0x7f03200d1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 >> v1:172.21.15.91:6789/0 conn(0x7f03200ceaa0 legacy=0x7f03200cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200d1080 con 0x7f032015d550 2023-11-24T08:06:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f031c001670 con 0x7f032015d550 2023-11-24T08:06:26.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 >> v1:172.21.15.64:6789/0 conn(0x7f032015d550 legacy=0x7f03200cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c003a10 0x7f030c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200d01c0 0x7f030c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c004d00 0x7f030c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c00bb10 con 0x7f030c003a10 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c00bc50 con 0x7f030c004d00 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c00be10 con 0x7f03200d01c0 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.363+0000 7f0325cf6640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c003a10 0x7f030c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.364+0000 7f0325cf6640 1 -- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200d01c0 msgr2=0x7f030c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.364+0000 7f0325cf6640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200d01c0 0x7f030c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.364+0000 7f0325cf6640 1 -- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c004d00 msgr2=0x7f030c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.364+0000 7f0325cf6640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c004d00 0x7f030c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.364+0000 7f0325cf6640 1 -- 172.21.15.64:0/249475786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c003520 con 0x7f030c003a10 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.365+0000 7f0325cf6640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c003a10 0x7f030c003e30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f031c001050 tx=0x7f030c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b0fb5cadcb5f39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.365+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c005ad0 con 0x7f030c003a10 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.365+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c005c30 con 0x7f030c003a10 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.365+0000 7f0324cf4640 1 -- 172.21.15.64:0/249475786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c02c890 con 0x7f030c003a10 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.366+0000 7f0326cf8640 1 -- 172.21.15.64:0/249475786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c003a10 msgr2=0x7f030c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.366+0000 7f0326cf8640 1 --2- 172.21.15.64:0/249475786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c003a10 0x7f030c003e30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f031c001050 tx=0x7f030c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.366+0000 7f03264f7640 1 -- 172.21.15.64:0/249475786 reap_dead start 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.366+0000 7f0326cf8640 1 -- 172.21.15.64:0/249475786 shutdown_connections 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.366+0000 7f0326cf8640 1 -- 172.21.15.64:0/249475786 >> 172.21.15.64:0/249475786 conn(0x7f03200d30a0 msgr2=0x7f03200d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.366+0000 7f0326cf8640 1 -- 172.21.15.64:0/249475786 shutdown_connections 2023-11-24T08:06:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.366+0000 7f0326cf8640 1 -- 172.21.15.64:0/249475786 wait complete. 2023-11-24T08:06:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.368+0000 7f0326cf8640 1 Processor -- start 2023-11-24T08:06:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.368+0000 7f0326cf8640 1 -- start start 2023-11-24T08:06:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f0326cf8640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c003a10 0x7f032014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f0326cf8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032015c1e0 0x7f03200b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f0326cf8640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200c7930 0x7f03200c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f0326cf8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032015e5a0 con 0x7f032015c1e0 2023-11-24T08:06:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f0326cf8640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032015d9d0 con 0x7f030c003a10 2023-11-24T08:06:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f0326cf8640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032015dc70 con 0x7f03200c7930 2023-11-24T08:06:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f03254f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032015c1e0 0x7f03200b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f03254f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032015c1e0 0x7f03200b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47750/0 (socket says 172.21.15.64:47750) 2023-11-24T08:06:26.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.369+0000 7f03254f5640 1 -- 172.21.15.64:0/224059726 learned_addr learned my addr 172.21.15.64:0/224059726 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:26.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03254f5640 1 -- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200c7930 msgr2=0x7f03200c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f0325cf6640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c003a10 0x7f032014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03264f7640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200c7930 0x7f03200c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03254f5640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200c7930 0x7f03200c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03254f5640 1 -- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c003a10 msgr2=0x7f032014f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03254f5640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c003a10 0x7f032014f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03254f5640 1 -- 172.21.15.64:0/224059726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032015df50 con 0x7f032015c1e0 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03264f7640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200c7930 0x7f03200c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f0325cf6640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c003a10 0x7f032014f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03254f5640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032015c1e0 0x7f03200b81f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f031000e7c0 tx=0x7f031000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03167fc640 1 -- 172.21.15.64:0/224059726 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031000eed0 con 0x7f032015c1e0 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.370+0000 7f03167fc640 1 -- 172.21.15.64:0/224059726 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031000c8e0 con 0x7f032015c1e0 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.371+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032015e230 con 0x7f032015c1e0 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.371+0000 7f03167fc640 1 -- 172.21.15.64:0/224059726 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031000cc70 con 0x7f032015c1e0 2023-11-24T08:06:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.371+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032015f7a0 con 0x7f032015c1e0 2023-11-24T08:06:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.372+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0320067960 con 0x7f032015c1e0 2023-11-24T08:06:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.373+0000 7f03167fc640 1 -- 172.21.15.64:0/224059726 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0310013470 con 0x7f032015c1e0 2023-11-24T08:06:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.374+0000 7f03167fc640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f030406af00 0x7f030406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.374+0000 7f03167fc640 1 -- 172.21.15.64:0/224059726 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0310089630 con 0x7f032015c1e0 2023-11-24T08:06:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.375+0000 7f0325cf6640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f030406af00 0x7f030406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.376+0000 7f0325cf6640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f030406af00 0x7f030406d3c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f031c034000 tx=0x7f031c037040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.379+0000 7f03167fc640 1 -- 172.21.15.64:0/224059726 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0310054660 con 0x7f032015c1e0 2023-11-24T08:06:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.601+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp934", "target": ["mon-mgr", ""]}) v1 -- 0x7f03200d1640 con 0x7f030406af00 2023-11-24T08:06:26.615 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:26.613+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:26.615 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:26.613+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:26.615 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:26.613+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:26.615 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:26.613+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:26.616 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:26.613+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:26.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.647+0000 7f03167fc640 1 -- 172.21.15.64:0/224059726 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f031c0015b0 con 0x7f030406af00 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f030406af00 msgr2=0x7f030406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f030406af00 0x7f030406d3c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f031c034000 tx=0x7f031c037040 comp rx=0 tx=0).stop 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032015c1e0 msgr2=0x7f03200b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032015c1e0 0x7f03200b81f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f031000e7c0 tx=0x7f031000ec80 comp rx=0 tx=0).stop 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 shutdown_connections 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f030406af00 0x7f030406d3c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03200c7930 0x7f03200c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032015c1e0 0x7f03200b81f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 --2- 172.21.15.64:0/224059726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f030c003a10 0x7f032014f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 >> 172.21.15.64:0/224059726 conn(0x7f03200d30a0 msgr2=0x7f03200b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:26.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 shutdown_connections 2023-11-24T08:06:26.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.650+0000 7f0326cf8640 1 -- 172.21.15.64:0/224059726 wait complete. 2023-11-24T08:06:26.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol9602 subvol_grp934 2023-11-24T08:06:26.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.905+0000 7f71c4c1c640 1 Processor -- start 2023-11-24T08:06:26.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.905+0000 7f71c4c1c640 1 -- start start 2023-11-24T08:06:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.907+0000 7f71c4c1c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f71c006bb60 0x7f71c006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.907+0000 7f71c4c1c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c0161ff0 con 0x7f71c0161b80 2023-11-24T08:06:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.907+0000 7f71c4c1c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c00b3ff0 con 0x7f71c015d3c0 2023-11-24T08:06:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.907+0000 7f71c4c1c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c00b3270 con 0x7f71c006bb60 2023-11-24T08:06:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bdd9b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f71c0161b80 0x7f71c006b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52464/0 (socket says 172.21.15.64:52464) 2023-11-24T08:06:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bdd9b640 1 -- 172.21.15.64:0/950131069 learned_addr learned my addr 172.21.15.64:0/950131069 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bed9d640 1 --2- 172.21.15.64:0/950131069 >> v2:172.21.15.91:3300/0 conn(0x7f71c006bb60 0x7f71c006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011475994 0 0) 0x7f71c0161ff0 con 0x7f71c0161b80 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a8003520 con 0x7f71c0161b80 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1507526594 0 0) 0x7f71a8003520 con 0x7f71c0161b80 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 >> v2:172.21.15.91:3300/0 conn(0x7f71c006bb60 msgr2=0x7f71c006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bd59a640 1 --2- 172.21.15.64:0/950131069 >> v2:172.21.15.91:3300/0 conn(0x7f71c006bb60 0x7f71c006cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.908+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 >> v1:172.21.15.91:6789/0 conn(0x7f71c015d3c0 legacy=0x7f71c00b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c00b3550 con 0x7f71c0161b80 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f71b4001670 con 0x7f71c0161b80 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 >> v1:172.21.15.64:6789/0 conn(0x7f71c0161b80 legacy=0x7f71c006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71a8003970 0x7f71a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71a8005180 0x7f71a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c00b3270 con 0x7f71a80047c0 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a800c020 con 0x7f71a8003970 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a800c1e0 con 0x7f71a8005180 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bdd9b640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71be59c640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71a8003970 0x7f71a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bed9d640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71a8005180 0x7f71a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bdd9b640 1 -- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71a8005180 msgr2=0x7f71a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bdd9b640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71a8005180 0x7f71a800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bdd9b640 1 -- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71a8003970 msgr2=0x7f71a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bdd9b640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71a8003970 0x7f71a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.909+0000 7f71bed9d640 1 -- 172.21.15.64:0/950131069 reap_dead start 2023-11-24T08:06:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.910+0000 7f71bdd9b640 1 -- 172.21.15.64:0/950131069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a8003520 con 0x7f71a80047c0 2023-11-24T08:06:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.910+0000 7f71be59c640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71a8003970 0x7f71a8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.911+0000 7f71bdd9b640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71a8004c40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c240 tx=0x7f71a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b42c2664aed9ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.911+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4005ad0 con 0x7f71a80047c0 2023-11-24T08:06:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.911+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4005c50 con 0x7f71a80047c0 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.911+0000 7f71bd59a640 1 -- 172.21.15.64:0/950131069 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b402c7c0 con 0x7f71a80047c0 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.911+0000 7f71c4c1c640 1 -- 172.21.15.64:0/950131069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 msgr2=0x7f71a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.911+0000 7f71c4c1c640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71a8004c40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c240 tx=0x7f71a80057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.912+0000 7f71c4c1c640 1 -- 172.21.15.64:0/950131069 shutdown_connections 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.912+0000 7f71c4c1c640 1 --2- 172.21.15.64:0/950131069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71a8004c40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.912+0000 7f71c4c1c640 1 -- 172.21.15.64:0/950131069 >> 172.21.15.64:0/950131069 conn(0x7f71c00cfd60 msgr2=0x7f71c0070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.912+0000 7f71c4c1c640 1 -- 172.21.15.64:0/950131069 shutdown_connections 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.912+0000 7f71c4c1c640 1 -- 172.21.15.64:0/950131069 wait complete. 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.914+0000 7f71c4c1c640 1 Processor -- start 2023-11-24T08:06:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.914+0000 7f71c4c1c640 1 -- start start 2023-11-24T08:06:26.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71c4c1c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71c015daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71c4c1c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71c00c6280 0x7f71c00c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71c4c1c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71c00c54f0 0x7f71c00c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71c4c1c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c00c6c40 con 0x7f71a80047c0 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71c4c1c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c014aa70 con 0x7f71c00c54f0 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71c4c1c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c00c4730 con 0x7f71c00c6280 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71be59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71c015daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71be59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71c015daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47776/0 (socket says 172.21.15.64:47776) 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71be59c640 1 -- 172.21.15.64:0/3739262387 learned_addr learned my addr 172.21.15.64:0/3739262387 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71bdd9b640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71c00c6280 0x7f71c00c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.915+0000 7f71bed9d640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71c00c54f0 0x7f71c00c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71be59c640 1 -- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71c00c6280 msgr2=0x7f71c00c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71be59c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71c00c6280 0x7f71c00c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71be59c640 1 -- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71c00c54f0 msgr2=0x7f71c00c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71be59c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71c00c54f0 0x7f71c00c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71be59c640 1 -- 172.21.15.64:0/3739262387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c00c4a10 con 0x7f71a80047c0 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71bed9d640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71c00c54f0 0x7f71c00c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71be59c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71c015daa0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f71b000d480 tx=0x7f71b000d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71aeffd640 1 -- 172.21.15.64:0/3739262387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0005480 con 0x7f71a80047c0 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71aeffd640 1 -- 172.21.15.64:0/3739262387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b0003610 con 0x7f71a80047c0 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71aeffd640 1 -- 172.21.15.64:0/3739262387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0003a00 con 0x7f71a80047c0 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.916+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c00c4cf0 con 0x7f71a80047c0 2023-11-24T08:06:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.917+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c00c5eb0 con 0x7f71a80047c0 2023-11-24T08:06:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.918+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c0072620 con 0x7f71a80047c0 2023-11-24T08:06:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.919+0000 7f71aeffd640 1 -- 172.21.15.64:0/3739262387 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f71b0003b60 con 0x7f71a80047c0 2023-11-24T08:06:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.920+0000 7f71aeffd640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f71a006aed0 0x7f71a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.920+0000 7f71aeffd640 1 -- 172.21.15.64:0/3739262387 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71b00896b0 con 0x7f71a80047c0 2023-11-24T08:06:26.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.921+0000 7f71bdd9b640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f71a006aed0 0x7f71a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:26.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.922+0000 7f71bdd9b640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f71a006aed0 0x7f71a006d390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f71b4034000 tx=0x7f71b4009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:26.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:26.925+0000 7f71aeffd640 1 -- 172.21.15.64:0/3739262387 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71b00546b0 con 0x7f71a80047c0 2023-11-24T08:06:27.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.151+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol9602", "group_name": "subvol_grp934", "target": ["mon-mgr", ""]}) v1 -- 0x7f71c00c48a0 con 0x7f71a006aed0 2023-11-24T08:06:27.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.213+0000 7f71aeffd640 1 -- 172.21.15.64:0/3739262387 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f71c00c48a0 con 0x7f71a006aed0 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.217+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f71a006aed0 msgr2=0x7f71a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.217+0000 7f71c4c1c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f71a006aed0 0x7f71a006d390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f71b4034000 tx=0x7f71b4009030 comp rx=0 tx=0).stop 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.217+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 msgr2=0x7f71c015daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.217+0000 7f71c4c1c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71c015daa0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f71b000d480 tx=0x7f71b000d940 comp rx=0 tx=0).stop 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.218+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 shutdown_connections 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.218+0000 7f71c4c1c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f71a006aed0 0x7f71a006d390 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.218+0000 7f71c4c1c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f71c00c54f0 0x7f71c00c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.218+0000 7f71c4c1c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f71c00c6280 0x7f71c00c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.218+0000 7f71c4c1c640 1 --2- 172.21.15.64:0/3739262387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71a80047c0 0x7f71c015daa0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.218+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 >> 172.21.15.64:0/3739262387 conn(0x7f71c00cfd60 msgr2=0x7f71c0070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:27.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.218+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 shutdown_connections 2023-11-24T08:06:27.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.218+0000 7f71c4c1c640 1 -- 172.21.15.64:0/3739262387 wait complete. 2023-11-24T08:06:27.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol9602 subvol_snap4432 subvol_grp934 2023-11-24T08:06:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.478+0000 7f364ffff640 1 Processor -- start 2023-11-24T08:06:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.478+0000 7f364ffff640 1 -- start start 2023-11-24T08:06:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.480+0000 7f364ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3650153ea0 0x7f36500ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.480+0000 7f364ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 0x7f3650071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.480+0000 7f364ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36500ce650 con 0x7f3650154280 2023-11-24T08:06:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.480+0000 7f364ffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3650154710 con 0x7f3650153ea0 2023-11-24T08:06:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.480+0000 7f364ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365015e0a0 con 0x7f3650070850 2023-11-24T08:06:27.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.481+0000 7f364e7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3650154280 0x7f365015d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46360/0 (socket says 172.21.15.64:46360) 2023-11-24T08:06:27.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.481+0000 7f364e7fc640 1 -- 172.21.15.64:0/1076309301 learned_addr learned my addr 172.21.15.64:0/1076309301 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:27.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.481+0000 7f364effd640 1 --2- 172.21.15.64:0/1076309301 >> v2:172.21.15.91:3300/0 conn(0x7f3650153ea0 0x7f36500ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:27.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.481+0000 7f364f7fe640 1 --2- 172.21.15.64:0/1076309301 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 0x7f3650071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:27.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.483+0000 7f364effd640 1 -- 172.21.15.64:0/1076309301 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 msgr2=0x7f3650071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:27.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.483+0000 7f364effd640 1 --2- 172.21.15.64:0/1076309301 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 0x7f3650071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.483+0000 7f364effd640 1 -- 172.21.15.64:0/1076309301 >> v1:172.21.15.64:6789/0 conn(0x7f3650154280 legacy=0x7f365015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:27.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.483+0000 7f364dffb640 1 -- 172.21.15.64:0/1076309301 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951800207 0 0) 0x7f36500ce650 con 0x7f3650154280 2023-11-24T08:06:27.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.483+0000 7f364effd640 1 -- 172.21.15.64:0/1076309301 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365015e380 con 0x7f3650153ea0 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.483+0000 7f364effd640 1 --2- 172.21.15.64:0/1076309301 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650153ea0 0x7f36500ad450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36400097a0 tx=0x7f364002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab650e9e41b6e48 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364dffb640 1 -- 172.21.15.64:0/1076309301 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364002d040 con 0x7f3650153ea0 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364dffb640 1 -- 172.21.15.64:0/1076309301 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640031810 con 0x7f3650153ea0 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364dffb640 1 -- 172.21.15.64:0/1076309301 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640031ba0 con 0x7f3650153ea0 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364ffff640 1 -- 172.21.15.64:0/1076309301 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650153ea0 msgr2=0x7f36500ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364ffff640 1 --2- 172.21.15.64:0/1076309301 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650153ea0 0x7f36500ad450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36400097a0 tx=0x7f364002b8b0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364ffff640 1 -- 172.21.15.64:0/1076309301 shutdown_connections 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364ffff640 1 --2- 172.21.15.64:0/1076309301 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 0x7f3650071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364ffff640 1 --2- 172.21.15.64:0/1076309301 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650153ea0 0x7f36500ad450 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364ffff640 1 -- 172.21.15.64:0/1076309301 >> 172.21.15.64:0/1076309301 conn(0x7f36500cec10 msgr2=0x7f36500cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.484+0000 7f364ffff640 1 -- 172.21.15.64:0/1076309301 shutdown_connections 2023-11-24T08:06:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.485+0000 7f364ffff640 1 -- 172.21.15.64:0/1076309301 wait complete. 2023-11-24T08:06:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.486+0000 7f364ffff640 1 Processor -- start 2023-11-24T08:06:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.486+0000 7f364ffff640 1 -- start start 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.487+0000 7f364ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 0x7f36501e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.487+0000 7f364ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3650153ea0 0x7f3650143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.487+0000 7f364ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650154280 0x7f365006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.487+0000 7f364ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501e9300 con 0x7f3650153ea0 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.487+0000 7f364ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501e8a10 con 0x7f3650154280 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.487+0000 7f364ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36501f3450 con 0x7f3650070850 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3650153ea0 0x7f3650143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3650153ea0 0x7f3650143550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56898/0 (socket says 172.21.15.64:56898) 2023-11-24T08:06:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364e7fc640 1 -- 172.21.15.64:0/3928302568 learned_addr learned my addr 172.21.15.64:0/3928302568 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364effd640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 0x7f36501e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364f7fe640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650154280 0x7f365006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364e7fc640 1 -- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 msgr2=0x7f36501e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364e7fc640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 0x7f36501e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364e7fc640 1 -- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650154280 msgr2=0x7f365006d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364e7fc640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650154280 0x7f365006d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364e7fc640 1 -- 172.21.15.64:0/3928302568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640009400 con 0x7f3650153ea0 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.488+0000 7f364f7fe640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650154280 0x7f365006d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.489+0000 7f364e7fc640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3650153ea0 0x7f3650143550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f364400c430 tx=0x7f364400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.489+0000 7f36377fe640 1 -- 172.21.15.64:0/3928302568 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36440074e0 con 0x7f3650153ea0 2023-11-24T08:06:27.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.489+0000 7f36377fe640 1 -- 172.21.15.64:0/3928302568 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644007660 con 0x7f3650153ea0 2023-11-24T08:06:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.489+0000 7f36377fe640 1 -- 172.21.15.64:0/3928302568 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644004a20 con 0x7f3650153ea0 2023-11-24T08:06:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.489+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36501e9720 con 0x7f3650153ea0 2023-11-24T08:06:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.489+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36501e8c40 con 0x7f3650153ea0 2023-11-24T08:06:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.491+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36501e9440 con 0x7f3650153ea0 2023-11-24T08:06:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.491+0000 7f36377fe640 1 -- 172.21.15.64:0/3928302568 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3644004b80 con 0x7f3650153ea0 2023-11-24T08:06:27.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.492+0000 7f36377fe640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f363c06aed0 0x7f363c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:27.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.492+0000 7f36377fe640 1 -- 172.21.15.64:0/3928302568 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36440893c0 con 0x7f3650153ea0 2023-11-24T08:06:27.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.493+0000 7f364effd640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f363c06aed0 0x7f363c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:27.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.494+0000 7f364effd640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f363c06aed0 0x7f363c06d390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3640036000 tx=0x7f364003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:27.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.497+0000 7f36377fe640 1 -- 172.21.15.64:0/3928302568 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36501e9440 con 0x7f3650153ea0 2023-11-24T08:06:27.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.723+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol9602", "snap_name": "subvol_snap4432", "group_name": "subvol_grp934", "target": ["mon-mgr", ""]}) v1 -- 0x7f3650070330 con 0x7f363c06aed0 2023-11-24T08:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.757+0000 7f36377fe640 1 -- 172.21.15.64:0/3928302568 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3640031b00 con 0x7f363c06aed0 2023-11-24T08:06:27.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f363c06aed0 msgr2=0x7f363c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:27.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f363c06aed0 0x7f363c06d390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3640036000 tx=0x7f364003c040 comp rx=0 tx=0).stop 2023-11-24T08:06:27.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3650153ea0 msgr2=0x7f3650143550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:27.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3650153ea0 0x7f3650143550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f364400c430 tx=0x7f364400c8f0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 shutdown_connections 2023-11-24T08:06:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f363c06aed0 0x7f363c06d390 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3650154280 0x7f365006d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3650153ea0 0x7f3650143550 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 --2- 172.21.15.64:0/3928302568 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3650070850 0x7f36501e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 >> 172.21.15.64:0/3928302568 conn(0x7f36500cec10 msgr2=0x7f36500ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 shutdown_connections 2023-11-24T08:06:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:27.760+0000 7f364ffff640 1 -- 172.21.15.64:0/3928302568 wait complete. 2023-11-24T08:06:27.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot metadata set cephfs subvol9602 subvol_snap4432 key value subvol_grp934 2023-11-24T08:06:28.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.020+0000 7f104b1ce640 1 Processor -- start 2023-11-24T08:06:28.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.020+0000 7f104b1ce640 1 -- start start 2023-11-24T08:06:28.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f104b1ce640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1044158f40 0x7f10440ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f104b1ce640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10440cb9d0 0x7f104406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f104b1ce640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10440cd2b0 con 0x7f10440cec70 2023-11-24T08:06:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f104b1ce640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10440d02e0 con 0x7f1044158f40 2023-11-24T08:06:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f104b1ce640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104406f670 con 0x7f10440cb9d0 2023-11-24T08:06:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f1043fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10440cb9d0 0x7f104406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f1048f6a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1044158f40 0x7f10440ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f1043fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10440cb9d0 0x7f104406f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:59706/0 (socket says 172.21.15.64:59706) 2023-11-24T08:06:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.022+0000 7f1043fff640 1 -- 172.21.15.64:0/1858385739 learned_addr learned my addr 172.21.15.64:0/1858385739 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.023+0000 7f1048f6a640 1 -- 172.21.15.64:0/1858385739 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10440cb9d0 msgr2=0x7f104406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.023+0000 7f1048f6a640 1 --2- 172.21.15.64:0/1858385739 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10440cb9d0 0x7f104406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.023+0000 7f1048f6a640 1 -- 172.21.15.64:0/1858385739 >> v1:172.21.15.91:6789/0 conn(0x7f10440cec70 legacy=0x7f104415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.023+0000 7f1048f6a640 1 -- 172.21.15.64:0/1858385739 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10440cbe30 con 0x7f1044158f40 2023-11-24T08:06:28.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.024+0000 7f1048f6a640 1 --2- 172.21.15.64:0/1858385739 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1044158f40 0x7f10440ce6b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f10380097a0 tx=0x7f103802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9661c16d7faafa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:28.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.025+0000 7f10437fe640 1 -- 172.21.15.64:0/1858385739 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038031830 con 0x7f1044158f40 2023-11-24T08:06:28.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.025+0000 7f10437fe640 1 -- 172.21.15.64:0/1858385739 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1038031990 con 0x7f1044158f40 2023-11-24T08:06:28.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.025+0000 7f10437fe640 1 -- 172.21.15.64:0/1858385739 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038031d20 con 0x7f1044158f40 2023-11-24T08:06:28.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.025+0000 7f104b1ce640 1 -- 172.21.15.64:0/1858385739 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1044158f40 msgr2=0x7f10440ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:28.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.025+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1858385739 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1044158f40 0x7f10440ce6b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f10380097a0 tx=0x7f103802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.026+0000 7f104b1ce640 1 -- 172.21.15.64:0/1858385739 shutdown_connections 2023-11-24T08:06:28.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.026+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1858385739 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f10440cb9d0 0x7f104406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.026+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1858385739 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1044158f40 0x7f10440ce6b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.026+0000 7f104b1ce640 1 -- 172.21.15.64:0/1858385739 >> 172.21.15.64:0/1858385739 conn(0x7f10440cdb20 msgr2=0x7f10440cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:28.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.026+0000 7f104b1ce640 1 -- 172.21.15.64:0/1858385739 shutdown_connections 2023-11-24T08:06:28.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.026+0000 7f104b1ce640 1 -- 172.21.15.64:0/1858385739 wait complete. 2023-11-24T08:06:28.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.028+0000 7f104b1ce640 1 Processor -- start 2023-11-24T08:06:28.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.028+0000 7f104b1ce640 1 -- start start 2023-11-24T08:06:28.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f104b1ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10440cb9d0 0x7f1044157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f104b1ce640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10440cec70 0x7f10441f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f104b1ce640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1044158f40 0x7f10440c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f104b1ce640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10440ca410 con 0x7f10440cb9d0 2023-11-24T08:06:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f104b1ce640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10440c4fb0 con 0x7f10440cec70 2023-11-24T08:06:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f104b1ce640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10440ca6b0 con 0x7f1044158f40 2023-11-24T08:06:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f1048f6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10440cb9d0 0x7f1044157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f1048f6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10440cb9d0 0x7f1044157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56906/0 (socket says 172.21.15.64:56906) 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.029+0000 7f1048f6a640 1 -- 172.21.15.64:0/1745762391 learned_addr learned my addr 172.21.15.64:0/1745762391 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.030+0000 7f1043fff640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10440cec70 0x7f10441f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.030+0000 7f104976b640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1044158f40 0x7f10440c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.030+0000 7f1048f6a640 1 -- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1044158f40 msgr2=0x7f10440c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.030+0000 7f1048f6a640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1044158f40 0x7f10440c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.030+0000 7f1048f6a640 1 -- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10440cec70 msgr2=0x7f10441f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.030+0000 7f1048f6a640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10440cec70 0x7f10441f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.030+0000 7f1048f6a640 1 -- 172.21.15.64:0/1745762391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038009400 con 0x7f10440cb9d0 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.030+0000 7f1048f6a640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10440cb9d0 0x7f1044157730 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f103802b910 tx=0x7f10380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.031+0000 7f10417fa640 1 -- 172.21.15.64:0/1745762391 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103803a8f0 con 0x7f10440cb9d0 2023-11-24T08:06:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.031+0000 7f10417fa640 1 -- 172.21.15.64:0/1745762391 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103803aa50 con 0x7f10440cb9d0 2023-11-24T08:06:28.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.031+0000 7f10417fa640 1 -- 172.21.15.64:0/1745762391 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103803ae00 con 0x7f10440cb9d0 2023-11-24T08:06:28.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.031+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10440ca990 con 0x7f10440cb9d0 2023-11-24T08:06:28.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.031+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10441f36e0 con 0x7f10440cb9d0 2023-11-24T08:06:28.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.032+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104406fb50 con 0x7f10440cb9d0 2023-11-24T08:06:28.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.033+0000 7f10417fa640 1 -- 172.21.15.64:0/1745762391 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f103804d450 con 0x7f10440cb9d0 2023-11-24T08:06:28.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.034+0000 7f10417fa640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102806aed0 0x7f102806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:28.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.034+0000 7f10417fa640 1 -- 172.21.15.64:0/1745762391 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10380ab7a0 con 0x7f10440cb9d0 2023-11-24T08:06:28.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.035+0000 7f1043fff640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102806aed0 0x7f102806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:28.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.036+0000 7f1043fff640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102806aed0 0x7f102806d390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f10440cb200 tx=0x7f1030008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.038+0000 7f10417fa640 1 -- 172.21.15.64:0/1745762391 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1038076750 con 0x7f10440cb9d0 2023-11-24T08:06:28.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.269+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot metadata set", "vol_name": "cephfs", "sub_name": "subvol9602", "snap_name": "subvol_snap4432", "key_name": "key", "value": "value", "group_name": "subvol_grp934", "target": ["mon-mgr", ""]}) v1 -- 0x7f1044067670 con 0x7f102806aed0 2023-11-24T08:06:28.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.306+0000 7f10417fa640 1 -- 172.21.15.64:0/1745762391 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1044067670 con 0x7f102806aed0 2023-11-24T08:06:28.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102806aed0 msgr2=0x7f102806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:28.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102806aed0 0x7f102806d390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f10440cb200 tx=0x7f1030008040 comp rx=0 tx=0).stop 2023-11-24T08:06:28.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10440cb9d0 msgr2=0x7f1044157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:28.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10440cb9d0 0x7f1044157730 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f103802b910 tx=0x7f10380046c0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 shutdown_connections 2023-11-24T08:06:28.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102806aed0 0x7f102806d390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1044158f40 0x7f10440c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f10440cec70 0x7f10441f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 --2- 172.21.15.64:0/1745762391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10440cb9d0 0x7f1044157730 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:28.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 >> 172.21.15.64:0/1745762391 conn(0x7f10440cdb20 msgr2=0x7f104406fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:28.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 shutdown_connections 2023-11-24T08:06:28.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:28.309+0000 7f104b1ce640 1 -- 172.21.15.64:0/1745762391 wait complete. 2023-11-24T08:06:28.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:28.330 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp934/subvol9602/.meta ./volumes/subvol_grp934/subvol9602/.meta.stale_snap_section) 2023-11-24T08:06:29.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol9602 subvol_snap4432 subvol_grp934 2023-11-24T08:06:29.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.994+0000 7f178359e640 1 Processor -- start 2023-11-24T08:06:29.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.994+0000 7f178359e640 1 -- start start 2023-11-24T08:06:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f178359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f17840b4140 0x7f1784071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f178359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17841439b0 0x7f1784070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f178359e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1784070d40 con 0x7f17841435d0 2023-11-24T08:06:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f178359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1784071020 con 0x7f17840b4140 2023-11-24T08:06:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f178359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1784071300 con 0x7f17841439b0 2023-11-24T08:06:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f177bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f17840b4140 0x7f1784071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f177bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f17840b4140 0x7f1784071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56918/0 (socket says 172.21.15.64:56918) 2023-11-24T08:06:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f177bfff640 1 -- 172.21.15.64:0/2193396996 learned_addr learned my addr 172.21.15.64:0/2193396996 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:29.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.996+0000 7f1782d9d640 1 --2- 172.21.15.64:0/2193396996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17841439b0 0x7f1784070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:29.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.997+0000 7f177bfff640 1 -- 172.21.15.64:0/2193396996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17841439b0 msgr2=0x7f1784070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:29.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.997+0000 7f177bfff640 1 --2- 172.21.15.64:0/2193396996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17841439b0 0x7f1784070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:29.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.997+0000 7f177bfff640 1 -- 172.21.15.64:0/2193396996 >> v1:172.21.15.91:6789/0 conn(0x7f17841435d0 legacy=0x7f17840b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:29.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.997+0000 7f177bfff640 1 -- 172.21.15.64:0/2193396996 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840715e0 con 0x7f17840b4140 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.998+0000 7f177bfff640 1 --2- 172.21.15.64:0/2193396996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17840b4140 0x7f1784071de0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f176c009870 tx=0x7f176c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3645903048e53a7d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.998+0000 7f1781d9b640 1 -- 172.21.15.64:0/2193396996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c031640 con 0x7f17840b4140 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.998+0000 7f1781d9b640 1 -- 172.21.15.64:0/2193396996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c0317a0 con 0x7f17840b4140 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.998+0000 7f1781d9b640 1 -- 172.21.15.64:0/2193396996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c031b30 con 0x7f17840b4140 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.999+0000 7f178359e640 1 -- 172.21.15.64:0/2193396996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17840b4140 msgr2=0x7f1784071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.999+0000 7f178359e640 1 --2- 172.21.15.64:0/2193396996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17840b4140 0x7f1784071de0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f176c009870 tx=0x7f176c02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.999+0000 7f178359e640 1 -- 172.21.15.64:0/2193396996 shutdown_connections 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.999+0000 7f178359e640 1 --2- 172.21.15.64:0/2193396996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17841439b0 0x7f1784070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.999+0000 7f178359e640 1 --2- 172.21.15.64:0/2193396996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17840b4140 0x7f1784071de0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.999+0000 7f178359e640 1 -- 172.21.15.64:0/2193396996 >> 172.21.15.64:0/2193396996 conn(0x7f17840cec10 msgr2=0x7f17840cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:30.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.999+0000 7f178359e640 1 -- 172.21.15.64:0/2193396996 shutdown_connections 2023-11-24T08:06:30.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:29.999+0000 7f178359e640 1 -- 172.21.15.64:0/2193396996 wait complete. 2023-11-24T08:06:30.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.001+0000 7f178359e640 1 Processor -- start 2023-11-24T08:06:30.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.001+0000 7f178359e640 1 -- start start 2023-11-24T08:06:30.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f178359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17840b4140 0x7f1784139550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f178359e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f17841435d0 0x7f178413b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f178359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17841439b0 0x7f178406cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f178359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178413cb60 con 0x7f17841439b0 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f178359e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178413bee0 con 0x7f17841435d0 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f178359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178413c1c0 con 0x7f17840b4140 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f1782d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17841439b0 0x7f178406cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f1782d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17841439b0 0x7f178406cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56930/0 (socket says 172.21.15.64:56930) 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.002+0000 7f1782d9d640 1 -- 172.21.15.64:0/2761070188 learned_addr learned my addr 172.21.15.64:0/2761070188 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f177bfff640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f17841435d0 0x7f178413b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f178259c640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17840b4140 0x7f1784139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f1782d9d640 1 -- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17840b4140 msgr2=0x7f1784139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f1782d9d640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17840b4140 0x7f1784139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f1782d9d640 1 -- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f17841435d0 msgr2=0x7f178413b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f1782d9d640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f17841435d0 0x7f178413b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f1782d9d640 1 -- 172.21.15.64:0/2761070188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0094d0 con 0x7f17841439b0 2023-11-24T08:06:30.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f1782d9d640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17841439b0 0x7f178406cba0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f177c009fd0 tx=0x7f177c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:30.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.003+0000 7f177affd640 1 -- 172.21.15.64:0/2761070188 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c014420 con 0x7f17841439b0 2023-11-24T08:06:30.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.004+0000 7f177affd640 1 -- 172.21.15.64:0/2761070188 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c014580 con 0x7f17841439b0 2023-11-24T08:06:30.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.004+0000 7f177affd640 1 -- 172.21.15.64:0/2761070188 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c014930 con 0x7f17841439b0 2023-11-24T08:06:30.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.004+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178413c4a0 con 0x7f17841439b0 2023-11-24T08:06:30.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.004+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178413cf80 con 0x7f17841439b0 2023-11-24T08:06:30.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.005+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178413cca0 con 0x7f17841439b0 2023-11-24T08:06:30.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.006+0000 7f177affd640 1 -- 172.21.15.64:0/2761070188 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f177c014a90 con 0x7f17841439b0 2023-11-24T08:06:30.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.007+0000 7f177affd640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f176406af60 0x7f176406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.007+0000 7f177affd640 1 -- 172.21.15.64:0/2761070188 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f177c08a000 con 0x7f17841439b0 2023-11-24T08:06:30.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.008+0000 7f178259c640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f176406af60 0x7f176406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:30.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.009+0000 7f178259c640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f176406af60 0x7f176406d420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1774004ea0 tx=0x7f1774009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:30.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.012+0000 7f177affd640 1 -- 172.21.15.64:0/2761070188 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f177c055030 con 0x7f17841439b0 2023-11-24T08:06:30.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.244+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol9602", "snap_name": "subvol_snap4432", "group_name": "subvol_grp934", "target": ["mon-mgr", ""]}) v1 -- 0x7f178404e840 con 0x7f176406af60 2023-11-24T08:06:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.316+0000 7f177affd640 1 -- 172.21.15.64:0/2761070188 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f178404e840 con 0x7f176406af60 2023-11-24T08:06:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.319+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f176406af60 msgr2=0x7f176406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.319+0000 7f178359e640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f176406af60 0x7f176406d420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1774004ea0 tx=0x7f1774009040 comp rx=0 tx=0).stop 2023-11-24T08:06:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17841439b0 msgr2=0x7f178406cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17841439b0 0x7f178406cba0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f177c009fd0 tx=0x7f177c00e4b0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 shutdown_connections 2023-11-24T08:06:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f176406af60 0x7f176406d420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17841439b0 0x7f178406cba0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f17841435d0 0x7f178413b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 --2- 172.21.15.64:0/2761070188 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f17840b4140 0x7f1784139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 >> 172.21.15.64:0/2761070188 conn(0x7f17840cec10 msgr2=0x7f17840acba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 shutdown_connections 2023-11-24T08:06:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.320+0000 7f178359e640 1 -- 172.21.15.64:0/2761070188 wait complete. 2023-11-24T08:06:30.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:30.343 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp934/subvol9602/.meta.stale_snap_section ./volumes/subvol_grp934/subvol9602/.meta) 2023-11-24T08:06:30.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:30.480 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap4432 ./volumes/subvol_grp934/subvol9602/.meta) 2023-11-24T08:06:30.576 INFO:teuthology.orchestra.run.smithi064.stdout:[SNAP_METADATA_subvol_snap4432] 2023-11-24T08:06:30.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol9602 subvol_grp934 2023-11-24T08:06:30.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.819+0000 7fdb0cc7c640 1 Processor -- start 2023-11-24T08:06:30.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.820+0000 7fdb0cc7c640 1 -- start start 2023-11-24T08:06:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.821+0000 7fdb0cc7c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdb0815b350 0x7fdb080abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.821+0000 7fdb0cc7c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdb0809cb30 0x7fdb080b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.821+0000 7fdb0cc7c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0815b730 con 0x7fdb080adb60 2023-11-24T08:06:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.821+0000 7fdb0cc7c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080ac740 con 0x7fdb0809cb30 2023-11-24T08:06:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.821+0000 7fdb0cc7c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080ac460 con 0x7fdb0815b350 2023-11-24T08:06:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.822+0000 7fdb05d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdb0809cb30 0x7fdb080b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.822+0000 7fdb06d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdb080adb60 0x7fdb080adf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46376/0 (socket says 172.21.15.64:46376) 2023-11-24T08:06:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.822+0000 7fdb06d9d640 1 -- 172.21.15.64:0/1299457320 learned_addr learned my addr 172.21.15.64:0/1299457320 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.822+0000 7fdb0659c640 1 --2- 172.21.15.64:0/1299457320 >> v2:172.21.15.91:3300/0 conn(0x7fdb0815b350 0x7fdb080abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.823+0000 7fdb05d9b640 1 -- 172.21.15.64:0/1299457320 >> v2:172.21.15.91:3300/0 conn(0x7fdb0815b350 msgr2=0x7fdb080abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.823+0000 7fdb05d9b640 1 --2- 172.21.15.64:0/1299457320 >> v2:172.21.15.91:3300/0 conn(0x7fdb0815b350 0x7fdb080abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.823+0000 7fdb05d9b640 1 -- 172.21.15.64:0/1299457320 >> v1:172.21.15.64:6789/0 conn(0x7fdb080adb60 legacy=0x7fdb080adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.823+0000 7fdb05d9b640 1 -- 172.21.15.64:0/1299457320 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080b2a50 con 0x7fdb0809cb30 2023-11-24T08:06:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.823+0000 7fdb0559a640 1 -- 172.21.15.64:0/1299457320 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688004063 0 0) 0x7fdb00001490 con 0x7fdb080adb60 2023-11-24T08:06:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.824+0000 7fdb05d9b640 1 --2- 172.21.15.64:0/1299457320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0809cb30 0x7fdb080b3530 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdb080b5100 tx=0x7fdafc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7171c52682c76d3d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.824+0000 7fdb0559a640 1 -- 172.21.15.64:0/1299457320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc031830 con 0x7fdb0809cb30 2023-11-24T08:06:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.824+0000 7fdb0559a640 1 -- 172.21.15.64:0/1299457320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc031990 con 0x7fdb0809cb30 2023-11-24T08:06:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.824+0000 7fdb0559a640 1 -- 172.21.15.64:0/1299457320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc031d20 con 0x7fdb0809cb30 2023-11-24T08:06:30.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.825+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/1299457320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0809cb30 msgr2=0x7fdb080b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.825+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/1299457320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0809cb30 0x7fdb080b3530 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdb080b5100 tx=0x7fdafc02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.825+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/1299457320 shutdown_connections 2023-11-24T08:06:30.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.825+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/1299457320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0809cb30 0x7fdb080b3530 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.825+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/1299457320 >> v2:172.21.15.91:3300/0 conn(0x7fdb0815b350 0x7fdb080abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.825+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/1299457320 >> 172.21.15.64:0/1299457320 conn(0x7fdb080b0ba0 msgr2=0x7fdb080b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:30.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.825+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/1299457320 shutdown_connections 2023-11-24T08:06:30.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.825+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/1299457320 wait complete. 2023-11-24T08:06:30.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.827+0000 7fdb0cc7c640 1 Processor -- start 2023-11-24T08:06:30.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.827+0000 7fdb0cc7c640 1 -- start start 2023-11-24T08:06:30.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb0cc7c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdb0809cb30 0x7fdb081f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb0cc7c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb080adb60 0x7fdb08153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb0cc7c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb0815b350 0x7fdb0815d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb0cc7c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb08154650 con 0x7fdb080adb60 2023-11-24T08:06:30.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb0cc7c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb08154930 con 0x7fdb0815b350 2023-11-24T08:06:30.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb0cc7c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb08154c10 con 0x7fdb0809cb30 2023-11-24T08:06:30.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb05d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb080adb60 0x7fdb08153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb05d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb080adb60 0x7fdb08153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56950/0 (socket says 172.21.15.64:56950) 2023-11-24T08:06:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb05d9b640 1 -- 172.21.15.64:0/2260612725 learned_addr learned my addr 172.21.15.64:0/2260612725 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb05d9b640 1 -- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdb0809cb30 msgr2=0x7fdb081f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb05d9b640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdb0809cb30 0x7fdb081f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb05d9b640 1 -- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb0815b350 msgr2=0x7fdb0815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb05d9b640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb0815b350 0x7fdb0815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.828+0000 7fdb05d9b640 1 -- 172.21.15.64:0/2260612725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009400 con 0x7fdb080adb60 2023-11-24T08:06:30.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.829+0000 7fdb05d9b640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb080adb60 0x7fdb08153ce0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009890 tx=0x7fdafc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:30.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.829+0000 7fdaf6ffd640 1 -- 172.21.15.64:0/2260612725 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc03a6f0 con 0x7fdb080adb60 2023-11-24T08:06:30.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.829+0000 7fdaf6ffd640 1 -- 172.21.15.64:0/2260612725 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc03a870 con 0x7fdb080adb60 2023-11-24T08:06:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.829+0000 7fdaf6ffd640 1 -- 172.21.15.64:0/2260612725 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc03ac20 con 0x7fdb080adb60 2023-11-24T08:06:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.829+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb08154ef0 con 0x7fdb080adb60 2023-11-24T08:06:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.829+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb08154220 con 0x7fdb080adb60 2023-11-24T08:06:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.831+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb081556f0 con 0x7fdb080adb60 2023-11-24T08:06:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.831+0000 7fdaf6ffd640 1 -- 172.21.15.64:0/2260612725 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdafc043410 con 0x7fdb080adb60 2023-11-24T08:06:30.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.833+0000 7fdaf6ffd640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdae806af60 0x7fdae806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:30.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.833+0000 7fdaf6ffd640 1 -- 172.21.15.64:0/2260612725 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdafc0ab530 con 0x7fdb080adb60 2023-11-24T08:06:30.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.833+0000 7fdb0659c640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdae806af60 0x7fdae806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.835+0000 7fdb0659c640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdae806af60 0x7fdae806d420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8006a90 tx=0x7fdaf8006a20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:30.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:30.838+0000 7fdaf6ffd640 1 -- 172.21.15.64:0/2260612725 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdafc0764e0 con 0x7fdb080adb60 2023-11-24T08:06:31.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.070+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol9602", "group_name": "subvol_grp934", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb080b3a70 con 0x7fdae806af60 2023-11-24T08:06:31.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.142+0000 7fdaf6ffd640 1 -- 172.21.15.64:0/2260612725 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+701 (secure 0 0 0) 0x7fdaf8009620 con 0x7fdae806af60 2023-11-24T08:06:31.143 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:06:31.143 INFO:teuthology.orchestra.run.smithi064.stdout: "atime": "2023-11-24 08:06:27", 2023-11-24T08:06:31.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bytes_pcent": "undefined", 2023-11-24T08:06:31.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bytes_quota": "infinite", 2023-11-24T08:06:31.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bytes_used": 0, 2023-11-24T08:06:31.143 INFO:teuthology.orchestra.run.smithi064.stdout: "created_at": "2023-11-24 08:06:27", 2023-11-24T08:06:31.143 INFO:teuthology.orchestra.run.smithi064.stdout: "ctime": "2023-11-24 08:06:27", 2023-11-24T08:06:31.143 INFO:teuthology.orchestra.run.smithi064.stdout: "data_pool": "cephfs_data", 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "features": [ 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot-clone", 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot-autoprotect", 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot-retention" 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 0, 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": 16877, 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "mon_addrs": [ 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "172.21.15.64:6789", 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "172.21.15.91:6789", 2023-11-24T08:06:31.144 INFO:teuthology.orchestra.run.smithi064.stdout: "172.21.15.91:6790" 2023-11-24T08:06:31.145 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-11-24T08:06:31.145 INFO:teuthology.orchestra.run.smithi064.stdout: "mtime": "2023-11-24 08:06:27", 2023-11-24T08:06:31.145 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/subvol_grp934/subvol9602/a825d5aa-f0f6-413e-9a64-c88ae8de51f6", 2023-11-24T08:06:31.145 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_namespace": "", 2023-11-24T08:06:31.145 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "complete", 2023-11-24T08:06:31.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "subvolume", 2023-11-24T08:06:31.145 INFO:teuthology.orchestra.run.smithi064.stdout: "uid": 0 2023-11-24T08:06:31.145 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:06:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdae806af60 msgr2=0x7fdae806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdae806af60 0x7fdae806d420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8006a90 tx=0x7fdaf8006a20 comp rx=0 tx=0).stop 2023-11-24T08:06:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb080adb60 msgr2=0x7fdb08153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb080adb60 0x7fdb08153ce0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009890 tx=0x7fdafc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 shutdown_connections 2023-11-24T08:06:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdae806af60 0x7fdae806d420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdb0815b350 0x7fdb0815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb080adb60 0x7fdb08153ce0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 --2- 172.21.15.64:0/2260612725 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdb0809cb30 0x7fdb081f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 >> 172.21.15.64:0/2260612725 conn(0x7fdb080b0ba0 msgr2=0x7fdb0809ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.145+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 shutdown_connections 2023-11-24T08:06:31.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.146+0000 7fdb0cc7c640 1 -- 172.21.15.64:0/2260612725 wait complete. 2023-11-24T08:06:31.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:31.166 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap4432 ./volumes/subvol_grp934/subvol9602/.meta) 2023-11-24T08:06:31.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:06:31.283 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol9602 subvol_grp934 2023-11-24T08:06:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.518+0000 7f705f59e640 1 Processor -- start 2023-11-24T08:06:31.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.518+0000 7f705f59e640 1 -- start start 2023-11-24T08:06:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.520+0000 7f705f59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f706006dae0 0x7f706015b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.520+0000 7f705f59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f706015bbd0 0x7f7060070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.520+0000 7f705f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7060144ca0 con 0x7f70601448c0 2023-11-24T08:06:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.520+0000 7f705f59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706015c450 con 0x7f706006dae0 2023-11-24T08:06:31.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.520+0000 7f705f59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70600715b0 con 0x7f706015bbd0 2023-11-24T08:06:31.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.520+0000 7f705dd9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f706006dae0 0x7f706015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:31.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.520+0000 7f705dd9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f706006dae0 0x7f706015b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56964/0 (socket says 172.21.15.64:56964) 2023-11-24T08:06:31.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.520+0000 7f705dd9b640 1 -- 172.21.15.64:0/4024192094 learned_addr learned my addr 172.21.15.64:0/4024192094 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:31.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.521+0000 7f705ed9d640 1 --2- 172.21.15.64:0/4024192094 >> v2:172.21.15.91:3300/0 conn(0x7f706015bbd0 0x7f7060070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:31.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.521+0000 7f705dd9b640 1 -- 172.21.15.64:0/4024192094 >> v2:172.21.15.91:3300/0 conn(0x7f706015bbd0 msgr2=0x7f7060070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:31.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.521+0000 7f705dd9b640 1 --2- 172.21.15.64:0/4024192094 >> v2:172.21.15.91:3300/0 conn(0x7f706015bbd0 0x7f7060070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.521+0000 7f705dd9b640 1 -- 172.21.15.64:0/4024192094 >> v1:172.21.15.64:6789/0 conn(0x7f70601448c0 legacy=0x7f706006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:31.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.521+0000 7f705dd9b640 1 -- 172.21.15.64:0/4024192094 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7060071890 con 0x7f706006dae0 2023-11-24T08:06:31.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.522+0000 7f705dd9b640 1 --2- 172.21.15.64:0/4024192094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706006dae0 0x7f706015b690 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f70600b41b0 tx=0x7f705402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e55af4f9d7d0b15 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.522+0000 7f705d59a640 1 -- 172.21.15.64:0/4024192094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054004230 con 0x7f706006dae0 2023-11-24T08:06:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.522+0000 7f705d59a640 1 -- 172.21.15.64:0/4024192094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054004390 con 0x7f706006dae0 2023-11-24T08:06:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.522+0000 7f705d59a640 1 -- 172.21.15.64:0/4024192094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054030840 con 0x7f706006dae0 2023-11-24T08:06:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.523+0000 7f705f59e640 1 -- 172.21.15.64:0/4024192094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706006dae0 msgr2=0x7f706015b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:31.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.523+0000 7f705f59e640 1 --2- 172.21.15.64:0/4024192094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706006dae0 0x7f706015b690 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f70600b41b0 tx=0x7f705402b980 comp rx=0 tx=0).stop 2023-11-24T08:06:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.523+0000 7f705f59e640 1 -- 172.21.15.64:0/4024192094 shutdown_connections 2023-11-24T08:06:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.523+0000 7f705f59e640 1 --2- 172.21.15.64:0/4024192094 >> v2:172.21.15.91:3300/0 conn(0x7f706015bbd0 0x7f7060070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.523+0000 7f705f59e640 1 --2- 172.21.15.64:0/4024192094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706006dae0 0x7f706015b690 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.523+0000 7f705f59e640 1 -- 172.21.15.64:0/4024192094 >> 172.21.15.64:0/4024192094 conn(0x7f70600b0aa0 msgr2=0x7f70600b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.523+0000 7f705f59e640 1 -- 172.21.15.64:0/4024192094 shutdown_connections 2023-11-24T08:06:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.523+0000 7f705f59e640 1 -- 172.21.15.64:0/4024192094 wait complete. 2023-11-24T08:06:31.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.525+0000 7f705f59e640 1 Processor -- start 2023-11-24T08:06:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.525+0000 7f705f59e640 1 -- start start 2023-11-24T08:06:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f706006dae0 0x7f706013bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70601448c0 0x7f70601f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706015bbd0 0x7f706013d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:31.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601f5df0 con 0x7f706015bbd0 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601f60d0 con 0x7f70601448c0 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70601f63b0 con 0x7f706006dae0 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706015bbd0 0x7f706013d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706015bbd0 0x7f706013d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56972/0 (socket says 172.21.15.64:56972) 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.526+0000 7f705ed9d640 1 -- 172.21.15.64:0/1587321465 learned_addr learned my addr 172.21.15.64:0/1587321465 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f705dd9b640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70601448c0 0x7f70601f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f705ed9d640 1 -- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f706006dae0 msgr2=0x7f706013bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f705ed9d640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f706006dae0 0x7f706013bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f705ed9d640 1 -- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70601448c0 msgr2=0x7f70601f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f705ed9d640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70601448c0 0x7f70601f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f705ed9d640 1 -- 172.21.15.64:0/1587321465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054009460 con 0x7f706015bbd0 2023-11-24T08:06:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f705dd9b640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70601448c0 0x7f70601f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f705ed9d640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706015bbd0 0x7f706013d680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7058007e60 tx=0x7f705800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f704effd640 1 -- 172.21.15.64:0/1587321465 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058014420 con 0x7f706015bbd0 2023-11-24T08:06:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f704effd640 1 -- 172.21.15.64:0/1587321465 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70580145a0 con 0x7f706015bbd0 2023-11-24T08:06:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.527+0000 7f704effd640 1 -- 172.21.15.64:0/1587321465 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058014950 con 0x7f706015bbd0 2023-11-24T08:06:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.528+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70600724f0 con 0x7f706015bbd0 2023-11-24T08:06:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.528+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70601f6970 con 0x7f706015bbd0 2023-11-24T08:06:31.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.529+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70600b1ea0 con 0x7f706015bbd0 2023-11-24T08:06:31.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.530+0000 7f704effd640 1 -- 172.21.15.64:0/1587321465 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f7058014ab0 con 0x7f706015bbd0 2023-11-24T08:06:31.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.531+0000 7f704effd640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f704006af60 0x7f704006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:31.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.531+0000 7f704effd640 1 -- 172.21.15.64:0/1587321465 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f705808a0f0 con 0x7f706015bbd0 2023-11-24T08:06:31.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.532+0000 7f705e59c640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f704006af60 0x7f704006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:31.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.532+0000 7f705e59c640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f704006af60 0x7f704006d420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7050000f60 tx=0x7f70500079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.535+0000 7f704effd640 1 -- 172.21.15.64:0/1587321465 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70600b1ea0 con 0x7f706015bbd0 2023-11-24T08:06:31.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.760+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol9602", "group_name": "subvol_grp934", "target": ["mon-mgr", ""]}) v1 -- 0x7f7060072370 con 0x7f704006af60 2023-11-24T08:06:31.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.804+0000 7f704effd640 1 -- 172.21.15.64:0/1587321465 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7050005080 con 0x7f704006af60 2023-11-24T08:06:31.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f704006af60 msgr2=0x7f704006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:31.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f704006af60 0x7f704006d420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7050000f60 tx=0x7f70500079b0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706015bbd0 msgr2=0x7f706013d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:31.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706015bbd0 0x7f706013d680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7058007e60 tx=0x7f705800e4b0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 shutdown_connections 2023-11-24T08:06:31.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f704006af60 0x7f704006d420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f706015bbd0 0x7f706013d680 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70601448c0 0x7f70601f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 --2- 172.21.15.64:0/1587321465 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f706006dae0 0x7f706013bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.807+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 >> 172.21.15.64:0/1587321465 conn(0x7f70600b0aa0 msgr2=0x7f706006ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.808+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 shutdown_connections 2023-11-24T08:06:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:31.808+0000 7f705f59e640 1 -- 172.21.15.64:0/1587321465 wait complete. 2023-11-24T08:06:31.815 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.814+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.816 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.814+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.816 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.814+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.816 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.814+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.816 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.814+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.832 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp934 2023-11-24T08:06:31.847 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.845+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.847 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.845+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.847 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.845+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.847 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.845+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:31.847 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:06:31.845+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:06:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.058+0000 7faff2d1a640 1 Processor -- start 2023-11-24T08:06:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.058+0000 7faff2d1a640 1 -- start start 2023-11-24T08:06:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.059+0000 7faff2d1a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fafec1435d0 0x7fafec071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.059+0000 7faff2d1a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec070d40 con 0x7fafec0b4140 2023-11-24T08:06:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.060+0000 7faff2d1a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec071020 con 0x7fafec1439b0 2023-11-24T08:06:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.060+0000 7faff2d1a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafec071300 con 0x7fafec1435d0 2023-11-24T08:06:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.060+0000 7faff2519640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fafec0b4140 0x7fafec070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46402/0 (socket says 172.21.15.64:46402) 2023-11-24T08:06:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.060+0000 7faff2519640 1 -- 172.21.15.64:0/2143603291 learned_addr learned my addr 172.21.15.64:0/2143603291 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.060+0000 7faff1d18640 1 --2- 172.21.15.64:0/2143603291 >> v2:172.21.15.91:3300/0 conn(0x7fafec1435d0 0x7fafec071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.060+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678304914 0 0) 0x7fafec070d40 con 0x7fafec0b4140 2023-11-24T08:06:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.060+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafd8003520 con 0x7fafec0b4140 2023-11-24T08:06:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 595986453 0 0) 0x7fafd8003520 con 0x7fafec0b4140 2023-11-24T08:06:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 >> v2:172.21.15.91:3300/0 conn(0x7fafec1435d0 msgr2=0x7fafec071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 --2- 172.21.15.64:0/2143603291 >> v2:172.21.15.91:3300/0 conn(0x7fafec1435d0 0x7fafec071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 >> v1:172.21.15.91:6789/0 conn(0x7fafec1439b0 legacy=0x7fafec0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec0715e0 con 0x7fafec0b4140 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fafe40019f0 con 0x7fafec0b4140 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff1d18640 1 --2- 172.21.15.64:0/2143603291 >> v2:172.21.15.91:3300/0 conn(0x7fafec1435d0 0x7fafec071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 >> v1:172.21.15.64:6789/0 conn(0x7fafec0b4140 legacy=0x7fafec070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd80039e0 0x7fafd8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fafd80047c0 0x7fafd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd80052a0 0x7fafd800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec071300 con 0x7fafd80039e0 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd800e400 con 0x7fafd80052a0 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd800e5c0 con 0x7fafd80047c0 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.061+0000 7faff1d18640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd80039e0 0x7fafd8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.062+0000 7faff2519640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd80052a0 0x7fafd800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.062+0000 7faff1d18640 1 -- 172.21.15.64:0/2143603291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fafd80047c0 msgr2=0x7fafd8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.062+0000 7faff1d18640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fafd80047c0 0x7fafd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.062+0000 7faff1d18640 1 -- 172.21.15.64:0/2143603291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd80052a0 msgr2=0x7fafd800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.062+0000 7faff1d18640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd80052a0 0x7fafd800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.062+0000 7faff1d18640 1 -- 172.21.15.64:0/2143603291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd8003520 con 0x7fafd80039e0 2023-11-24T08:06:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.062+0000 7faff2519640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafd80052a0 0x7fafd800db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.062+0000 7faff2519640 1 -- 172.21.15.64:0/2143603291 reap_dead start 2023-11-24T08:06:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.063+0000 7faff1d18640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd80039e0 0x7fafd8003e00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fafe8006fd0 tx=0x7fafd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7104d31c239af6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.063+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8004030 con 0x7fafd80039e0 2023-11-24T08:06:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.063+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe8004190 con 0x7fafd80039e0 2023-11-24T08:06:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.063+0000 7faff0d16640 1 -- 172.21.15.64:0/2143603291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8003610 con 0x7fafd80039e0 2023-11-24T08:06:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.064+0000 7faff2d1a640 1 -- 172.21.15.64:0/2143603291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd80039e0 msgr2=0x7fafd8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:32.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.064+0000 7faff2d1a640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd80039e0 0x7fafd8003e00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fafe8006fd0 tx=0x7fafd80057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.064+0000 7faff2d1a640 1 -- 172.21.15.64:0/2143603291 shutdown_connections 2023-11-24T08:06:32.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.064+0000 7faff2d1a640 1 --2- 172.21.15.64:0/2143603291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd80039e0 0x7fafd8003e00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.064+0000 7faff2d1a640 1 -- 172.21.15.64:0/2143603291 >> 172.21.15.64:0/2143603291 conn(0x7fafec0cec10 msgr2=0x7fafec0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:32.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.064+0000 7faff2d1a640 1 -- 172.21.15.64:0/2143603291 shutdown_connections 2023-11-24T08:06:32.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.064+0000 7faff2d1a640 1 -- 172.21.15.64:0/2143603291 wait complete. 2023-11-24T08:06:32.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.066+0000 7faff2d1a640 1 Processor -- start 2023-11-24T08:06:32.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.066+0000 7faff2d1a640 1 -- start start 2023-11-24T08:06:32.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2d1a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fafd80039e0 0x7fafec15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:32.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2d1a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafec1e97f0 0x7fafec06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:32.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2d1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec1e6ef0 0x7fafec1e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:32.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2d1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec1ea9e0 con 0x7fafec1e6ef0 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2d1a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec142a00 con 0x7fafec1e97f0 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2d1a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec15c190 con 0x7fafd80039e0 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2519640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec1e6ef0 0x7fafec1e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2519640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec1e6ef0 0x7fafec1e7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56984/0 (socket says 172.21.15.64:56984) 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff2519640 1 -- 172.21.15.64:0/3861895754 learned_addr learned my addr 172.21.15.64:0/3861895754 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.067+0000 7faff1d18640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fafd80039e0 0x7fafec15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff2519640 1 -- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fafd80039e0 msgr2=0x7fafec15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff1517640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafec1e97f0 0x7fafec06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff2519640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fafd80039e0 0x7fafec15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff2519640 1 -- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafec1e97f0 msgr2=0x7fafec06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:32.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff2519640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafec1e97f0 0x7fafec06ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff2519640 1 -- 172.21.15.64:0/3861895754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec1e78b0 con 0x7fafec1e6ef0 2023-11-24T08:06:32.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff1517640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafec1e97f0 0x7fafec06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:32.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff2519640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec1e6ef0 0x7fafec1e7370 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fafec15db60 tx=0x7fafe400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:32.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7fafe27fc640 1 -- 172.21.15.64:0/3861895754 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4004d60 con 0x7fafec1e6ef0 2023-11-24T08:06:32.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7fafe27fc640 1 -- 172.21.15.64:0/3861895754 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe400bb20 con 0x7fafec1e6ef0 2023-11-24T08:06:32.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7fafe27fc640 1 -- 172.21.15.64:0/3861895754 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400bed0 con 0x7fafec1e6ef0 2023-11-24T08:06:32.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.068+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec1e9d60 con 0x7fafec1e6ef0 2023-11-24T08:06:32.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.069+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec1ea320 con 0x7fafec1e6ef0 2023-11-24T08:06:32.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.070+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec12e090 con 0x7fafec1e6ef0 2023-11-24T08:06:32.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.071+0000 7fafe27fc640 1 -- 172.21.15.64:0/3861895754 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fafe4007070 con 0x7fafec1e6ef0 2023-11-24T08:06:32.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.072+0000 7fafe27fc640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fafd006af90 0x7fafd006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:32.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.072+0000 7fafe27fc640 1 -- 172.21.15.64:0/3861895754 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafe40899f0 con 0x7fafec1e6ef0 2023-11-24T08:06:32.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.072+0000 7faff1d18640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fafd006af90 0x7fafd006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.074+0000 7faff1d18640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fafd006af90 0x7fafd006d450 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fafe8035000 tx=0x7fafe803a040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:32.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.076+0000 7fafe27fc640 1 -- 172.21.15.64:0/3861895754 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fafec12e090 con 0x7fafec1e6ef0 2023-11-24T08:06:32.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.300+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp934", "target": ["mon-mgr", ""]}) v1 -- 0x7fafec12e090 con 0x7fafd006af90 2023-11-24T08:06:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.309+0000 7fafe27fc640 1 -- 172.21.15.64:0/3861895754 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fafec12e090 con 0x7fafd006af90 2023-11-24T08:06:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fafd006af90 msgr2=0x7fafd006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fafd006af90 0x7fafd006d450 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fafe8035000 tx=0x7fafe803a040 comp rx=0 tx=0).stop 2023-11-24T08:06:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec1e6ef0 msgr2=0x7fafec1e7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec1e6ef0 0x7fafec1e7370 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fafec15db60 tx=0x7fafe400b4a0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 shutdown_connections 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fafd006af90 0x7fafd006d450 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec1e6ef0 0x7fafec1e7370 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fafec1e97f0 0x7fafec06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 --2- 172.21.15.64:0/3861895754 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fafd80039e0 0x7fafec15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 >> 172.21.15.64:0/3861895754 conn(0x7fafec0cec10 msgr2=0x7fafec0ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 shutdown_connections 2023-11-24T08:06:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:32.312+0000 7faff2d1a640 1 -- 172.21.15.64:0/3861895754 wait complete. 2023-11-24T08:06:32.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:32.332 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T08:06:32.398 INFO:teuthology.orchestra.run.smithi064.stdout:2 2023-11-24T08:06:32.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:32.400 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rm -f ./volumes/subvol_grp934/subvol9602/.meta.stale_snap_section) 2023-11-24T08:06:32.505 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:06:32.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:06:32.626 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:06:32.743 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:06:32.744 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:06:32.744 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:06:32.808 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.044+0000 7fad65d63640 1 Processor -- start 2023-11-24T08:06:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.044+0000 7fad65d63640 1 -- start start 2023-11-24T08:06:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad65d63640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad6013dfd0 0x7fad6015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad65d63640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad600cf050 0x7fad600cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad65d63640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad600ce7c0 con 0x7fad6006c130 2023-11-24T08:06:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad65d63640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad600cfa10 con 0x7fad6013dfd0 2023-11-24T08:06:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad65d63640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad60140900 con 0x7fad600cf050 2023-11-24T08:06:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad64d61640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad6013dfd0 0x7fad6015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad64d61640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad6013dfd0 0x7fad6015be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56996/0 (socket says 172.21.15.64:56996) 2023-11-24T08:06:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad64d61640 1 -- 172.21.15.64:0/265239616 learned_addr learned my addr 172.21.15.64:0/265239616 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad64d61640 1 -- 172.21.15.64:0/265239616 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad600cf050 msgr2=0x7fad600cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad64d61640 1 --2- 172.21.15.64:0/265239616 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad600cf050 0x7fad600cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.046+0000 7fad64d61640 1 -- 172.21.15.64:0/265239616 >> v1:172.21.15.91:6789/0 conn(0x7fad6006c130 legacy=0x7fad6006c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.047+0000 7fad64d61640 1 -- 172.21.15.64:0/265239616 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6015fa70 con 0x7fad6013dfd0 2023-11-24T08:06:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.048+0000 7fad64d61640 1 --2- 172.21.15.64:0/265239616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad6015be70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fad54009870 tx=0x7fad5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf83d02c4fa3c400 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.048+0000 7fad5f7fe640 1 -- 172.21.15.64:0/265239616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54031640 con 0x7fad6013dfd0 2023-11-24T08:06:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.048+0000 7fad5f7fe640 1 -- 172.21.15.64:0/265239616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad540317a0 con 0x7fad6013dfd0 2023-11-24T08:06:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.048+0000 7fad5f7fe640 1 -- 172.21.15.64:0/265239616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54031b30 con 0x7fad6013dfd0 2023-11-24T08:06:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.049+0000 7fad65d63640 1 -- 172.21.15.64:0/265239616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 msgr2=0x7fad6015be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.049+0000 7fad65d63640 1 --2- 172.21.15.64:0/265239616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad6015be70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fad54009870 tx=0x7fad5402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.049+0000 7fad65d63640 1 -- 172.21.15.64:0/265239616 shutdown_connections 2023-11-24T08:06:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.049+0000 7fad65d63640 1 --2- 172.21.15.64:0/265239616 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad600cf050 0x7fad600cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.049+0000 7fad65d63640 1 --2- 172.21.15.64:0/265239616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad6015be70 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.049+0000 7fad65d63640 1 -- 172.21.15.64:0/265239616 >> 172.21.15.64:0/265239616 conn(0x7fad6006a860 msgr2=0x7fad6006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.049+0000 7fad65d63640 1 -- 172.21.15.64:0/265239616 shutdown_connections 2023-11-24T08:06:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.049+0000 7fad65d63640 1 -- 172.21.15.64:0/265239616 wait complete. 2023-11-24T08:06:33.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.051+0000 7fad65d63640 1 Processor -- start 2023-11-24T08:06:33.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.051+0000 7fad65d63640 1 -- start start 2023-11-24T08:06:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65d63640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad6006c130 0x7fad600ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65d63640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad600cf050 0x7fad60130440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65d63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad60159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65d63640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad60158ad0 con 0x7fad6013dfd0 2023-11-24T08:06:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65d63640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad60130980 con 0x7fad600cf050 2023-11-24T08:06:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65d63640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad600cac10 con 0x7fad6006c130 2023-11-24T08:06:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65562640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad60159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65562640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad60159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56998/0 (socket says 172.21.15.64:56998) 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65562640 1 -- 172.21.15.64:0/2137511263 learned_addr learned my addr 172.21.15.64:0/2137511263 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad64d61640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad6006c130 0x7fad600ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65562640 1 -- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad6006c130 msgr2=0x7fad600ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65562640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad6006c130 0x7fad600ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65562640 1 -- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad600cf050 msgr2=0x7fad60130440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65562640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad600cf050 0x7fad60130440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.052+0000 7fad65562640 1 -- 172.21.15.64:0/2137511263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540094d0 con 0x7fad6013dfd0 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.053+0000 7fad65562640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad60159400 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fad5800c780 tx=0x7fad5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.053+0000 7fad5d7fa640 1 -- 172.21.15.64:0/2137511263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58004b60 con 0x7fad6013dfd0 2023-11-24T08:06:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.053+0000 7fad5d7fa640 1 -- 172.21.15.64:0/2137511263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5800ce90 con 0x7fad6013dfd0 2023-11-24T08:06:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.053+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad600caef0 con 0x7fad6013dfd0 2023-11-24T08:06:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.053+0000 7fad5d7fa640 1 -- 172.21.15.64:0/2137511263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5800a850 con 0x7fad6013dfd0 2023-11-24T08:06:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.053+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad60155dd0 con 0x7fad6013dfd0 2023-11-24T08:06:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.054+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6004e840 con 0x7fad6013dfd0 2023-11-24T08:06:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.055+0000 7fad5d7fa640 1 -- 172.21.15.64:0/2137511263 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fad5800a9b0 con 0x7fad6013dfd0 2023-11-24T08:06:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.056+0000 7fad5d7fa640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad4406b080 0x7fad4406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.057+0000 7fad5d7fa640 1 -- 172.21.15.64:0/2137511263 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad58089cc0 con 0x7fad6013dfd0 2023-11-24T08:06:33.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.057+0000 7fad64d61640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad4406b080 0x7fad4406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.058+0000 7fad64d61640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad4406b080 0x7fad4406d540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fad54036000 tx=0x7fad5403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.061+0000 7fad5d7fa640 1 -- 172.21.15.64:0/2137511263 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad58090080 con 0x7fad6013dfd0 2023-11-24T08:06:33.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.258+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad600ce9a0 con 0x7fad6013dfd0 2023-11-24T08:06:33.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.260+0000 7fad5d7fa640 1 -- 172.21.15.64:0/2137511263 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16465 (secure 0 0 0) 0x7fad58054970 con 0x7fad6013dfd0 2023-11-24T08:06:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:33.261 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:06.076753+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:02.427519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:03.560696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad4406b080 msgr2=0x7fad4406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad4406b080 0x7fad4406d540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fad54036000 tx=0x7fad5403c040 comp rx=0 tx=0).stop 2023-11-24T08:06:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 msgr2=0x7fad60159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad60159400 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fad5800c780 tx=0x7fad5800cc40 comp rx=0 tx=0).stop 2023-11-24T08:06:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 shutdown_connections 2023-11-24T08:06:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fad4406b080 0x7fad4406d540 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad6013dfd0 0x7fad60159400 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fad600cf050 0x7fad60130440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 --2- 172.21.15.64:0/2137511263 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fad6006c130 0x7fad600ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 >> 172.21.15.64:0/2137511263 conn(0x7fad6006a860 msgr2=0x7fad6015aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 shutdown_connections 2023-11-24T08:06:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.263+0000 7fad65d63640 1 -- 172.21.15.64:0/2137511263 wait complete. 2023-11-24T08:06:33.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:06:02.427519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:06:03.560696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:06:33.283 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:06:33.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.501+0000 7f2fe3b97640 1 Processor -- start 2023-11-24T08:06:33.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.501+0000 7f2fe3b97640 1 -- start start 2023-11-24T08:06:33.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.502+0000 7f2fe3b97640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2fdc06dc30 0x7f2fdc0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.502+0000 7f2fe3b97640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2fdc070490 0x7f2fdc070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.502+0000 7f2fe3b97640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc06d7a0 con 0x7f2fdc0acd70 2023-11-24T08:06:33.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.502+0000 7f2fe3b97640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc15a5b0 con 0x7f2fdc06dc30 2023-11-24T08:06:33.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.502+0000 7f2fe3b97640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc0ae9d0 con 0x7f2fdc070490 2023-11-24T08:06:33.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.503+0000 7f2fe1132640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2fdc06dc30 0x7f2fdc0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.503+0000 7f2fe1132640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2fdc06dc30 0x7f2fdc0ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57000/0 (socket says 172.21.15.64:57000) 2023-11-24T08:06:33.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.503+0000 7f2fe1132640 1 -- 172.21.15.64:0/4142980837 learned_addr learned my addr 172.21.15.64:0/4142980837 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:33.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.503+0000 7f2fe2134640 1 --2- 172.21.15.64:0/4142980837 >> v2:172.21.15.91:3300/0 conn(0x7f2fdc070490 0x7f2fdc070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.503+0000 7f2fe1132640 1 -- 172.21.15.64:0/4142980837 >> v2:172.21.15.91:3300/0 conn(0x7f2fdc070490 msgr2=0x7f2fdc070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:33.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.503+0000 7f2fe1132640 1 --2- 172.21.15.64:0/4142980837 >> v2:172.21.15.91:3300/0 conn(0x7f2fdc070490 0x7f2fdc070900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.503+0000 7f2fe1132640 1 -- 172.21.15.64:0/4142980837 >> v1:172.21.15.91:6789/0 conn(0x7f2fdc0acd70 legacy=0x7f2fdc0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:33.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.503+0000 7f2fe1132640 1 -- 172.21.15.64:0/4142980837 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc06ce70 con 0x7f2fdc06dc30 2023-11-24T08:06:33.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.505+0000 7f2fe1132640 1 --2- 172.21.15.64:0/4142980837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc06dc30 0x7f2fdc0ae490 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc067e70 tx=0x7f2fcc02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc48544c100e20c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.505+0000 7f2fe0931640 1 -- 172.21.15.64:0/4142980837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc02ee90 con 0x7f2fdc06dc30 2023-11-24T08:06:33.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.505+0000 7f2fe0931640 1 -- 172.21.15.64:0/4142980837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc036440 con 0x7f2fdc06dc30 2023-11-24T08:06:33.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.505+0000 7f2fe0931640 1 -- 172.21.15.64:0/4142980837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0367d0 con 0x7f2fdc06dc30 2023-11-24T08:06:33.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.506+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4142980837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc06dc30 msgr2=0x7f2fdc0ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.506+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4142980837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc06dc30 0x7f2fdc0ae490 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc067e70 tx=0x7f2fcc02e0e0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.506+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4142980837 shutdown_connections 2023-11-24T08:06:33.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.506+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4142980837 >> v2:172.21.15.91:3300/0 conn(0x7f2fdc070490 0x7f2fdc070900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.506+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4142980837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc06dc30 0x7f2fdc0ae490 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.506+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4142980837 >> 172.21.15.64:0/4142980837 conn(0x7f2fdc0b0ba0 msgr2=0x7f2fdc0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:33.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.506+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4142980837 shutdown_connections 2023-11-24T08:06:33.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.506+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4142980837 wait complete. 2023-11-24T08:06:33.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.508+0000 7f2fe3b97640 1 Processor -- start 2023-11-24T08:06:33.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.508+0000 7f2fe3b97640 1 -- start start 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe3b97640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fdc06dc30 0x7f2fdc159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe3b97640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2fdc070490 0x7f2fdc145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe3b97640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc0acd70 0x7f2fdc0aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe3b97640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc146080 con 0x7f2fdc0acd70 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe3b97640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1593b0 con 0x7f2fdc06dc30 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe3b97640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1587a0 con 0x7f2fdc070490 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe2134640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc0acd70 0x7f2fdc0aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe2134640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc0acd70 0x7f2fdc0aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57006/0 (socket says 172.21.15.64:57006) 2023-11-24T08:06:33.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe2134640 1 -- 172.21.15.64:0/4172871521 learned_addr learned my addr 172.21.15.64:0/4172871521 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:33.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe1933640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fdc06dc30 0x7f2fdc159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe1132640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2fdc070490 0x7f2fdc145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe2134640 1 -- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2fdc070490 msgr2=0x7f2fdc145b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe2134640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2fdc070490 0x7f2fdc145b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe2134640 1 -- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fdc06dc30 msgr2=0x7f2fdc159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe2134640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fdc06dc30 0x7f2fdc159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe2134640 1 -- 172.21.15.64:0/4172871521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc00b770 con 0x7f2fdc0acd70 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.509+0000 7f2fe1933640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fdc06dc30 0x7f2fdc159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.510+0000 7f2fe2134640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc0acd70 0x7f2fdc0aec30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4007e40 tx=0x7f2fd400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.510+0000 7f2fd27fc640 1 -- 172.21.15.64:0/4172871521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4014620 con 0x7f2fdc0acd70 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.510+0000 7f2fd27fc640 1 -- 172.21.15.64:0/4172871521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd400ee90 con 0x7f2fdc0acd70 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.510+0000 7f2fd27fc640 1 -- 172.21.15.64:0/4172871521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40149e0 con 0x7f2fdc0acd70 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.510+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc158a80 con 0x7f2fdc0acd70 2023-11-24T08:06:33.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.510+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc159040 con 0x7f2fdc0acd70 2023-11-24T08:06:33.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.511+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc1591d0 con 0x7f2fdc0acd70 2023-11-24T08:06:33.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.512+0000 7f2fd27fc640 1 -- 172.21.15.64:0/4172871521 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2fd4014b40 con 0x7f2fdc0acd70 2023-11-24T08:06:33.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.513+0000 7f2fd27fc640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2fc006b080 0x7f2fc006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.514+0000 7f2fd27fc640 1 -- 172.21.15.64:0/4172871521 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fd408a320 con 0x7f2fdc0acd70 2023-11-24T08:06:33.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.514+0000 7f2fe1933640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2fc006b080 0x7f2fc006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.515+0000 7f2fe1933640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2fc006b080 0x7f2fc006d540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8003eb0 tx=0x7f2fd8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.518+0000 7f2fd27fc640 1 -- 172.21.15.64:0/4172871521 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fd4055140 con 0x7f2fdc0acd70 2023-11-24T08:06:33.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.711+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2fdc0ad810 con 0x7f2fdc0acd70 2023-11-24T08:06:33.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.712+0000 7f2fd27fc640 1 -- 172.21.15.64:0/4172871521 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2fd4058760 con 0x7f2fdc0acd70 2023-11-24T08:06:33.714 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:06:33.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2fc006b080 msgr2=0x7f2fc006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2fc006b080 0x7f2fc006d540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8003eb0 tx=0x7f2fd8009040 comp rx=0 tx=0).stop 2023-11-24T08:06:33.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc0acd70 msgr2=0x7f2fdc0aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc0acd70 0x7f2fdc0aec30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4007e40 tx=0x7f2fd400e4b0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 shutdown_connections 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2fc006b080 0x7f2fc006d540 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fdc0acd70 0x7f2fdc0aec30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2fdc070490 0x7f2fdc145b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 --2- 172.21.15.64:0/4172871521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2fdc06dc30 0x7f2fdc159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.714+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 >> 172.21.15.64:0/4172871521 conn(0x7f2fdc0b0ba0 msgr2=0x7f2fdc0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.715+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 shutdown_connections 2023-11-24T08:06:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.715+0000 7f2fe3b97640 1 -- 172.21.15.64:0/4172871521 wait complete. 2023-11-24T08:06:33.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:06:33.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.957+0000 7f21a4ba0640 1 Processor -- start 2023-11-24T08:06:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.958+0000 7f21a4ba0640 1 -- start start 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.959+0000 7f21a4ba0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 0x7f21a00cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f21a4ba0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21a006d9e0 0x7f21a00cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f21a4ba0640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a015f930 con 0x7f21a0144770 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f21a4ba0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a00b43b0 con 0x7f21a006d9e0 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f21a4ba0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00b3b10 con 0x7f21a015f270 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f219ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21a006d9e0 0x7f21a00cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f219ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21a006d9e0 0x7f21a00cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57008/0 (socket says 172.21.15.64:57008) 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f219ffff640 1 -- 172.21.15.64:0/2109328663 learned_addr learned my addr 172.21.15.64:0/2109328663 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f2197fff640 1 --2- 172.21.15.64:0/2109328663 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 0x7f21a00cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f219ffff640 1 -- 172.21.15.64:0/2109328663 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 msgr2=0x7f21a00cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:33.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.960+0000 7f219ffff640 1 --2- 172.21.15.64:0/2109328663 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 0x7f21a00cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.961+0000 7f219ffff640 1 -- 172.21.15.64:0/2109328663 >> v1:172.21.15.91:6789/0 conn(0x7f21a0144770 legacy=0x7f21a015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.961+0000 7f219ffff640 1 -- 172.21.15.64:0/2109328663 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a00b3df0 con 0x7f21a006d9e0 2023-11-24T08:06:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.962+0000 7f219ffff640 1 --2- 172.21.15.64:0/2109328663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a006d9e0 0x7f21a00cd9e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2198009e50 tx=0x7f219802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6928d81a4c9ea94 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.962+0000 7f219effd640 1 -- 172.21.15.64:0/2109328663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219802cb00 con 0x7f21a006d9e0 2023-11-24T08:06:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.962+0000 7f219effd640 1 -- 172.21.15.64:0/2109328663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219802cc60 con 0x7f21a006d9e0 2023-11-24T08:06:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.962+0000 7f219effd640 1 -- 172.21.15.64:0/2109328663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21980343f0 con 0x7f21a006d9e0 2023-11-24T08:06:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.962+0000 7f21a4ba0640 1 -- 172.21.15.64:0/2109328663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a006d9e0 msgr2=0x7f21a00cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.962+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/2109328663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a006d9e0 0x7f21a00cd9e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2198009e50 tx=0x7f219802bf60 comp rx=0 tx=0).stop 2023-11-24T08:06:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.963+0000 7f21a4ba0640 1 -- 172.21.15.64:0/2109328663 shutdown_connections 2023-11-24T08:06:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.963+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/2109328663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a006d9e0 0x7f21a00cd9e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.963+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/2109328663 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 0x7f21a00cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.963+0000 7f21a4ba0640 1 -- 172.21.15.64:0/2109328663 >> 172.21.15.64:0/2109328663 conn(0x7f21a00cec10 msgr2=0x7f21a00cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.963+0000 7f21a4ba0640 1 -- 172.21.15.64:0/2109328663 shutdown_connections 2023-11-24T08:06:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.963+0000 7f21a4ba0640 1 -- 172.21.15.64:0/2109328663 wait complete. 2023-11-24T08:06:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.964+0000 7f21a4ba0640 1 Processor -- start 2023-11-24T08:06:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.965+0000 7f21a4ba0640 1 -- start start 2023-11-24T08:06:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.965+0000 7f21a4ba0640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21a006d9e0 0x7f21a013a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f21a4ba0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0144770 0x7f21a0141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f21a4ba0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 0x7f21a01f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f21a4ba0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a013f570 con 0x7f21a0144770 2023-11-24T08:06:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f21a4ba0640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a01f45f0 con 0x7f21a006d9e0 2023-11-24T08:06:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f21a4ba0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a013e960 con 0x7f21a015f270 2023-11-24T08:06:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f2197fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0144770 0x7f21a0141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f2197fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0144770 0x7f21a0141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57012/0 (socket says 172.21.15.64:57012) 2023-11-24T08:06:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f2197fff640 1 -- 172.21.15.64:0/1620314415 learned_addr learned my addr 172.21.15.64:0/1620314415 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:33.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f219f7fe640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21a006d9e0 0x7f21a013a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f219ffff640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 0x7f21a01f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f2197fff640 1 -- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 msgr2=0x7f21a01f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f2197fff640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 0x7f21a01f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f2197fff640 1 -- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21a006d9e0 msgr2=0x7f21a013a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:33.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f2197fff640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21a006d9e0 0x7f21a013a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:33.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f2197fff640 1 -- 172.21.15.64:0/1620314415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2198009b30 con 0x7f21a0144770 2023-11-24T08:06:33.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.966+0000 7f219f7fe640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21a006d9e0 0x7f21a013a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:33.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.967+0000 7f2197fff640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0144770 0x7f21a0141160 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f218800d5d0 tx=0x7f218800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.967+0000 7f219cff9640 1 -- 172.21.15.64:0/1620314415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21880125f0 con 0x7f21a0144770 2023-11-24T08:06:33.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.967+0000 7f219cff9640 1 -- 172.21.15.64:0/1620314415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2188012770 con 0x7f21a0144770 2023-11-24T08:06:33.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.967+0000 7f219cff9640 1 -- 172.21.15.64:0/1620314415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188012b20 con 0x7f21a0144770 2023-11-24T08:06:33.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.967+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a013ec40 con 0x7f21a0144770 2023-11-24T08:06:33.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.967+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a013f200 con 0x7f21a0144770 2023-11-24T08:06:33.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.968+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a013f720 con 0x7f21a0144770 2023-11-24T08:06:33.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.969+0000 7f219cff9640 1 -- 172.21.15.64:0/1620314415 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2188012c80 con 0x7f21a0144770 2023-11-24T08:06:33.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.970+0000 7f219cff9640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f218006b080 0x7f218006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:33.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.970+0000 7f219cff9640 1 -- 172.21.15.64:0/1620314415 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2188089fe0 con 0x7f21a0144770 2023-11-24T08:06:33.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.971+0000 7f219f7fe640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f218006b080 0x7f218006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:33.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.972+0000 7f219f7fe640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f218006b080 0x7f218006d540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2190000f60 tx=0x7f2190009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:33.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:33.974+0000 7f219cff9640 1 -- 172.21.15.64:0/1620314415 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21a013f720 con 0x7f21a0144770 2023-11-24T08:06:34.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.173+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21a004e840 con 0x7f21a0144770 2023-11-24T08:06:34.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.174+0000 7f219cff9640 1 -- 172.21.15.64:0/1620314415 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2188054cb0 con 0x7f21a0144770 2023-11-24T08:06:34.176 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:06:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f218006b080 msgr2=0x7f218006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f218006b080 0x7f218006d540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2190000f60 tx=0x7f2190009040 comp rx=0 tx=0).stop 2023-11-24T08:06:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0144770 msgr2=0x7f21a0141160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0144770 0x7f21a0141160 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f218800d5d0 tx=0x7f218800da90 comp rx=0 tx=0).stop 2023-11-24T08:06:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 shutdown_connections 2023-11-24T08:06:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f218006b080 0x7f218006d540 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21a015f270 0x7f21a01f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0144770 0x7f21a0141160 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 --2- 172.21.15.64:0/1620314415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f21a006d9e0 0x7f21a013a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 >> 172.21.15.64:0/1620314415 conn(0x7f21a00cec10 msgr2=0x7f21a00b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 shutdown_connections 2023-11-24T08:06:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.177+0000 7f21a4ba0640 1 -- 172.21.15.64:0/1620314415 wait complete. 2023-11-24T08:06:34.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:06:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.414+0000 7f3803fff640 1 Processor -- start 2023-11-24T08:06:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.414+0000 7f3803fff640 1 -- start start 2023-11-24T08:06:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.416+0000 7f3803fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f380415d730 0x7f38040cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.416+0000 7f3803fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406c480 0x7f38040cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.416+0000 7f3803fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38040d0660 con 0x7f380406b8f0 2023-11-24T08:06:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.416+0000 7f3803fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38040d0380 con 0x7f380415d730 2023-11-24T08:06:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.416+0000 7f3803fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38040cb530 con 0x7f380406c480 2023-11-24T08:06:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.417+0000 7f38037fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406c480 0x7f38040cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.417+0000 7f38037fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406c480 0x7f38040cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:59828/0 (socket says 172.21.15.64:59828) 2023-11-24T08:06:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.417+0000 7f38037fe640 1 -- 172.21.15.64:0/2641311030 learned_addr learned my addr 172.21.15.64:0/2641311030 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.417+0000 7f3802ffd640 1 --2- 172.21.15.64:0/2641311030 >> v2:172.21.15.91:3300/0 conn(0x7f380415d730 0x7f38040cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.418+0000 7f3801ffb640 1 -- 172.21.15.64:0/2641311030 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943226530 0 0) 0x7f38040d0660 con 0x7f380406b8f0 2023-11-24T08:06:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.418+0000 7f3801ffb640 1 -- 172.21.15.64:0/2641311030 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ec003520 con 0x7f380406b8f0 2023-11-24T08:06:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.418+0000 7f38037fe640 1 -- 172.21.15.64:0/2641311030 >> v2:172.21.15.91:3300/0 conn(0x7f380415d730 msgr2=0x7f38040cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.418+0000 7f38037fe640 1 --2- 172.21.15.64:0/2641311030 >> v2:172.21.15.91:3300/0 conn(0x7f380415d730 0x7f38040cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.418+0000 7f38037fe640 1 -- 172.21.15.64:0/2641311030 >> v1:172.21.15.91:6789/0 conn(0x7f380406b8f0 legacy=0x7f380406bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.418+0000 7f38037fe640 1 -- 172.21.15.64:0/2641311030 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040cb810 con 0x7f380406c480 2023-11-24T08:06:34.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.418+0000 7f3802ffd640 1 --2- 172.21.15.64:0/2641311030 >> v2:172.21.15.91:3300/0 conn(0x7f380415d730 0x7f38040cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:06:34.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.420+0000 7f38037fe640 1 --2- 172.21.15.64:0/2641311030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406c480 0x7f38040cc010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009e50 tx=0x7f37fc02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d55b9eab925013 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:34.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.420+0000 7f3801ffb640 1 -- 172.21.15.64:0/2641311030 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc02c9e0 con 0x7f380406c480 2023-11-24T08:06:34.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.420+0000 7f3801ffb640 1 -- 172.21.15.64:0/2641311030 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc02cb40 con 0x7f380406c480 2023-11-24T08:06:34.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.420+0000 7f3801ffb640 1 -- 172.21.15.64:0/2641311030 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0343f0 con 0x7f380406c480 2023-11-24T08:06:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.421+0000 7f3803fff640 1 -- 172.21.15.64:0/2641311030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406c480 msgr2=0x7f38040cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.421+0000 7f3803fff640 1 --2- 172.21.15.64:0/2641311030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406c480 0x7f38040cc010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009e50 tx=0x7f37fc02be40 comp rx=0 tx=0).stop 2023-11-24T08:06:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.421+0000 7f3803fff640 1 -- 172.21.15.64:0/2641311030 shutdown_connections 2023-11-24T08:06:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.421+0000 7f3803fff640 1 --2- 172.21.15.64:0/2641311030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406c480 0x7f38040cc010 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.421+0000 7f3803fff640 1 --2- 172.21.15.64:0/2641311030 >> v2:172.21.15.91:3300/0 conn(0x7f380415d730 0x7f38040cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.421+0000 7f3803fff640 1 -- 172.21.15.64:0/2641311030 >> 172.21.15.64:0/2641311030 conn(0x7f38040d0e40 msgr2=0x7f38040d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.421+0000 7f3803fff640 1 -- 172.21.15.64:0/2641311030 shutdown_connections 2023-11-24T08:06:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.421+0000 7f3803fff640 1 -- 172.21.15.64:0/2641311030 wait complete. 2023-11-24T08:06:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.424+0000 7f3803fff640 1 Processor -- start 2023-11-24T08:06:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.424+0000 7f3803fff640 1 -- start start 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.425+0000 7f3803fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406b8f0 0x7f38040c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.425+0000 7f3803fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380406c480 0x7f3804158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.425+0000 7f3803fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f380415d730 0x7f3804158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.425+0000 7f3803fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38040c8a30 con 0x7f380415d730 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.425+0000 7f3803fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38040c9960 con 0x7f380406c480 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.425+0000 7f3803fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38040c75b0 con 0x7f380406b8f0 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f380415d730 0x7f3804158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f380415d730 0x7f3804158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57022/0 (socket says 172.21.15.64:57022) 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 -- 172.21.15.64:0/864980211 learned_addr learned my addr 172.21.15.64:0/864980211 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f3802ffd640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406b8f0 0x7f38040c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 -- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406b8f0 msgr2=0x7f38040c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406b8f0 0x7f38040c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 -- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380406c480 msgr2=0x7f3804158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380406c480 0x7f3804158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 -- 172.21.15.64:0/864980211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc009b30 con 0x7f380415d730 2023-11-24T08:06:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.426+0000 7f38037fe640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f380415d730 0x7f3804158ca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f37fc036040 tx=0x7f37fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.427+0000 7f37ebfff640 1 -- 172.21.15.64:0/864980211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc03e450 con 0x7f380415d730 2023-11-24T08:06:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.427+0000 7f37ebfff640 1 -- 172.21.15.64:0/864980211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc03e5d0 con 0x7f380415d730 2023-11-24T08:06:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.427+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38040c69a0 con 0x7f380415d730 2023-11-24T08:06:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.427+0000 7f37ebfff640 1 -- 172.21.15.64:0/864980211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc03e980 con 0x7f380415d730 2023-11-24T08:06:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.427+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38040c6f60 con 0x7f380415d730 2023-11-24T08:06:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.428+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380415dba0 con 0x7f380415d730 2023-11-24T08:06:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.429+0000 7f37ebfff640 1 -- 172.21.15.64:0/864980211 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f37fc03eae0 con 0x7f380415d730 2023-11-24T08:06:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.430+0000 7f37ebfff640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37f006b080 0x7f37f006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.430+0000 7f37ebfff640 1 -- 172.21.15.64:0/864980211 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37fc0ac850 con 0x7f380415d730 2023-11-24T08:06:34.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.431+0000 7f3802ffd640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37f006b080 0x7f37f006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.432+0000 7f3802ffd640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37f006b080 0x7f37f006d540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f37f40096d0 tx=0x7f37f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:34.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.435+0000 7f37ebfff640 1 -- 172.21.15.64:0/864980211 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f380415dba0 con 0x7f380415d730 2023-11-24T08:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.633+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f380415dba0 con 0x7f380415d730 2023-11-24T08:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.634+0000 7f37ebfff640 1 -- 172.21.15.64:0/864980211 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f380415dba0 con 0x7f380415d730 2023-11-24T08:06:34.636 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:06:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37f006b080 msgr2=0x7f37f006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37f006b080 0x7f37f006d540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f37f40096d0 tx=0x7f37f4008040 comp rx=0 tx=0).stop 2023-11-24T08:06:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f380415d730 msgr2=0x7f3804158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f380415d730 0x7f3804158ca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f37fc036040 tx=0x7f37fc004a40 comp rx=0 tx=0).stop 2023-11-24T08:06:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 shutdown_connections 2023-11-24T08:06:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f37f006b080 0x7f37f006d540 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f380415d730 0x7f3804158ca0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f380406c480 0x7f3804158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 --2- 172.21.15.64:0/864980211 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f380406b8f0 0x7f38040c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.636+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 >> 172.21.15.64:0/864980211 conn(0x7f38040d0e40 msgr2=0x7f38040cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.637+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 shutdown_connections 2023-11-24T08:06:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.637+0000 7f3803fff640 1 -- 172.21.15.64:0/864980211 wait complete. 2023-11-24T08:06:34.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:06:34.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.884+0000 7f6835fb3640 1 Processor -- start 2023-11-24T08:06:34.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.885+0000 7f6835fb3640 1 -- start start 2023-11-24T08:06:34.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.886+0000 7f6835fb3640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f68300b2550 0x7f6830070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.886+0000 7f6835fb3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6830161700 0x7f68300ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.886+0000 7f6835fb3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6830070a90 con 0x7f68300b2930 2023-11-24T08:06:34.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.886+0000 7f6835fb3640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6830070cf0 con 0x7f6830161700 2023-11-24T08:06:34.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.886+0000 7f6835fb3640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6830070fd0 con 0x7f68300b2550 2023-11-24T08:06:34.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f68357b2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6830161700 0x7f68300ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f68357b2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6830161700 0x7f68300ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57024/0 (socket says 172.21.15.64:57024) 2023-11-24T08:06:34.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f68357b2640 1 -- 172.21.15.64:0/2369358786 learned_addr learned my addr 172.21.15.64:0/2369358786 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:34.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f6834fb1640 1 --2- 172.21.15.64:0/2369358786 >> v2:172.21.15.91:3300/0 conn(0x7f68300b2550 0x7f6830070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f68357b2640 1 -- 172.21.15.64:0/2369358786 >> v2:172.21.15.91:3300/0 conn(0x7f68300b2550 msgr2=0x7f6830070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:34.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f68357b2640 1 --2- 172.21.15.64:0/2369358786 >> v2:172.21.15.91:3300/0 conn(0x7f68300b2550 0x7f6830070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f68357b2640 1 -- 172.21.15.64:0/2369358786 >> v1:172.21.15.91:6789/0 conn(0x7f68300b2930 legacy=0x7f6830160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f68357b2640 1 -- 172.21.15.64:0/2369358786 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300712b0 con 0x7f6830161700 2023-11-24T08:06:34.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.887+0000 7f6834fb1640 1 --2- 172.21.15.64:0/2369358786 >> v2:172.21.15.91:3300/0 conn(0x7f68300b2550 0x7f6830070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:06:34.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.889+0000 7f68357b2640 1 --2- 172.21.15.64:0/2369358786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6830161700 0x7f68300ac900 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f683006fdc0 tx=0x7f682802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e15abfd897f9d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:34.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.889+0000 7f682f7fe640 1 -- 172.21.15.64:0/2369358786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682802cb10 con 0x7f6830161700 2023-11-24T08:06:34.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.889+0000 7f682f7fe640 1 -- 172.21.15.64:0/2369358786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682802cc70 con 0x7f6830161700 2023-11-24T08:06:34.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.889+0000 7f682f7fe640 1 -- 172.21.15.64:0/2369358786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280343f0 con 0x7f6830161700 2023-11-24T08:06:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.889+0000 7f6835fb3640 1 -- 172.21.15.64:0/2369358786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6830161700 msgr2=0x7f68300ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.889+0000 7f6835fb3640 1 --2- 172.21.15.64:0/2369358786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6830161700 0x7f68300ac900 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f683006fdc0 tx=0x7f682802bf70 comp rx=0 tx=0).stop 2023-11-24T08:06:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.890+0000 7f6835fb3640 1 -- 172.21.15.64:0/2369358786 shutdown_connections 2023-11-24T08:06:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.890+0000 7f6835fb3640 1 --2- 172.21.15.64:0/2369358786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6830161700 0x7f68300ac900 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.890+0000 7f6835fb3640 1 --2- 172.21.15.64:0/2369358786 >> v2:172.21.15.91:3300/0 conn(0x7f68300b2550 0x7f6830070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.890+0000 7f6835fb3640 1 -- 172.21.15.64:0/2369358786 >> 172.21.15.64:0/2369358786 conn(0x7f68300b0840 msgr2=0x7f68300b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.890+0000 7f6835fb3640 1 -- 172.21.15.64:0/2369358786 shutdown_connections 2023-11-24T08:06:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.890+0000 7f6835fb3640 1 -- 172.21.15.64:0/2369358786 wait complete. 2023-11-24T08:06:34.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.891+0000 7f6835fb3640 1 Processor -- start 2023-11-24T08:06:34.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.892+0000 7f6835fb3640 1 -- start start 2023-11-24T08:06:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.892+0000 7f6835fb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68300b2550 0x7f6830154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.892+0000 7f6835fb3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68300b2930 0x7f68301f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.892+0000 7f6835fb3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6830161700 0x7f683015a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6835fb3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830158130 con 0x7f68300b2550 2023-11-24T08:06:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6835fb3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301583a0 con 0x7f6830161700 2023-11-24T08:06:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6835fb3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6830158680 con 0x7f68300b2930 2023-11-24T08:06:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6834fb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68300b2550 0x7f6830154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6834fb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68300b2550 0x7f6830154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57028/0 (socket says 172.21.15.64:57028) 2023-11-24T08:06:34.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6834fb1640 1 -- 172.21.15.64:0/4013091810 learned_addr learned my addr 172.21.15.64:0/4013091810 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:34.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f68357b2640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6830161700 0x7f683015a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f682ffff640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68300b2930 0x7f68301f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6834fb1640 1 -- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68300b2930 msgr2=0x7f68301f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6834fb1640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68300b2930 0x7f68301f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6834fb1640 1 -- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6830161700 msgr2=0x7f683015a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:34.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6834fb1640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6830161700 0x7f683015a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:34.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.893+0000 7f6834fb1640 1 -- 172.21.15.64:0/4013091810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009b30 con 0x7f68300b2550 2023-11-24T08:06:34.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.894+0000 7f6834fb1640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68300b2550 0x7f6830154b00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f682400d7d0 tx=0x7f682400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:34.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.894+0000 7f682d7fa640 1 -- 172.21.15.64:0/4013091810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824004030 con 0x7f68300b2550 2023-11-24T08:06:34.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.894+0000 7f682d7fa640 1 -- 172.21.15.64:0/4013091810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6824004190 con 0x7f68300b2550 2023-11-24T08:06:34.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.894+0000 7f682d7fa640 1 -- 172.21.15.64:0/4013091810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824004520 con 0x7f68300b2550 2023-11-24T08:06:34.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.894+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6830158960 con 0x7f68300b2550 2023-11-24T08:06:34.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.894+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683015af30 con 0x7f68300b2550 2023-11-24T08:06:34.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.895+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683006d6a0 con 0x7f68300b2550 2023-11-24T08:06:34.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.896+0000 7f682d7fa640 1 -- 172.21.15.64:0/4013091810 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f68240096a0 con 0x7f68300b2550 2023-11-24T08:06:34.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.897+0000 7f682d7fa640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f681406b0b0 0x7f681406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:34.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.897+0000 7f682d7fa640 1 -- 172.21.15.64:0/4013091810 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6824089b90 con 0x7f68300b2550 2023-11-24T08:06:34.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.898+0000 7f682ffff640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f681406b0b0 0x7f681406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:34.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.899+0000 7f682ffff640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f681406b0b0 0x7f681406d570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6830070820 tx=0x7f681c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:34.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:34.901+0000 7f682d7fa640 1 -- 172.21.15.64:0/4013091810 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68240548d0 con 0x7f68300b2550 2023-11-24T08:06:35.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.118+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f683004e840 con 0x7f68300b2550 2023-11-24T08:06:35.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.119+0000 7f682d7fa640 1 -- 172.21.15.64:0/4013091810 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6824057ef0 con 0x7f68300b2550 2023-11-24T08:06:35.121 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:06:35.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f681406b0b0 msgr2=0x7f681406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:35.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f681406b0b0 0x7f681406d570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6830070820 tx=0x7f681c009040 comp rx=0 tx=0).stop 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68300b2550 msgr2=0x7f6830154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68300b2550 0x7f6830154b00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f682400d7d0 tx=0x7f682400dc90 comp rx=0 tx=0).stop 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 shutdown_connections 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f681406b0b0 0x7f681406d570 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6830161700 0x7f683015a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f68300b2930 0x7f68301f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.122+0000 7f6835fb3640 1 --2- 172.21.15.64:0/4013091810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68300b2550 0x7f6830154b00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.123+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 >> 172.21.15.64:0/4013091810 conn(0x7f68300b0840 msgr2=0x7f683006c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.123+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 shutdown_connections 2023-11-24T08:06:35.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.123+0000 7f6835fb3640 1 -- 172.21.15.64:0/4013091810 wait complete. 2023-11-24T08:06:35.142 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T08:06:35.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:35.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T08:06:35.235 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[105745]: fuse finished with error 0 and tester_r 0 2023-11-24T08:06:35.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:06:35.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:35.241 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:06:35.305 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:06:35.305 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:06:35.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:35.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.597+0000 7fab6fdc9640 1 Processor -- start 2023-11-24T08:06:35.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.597+0000 7fab6fdc9640 1 -- start start 2023-11-24T08:06:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.599+0000 7fab6fdc9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6815d4a0 0x7fab680af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.599+0000 7fab6fdc9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab680af6c0 0x7fab6806a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.599+0000 7fab6fdc9640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fab6806ab40 0x7fab68160f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.599+0000 7fab6fdc9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab680afb00 con 0x7fab680af6c0 2023-11-24T08:06:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.599+0000 7fab6fdc9640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6806afd0 con 0x7fab6806ab40 2023-11-24T08:06:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.599+0000 7fab6fdc9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6815d880 con 0x7fab6815d4a0 2023-11-24T08:06:35.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6d364640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab680af6c0 0x7fab6806a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:35.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6d364640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab680af6c0 0x7fab6806a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57032/0 (socket says 172.21.15.64:57032) 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6d364640 1 -- 172.21.15.64:0/3490012524 learned_addr learned my addr 172.21.15.64:0/3490012524 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6db65640 1 --2- 172.21.15.64:0/3490012524 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6815d4a0 0x7fab680af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6e366640 1 --2- 172.21.15.64:0/3490012524 >> v2:172.21.15.91:3300/0 conn(0x7fab6806ab40 0x7fab68160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6d364640 1 -- 172.21.15.64:0/3490012524 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6815d4a0 msgr2=0x7fab680af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6d364640 1 --2- 172.21.15.64:0/3490012524 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6815d4a0 0x7fab680af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6d364640 1 -- 172.21.15.64:0/3490012524 >> v2:172.21.15.91:3300/0 conn(0x7fab6806ab40 msgr2=0x7fab68160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6d364640 1 --2- 172.21.15.64:0/3490012524 >> v2:172.21.15.91:3300/0 conn(0x7fab6806ab40 0x7fab68160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6d364640 1 -- 172.21.15.64:0/3490012524 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6806fba0 con 0x7fab680af6c0 2023-11-24T08:06:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.600+0000 7fab6e366640 1 --2- 172.21.15.64:0/3490012524 >> v2:172.21.15.91:3300/0 conn(0x7fab6806ab40 0x7fab68160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.601+0000 7fab6d364640 1 --2- 172.21.15.64:0/3490012524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab680af6c0 0x7fab6806a600 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fab680b4640 tx=0x7fab5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf92c91abd90c62 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.602+0000 7fab6cb63640 1 -- 172.21.15.64:0/3490012524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031870 con 0x7fab680af6c0 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.602+0000 7fab6cb63640 1 -- 172.21.15.64:0/3490012524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c0319d0 con 0x7fab680af6c0 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.602+0000 7fab6cb63640 1 -- 172.21.15.64:0/3490012524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031d60 con 0x7fab680af6c0 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.602+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3490012524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab680af6c0 msgr2=0x7fab6806a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.602+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3490012524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab680af6c0 0x7fab6806a600 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fab680b4640 tx=0x7fab5c02b910 comp rx=0 tx=0).stop 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.603+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3490012524 shutdown_connections 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.603+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3490012524 >> v2:172.21.15.91:3300/0 conn(0x7fab6806ab40 0x7fab68160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.603+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3490012524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab680af6c0 0x7fab6806a600 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.603+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3490012524 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6815d4a0 0x7fab680af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.603+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3490012524 >> 172.21.15.64:0/3490012524 conn(0x7fab6806d490 msgr2=0x7fab6806d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.603+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3490012524 shutdown_connections 2023-11-24T08:06:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.603+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3490012524 wait complete. 2023-11-24T08:06:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.605+0000 7fab6fdc9640 1 Processor -- start 2023-11-24T08:06:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.605+0000 7fab6fdc9640 1 -- start start 2023-11-24T08:06:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6fdc9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6806ab40 0x7fab68148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6fdc9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab680af6c0 0x7fab68156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6fdc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6815d4a0 0x7fab68159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6fdc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680cf9c0 con 0x7fab6815d4a0 2023-11-24T08:06:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6fdc9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680ce390 con 0x7fab680af6c0 2023-11-24T08:06:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6fdc9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab680ce0b0 con 0x7fab6806ab40 2023-11-24T08:06:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6e366640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6815d4a0 0x7fab68159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6e366640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6815d4a0 0x7fab68159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57044/0 (socket says 172.21.15.64:57044) 2023-11-24T08:06:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6e366640 1 -- 172.21.15.64:0/3004271905 learned_addr learned my addr 172.21.15.64:0/3004271905 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6db65640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6806ab40 0x7fab68148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.606+0000 7fab6d364640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab680af6c0 0x7fab68156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:35.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab6e366640 1 -- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6806ab40 msgr2=0x7fab68148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab6e366640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6806ab40 0x7fab68148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab6e366640 1 -- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab680af6c0 msgr2=0x7fab68156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab6e366640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab680af6c0 0x7fab68156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab6e366640 1 -- 172.21.15.64:0/3004271905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0094d0 con 0x7fab6815d4a0 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab6e366640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6815d4a0 0x7fab68159970 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fab6000c7f0 tx=0x7fab6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab667fc640 1 -- 172.21.15.64:0/3004271905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60007820 con 0x7fab6815d4a0 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab667fc640 1 -- 172.21.15.64:0/3004271905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab60007980 con 0x7fab6815d4a0 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab667fc640 1 -- 172.21.15.64:0/3004271905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60007d30 con 0x7fab6815d4a0 2023-11-24T08:06:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.607+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab681575b0 con 0x7fab6815d4a0 2023-11-24T08:06:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.608+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab68157b70 con 0x7fab6815d4a0 2023-11-24T08:06:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.609+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab68161d70 con 0x7fab6815d4a0 2023-11-24T08:06:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.610+0000 7fab667fc640 1 -- 172.21.15.64:0/3004271905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fab600113f0 con 0x7fab6815d4a0 2023-11-24T08:06:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.610+0000 7fab667fc640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab5406b080 0x7fab5406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.611+0000 7fab667fc640 1 -- 172.21.15.64:0/3004271905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fab60089e40 con 0x7fab6815d4a0 2023-11-24T08:06:35.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.611+0000 7fab6db65640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab5406b080 0x7fab5406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:35.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.612+0000 7fab6db65640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab5406b080 0x7fab5406d540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fab580099c0 tx=0x7fab58003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.615+0000 7fab667fc640 1 -- 172.21.15.64:0/3004271905 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab60054b00 con 0x7fab6815d4a0 2023-11-24T08:06:35.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.808+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab68157d60 con 0x7fab6815d4a0 2023-11-24T08:06:35.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.811+0000 7fab667fc640 1 -- 172.21.15.64:0/3004271905 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16573 (secure 0 0 0) 0x7fab6001f090 con 0x7fab6815d4a0 2023-11-24T08:06:35.814 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:35.814 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:35.008634+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:02.427519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:03.560696+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab5406b080 msgr2=0x7fab5406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab5406b080 0x7fab5406d540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fab580099c0 tx=0x7fab58003990 comp rx=0 tx=0).stop 2023-11-24T08:06:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6815d4a0 msgr2=0x7fab68159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6815d4a0 0x7fab68159970 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fab6000c7f0 tx=0x7fab6000ccb0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 shutdown_connections 2023-11-24T08:06:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab5406b080 0x7fab5406d540 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6815d4a0 0x7fab68159970 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab680af6c0 0x7fab68156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 --2- 172.21.15.64:0/3004271905 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab6806ab40 0x7fab68148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:35.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 >> 172.21.15.64:0/3004271905 conn(0x7fab6806d490 msgr2=0x7fab680b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:35.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 shutdown_connections 2023-11-24T08:06:35.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:35.814+0000 7fab6fdc9640 1 -- 172.21.15.64:0/3004271905 wait complete. 2023-11-24T08:06:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:06:02.427519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:06:03.560696+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 46, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:06:35.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:06:36.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.064+0000 7fd69eea6640 1 Processor -- start 2023-11-24T08:06:36.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.064+0000 7fd69eea6640 1 -- start start 2023-11-24T08:06:36.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.066+0000 7fd69eea6640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd6980ae270 0x7fd6980abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.066+0000 7fd69eea6640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69815b730 con 0x7fd69815b350 2023-11-24T08:06:36.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.066+0000 7fd69eea6640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980ac740 con 0x7fd69809cb30 2023-11-24T08:06:36.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.066+0000 7fd69eea6640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980ac460 con 0x7fd6980ae270 2023-11-24T08:06:36.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.066+0000 7fd69cc42640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd69815b350 0x7fd6980b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46408/0 (socket says 172.21.15.64:46408) 2023-11-24T08:06:36.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.066+0000 7fd69cc42640 1 -- 172.21.15.64:0/678311551 learned_addr learned my addr 172.21.15.64:0/678311551 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126632545 0 0) 0x7fd694001490 con 0x7fd69815b350 2023-11-24T08:06:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd680003520 con 0x7fd69815b350 2023-11-24T08:06:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd69d443640 1 --2- 172.21.15.64:0/678311551 >> v2:172.21.15.91:3300/0 conn(0x7fd6980ae270 0x7fd6980abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2538980479 0 0) 0x7fd680003520 con 0x7fd69815b350 2023-11-24T08:06:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 >> v2:172.21.15.91:3300/0 conn(0x7fd6980ae270 msgr2=0x7fd6980abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 --2- 172.21.15.64:0/678311551 >> v2:172.21.15.91:3300/0 conn(0x7fd6980ae270 0x7fd6980abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 >> v1:172.21.15.91:6789/0 conn(0x7fd69809cb30 legacy=0x7fd6980adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980b2a50 con 0x7fd69815b350 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fd694001b80 con 0x7fd69815b350 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 >> v1:172.21.15.64:6789/0 conn(0x7fd69815b350 legacy=0x7fd6980b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6800039c0 0x7fd680003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6800047c0 0x7fd680004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800051d0 0x7fd68000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980ac460 con 0x7fd6800039c0 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000c120 con 0x7fd6800051d0 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.067+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000c2e0 con 0x7fd6800047c0 2023-11-24T08:06:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69cc42640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6800039c0 0x7fd680003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69d443640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800051d0 0x7fd68000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd68ffff640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6800047c0 0x7fd680004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69cc42640 1 -- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6800047c0 msgr2=0x7fd680004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69cc42640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd6800047c0 0x7fd680004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69cc42640 1 -- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800051d0 msgr2=0x7fd68000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69cc42640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800051d0 0x7fd68000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69cc42640 1 -- 172.21.15.64:0/678311551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680003520 con 0x7fd6800039c0 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69d443640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800051d0 0x7fd68000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.068+0000 7fd69d443640 1 -- 172.21.15.64:0/678311551 reap_dead start 2023-11-24T08:06:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.069+0000 7fd69cc42640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6800039c0 0x7fd680003de0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd69400a730 tx=0x7fd6800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c46f684fce1978 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.069+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6940072d0 con 0x7fd6800039c0 2023-11-24T08:06:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.069+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69402f070 con 0x7fd6800039c0 2023-11-24T08:06:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.069+0000 7fd68f7fe640 1 -- 172.21.15.64:0/678311551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6940076d0 con 0x7fd6800039c0 2023-11-24T08:06:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.070+0000 7fd69eea6640 1 -- 172.21.15.64:0/678311551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6800039c0 msgr2=0x7fd680003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.070+0000 7fd69eea6640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6800039c0 0x7fd680003de0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd69400a730 tx=0x7fd6800057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.070+0000 7fd69eea6640 1 -- 172.21.15.64:0/678311551 shutdown_connections 2023-11-24T08:06:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.070+0000 7fd69eea6640 1 --2- 172.21.15.64:0/678311551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6800039c0 0x7fd680003de0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.070+0000 7fd69eea6640 1 -- 172.21.15.64:0/678311551 >> 172.21.15.64:0/678311551 conn(0x7fd6980b0ba0 msgr2=0x7fd6980b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.070+0000 7fd69eea6640 1 -- 172.21.15.64:0/678311551 shutdown_connections 2023-11-24T08:06:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.070+0000 7fd69eea6640 1 -- 172.21.15.64:0/678311551 wait complete. 2023-11-24T08:06:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.072+0000 7fd69eea6640 1 Processor -- start 2023-11-24T08:06:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.072+0000 7fd69eea6640 1 -- start start 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69eea6640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800039c0 0x7fd698152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69eea6640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd698159450 0x7fd6981f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69eea6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6981598d0 0x7fd6981f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69eea6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69814c400 con 0x7fd6981598d0 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69eea6640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6981f2520 con 0x7fd6800039c0 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69eea6640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6981f2800 con 0x7fd698159450 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69d443640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6981598d0 0x7fd6981f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69d443640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6981598d0 0x7fd6981f1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57056/0 (socket says 172.21.15.64:57056) 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69d443640 1 -- 172.21.15.64:0/2666143508 learned_addr learned my addr 172.21.15.64:0/2666143508 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd68ffff640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd698159450 0x7fd6981f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69d443640 1 -- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd698159450 msgr2=0x7fd6981f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69d443640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd698159450 0x7fd6981f19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69d443640 1 -- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800039c0 msgr2=0x7fd698152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69d443640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800039c0 0x7fd698152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd69d443640 1 -- 172.21.15.64:0/2666143508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6981f2ae0 con 0x7fd6981598d0 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.074+0000 7fd68ffff640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd698159450 0x7fd6981f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.075+0000 7fd69d443640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6981598d0 0x7fd6981f1ef0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd698072730 tx=0x7fd69000e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.075+0000 7fd68d7fa640 1 -- 172.21.15.64:0/2666143508 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690004230 con 0x7fd6981598d0 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.075+0000 7fd68d7fa640 1 -- 172.21.15.64:0/2666143508 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6900043b0 con 0x7fd6981598d0 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.075+0000 7fd68d7fa640 1 -- 172.21.15.64:0/2666143508 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690004520 con 0x7fd6981598d0 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.075+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6981f2dc0 con 0x7fd6981598d0 2023-11-24T08:06:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.075+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69813e6a0 con 0x7fd6981598d0 2023-11-24T08:06:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.076+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6981f32e0 con 0x7fd6981598d0 2023-11-24T08:06:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.077+0000 7fd68d7fa640 1 -- 172.21.15.64:0/2666143508 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd690003850 con 0x7fd6981598d0 2023-11-24T08:06:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.078+0000 7fd68d7fa640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd68406b0d0 0x7fd68406d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.078+0000 7fd68d7fa640 1 -- 172.21.15.64:0/2666143508 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd690088cc0 con 0x7fd6981598d0 2023-11-24T08:06:36.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.079+0000 7fd69cc42640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd68406b0d0 0x7fd68406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.080+0000 7fd69cc42640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd68406b0d0 0x7fd68406d590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd694034000 tx=0x7fd69402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:36.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.083+0000 7fd68d7fa640 1 -- 172.21.15.64:0/2666143508 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6981f32e0 con 0x7fd6981598d0 2023-11-24T08:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.277+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6981f32e0 con 0x7fd6981598d0 2023-11-24T08:06:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.278+0000 7fd68d7fa640 1 -- 172.21.15.64:0/2666143508 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6981f32e0 con 0x7fd6981598d0 2023-11-24T08:06:36.280 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.280+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd68406b0d0 msgr2=0x7fd68406d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.280+0000 7fd69eea6640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd68406b0d0 0x7fd68406d590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd694034000 tx=0x7fd69402f000 comp rx=0 tx=0).stop 2023-11-24T08:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.280+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6981598d0 msgr2=0x7fd6981f1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.280+0000 7fd69eea6640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6981598d0 0x7fd6981f1ef0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd698072730 tx=0x7fd69000e830 comp rx=0 tx=0).stop 2023-11-24T08:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.281+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 shutdown_connections 2023-11-24T08:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.281+0000 7fd69eea6640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd68406b0d0 0x7fd68406d590 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.281+0000 7fd69eea6640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6981598d0 0x7fd6981f1ef0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.281+0000 7fd69eea6640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd698159450 0x7fd6981f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.281+0000 7fd69eea6640 1 --2- 172.21.15.64:0/2666143508 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd6800039c0 0x7fd698152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.281+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 >> 172.21.15.64:0/2666143508 conn(0x7fd6980b0ba0 msgr2=0x7fd6980b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.281+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 shutdown_connections 2023-11-24T08:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.281+0000 7fd69eea6640 1 -- 172.21.15.64:0/2666143508 wait complete. 2023-11-24T08:06:36.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:06:36.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.533+0000 7fb8aa2bb640 1 Processor -- start 2023-11-24T08:06:36.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.533+0000 7fb8aa2bb640 1 -- start start 2023-11-24T08:06:36.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.534+0000 7fb8aa2bb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb8a4144770 0x7fb8a415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8aa2bb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb8a40b35a0 0x7fb8a415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8aa2bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a406c9d0 0x7fb8a406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8aa2bb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a40b43b0 con 0x7fb8a4144770 2023-11-24T08:06:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8aa2bb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a415f930 con 0x7fb8a40b35a0 2023-11-24T08:06:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8aa2bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40b3b10 con 0x7fb8a406c9d0 2023-11-24T08:06:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a92b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb8a4144770 0x7fb8a415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a92b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb8a4144770 0x7fb8a415eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57070/0 (socket says 172.21.15.64:57070) 2023-11-24T08:06:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a92b9640 1 -- 172.21.15.64:0/706502284 learned_addr learned my addr 172.21.15.64:0/706502284 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:36.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a92b9640 1 -- 172.21.15.64:0/706502284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a406c9d0 msgr2=0x7fb8a406ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:36.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a9aba640 1 --2- 172.21.15.64:0/706502284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a406c9d0 0x7fb8a406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a8ab8640 1 --2- 172.21.15.64:0/706502284 >> v2:172.21.15.91:3300/0 conn(0x7fb8a40b35a0 0x7fb8a415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a92b9640 1 --2- 172.21.15.64:0/706502284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a406c9d0 0x7fb8a406ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a92b9640 1 -- 172.21.15.64:0/706502284 >> v2:172.21.15.91:3300/0 conn(0x7fb8a40b35a0 msgr2=0x7fb8a415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a92b9640 1 --2- 172.21.15.64:0/706502284 >> v2:172.21.15.91:3300/0 conn(0x7fb8a40b35a0 0x7fb8a415f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a92b9640 1 -- 172.21.15.64:0/706502284 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a40b3df0 con 0x7fb8a4144770 2023-11-24T08:06:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.535+0000 7fb8a8ab8640 1 --2- 172.21.15.64:0/706502284 >> v2:172.21.15.91:3300/0 conn(0x7fb8a40b35a0 0x7fb8a415f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.536+0000 7fb8a92b9640 1 --2- 172.21.15.64:0/706502284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a4144770 0x7fb8a415eb60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40ae900 tx=0x7fb8a002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec3e75b640783d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:36.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.537+0000 7fb89bfff640 1 -- 172.21.15.64:0/706502284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0031860 con 0x7fb8a4144770 2023-11-24T08:06:36.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.537+0000 7fb89bfff640 1 -- 172.21.15.64:0/706502284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a00319c0 con 0x7fb8a4144770 2023-11-24T08:06:36.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.537+0000 7fb89bfff640 1 -- 172.21.15.64:0/706502284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0031d50 con 0x7fb8a4144770 2023-11-24T08:06:36.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.537+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/706502284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a4144770 msgr2=0x7fb8a415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.537+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/706502284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a4144770 0x7fb8a415eb60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40ae900 tx=0x7fb8a002b900 comp rx=0 tx=0).stop 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.538+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/706502284 shutdown_connections 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.538+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/706502284 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a406c9d0 0x7fb8a406ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.538+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/706502284 >> v2:172.21.15.91:3300/0 conn(0x7fb8a40b35a0 0x7fb8a415f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.538+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/706502284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a4144770 0x7fb8a415eb60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.538+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/706502284 >> 172.21.15.64:0/706502284 conn(0x7fb8a40cec10 msgr2=0x7fb8a40cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.538+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/706502284 shutdown_connections 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.538+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/706502284 wait complete. 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.539+0000 7fb8aa2bb640 1 Processor -- start 2023-11-24T08:06:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.540+0000 7fb8aa2bb640 1 -- start start 2023-11-24T08:06:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.540+0000 7fb8aa2bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a406c9d0 0x7fb8a40aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.540+0000 7fb8aa2bb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8a40b35a0 0x7fb8a40ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8aa2bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a4144770 0x7fb8a414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8aa2bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41431e0 con 0x7fb8a406c9d0 2023-11-24T08:06:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8aa2bb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41434c0 con 0x7fb8a40b35a0 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8aa2bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a414b6e0 con 0x7fb8a4144770 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a92b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a406c9d0 0x7fb8a40aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a92b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a406c9d0 0x7fb8a40aab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57074/0 (socket says 172.21.15.64:57074) 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a92b9640 1 -- 172.21.15.64:0/214251533 learned_addr learned my addr 172.21.15.64:0/214251533 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a92b9640 1 -- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a4144770 msgr2=0x7fb8a414d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a9aba640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a4144770 0x7fb8a414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a8ab8640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8a40b35a0 0x7fb8a40ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a92b9640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a4144770 0x7fb8a414d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a92b9640 1 -- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8a40b35a0 msgr2=0x7fb8a40ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a92b9640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8a40b35a0 0x7fb8a40ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a92b9640 1 -- 172.21.15.64:0/214251533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a00094d0 con 0x7fb8a406c9d0 2023-11-24T08:06:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.541+0000 7fb8a8ab8640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8a40b35a0 0x7fb8a40ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.542+0000 7fb8a92b9640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a406c9d0 0x7fb8a40aab90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0000c00 tx=0x7fb8a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.542+0000 7fb899ffb640 1 -- 172.21.15.64:0/214251533 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a003ad10 con 0x7fb8a406c9d0 2023-11-24T08:06:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.542+0000 7fb899ffb640 1 -- 172.21.15.64:0/214251533 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a003ae70 con 0x7fb8a406c9d0 2023-11-24T08:06:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.542+0000 7fb899ffb640 1 -- 172.21.15.64:0/214251533 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0044610 con 0x7fb8a406c9d0 2023-11-24T08:06:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.542+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a414b9c0 con 0x7fb8a406c9d0 2023-11-24T08:06:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.542+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a414dc80 con 0x7fb8a406c9d0 2023-11-24T08:06:36.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.543+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a414b820 con 0x7fb8a406c9d0 2023-11-24T08:06:36.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.544+0000 7fb899ffb640 1 -- 172.21.15.64:0/214251533 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fb8a004d450 con 0x7fb8a406c9d0 2023-11-24T08:06:36.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.545+0000 7fb899ffb640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb88806b0b0 0x7fb88806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.545+0000 7fb899ffb640 1 -- 172.21.15.64:0/214251533 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb8a00ac250 con 0x7fb8a406c9d0 2023-11-24T08:06:36.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.546+0000 7fb8a8ab8640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb88806b0b0 0x7fb88806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.547+0000 7fb8a8ab8640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb88806b0b0 0x7fb88806d570 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40cc290 tx=0x7fb894008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.549+0000 7fb899ffb640 1 -- 172.21.15.64:0/214251533 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8a414b820 con 0x7fb8a406c9d0 2023-11-24T08:06:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.738+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8a414b820 con 0x7fb8a406c9d0 2023-11-24T08:06:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.739+0000 7fb899ffb640 1 -- 172.21.15.64:0/214251533 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8a414b820 con 0x7fb8a406c9d0 2023-11-24T08:06:36.741 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:06:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb88806b0b0 msgr2=0x7fb88806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb88806b0b0 0x7fb88806d570 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40cc290 tx=0x7fb894008040 comp rx=0 tx=0).stop 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a406c9d0 msgr2=0x7fb8a40aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a406c9d0 0x7fb8a40aab90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0000c00 tx=0x7fb8a00046c0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 shutdown_connections 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb88806b0b0 0x7fb88806d570 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb8a4144770 0x7fb8a414d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb8a40b35a0 0x7fb8a40ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 --2- 172.21.15.64:0/214251533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a406c9d0 0x7fb8a40aab90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.741+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 >> 172.21.15.64:0/214251533 conn(0x7fb8a40cec10 msgr2=0x7fb8a406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.742+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 shutdown_connections 2023-11-24T08:06:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.742+0000 7fb8aa2bb640 1 -- 172.21.15.64:0/214251533 wait complete. 2023-11-24T08:06:36.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:06:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.985+0000 7f70df5a9640 1 Processor -- start 2023-11-24T08:06:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.986+0000 7f70df5a9640 1 -- start start 2023-11-24T08:06:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70df5a9640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f70d809cb30 0x7f70d80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70df5a9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70d80adb60 0x7f70d80adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70df5a9640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d815b730 con 0x7f70d815b350 2023-11-24T08:06:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70df5a9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d80ac740 con 0x7f70d80adb60 2023-11-24T08:06:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70df5a9640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d80ac460 con 0x7f70d809cb30 2023-11-24T08:06:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70ddb46640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70d80adb60 0x7f70d80adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70ddb46640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70d80adb60 0x7f70d80adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57090/0 (socket says 172.21.15.64:57090) 2023-11-24T08:06:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70ddb46640 1 -- 172.21.15.64:0/1175947089 learned_addr learned my addr 172.21.15.64:0/1175947089 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70dcb44640 1 --2- 172.21.15.64:0/1175947089 >> v2:172.21.15.91:3300/0 conn(0x7f70d809cb30 0x7f70d80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.988+0000 7f70ddb46640 1 -- 172.21.15.64:0/1175947089 >> v2:172.21.15.91:3300/0 conn(0x7f70d809cb30 msgr2=0x7f70d80abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.989+0000 7f70ddb46640 1 --2- 172.21.15.64:0/1175947089 >> v2:172.21.15.91:3300/0 conn(0x7f70d809cb30 0x7f70d80abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.989+0000 7f70ddb46640 1 -- 172.21.15.64:0/1175947089 >> v1:172.21.15.91:6789/0 conn(0x7f70d815b350 legacy=0x7f70d80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:36.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.989+0000 7f70ddb46640 1 -- 172.21.15.64:0/1175947089 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d80b2a50 con 0x7f70d80adb60 2023-11-24T08:06:36.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.990+0000 7f70ddb46640 1 --2- 172.21.15.64:0/1175947089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d80adb60 0x7f70d80adf70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f70d80d0060 tx=0x7f70d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597b1d13dd7a0cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.990+0000 7f70cffff640 1 -- 172.21.15.64:0/1175947089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d002ca80 con 0x7f70d80adb60 2023-11-24T08:06:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.990+0000 7f70cffff640 1 -- 172.21.15.64:0/1175947089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d002cbe0 con 0x7f70d80adb60 2023-11-24T08:06:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.990+0000 7f70cffff640 1 -- 172.21.15.64:0/1175947089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d00343f0 con 0x7f70d80adb60 2023-11-24T08:06:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.991+0000 7f70df5a9640 1 -- 172.21.15.64:0/1175947089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d80adb60 msgr2=0x7f70d80adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.991+0000 7f70df5a9640 1 --2- 172.21.15.64:0/1175947089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d80adb60 0x7f70d80adf70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f70d80d0060 tx=0x7f70d002bee0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.991+0000 7f70df5a9640 1 -- 172.21.15.64:0/1175947089 shutdown_connections 2023-11-24T08:06:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.991+0000 7f70df5a9640 1 --2- 172.21.15.64:0/1175947089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d80adb60 0x7f70d80adf70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.991+0000 7f70df5a9640 1 --2- 172.21.15.64:0/1175947089 >> v2:172.21.15.91:3300/0 conn(0x7f70d809cb30 0x7f70d80abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.991+0000 7f70df5a9640 1 -- 172.21.15.64:0/1175947089 >> 172.21.15.64:0/1175947089 conn(0x7f70d80b0ba0 msgr2=0x7f70d80b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.991+0000 7f70df5a9640 1 -- 172.21.15.64:0/1175947089 shutdown_connections 2023-11-24T08:06:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.991+0000 7f70df5a9640 1 -- 172.21.15.64:0/1175947089 wait complete. 2023-11-24T08:06:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.993+0000 7f70df5a9640 1 Processor -- start 2023-11-24T08:06:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.993+0000 7f70df5a9640 1 -- start start 2023-11-24T08:06:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70df5a9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70d809cb30 0x7f70d8155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70df5a9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70d80adb60 0x7f70d8158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70df5a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d815b350 0x7f70d81379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70df5a9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8159300 con 0x7f70d815b350 2023-11-24T08:06:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70df5a9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81542a0 con 0x7f70d80adb60 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70df5a9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8154580 con 0x7f70d809cb30 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70ddb46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d815b350 0x7f70d81379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70ddb46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d815b350 0x7f70d81379b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57104/0 (socket says 172.21.15.64:57104) 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70ddb46640 1 -- 172.21.15.64:0/3343515594 learned_addr learned my addr 172.21.15.64:0/3343515594 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70dcb44640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70d80adb60 0x7f70d8158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70ddb46640 1 -- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70d809cb30 msgr2=0x7f70d8155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70dd345640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70d809cb30 0x7f70d8155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70ddb46640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70d809cb30 0x7f70d8155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70ddb46640 1 -- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70d80adb60 msgr2=0x7f70d8158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70ddb46640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70d80adb60 0x7f70d8158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70ddb46640 1 -- 172.21.15.64:0/3343515594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0009b30 con 0x7f70d815b350 2023-11-24T08:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.994+0000 7f70dcb44640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70d80adb60 0x7f70d8158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.995+0000 7f70ddb46640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d815b350 0x7f70d81379b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f70d0036040 tx=0x7f70d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.995+0000 7f70cdffb640 1 -- 172.21.15.64:0/3343515594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d003e450 con 0x7f70d815b350 2023-11-24T08:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.995+0000 7f70cdffb640 1 -- 172.21.15.64:0/3343515594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d003e5d0 con 0x7f70d815b350 2023-11-24T08:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.995+0000 7f70cdffb640 1 -- 172.21.15.64:0/3343515594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d003e980 con 0x7f70d815b350 2023-11-24T08:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.995+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d8154860 con 0x7f70d815b350 2023-11-24T08:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.995+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d8154e20 con 0x7f70d815b350 2023-11-24T08:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.996+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d809d200 con 0x7f70d815b350 2023-11-24T08:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.997+0000 7f70cdffb640 1 -- 172.21.15.64:0/3343515594 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f70d002cd50 con 0x7f70d815b350 2023-11-24T08:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.998+0000 7f70cdffb640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70bc06b080 0x7f70bc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.998+0000 7f70cdffb640 1 -- 172.21.15.64:0/3343515594 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f70d00ac440 con 0x7f70d815b350 2023-11-24T08:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:36.999+0000 7f70dd345640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70bc06b080 0x7f70bc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.000+0000 7f70dd345640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70bc06b080 0x7f70bc06d540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70d4003d90 tx=0x7f70d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.003+0000 7f70cdffb640 1 -- 172.21.15.64:0/3343515594 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70d00773b0 con 0x7f70d815b350 2023-11-24T08:06:37.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.202+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70d815b730 con 0x7f70d815b350 2023-11-24T08:06:37.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.203+0000 7f70cdffb640 1 -- 172.21.15.64:0/3343515594 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70d815b730 con 0x7f70d815b350 2023-11-24T08:06:37.205 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:06:37.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70bc06b080 msgr2=0x7f70bc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:37.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70bc06b080 0x7f70bc06d540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70d4003d90 tx=0x7f70d4009040 comp rx=0 tx=0).stop 2023-11-24T08:06:37.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d815b350 msgr2=0x7f70d81379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d815b350 0x7f70d81379b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f70d0036040 tx=0x7f70d0004a40 comp rx=0 tx=0).stop 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 shutdown_connections 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f70bc06b080 0x7f70bc06d540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d815b350 0x7f70d81379b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f70d80adb60 0x7f70d8158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 --2- 172.21.15.64:0/3343515594 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f70d809cb30 0x7f70d8155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 >> 172.21.15.64:0/3343515594 conn(0x7f70d80b0ba0 msgr2=0x7f70d806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 shutdown_connections 2023-11-24T08:06:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.205+0000 7f70df5a9640 1 -- 172.21.15.64:0/3343515594 wait complete. 2023-11-24T08:06:37.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:06:37.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.443+0000 7f3065147640 1 Processor -- start 2023-11-24T08:06:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.444+0000 7f3065147640 1 -- start start 2023-11-24T08:06:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.445+0000 7f3065147640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060071620 0x7f306015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.445+0000 7f3065147640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3060071a00 0x7f30600ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.445+0000 7f3065147640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3060070e90 0x7f30600cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.445+0000 7f3065147640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30600ce670 con 0x7f3060071a00 2023-11-24T08:06:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.445+0000 7f3065147640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30600ce8d0 con 0x7f3060070e90 2023-11-24T08:06:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.445+0000 7f3065147640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600700d0 con 0x7f3060071620 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3057fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3060071a00 0x7f30600ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3057fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3060071a00 0x7f30600ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52446/0 (socket says 172.21.15.64:52446) 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3057fff640 1 -- 172.21.15.64:0/334608557 learned_addr learned my addr 172.21.15.64:0/334608557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f305ffff640 1 --2- 172.21.15.64:0/334608557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060071620 0x7f306015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3064946640 1 --2- 172.21.15.64:0/334608557 >> v2:172.21.15.91:3300/0 conn(0x7f3060070e90 0x7f30600cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3057fff640 1 -- 172.21.15.64:0/334608557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060071620 msgr2=0x7f306015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3057fff640 1 --2- 172.21.15.64:0/334608557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060071620 0x7f306015e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3057fff640 1 -- 172.21.15.64:0/334608557 >> v2:172.21.15.91:3300/0 conn(0x7f3060070e90 msgr2=0x7f30600cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3057fff640 1 --2- 172.21.15.64:0/334608557 >> v2:172.21.15.91:3300/0 conn(0x7f3060070e90 0x7f30600cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3057fff640 1 -- 172.21.15.64:0/334608557 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600703b0 con 0x7f3060071a00 2023-11-24T08:06:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.446+0000 7f3064946640 1 --2- 172.21.15.64:0/334608557 >> v2:172.21.15.91:3300/0 conn(0x7f3060070e90 0x7f30600cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.447+0000 7f3057fff640 1 --2- 172.21.15.64:0/334608557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f30600ce130 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f306006c930 tx=0x7f304802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3747414d710444e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f305f7fe640 1 -- 172.21.15.64:0/334608557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480317d0 con 0x7f3060071a00 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f305f7fe640 1 -- 172.21.15.64:0/334608557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048031930 con 0x7f3060071a00 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f305f7fe640 1 -- 172.21.15.64:0/334608557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048031cc0 con 0x7f3060071a00 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f3065147640 1 -- 172.21.15.64:0/334608557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 msgr2=0x7f30600ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f3065147640 1 --2- 172.21.15.64:0/334608557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f30600ce130 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f306006c930 tx=0x7f304802b870 comp rx=0 tx=0).stop 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f3065147640 1 -- 172.21.15.64:0/334608557 shutdown_connections 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f3065147640 1 --2- 172.21.15.64:0/334608557 >> v2:172.21.15.91:3300/0 conn(0x7f3060070e90 0x7f30600cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f3065147640 1 --2- 172.21.15.64:0/334608557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f30600ce130 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f3065147640 1 --2- 172.21.15.64:0/334608557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060071620 0x7f306015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.448+0000 7f3065147640 1 -- 172.21.15.64:0/334608557 >> 172.21.15.64:0/334608557 conn(0x7f30600b0840 msgr2=0x7f30600b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:37.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.449+0000 7f3065147640 1 -- 172.21.15.64:0/334608557 shutdown_connections 2023-11-24T08:06:37.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.449+0000 7f3065147640 1 -- 172.21.15.64:0/334608557 wait complete. 2023-11-24T08:06:37.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.450+0000 7f3065147640 1 Processor -- start 2023-11-24T08:06:37.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.450+0000 7f3065147640 1 -- start start 2023-11-24T08:06:37.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.451+0000 7f3065147640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060070e90 0x7f3060156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:37.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.451+0000 7f3065147640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3060071620 0x7f306014a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:37.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.451+0000 7f3065147640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f306014dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.451+0000 7f3065147640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060155810 con 0x7f3060071a00 2023-11-24T08:06:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.451+0000 7f3065147640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306014e670 con 0x7f3060071620 2023-11-24T08:06:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.451+0000 7f3065147640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3060156370 con 0x7f3060070e90 2023-11-24T08:06:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3064946640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f306014dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3064946640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f306014dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52460/0 (socket says 172.21.15.64:52460) 2023-11-24T08:06:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3064946640 1 -- 172.21.15.64:0/2120786582 learned_addr learned my addr 172.21.15.64:0/2120786582 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f305ffff640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060070e90 0x7f3060156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3057fff640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3060071620 0x7f306014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3064946640 1 -- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060070e90 msgr2=0x7f3060156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3064946640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060070e90 0x7f3060156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3064946640 1 -- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3060071620 msgr2=0x7f306014a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3064946640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3060071620 0x7f306014a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3064946640 1 -- 172.21.15.64:0/2120786582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30480094d0 con 0x7f3060071a00 2023-11-24T08:06:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f3057fff640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3060071620 0x7f306014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.452+0000 7f305ffff640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060070e90 0x7f3060156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.453+0000 7f3064946640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f306014dcb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f305800cf20 tx=0x7f305800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:37.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.453+0000 7f305d7fa640 1 -- 172.21.15.64:0/2120786582 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305800d840 con 0x7f3060071a00 2023-11-24T08:06:37.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.453+0000 7f305d7fa640 1 -- 172.21.15.64:0/2120786582 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305800d9a0 con 0x7f3060071a00 2023-11-24T08:06:37.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.453+0000 7f305d7fa640 1 -- 172.21.15.64:0/2120786582 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305800dd50 con 0x7f3060071a00 2023-11-24T08:06:37.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.453+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3060155a40 con 0x7f3060071a00 2023-11-24T08:06:37.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.453+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3060156000 con 0x7f3060071a00 2023-11-24T08:06:37.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.454+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306004e840 con 0x7f3060071a00 2023-11-24T08:06:37.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.455+0000 7f305d7fa640 1 -- 172.21.15.64:0/2120786582 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f305801e3b0 con 0x7f3060071a00 2023-11-24T08:06:37.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.456+0000 7f305d7fa640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f304006b080 0x7f304006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:37.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.456+0000 7f305d7fa640 1 -- 172.21.15.64:0/2120786582 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f305808a610 con 0x7f3060071a00 2023-11-24T08:06:37.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.456+0000 7f305ffff640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f304006b080 0x7f304006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:37.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.458+0000 7f305ffff640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f304006b080 0x7f304006d540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3050006fd0 tx=0x7f3050008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:37.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.460+0000 7f305d7fa640 1 -- 172.21.15.64:0/2120786582 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3058055260 con 0x7f3060071a00 2023-11-24T08:06:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.667+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f30601564b0 con 0x7f3060071a00 2023-11-24T08:06:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.668+0000 7f305d7fa640 1 -- 172.21.15.64:0/2120786582 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3058088e60 con 0x7f3060071a00 2023-11-24T08:06:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:06:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.670+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f304006b080 msgr2=0x7f304006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.670+0000 7f3065147640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f304006b080 0x7f304006d540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3050006fd0 tx=0x7f3050008040 comp rx=0 tx=0).stop 2023-11-24T08:06:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.670+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 msgr2=0x7f306014dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.670+0000 7f3065147640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f306014dcb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f305800cf20 tx=0x7f305800a7d0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.671+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 shutdown_connections 2023-11-24T08:06:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.671+0000 7f3065147640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f304006b080 0x7f304006d540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.671+0000 7f3065147640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3060071a00 0x7f306014dcb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.671+0000 7f3065147640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3060071620 0x7f306014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.671+0000 7f3065147640 1 --2- 172.21.15.64:0/2120786582 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3060070e90 0x7f3060156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.671+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 >> 172.21.15.64:0/2120786582 conn(0x7f30600b0840 msgr2=0x7f306015b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.671+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 shutdown_connections 2023-11-24T08:06:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:37.671+0000 7f3065147640 1 -- 172.21.15.64:0/2120786582 wait complete. 2023-11-24T08:06:37.692 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi091.front.sepia.ceph.com... 2023-11-24T08:06:37.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:37.693 DEBUG:teuthology.orchestra.run.smithi091:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T08:06:37.791 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[96513]: fuse finished with error 0 and tester_r 0 2023-11-24T08:06:37.795 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:06:37.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:37.796 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:06:37.971 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:06:37.971 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:06:38.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:06:38.035 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:06:38.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:38.036 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T08:06:38.095 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T08:06:38.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:06:38.096 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:06:38.097 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:06:38.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:06:38.157 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:06:38.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:06:38.157 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T08:06:38.259 INFO:teuthology.orchestra.run.smithi091.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T08:06:38.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:06:38.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:06:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.493+0000 7f5955883640 1 Processor -- start 2023-11-24T08:06:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.494+0000 7f5955883640 1 -- start start 2023-11-24T08:06:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.495+0000 7f5955883640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59500cf050 0x7f59500cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.495+0000 7f5955883640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5950140900 con 0x7f59500cd7b0 2023-11-24T08:06:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.495+0000 7f5955883640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595015fa70 con 0x7f595013dfd0 2023-11-24T08:06:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.495+0000 7f5955883640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595015c580 con 0x7f59500cf050 2023-11-24T08:06:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.495+0000 7f5955082640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f59500cd7b0 0x7f59500cdc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45676/0 (socket says 172.21.15.64:45676) 2023-11-24T08:06:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.495+0000 7f5955082640 1 -- 172.21.15.64:0/3460308564 learned_addr learned my addr 172.21.15.64:0/3460308564 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594ffff640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59500cf050 0x7f59500cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628144029 0 0) 0x7f5950140900 con 0x7f59500cd7b0 2023-11-24T08:06:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5940003520 con 0x7f59500cd7b0 2023-11-24T08:06:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 248065735 0 0) 0x7f5940003520 con 0x7f59500cd7b0 2023-11-24T08:06:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59500cf050 msgr2=0x7f59500cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59500cf050 0x7f59500cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 >> v1:172.21.15.91:6789/0 conn(0x7f595013dfd0 legacy=0x7f595015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595015ee60 con 0x7f59500cd7b0 2023-11-24T08:06:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f59480019f0 con 0x7f59500cd7b0 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 >> v1:172.21.15.64:6789/0 conn(0x7f59500cd7b0 legacy=0x7f59500cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5940003a10 0x7f5940003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.496+0000 7f594f7fe640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59500cf050 0x7f59400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f594f7fe640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940004d00 0x7f5940005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595015c580 con 0x7f5940003a10 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594000bb10 con 0x7f5940004d00 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594000bca0 con 0x7f59500cf050 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f5954881640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5940003a10 0x7f5940003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f5955082640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940004d00 0x7f5940005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f594ffff640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59500cf050 0x7f59400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f5954881640 1 -- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59500cf050 msgr2=0x7f59400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f5954881640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f59500cf050 0x7f59400047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f5954881640 1 -- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940004d00 msgr2=0x7f5940005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f5954881640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940004d00 0x7f5940005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.497+0000 7f5954881640 1 -- 172.21.15.64:0/3460308564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940003520 con 0x7f5940003a10 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.498+0000 7f5954881640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5940003a10 0x7f5940003e30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5944001750 tx=0x7f59400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1732b33ba89f0f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440052d0 con 0x7f5940003a10 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594402e050 con 0x7f5940003a10 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f594f7fe640 1 -- 172.21.15.64:0/3460308564 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440035f0 con 0x7f5940003a10 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f5955883640 1 -- 172.21.15.64:0/3460308564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5940003a10 msgr2=0x7f5940003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f5955883640 1 --2- 172.21.15.64:0/3460308564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5940003a10 0x7f5940003e30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5944001750 tx=0x7f59400057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f5955082640 1 -- 172.21.15.64:0/3460308564 reap_dead start 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f5955883640 1 -- 172.21.15.64:0/3460308564 shutdown_connections 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f5955883640 1 -- 172.21.15.64:0/3460308564 >> 172.21.15.64:0/3460308564 conn(0x7f595006a860 msgr2=0x7f595006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f5955883640 1 -- 172.21.15.64:0/3460308564 shutdown_connections 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.499+0000 7f5955883640 1 -- 172.21.15.64:0/3460308564 wait complete. 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.501+0000 7f5955883640 1 Processor -- start 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.501+0000 7f5955883640 1 -- start start 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.502+0000 7f5955883640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940003a10 0x7f59501528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.502+0000 7f5955883640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59501545b0 0x7f595015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.502+0000 7f5955883640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5950154a30 0x7f595015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.502+0000 7f5955883640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950159f00 con 0x7f59501545b0 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.502+0000 7f5955883640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950159610 con 0x7f5940003a10 2023-11-24T08:06:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.502+0000 7f5955883640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59501598b0 con 0x7f5950154a30 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59501545b0 0x7f595015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59501545b0 0x7f595015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52484/0 (socket says 172.21.15.64:52484) 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 -- 172.21.15.64:0/2948562634 learned_addr learned my addr 172.21.15.64:0/2948562634 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f5954881640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940003a10 0x7f59501528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f5955082640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5950154a30 0x7f595015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 -- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5950154a30 msgr2=0x7f595015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5950154a30 0x7f595015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 -- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940003a10 msgr2=0x7f59501528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940003a10 0x7f59501528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 -- 172.21.15.64:0/2948562634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950159b90 con 0x7f59501545b0 2023-11-24T08:06:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f5955082640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5950154a30 0x7f595015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.503+0000 7f594ffff640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59501545b0 0x7f595015a390 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f593c010860 tx=0x7f593c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.504+0000 7f594d7fa640 1 -- 172.21.15.64:0/2948562634 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c009610 con 0x7f59501545b0 2023-11-24T08:06:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.504+0000 7f594d7fa640 1 -- 172.21.15.64:0/2948562634 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c00eb60 con 0x7f59501545b0 2023-11-24T08:06:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.504+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5950152e30 con 0x7f59501545b0 2023-11-24T08:06:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.504+0000 7f594d7fa640 1 -- 172.21.15.64:0/2948562634 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c01c400 con 0x7f59501545b0 2023-11-24T08:06:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.504+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5950155090 con 0x7f59501545b0 2023-11-24T08:06:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.505+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595004e840 con 0x7f59501545b0 2023-11-24T08:06:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.506+0000 7f594d7fa640 1 -- 172.21.15.64:0/2948562634 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f593c01c560 con 0x7f59501545b0 2023-11-24T08:06:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.507+0000 7f594d7fa640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f593406b080 0x7f593406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.507+0000 7f594d7fa640 1 -- 172.21.15.64:0/2948562634 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f593c089ff0 con 0x7f59501545b0 2023-11-24T08:06:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.508+0000 7f5954881640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f593406b080 0x7f593406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.509+0000 7f5954881640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f593406b080 0x7f593406d540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5944033000 tx=0x7f5944009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:38.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.512+0000 7f594d7fa640 1 -- 172.21.15.64:0/2948562634 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f593c054d10 con 0x7f59501545b0 2023-11-24T08:06:38.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.747+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59500ce9a0 con 0x7f59501545b0 2023-11-24T08:06:38.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.749+0000 7f594d7fa640 1 -- 172.21.15.64:0/2948562634 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f593c058330 con 0x7f59501545b0 2023-11-24T08:06:38.751 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:38.751 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/618780376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":618780376},{"type":"v1","addr":"172.21.15.64:6835","nonce":618780376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1212789084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1212789084},{"type":"v1","addr":"172.21.15.91:6836","nonce":1212789084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1895191218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1895191218},{"type":"v1","addr":"172.21.15.91:6837","nonce":1895191218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:04.548725+0000","modified":"2023-11-24T08:06:07.601727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/1271535459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1271535459},{"type":"v1","addr":"172.21.15.64:6834","nonce":1271535459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:06:38.752 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2023-11-24T08:06:38.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.751+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f593406b080 msgr2=0x7f593406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:38.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.751+0000 7f5955883640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f593406b080 0x7f593406d540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5944033000 tx=0x7f5944009030 comp rx=0 tx=0).stop 2023-11-24T08:06:38.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.751+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59501545b0 msgr2=0x7f595015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:38.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.751+0000 7f5955883640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59501545b0 0x7f595015a390 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f593c010860 tx=0x7f593c010d20 comp rx=0 tx=0).stop 2023-11-24T08:06:38.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.752+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 shutdown_connections 2023-11-24T08:06:38.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.752+0000 7f5955883640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f593406b080 0x7f593406d540 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.752+0000 7f5955883640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5950154a30 0x7f595015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.752+0000 7f5955883640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59501545b0 0x7f595015a390 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.752+0000 7f5955883640 1 --2- 172.21.15.64:0/2948562634 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5940003a10 0x7f59501528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:38.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.752+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 >> 172.21.15.64:0/2948562634 conn(0x7f595006a860 msgr2=0x7f5950140100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:38.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.752+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 shutdown_connections 2023-11-24T08:06:38.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:38.752+0000 7f5955883640 1 -- 172.21.15.64:0/2948562634 wait complete. 2023-11-24T08:06:38.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:39.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.004+0000 7ff1c22cf640 1 Processor -- start 2023-11-24T08:06:39.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.004+0000 7ff1c22cf640 1 -- start start 2023-11-24T08:06:39.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.005+0000 7ff1c22cf640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff1bc160970 0x7ff1bc0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.005+0000 7ff1c22cf640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc0b3140 0x7ff1bc06d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.005+0000 7ff1c22cf640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1bc071cd0 con 0x7ff1bc070780 2023-11-24T08:06:39.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.005+0000 7ff1c22cf640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1bc0b2120 con 0x7ff1bc160970 2023-11-24T08:06:39.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.005+0000 7ff1c22cf640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc0b2400 con 0x7ff1bc0b3140 2023-11-24T08:06:39.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.006+0000 7ff1c086c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff1bc070780 0x7ff1bc0acbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45682/0 (socket says 172.21.15.64:45682) 2023-11-24T08:06:39.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.006+0000 7ff1c086c640 1 -- 172.21.15.64:0/1856420091 learned_addr learned my addr 172.21.15.64:0/1856420091 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:39.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.006+0000 7ff1bbfff640 1 --2- 172.21.15.64:0/1856420091 >> v2:172.21.15.91:3300/0 conn(0x7ff1bc160970 0x7ff1bc0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.006+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc0b3140 0x7ff1bc06d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.006+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793241440 0 0) 0x7ff1bc071cd0 con 0x7ff1bc070780 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.006+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a8003520 con 0x7ff1bc070780 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088294113 0 0) 0x7ff1a8003520 con 0x7ff1bc070780 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc0b3140 msgr2=0x7ff1bc06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc0b3140 0x7ff1bc06d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 >> v2:172.21.15.91:3300/0 conn(0x7ff1bc160970 msgr2=0x7ff1bc0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 --2- 172.21.15.64:0/1856420091 >> v2:172.21.15.91:3300/0 conn(0x7ff1bc160970 0x7ff1bc0b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0b26e0 con 0x7ff1bc070780 2023-11-24T08:06:39.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7ff1b4001670 con 0x7ff1bc070780 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 >> v1:172.21.15.64:6789/0 conn(0x7ff1bc070780 legacy=0x7ff1bc0acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc0b3140 0x7ff1a8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1a8004d00 0x7ff1a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc0b2120 con 0x7ff1a8003f40 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc0b2400 con 0x7ff1a8004d00 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a800ba30 con 0x7ff1bc0b3140 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1bbfff640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc0b3140 0x7ff1a8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1c086c640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1a8004d00 0x7ff1a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1bb7fe640 1 -- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc0b3140 msgr2=0x7ff1a8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.007+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc0b3140 0x7ff1a8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.008+0000 7ff1bb7fe640 1 -- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1a8004d00 msgr2=0x7ff1a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.008+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1a8004d00 0x7ff1a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.008+0000 7ff1bb7fe640 1 -- 172.21.15.64:0/1856420091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8003520 con 0x7ff1a8003f40 2023-11-24T08:06:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.008+0000 7ff1c086c640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1a8004d00 0x7ff1a8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.009+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1a80047c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009a90 tx=0x7ff1a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1737a67e313ef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.009+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a402c680 con 0x7ff1a8003f40 2023-11-24T08:06:39.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.009+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a402c7e0 con 0x7ff1a8003f40 2023-11-24T08:06:39.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.009+0000 7ff1baffd640 1 -- 172.21.15.64:0/1856420091 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a402cb90 con 0x7ff1a8003f40 2023-11-24T08:06:39.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.010+0000 7ff1c22cf640 1 -- 172.21.15.64:0/1856420091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 msgr2=0x7ff1a80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.010+0000 7ff1c22cf640 1 --2- 172.21.15.64:0/1856420091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1a80047c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009a90 tx=0x7ff1a80057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.010+0000 7ff1c086c640 1 -- 172.21.15.64:0/1856420091 reap_dead start 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.010+0000 7ff1c22cf640 1 -- 172.21.15.64:0/1856420091 shutdown_connections 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.010+0000 7ff1c22cf640 1 -- 172.21.15.64:0/1856420091 >> 172.21.15.64:0/1856420091 conn(0x7ff1bc0b0d80 msgr2=0x7ff1bc0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.010+0000 7ff1c22cf640 1 -- 172.21.15.64:0/1856420091 shutdown_connections 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.010+0000 7ff1c22cf640 1 -- 172.21.15.64:0/1856420091 wait complete. 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.012+0000 7ff1c22cf640 1 Processor -- start 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.012+0000 7ff1c22cf640 1 -- start start 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1c22cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1bc15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1c22cf640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1bc15dcd0 0x7ff1bc099840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1c22cf640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc099d80 0x7ff1bc09c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1c22cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc09a200 con 0x7ff1a8003f40 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1c22cf640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc15cc30 con 0x7ff1bc15dcd0 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1c22cf640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc15cf10 con 0x7ff1bc099d80 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1bc15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1bc15c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52496/0 (socket says 172.21.15.64:52496) 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.013+0000 7ff1bbfff640 1 -- 172.21.15.64:0/826016920 learned_addr learned my addr 172.21.15.64:0/826016920 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1bc15dcd0 0x7ff1bc099840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1c086c640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc099d80 0x7ff1bc09c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1bbfff640 1 -- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc099d80 msgr2=0x7ff1bc09c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1bbfff640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc099d80 0x7ff1bc09c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1bbfff640 1 -- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1bc15dcd0 msgr2=0x7ff1bc099840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1bbfff640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1bc15dcd0 0x7ff1bc099840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1bbfff640 1 -- 172.21.15.64:0/826016920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc15d1f0 con 0x7ff1a8003f40 2023-11-24T08:06:39.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1bc15dcd0 0x7ff1bc099840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:39.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1c086c640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc099d80 0x7ff1bc09c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:39.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1bbfff640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1bc15c570 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac007f30 tx=0x7ff1ac011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1b8ff9640 1 -- 172.21.15.64:0/826016920 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac00e030 con 0x7ff1a8003f40 2023-11-24T08:06:39.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1b8ff9640 1 -- 172.21.15.64:0/826016920 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac016470 con 0x7ff1a8003f40 2023-11-24T08:06:39.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1b8ff9640 1 -- 172.21.15.64:0/826016920 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac016820 con 0x7ff1a8003f40 2023-11-24T08:06:39.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.014+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc15d4d0 con 0x7ff1a8003f40 2023-11-24T08:06:39.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.015+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc09cb80 con 0x7ff1a8003f40 2023-11-24T08:06:39.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.016+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc0676c0 con 0x7ff1a8003f40 2023-11-24T08:06:39.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.017+0000 7ff1b8ff9640 1 -- 172.21.15.64:0/826016920 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff1ac016980 con 0x7ff1a8003f40 2023-11-24T08:06:39.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.018+0000 7ff1b8ff9640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff19c06b080 0x7ff19c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.018+0000 7ff1b8ff9640 1 -- 172.21.15.64:0/826016920 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff1ac00f070 con 0x7ff1a8003f40 2023-11-24T08:06:39.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.018+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff19c06b080 0x7ff19c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.019+0000 7ff1bb7fe640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff19c06b080 0x7ff19c06d540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4036000 tx=0x7ff1a4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.022+0000 7ff1b8ff9640 1 -- 172.21.15.64:0/826016920 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1ac055840 con 0x7ff1a8003f40 2023-11-24T08:06:39.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.218+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1bc067ee0 con 0x7ff1a8003f40 2023-11-24T08:06:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.220+0000 7ff1b8ff9640 1 -- 172.21.15.64:0/826016920 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16620 (secure 0 0 0) 0x7ff1ac058e60 con 0x7ff1a8003f40 2023-11-24T08:06:39.222 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:39.222 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":48,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:38.040815+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:02.427519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:03.560696+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.9","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.222+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff19c06b080 msgr2=0x7ff19c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.222+0000 7ff1c22cf640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff19c06b080 0x7ff19c06d540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4036000 tx=0x7ff1a4030000 comp rx=0 tx=0).stop 2023-11-24T08:06:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.222+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 msgr2=0x7ff1bc15c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.222+0000 7ff1c22cf640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1bc15c570 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac007f30 tx=0x7ff1ac011660 comp rx=0 tx=0).stop 2023-11-24T08:06:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.223+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 shutdown_connections 2023-11-24T08:06:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.223+0000 7ff1c22cf640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff19c06b080 0x7ff19c06d540 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.223+0000 7ff1c22cf640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1bc099d80 0x7ff1bc09c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.223+0000 7ff1c22cf640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1bc15dcd0 0x7ff1bc099840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.223+0000 7ff1c22cf640 1 --2- 172.21.15.64:0/826016920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1a8003f40 0x7ff1bc15c570 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.223+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 >> 172.21.15.64:0/826016920 conn(0x7ff1bc0b0d80 msgr2=0x7ff1bc1521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.223+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 shutdown_connections 2023-11-24T08:06:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.223+0000 7ff1c22cf640 1 -- 172.21.15.64:0/826016920 wait complete. 2023-11-24T08:06:39.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:06:02.427519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:06:03.560696+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 46, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:06:39.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:06:39.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.474+0000 7fdf407dc640 1 Processor -- start 2023-11-24T08:06:39.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.474+0000 7fdf407dc640 1 -- start start 2023-11-24T08:06:39.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.476+0000 7fdf407dc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf380ae490 0x7fdf38070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.476+0000 7fdf407dc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3806d7a0 con 0x7fdf380acd70 2023-11-24T08:06:39.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.476+0000 7fdf407dc640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf380ae900 con 0x7fdf3806dc30 2023-11-24T08:06:39.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.476+0000 7fdf407dc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3815a5b0 con 0x7fdf380ae490 2023-11-24T08:06:39.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.476+0000 7fdf3e578640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdf380acd70 0x7fdf380ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45692/0 (socket says 172.21.15.64:45692) 2023-11-24T08:06:39.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.476+0000 7fdf3e578640 1 -- 172.21.15.64:0/619149356 learned_addr learned my addr 172.21.15.64:0/619149356 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:39.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.476+0000 7fdf3ed79640 1 --2- 172.21.15.64:0/619149356 >> v2:172.21.15.64:3300/0 conn(0x7fdf380ae490 0x7fdf38070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.477+0000 7fdf3ed79640 1 -- 172.21.15.64:0/619149356 >> v1:172.21.15.91:6789/0 conn(0x7fdf3806dc30 legacy=0x7fdf38070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.477+0000 7fdf3d576640 1 -- 172.21.15.64:0/619149356 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354426573 0 0) 0x7fdf3806d7a0 con 0x7fdf380acd70 2023-11-24T08:06:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.477+0000 7fdf3ed79640 1 -- 172.21.15.64:0/619149356 >> v1:172.21.15.64:6789/0 conn(0x7fdf380acd70 legacy=0x7fdf380ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.477+0000 7fdf3ed79640 1 -- 172.21.15.64:0/619149356 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3806ce70 con 0x7fdf380ae490 2023-11-24T08:06:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.478+0000 7fdf3ed79640 1 --2- 172.21.15.64:0/619149356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf38070ba0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009d90 tx=0x7fdf3002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e0acb3cd54182d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.478+0000 7fdf3d576640 1 -- 172.21.15.64:0/619149356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3002cd00 con 0x7fdf380ae490 2023-11-24T08:06:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.478+0000 7fdf3d576640 1 -- 172.21.15.64:0/619149356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3002ce60 con 0x7fdf380ae490 2023-11-24T08:06:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.478+0000 7fdf3d576640 1 -- 172.21.15.64:0/619149356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30034660 con 0x7fdf380ae490 2023-11-24T08:06:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.479+0000 7fdf407dc640 1 -- 172.21.15.64:0/619149356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 msgr2=0x7fdf38070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.479+0000 7fdf407dc640 1 --2- 172.21.15.64:0/619149356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf38070ba0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009d90 tx=0x7fdf3002bf50 comp rx=0 tx=0).stop 2023-11-24T08:06:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.479+0000 7fdf407dc640 1 -- 172.21.15.64:0/619149356 shutdown_connections 2023-11-24T08:06:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.479+0000 7fdf407dc640 1 --2- 172.21.15.64:0/619149356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf38070ba0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.479+0000 7fdf407dc640 1 -- 172.21.15.64:0/619149356 >> 172.21.15.64:0/619149356 conn(0x7fdf380b0ba0 msgr2=0x7fdf380b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.479+0000 7fdf407dc640 1 -- 172.21.15.64:0/619149356 shutdown_connections 2023-11-24T08:06:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.479+0000 7fdf407dc640 1 -- 172.21.15.64:0/619149356 wait complete. 2023-11-24T08:06:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.481+0000 7fdf407dc640 1 Processor -- start 2023-11-24T08:06:39.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.481+0000 7fdf407dc640 1 -- start start 2023-11-24T08:06:39.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf407dc640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3806dc30 0x7fdf3815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf407dc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf380acd70 0x7fdf38158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf407dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf381e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf407dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38157cc0 con 0x7fdf380ae490 2023-11-24T08:06:39.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf407dc640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38158280 con 0x7fdf3806dc30 2023-11-24T08:06:39.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf407dc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf381593c0 con 0x7fdf380acd70 2023-11-24T08:06:39.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf3ed79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf381e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf3ed79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf381e6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52514/0 (socket says 172.21.15.64:52514) 2023-11-24T08:06:39.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.482+0000 7fdf3ed79640 1 -- 172.21.15.64:0/3874829611 learned_addr learned my addr 172.21.15.64:0/3874829611 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:39.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3e578640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3806dc30 0x7fdf3815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3dd77640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf380acd70 0x7fdf38158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3ed79640 1 -- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf380acd70 msgr2=0x7fdf38158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3ed79640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf380acd70 0x7fdf38158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3ed79640 1 -- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3806dc30 msgr2=0x7fdf3815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3ed79640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3806dc30 0x7fdf3815baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3ed79640 1 -- 172.21.15.64:0/3874829611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30009a70 con 0x7fdf380ae490 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3dd77640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf380acd70 0x7fdf38158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.483+0000 7fdf3ed79640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf381e6d20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009d90 tx=0x7fdf30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.484+0000 7fdf2effd640 1 -- 172.21.15.64:0/3874829611 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3003d600 con 0x7fdf380ae490 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.484+0000 7fdf2effd640 1 -- 172.21.15.64:0/3874829611 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3003d780 con 0x7fdf380ae490 2023-11-24T08:06:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.484+0000 7fdf2effd640 1 -- 172.21.15.64:0/3874829611 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3003db30 con 0x7fdf380ae490 2023-11-24T08:06:39.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.484+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf38151d00 con 0x7fdf380ae490 2023-11-24T08:06:39.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.484+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3815c460 con 0x7fdf380ae490 2023-11-24T08:06:39.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.485+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf381583f0 con 0x7fdf380ae490 2023-11-24T08:06:39.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.486+0000 7fdf2effd640 1 -- 172.21.15.64:0/3874829611 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fdf3003b040 con 0x7fdf380ae490 2023-11-24T08:06:39.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.487+0000 7fdf2effd640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdf1c06b080 0x7fdf1c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.487+0000 7fdf2effd640 1 -- 172.21.15.64:0/3874829611 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdf300abbb0 con 0x7fdf380ae490 2023-11-24T08:06:39.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.487+0000 7fdf3e578640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdf1c06b080 0x7fdf1c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.489+0000 7fdf3e578640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdf1c06b080 0x7fdf1c06d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdf34000f60 tx=0x7fdf34007d20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.491+0000 7fdf2effd640 1 -- 172.21.15.64:0/3874829611 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf30076800 con 0x7fdf380ae490 2023-11-24T08:06:39.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.687+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf3809d200 con 0x7fdf380ae490 2023-11-24T08:06:39.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.688+0000 7fdf2effd640 1 -- 172.21.15.64:0/3874829611 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf300390a0 con 0x7fdf380ae490 2023-11-24T08:06:39.689 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:06:39.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdf1c06b080 msgr2=0x7fdf1c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdf1c06b080 0x7fdf1c06d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdf34000f60 tx=0x7fdf34007d20 comp rx=0 tx=0).stop 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 msgr2=0x7fdf381e6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf381e6d20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009d90 tx=0x7fdf30004a40 comp rx=0 tx=0).stop 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 shutdown_connections 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdf1c06b080 0x7fdf1c06d540 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380ae490 0x7fdf381e6d20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf380acd70 0x7fdf38158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 --2- 172.21.15.64:0/3874829611 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf3806dc30 0x7fdf3815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 >> 172.21.15.64:0/3874829611 conn(0x7fdf380b0ba0 msgr2=0x7fdf380af260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.690+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 shutdown_connections 2023-11-24T08:06:39.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.691+0000 7fdf407dc640 1 -- 172.21.15.64:0/3874829611 wait complete. 2023-11-24T08:06:39.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:06:39.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.941+0000 7f843e127640 1 Processor -- start 2023-11-24T08:06:39.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.942+0000 7f843e127640 1 -- start start 2023-11-24T08:06:39.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.943+0000 7f843e127640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f843806c9d0 0x7f8438070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.944+0000 7f843e127640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84380b2540 con 0x7f8438071310 2023-11-24T08:06:39.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.944+0000 7f843e127640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8438071710 con 0x7f84380cbb40 2023-11-24T08:06:39.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.944+0000 7f843e127640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843806ce40 con 0x7f843806c9d0 2023-11-24T08:06:39.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.944+0000 7f843d926640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8438071310 0x7f84380b3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45704/0 (socket says 172.21.15.64:45704) 2023-11-24T08:06:39.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.944+0000 7f843d926640 1 -- 172.21.15.64:0/924976664 learned_addr learned my addr 172.21.15.64:0/924976664 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.944+0000 7f843c924640 1 --2- 172.21.15.64:0/924976664 >> v2:172.21.15.91:3300/0 conn(0x7f843806c9d0 0x7f8438070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.944+0000 7f842ffff640 1 -- 172.21.15.64:0/924976664 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103001319 0 0) 0x7f84380b2540 con 0x7f8438071310 2023-11-24T08:06:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.945+0000 7f842ffff640 1 -- 172.21.15.64:0/924976664 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8424003520 con 0x7f8438071310 2023-11-24T08:06:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.945+0000 7f842ffff640 1 -- 172.21.15.64:0/924976664 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3857796028 0 0) 0x7f8424003520 con 0x7f8438071310 2023-11-24T08:06:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.945+0000 7f843c924640 1 -- 172.21.15.64:0/924976664 >> v1:172.21.15.91:6789/0 conn(0x7f84380cbb40 legacy=0x7f843815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.945+0000 7f843c924640 1 -- 172.21.15.64:0/924976664 >> v1:172.21.15.64:6789/0 conn(0x7f8438071310 legacy=0x7f84380b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.945+0000 7f843c924640 1 -- 172.21.15.64:0/924976664 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843815b3a0 con 0x7f843806c9d0 2023-11-24T08:06:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.946+0000 7f843c924640 1 --2- 172.21.15.64:0/924976664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 0x7f8438070dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8428009790 tx=0x7f842802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a8848f9e8bd6322 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.947+0000 7f842ffff640 1 -- 172.21.15.64:0/924976664 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842802f050 con 0x7f843806c9d0 2023-11-24T08:06:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.947+0000 7f842ffff640 1 -- 172.21.15.64:0/924976664 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8428031500 con 0x7f843806c9d0 2023-11-24T08:06:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.947+0000 7f842ffff640 1 -- 172.21.15.64:0/924976664 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300019f0 con 0x7f843806c9d0 2023-11-24T08:06:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.948+0000 7f843e127640 1 -- 172.21.15.64:0/924976664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 msgr2=0x7f8438070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.948+0000 7f843e127640 1 --2- 172.21.15.64:0/924976664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 0x7f8438070dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8428009790 tx=0x7f842802b820 comp rx=0 tx=0).stop 2023-11-24T08:06:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.948+0000 7f843e127640 1 -- 172.21.15.64:0/924976664 shutdown_connections 2023-11-24T08:06:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.948+0000 7f843e127640 1 --2- 172.21.15.64:0/924976664 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 0x7f8438070dd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.948+0000 7f843e127640 1 -- 172.21.15.64:0/924976664 >> 172.21.15.64:0/924976664 conn(0x7f84380b15c0 msgr2=0x7f84380b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.948+0000 7f843e127640 1 -- 172.21.15.64:0/924976664 shutdown_connections 2023-11-24T08:06:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.948+0000 7f843e127640 1 -- 172.21.15.64:0/924976664 wait complete. 2023-11-24T08:06:39.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.950+0000 7f843e127640 1 Processor -- start 2023-11-24T08:06:39.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.950+0000 7f843e127640 1 -- start start 2023-11-24T08:06:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843e127640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 0x7f843814d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843e127640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8438071310 0x7f84381f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843e127640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84380cbb40 0x7f84380cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843e127640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843814d600 con 0x7f84380cbb40 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843e127640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84381f3330 con 0x7f843806c9d0 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843e127640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8438151580 con 0x7f8438071310 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843d926640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84380cbb40 0x7f84380cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843d926640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84380cbb40 0x7f84380cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52526/0 (socket says 172.21.15.64:52526) 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843d926640 1 -- 172.21.15.64:0/1435708302 learned_addr learned my addr 172.21.15.64:0/1435708302 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843d125640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 0x7f843814d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.951+0000 7f843c924640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8438071310 0x7f84381f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f843d926640 1 -- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8438071310 msgr2=0x7f84381f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f843d926640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8438071310 0x7f84381f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f843d926640 1 -- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 msgr2=0x7f843814d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f843d926640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 0x7f843814d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f843d926640 1 -- 172.21.15.64:0/1435708302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84280093a0 con 0x7f84380cbb40 2023-11-24T08:06:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f843d926640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84380cbb40 0x7f84380cdab0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8430001830 tx=0x7f843000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f842dffb640 1 -- 172.21.15.64:0/1435708302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430012400 con 0x7f84380cbb40 2023-11-24T08:06:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f842dffb640 1 -- 172.21.15.64:0/1435708302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430012560 con 0x7f84380cbb40 2023-11-24T08:06:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f842dffb640 1 -- 172.21.15.64:0/1435708302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430012910 con 0x7f84380cbb40 2023-11-24T08:06:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8438151860 con 0x7f84380cbb40 2023-11-24T08:06:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.952+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8438151e20 con 0x7f84380cbb40 2023-11-24T08:06:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.954+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843804e840 con 0x7f84380cbb40 2023-11-24T08:06:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.955+0000 7f842dffb640 1 -- 172.21.15.64:0/1435708302 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8430012a70 con 0x7f84380cbb40 2023-11-24T08:06:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.955+0000 7f842dffb640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f841c06b080 0x7f841c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.956+0000 7f842dffb640 1 -- 172.21.15.64:0/1435708302 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f843008a250 con 0x7f84380cbb40 2023-11-24T08:06:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.956+0000 7f843d125640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f841c06b080 0x7f841c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:39.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.957+0000 7f843d125640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f841c06b080 0x7f841c06d540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8434000f60 tx=0x7f8434009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:39.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:39.960+0000 7f842dffb640 1 -- 172.21.15.64:0/1435708302 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8430054f30 con 0x7f84380cbb40 2023-11-24T08:06:40.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.157+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f843806baa0 con 0x7f84380cbb40 2023-11-24T08:06:40.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.158+0000 7f842dffb640 1 -- 172.21.15.64:0/1435708302 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8430058550 con 0x7f84380cbb40 2023-11-24T08:06:40.159 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.160+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f841c06b080 msgr2=0x7f841c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.160+0000 7f843e127640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f841c06b080 0x7f841c06d540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8434000f60 tx=0x7f8434009040 comp rx=0 tx=0).stop 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84380cbb40 msgr2=0x7f84380cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84380cbb40 0x7f84380cdab0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8430001830 tx=0x7f843000a430 comp rx=0 tx=0).stop 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 shutdown_connections 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f841c06b080 0x7f841c06d540 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84380cbb40 0x7f84380cdab0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8438071310 0x7f84381f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 --2- 172.21.15.64:0/1435708302 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f843806c9d0 0x7f843814d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 >> 172.21.15.64:0/1435708302 conn(0x7f84380b15c0 msgr2=0x7f843815abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:40.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 shutdown_connections 2023-11-24T08:06:40.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.161+0000 7f843e127640 1 -- 172.21.15.64:0/1435708302 wait complete. 2023-11-24T08:06:40.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:06:40.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.406+0000 7fd522911640 1 Processor -- start 2023-11-24T08:06:40.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.406+0000 7fd522911640 1 -- start start 2023-11-24T08:06:40.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.407+0000 7fd522911640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd51c15afc0 0x7fd51c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.407+0000 7fd522911640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c0cb870 con 0x7fd51c0d0d10 2023-11-24T08:06:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.408+0000 7fd522911640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c0ced50 con 0x7fd51c0cbd10 2023-11-24T08:06:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.408+0000 7fd522911640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0cdcc0 con 0x7fd51c15afc0 2023-11-24T08:06:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.408+0000 7fd52110e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd51c0d0d10 0x7fd51c0d11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45712/0 (socket says 172.21.15.64:45712) 2023-11-24T08:06:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.408+0000 7fd52190f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd51c15afc0 0x7fd51c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.408+0000 7fd52110e640 1 -- 172.21.15.64:0/2251751806 learned_addr learned my addr 172.21.15.64:0/2251751806 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.408+0000 7fd52190f640 1 -- 172.21.15.64:0/2251751806 >> v1:172.21.15.91:6789/0 conn(0x7fd51c0cbd10 legacy=0x7fd51c0cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.408+0000 7fd52190f640 1 -- 172.21.15.64:0/2251751806 >> v1:172.21.15.64:6789/0 conn(0x7fd51c0d0d10 legacy=0x7fd51c0d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.409+0000 7fd52190f640 1 -- 172.21.15.64:0/2251751806 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0cdfa0 con 0x7fd51c15afc0 2023-11-24T08:06:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.409+0000 7fd52090d640 1 -- 172.21.15.64:0/2251751806 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713325068 0 0) 0x7fd51c0cb870 con 0x7fd51c0d0d10 2023-11-24T08:06:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.410+0000 7fd52190f640 1 --2- 172.21.15.64:0/2251751806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c15afc0 0x7fd51c0b21e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd5180097a0 tx=0x7fd51802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c608ff15d93f822 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:40.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.410+0000 7fd52090d640 1 -- 172.21.15.64:0/2251751806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518031830 con 0x7fd51c15afc0 2023-11-24T08:06:40.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.410+0000 7fd52090d640 1 -- 172.21.15.64:0/2251751806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518031990 con 0x7fd51c15afc0 2023-11-24T08:06:40.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.410+0000 7fd52090d640 1 -- 172.21.15.64:0/2251751806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518031d20 con 0x7fd51c15afc0 2023-11-24T08:06:40.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.411+0000 7fd522911640 1 -- 172.21.15.64:0/2251751806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c15afc0 msgr2=0x7fd51c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.411+0000 7fd522911640 1 --2- 172.21.15.64:0/2251751806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c15afc0 0x7fd51c0b21e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd5180097a0 tx=0x7fd51802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.411+0000 7fd522911640 1 -- 172.21.15.64:0/2251751806 shutdown_connections 2023-11-24T08:06:40.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.411+0000 7fd522911640 1 --2- 172.21.15.64:0/2251751806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c15afc0 0x7fd51c0b21e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.411+0000 7fd522911640 1 -- 172.21.15.64:0/2251751806 >> 172.21.15.64:0/2251751806 conn(0x7fd51c099a00 msgr2=0x7fd51c099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:40.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.411+0000 7fd522911640 1 -- 172.21.15.64:0/2251751806 shutdown_connections 2023-11-24T08:06:40.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.411+0000 7fd522911640 1 -- 172.21.15.64:0/2251751806 wait complete. 2023-11-24T08:06:40.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.413+0000 7fd522911640 1 Processor -- start 2023-11-24T08:06:40.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.413+0000 7fd522911640 1 -- start start 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd522911640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c0cbd10 0x7fd51c1f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd522911640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd51c0d0d10 0x7fd51c155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd522911640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd51c15afc0 0x7fd51c06c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd522911640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c1f74c0 con 0x7fd51c0cbd10 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd522911640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c157ac0 con 0x7fd51c0d0d10 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd522911640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0c9c30 con 0x7fd51c15afc0 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd52190f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c0cbd10 0x7fd51c1f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd52190f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c0cbd10 0x7fd51c1f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52540/0 (socket says 172.21.15.64:52540) 2023-11-24T08:06:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd52190f640 1 -- 172.21.15.64:0/1259801308 learned_addr learned my addr 172.21.15.64:0/1259801308 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:40.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd52190f640 1 -- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd51c15afc0 msgr2=0x7fd51c06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd52190f640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd51c15afc0 0x7fd51c06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd52190f640 1 -- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd51c0d0d10 msgr2=0x7fd51c155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd52190f640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd51c0d0d10 0x7fd51c155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.414+0000 7fd52190f640 1 -- 172.21.15.64:0/1259801308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518009400 con 0x7fd51c0cbd10 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.415+0000 7fd52190f640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c0cbd10 0x7fd51c1f6f80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd5180098c0 tx=0x7fd5180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.415+0000 7fd5127fc640 1 -- 172.21.15.64:0/1259801308 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51803aa60 con 0x7fd51c0cbd10 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.415+0000 7fd5127fc640 1 -- 172.21.15.64:0/1259801308 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51803abc0 con 0x7fd51c0cbd10 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.415+0000 7fd5127fc640 1 -- 172.21.15.64:0/1259801308 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180443f0 con 0x7fd51c0cbd10 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.415+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c1f3540 con 0x7fd51c0cbd10 2023-11-24T08:06:40.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.415+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c1f3b00 con 0x7fd51c0cbd10 2023-11-24T08:06:40.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.416+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c0c9e00 con 0x7fd51c0cbd10 2023-11-24T08:06:40.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.417+0000 7fd5127fc640 1 -- 172.21.15.64:0/1259801308 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd51804d450 con 0x7fd51c0cbd10 2023-11-24T08:06:40.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.418+0000 7fd5127fc640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd50006b080 0x7fd50006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.418+0000 7fd5127fc640 1 -- 172.21.15.64:0/1259801308 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd5180abdf0 con 0x7fd51c0cbd10 2023-11-24T08:06:40.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.419+0000 7fd52110e640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd50006b080 0x7fd50006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.420+0000 7fd52110e640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd50006b080 0x7fd50006d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd50c007fd0 tx=0x7fd50c007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:40.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.422+0000 7fd5127fc640 1 -- 172.21.15.64:0/1259801308 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd51c0c9e00 con 0x7fd51c0cbd10 2023-11-24T08:06:40.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.618+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd51c0c9e00 con 0x7fd51c0cbd10 2023-11-24T08:06:40.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.620+0000 7fd5127fc640 1 -- 172.21.15.64:0/1259801308 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd51c0c9e00 con 0x7fd51c0cbd10 2023-11-24T08:06:40.621 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.622+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd50006b080 msgr2=0x7fd50006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.622+0000 7fd522911640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd50006b080 0x7fd50006d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd50c007fd0 tx=0x7fd50c007920 comp rx=0 tx=0).stop 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.622+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c0cbd10 msgr2=0x7fd51c1f6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.622+0000 7fd522911640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c0cbd10 0x7fd51c1f6f80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd5180098c0 tx=0x7fd5180046c0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.623+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 shutdown_connections 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.623+0000 7fd522911640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd50006b080 0x7fd50006d540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.623+0000 7fd522911640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd51c15afc0 0x7fd51c06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.623+0000 7fd522911640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd51c0d0d10 0x7fd51c155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.623+0000 7fd522911640 1 --2- 172.21.15.64:0/1259801308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd51c0cbd10 0x7fd51c1f6f80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.623+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 >> 172.21.15.64:0/1259801308 conn(0x7fd51c099a00 msgr2=0x7fd51c159750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:40.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.623+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 shutdown_connections 2023-11-24T08:06:40.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.623+0000 7fd522911640 1 -- 172.21.15.64:0/1259801308 wait complete. 2023-11-24T08:06:40.645 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:06:40.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.863+0000 7f126a8ea640 1 Processor -- start 2023-11-24T08:06:40.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.864+0000 7f126a8ea640 1 -- start start 2023-11-24T08:06:40.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.865+0000 7f126a8ea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 0x7f12640d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.865+0000 7f126a8ea640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f12640b1650 0x7f12640d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.865+0000 7f126a8ea640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12640cf0c0 con 0x7f12640d14a0 2023-11-24T08:06:40.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.865+0000 7f126a8ea640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12640b01b0 con 0x7f12640b1650 2023-11-24T08:06:40.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.865+0000 7f126a8ea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126415d990 con 0x7f126415d5b0 2023-11-24T08:06:40.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.865+0000 7f12690e7640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f12640d14a0 0x7f12640cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45716/0 (socket says 172.21.15.64:45716) 2023-11-24T08:06:40.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.865+0000 7f12690e7640 1 -- 172.21.15.64:0/4025107231 learned_addr learned my addr 172.21.15.64:0/4025107231 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f126a0e9640 1 --2- 172.21.15.64:0/4025107231 >> v2:172.21.15.91:3300/0 conn(0x7f12640b1650 0x7f12640d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959997836 0 0) 0x7f12640cf0c0 con 0x7f12640d14a0 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12698e8640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 0x7f12640d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1250003520 con 0x7f12640d14a0 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242992987 0 0) 0x7f1250003520 con 0x7f12640d14a0 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 msgr2=0x7f12640d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 0x7f12640d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 >> v2:172.21.15.91:3300/0 conn(0x7f12640b1650 msgr2=0x7f12640d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 --2- 172.21.15.64:0/4025107231 >> v2:172.21.15.91:3300/0 conn(0x7f12640b1650 0x7f12640d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12640d1e40 con 0x7f12640d14a0 2023-11-24T08:06:40.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f1254001670 con 0x7f12640d14a0 2023-11-24T08:06:40.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12698e8640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 0x7f12640d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:06:40.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 >> v1:172.21.15.64:6789/0 conn(0x7f12640d14a0 legacy=0x7f12640cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.866+0000 7f12688e6640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12500039e0 0x7f1250003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f12688e6640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 0x7f12500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f12688e6640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1250004d00 0x7f1250005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12640b01b0 con 0x7f1250004d00 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126415d990 con 0x7f12500039e0 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125000b9b0 con 0x7f126415d5b0 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f126a0e9640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1250004d00 0x7f1250005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f12698e8640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12500039e0 0x7f1250003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f12690e7640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 0x7f12500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f126a0e9640 1 -- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 msgr2=0x7f12500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f126a0e9640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f126415d5b0 0x7f12500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f126a0e9640 1 -- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12500039e0 msgr2=0x7f1250003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f126a0e9640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12500039e0 0x7f1250003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.867+0000 7f126a0e9640 1 -- 172.21.15.64:0/4025107231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1250003520 con 0x7f1250004d00 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.869+0000 7f126a0e9640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1250004d00 0x7f1250005180 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f125c009fc0 tx=0x7f12500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ced9d62744d879 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.869+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c02cc60 con 0x7f1250004d00 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.869+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c00f040 con 0x7f1250004d00 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.869+0000 7f12688e6640 1 -- 172.21.15.64:0/4025107231 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c034440 con 0x7f1250004d00 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.869+0000 7f126a8ea640 1 -- 172.21.15.64:0/4025107231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1250004d00 msgr2=0x7f1250005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.869+0000 7f126a8ea640 1 --2- 172.21.15.64:0/4025107231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1250004d00 0x7f1250005180 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f125c009fc0 tx=0x7f12500057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.869+0000 7f126a0e9640 1 -- 172.21.15.64:0/4025107231 reap_dead start 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.870+0000 7f126a8ea640 1 -- 172.21.15.64:0/4025107231 shutdown_connections 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.870+0000 7f126a8ea640 1 -- 172.21.15.64:0/4025107231 >> 172.21.15.64:0/4025107231 conn(0x7f126409c3b0 msgr2=0x7f12640d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.870+0000 7f126a8ea640 1 -- 172.21.15.64:0/4025107231 shutdown_connections 2023-11-24T08:06:40.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.870+0000 7f126a8ea640 1 -- 172.21.15.64:0/4025107231 wait complete. 2023-11-24T08:06:40.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.872+0000 7f126a8ea640 1 Processor -- start 2023-11-24T08:06:40.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.872+0000 7f126a8ea640 1 -- start start 2023-11-24T08:06:40.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f126a8ea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1250004d00 0x7f126415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f126a8ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12640b7870 0x7f12640c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f126a8ea640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12640c81e0 0x7f12640b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f126a8ea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12640c8660 con 0x7f12640b7870 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f126a8ea640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12640b73e0 con 0x7f12640c81e0 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f126a8ea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12640b67d0 con 0x7f1250004d00 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f12690e7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12640b7870 0x7f12640c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f12690e7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12640b7870 0x7f12640c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52562/0 (socket says 172.21.15.64:52562) 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f12690e7640 1 -- 172.21.15.64:0/3771212142 learned_addr learned my addr 172.21.15.64:0/3771212142 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f12698e8640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1250004d00 0x7f126415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f12690e7640 1 -- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1250004d00 msgr2=0x7f126415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f12690e7640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1250004d00 0x7f126415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f126a0e9640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12640c81e0 0x7f12640b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.873+0000 7f12690e7640 1 -- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12640c81e0 msgr2=0x7f12640b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:40.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.874+0000 7f12690e7640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12640c81e0 0x7f12640b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:40.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.874+0000 7f12690e7640 1 -- 172.21.15.64:0/3771212142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12640b6ab0 con 0x7f12640b7870 2023-11-24T08:06:40.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.874+0000 7f12690e7640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12640b7870 0x7f12640c7ca0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f125400e840 tx=0x7f125400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:40.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.874+0000 7f125a7fc640 1 -- 172.21.15.64:0/3771212142 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125400c8d0 con 0x7f12640b7870 2023-11-24T08:06:40.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.874+0000 7f125a7fc640 1 -- 172.21.15.64:0/3771212142 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125400ca50 con 0x7f12640b7870 2023-11-24T08:06:40.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.874+0000 7f125a7fc640 1 -- 172.21.15.64:0/3771212142 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125400ce00 con 0x7f12640b7870 2023-11-24T08:06:40.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.874+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12640b6d90 con 0x7f12640b7870 2023-11-24T08:06:40.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.874+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12640b8550 con 0x7f12640b7870 2023-11-24T08:06:40.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.876+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126404e840 con 0x7f12640b7870 2023-11-24T08:06:40.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.877+0000 7f125a7fc640 1 -- 172.21.15.64:0/3771212142 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f125401d460 con 0x7f12640b7870 2023-11-24T08:06:40.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.877+0000 7f125a7fc640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f124806b080 0x7f124806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:40.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.878+0000 7f125a7fc640 1 -- 172.21.15.64:0/3771212142 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1254089ad0 con 0x7f12640b7870 2023-11-24T08:06:40.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.878+0000 7f12698e8640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f124806b080 0x7f124806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.879+0000 7f12698e8640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f124806b080 0x7f124806d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1260006fd0 tx=0x7f1260008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:40.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:40.882+0000 7f125a7fc640 1 -- 172.21.15.64:0/3771212142 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12540547f0 con 0x7f12640b7870 2023-11-24T08:06:41.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.114+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12640678d0 con 0x7f12640b7870 2023-11-24T08:06:41.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.115+0000 7f125a7fc640 1 -- 172.21.15.64:0/3771212142 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f1254057e10 con 0x7f12640b7870 2023-11-24T08:06:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:41.117 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/618780376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":618780376},{"type":"v1","addr":"172.21.15.64:6835","nonce":618780376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1212789084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1212789084},{"type":"v1","addr":"172.21.15.91:6836","nonce":1212789084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1895191218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1895191218},{"type":"v1","addr":"172.21.15.91:6837","nonce":1895191218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:04.548725+0000","modified":"2023-11-24T08:06:07.601727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/1271535459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1271535459},{"type":"v1","addr":"172.21.15.64:6834","nonce":1271535459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:06:41.118 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2023-11-24T08:06:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f124806b080 msgr2=0x7f124806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f124806b080 0x7f124806d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1260006fd0 tx=0x7f1260008040 comp rx=0 tx=0).stop 2023-11-24T08:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12640b7870 msgr2=0x7f12640c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12640b7870 0x7f12640c7ca0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f125400e840 tx=0x7f125400ed00 comp rx=0 tx=0).stop 2023-11-24T08:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 shutdown_connections 2023-11-24T08:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f124806b080 0x7f124806d540 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f12640c81e0 0x7f12640b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12640b7870 0x7f12640c7ca0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 --2- 172.21.15.64:0/3771212142 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1250004d00 0x7f126415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.117+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 >> 172.21.15.64:0/3771212142 conn(0x7f126409c3b0 msgr2=0x7f12640d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:41.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.118+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 shutdown_connections 2023-11-24T08:06:41.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.118+0000 7f126a8ea640 1 -- 172.21.15.64:0/3771212142 wait complete. 2023-11-24T08:06:41.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:41.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.354+0000 7f1ef164b640 1 Processor -- start 2023-11-24T08:06:41.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.354+0000 7f1ef164b640 1 -- start start 2023-11-24T08:06:41.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.356+0000 7f1ef164b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1eec13dfd0 0x7f1eec15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.356+0000 7f1ef164b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1eec0cf500 0x7f1eec06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.356+0000 7f1ef164b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec0ce7c0 con 0x7f1eec0cf050 2023-11-24T08:06:41.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.356+0000 7f1ef164b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec140900 con 0x7f1eec0cf500 2023-11-24T08:06:41.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.356+0000 7f1ef164b640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec15fa70 con 0x7f1eec13dfd0 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.356+0000 7f1ef0e4a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1eec0cf500 0x7f1eec06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.356+0000 7f1eeb7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1eec0cf050 0x7f1eec06c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45726/0 (socket says 172.21.15.64:45726) 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.356+0000 7f1eeb7fe640 1 -- 172.21.15.64:0/2138670390 learned_addr learned my addr 172.21.15.64:0/2138670390 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.357+0000 7f1eebfff640 1 --2- 172.21.15.64:0/2138670390 >> v2:172.21.15.91:3300/0 conn(0x7f1eec13dfd0 0x7f1eec15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.357+0000 7f1ef0e4a640 1 -- 172.21.15.64:0/2138670390 >> v2:172.21.15.91:3300/0 conn(0x7f1eec13dfd0 msgr2=0x7f1eec15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.357+0000 7f1ef0e4a640 1 --2- 172.21.15.64:0/2138670390 >> v2:172.21.15.91:3300/0 conn(0x7f1eec13dfd0 0x7f1eec15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.357+0000 7f1ef0e4a640 1 -- 172.21.15.64:0/2138670390 >> v1:172.21.15.64:6789/0 conn(0x7f1eec0cf050 legacy=0x7f1eec06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.357+0000 7f1eeaffd640 1 -- 172.21.15.64:0/2138670390 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721226074 0 0) 0x7f1eec0ce7c0 con 0x7f1eec0cf050 2023-11-24T08:06:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.357+0000 7f1ef0e4a640 1 -- 172.21.15.64:0/2138670390 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec15ee60 con 0x7f1eec0cf500 2023-11-24T08:06:41.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.358+0000 7f1ef0e4a640 1 --2- 172.21.15.64:0/2138670390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf500 0x7f1eec06c840 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0cfc10 tx=0x7f1ee402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fca227900365c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:41.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.358+0000 7f1eeaffd640 1 -- 172.21.15.64:0/2138670390 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee402cc70 con 0x7f1eec0cf500 2023-11-24T08:06:41.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.358+0000 7f1eeaffd640 1 -- 172.21.15.64:0/2138670390 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee402cdd0 con 0x7f1eec0cf500 2023-11-24T08:06:41.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.358+0000 7f1eeaffd640 1 -- 172.21.15.64:0/2138670390 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4034660 con 0x7f1eec0cf500 2023-11-24T08:06:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.359+0000 7f1ef164b640 1 -- 172.21.15.64:0/2138670390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf500 msgr2=0x7f1eec06c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.359+0000 7f1ef164b640 1 --2- 172.21.15.64:0/2138670390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf500 0x7f1eec06c840 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0cfc10 tx=0x7f1ee402bec0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.359+0000 7f1ef164b640 1 -- 172.21.15.64:0/2138670390 shutdown_connections 2023-11-24T08:06:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.359+0000 7f1ef164b640 1 --2- 172.21.15.64:0/2138670390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf500 0x7f1eec06c840 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.359+0000 7f1ef164b640 1 --2- 172.21.15.64:0/2138670390 >> v2:172.21.15.91:3300/0 conn(0x7f1eec13dfd0 0x7f1eec15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.359+0000 7f1ef164b640 1 -- 172.21.15.64:0/2138670390 >> 172.21.15.64:0/2138670390 conn(0x7f1eec06a860 msgr2=0x7f1eec06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.359+0000 7f1ef164b640 1 -- 172.21.15.64:0/2138670390 shutdown_connections 2023-11-24T08:06:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.359+0000 7f1ef164b640 1 -- 172.21.15.64:0/2138670390 wait complete. 2023-11-24T08:06:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.361+0000 7f1ef164b640 1 Processor -- start 2023-11-24T08:06:41.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.361+0000 7f1ef164b640 1 -- start start 2023-11-24T08:06:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1ef164b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf050 0x7f1eec137090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1ef164b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1eec0cf500 0x7f1eec157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1ef164b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec13dfd0 0x7f1eec0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1ef164b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec1f5030 con 0x7f1eec0cf050 2023-11-24T08:06:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1ef164b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec1375d0 con 0x7f1eec13dfd0 2023-11-24T08:06:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1ef164b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec1f44b0 con 0x7f1eec0cf500 2023-11-24T08:06:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1eebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf050 0x7f1eec137090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1eebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf050 0x7f1eec137090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52592/0 (socket says 172.21.15.64:52592) 2023-11-24T08:06:41.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.362+0000 7f1eebfff640 1 -- 172.21.15.64:0/910528727 learned_addr learned my addr 172.21.15.64:0/910528727 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:41.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1eebfff640 1 -- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1eec0cf500 msgr2=0x7f1eec157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1eebfff640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1eec0cf500 0x7f1eec157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1eebfff640 1 -- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec13dfd0 msgr2=0x7f1eec0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1ef0e4a640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec13dfd0 0x7f1eec0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1eebfff640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec13dfd0 0x7f1eec0b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1eebfff640 1 -- 172.21.15.64:0/910528727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee4009a70 con 0x7f1eec0cf050 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1eebfff640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf050 0x7f1eec137090 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00d5f0 tx=0x7f1edc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1ee8ff9640 1 -- 172.21.15.64:0/910528727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0125f0 con 0x7f1eec0cf050 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1ee8ff9640 1 -- 172.21.15.64:0/910528727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc012750 con 0x7f1eec0cf050 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1ee8ff9640 1 -- 172.21.15.64:0/910528727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc012b00 con 0x7f1eec0cf050 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.363+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec1f4790 con 0x7f1eec0cf050 2023-11-24T08:06:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.364+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec1f4d50 con 0x7f1eec0cf050 2023-11-24T08:06:41.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.365+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec04e840 con 0x7f1eec0cf050 2023-11-24T08:06:41.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.366+0000 7f1ee8ff9640 1 -- 172.21.15.64:0/910528727 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f1edc012c60 con 0x7f1eec0cf050 2023-11-24T08:06:41.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.367+0000 7f1ee8ff9640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b080 0x7f1ed806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.367+0000 7f1ee8ff9640 1 -- 172.21.15.64:0/910528727 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1edc089fa0 con 0x7f1eec0cf050 2023-11-24T08:06:41.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.367+0000 7f1eeb7fe640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b080 0x7f1ed806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.368+0000 7f1eeb7fe640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b080 0x7f1ed806d540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0007fd0 tx=0x7f1ee0007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:41.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.371+0000 7f1ee8ff9640 1 -- 172.21.15.64:0/910528727 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1edc054c80 con 0x7f1eec0cf050 2023-11-24T08:06:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.571+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1eec0ce9a0 con 0x7f1eec0cf050 2023-11-24T08:06:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.573+0000 7f1ee8ff9640 1 -- 172.21.15.64:0/910528727 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f1edc0582a0 con 0x7f1eec0cf050 2023-11-24T08:06:41.576 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:41.576 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":48,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:38.040815+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:02.427519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:03.560696+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.9","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:41.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.576+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b080 msgr2=0x7f1ed806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b080 0x7f1ed806d540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0007fd0 tx=0x7f1ee0007920 comp rx=0 tx=0).stop 2023-11-24T08:06:41.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf050 msgr2=0x7f1eec137090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf050 0x7f1eec137090 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00d5f0 tx=0x7f1edc00dab0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 shutdown_connections 2023-11-24T08:06:41.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b080 0x7f1ed806d540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1eec13dfd0 0x7f1eec0b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1eec0cf500 0x7f1eec157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 --2- 172.21.15.64:0/910528727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eec0cf050 0x7f1eec137090 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 >> 172.21.15.64:0/910528727 conn(0x7f1eec06a860 msgr2=0x7f1eec1596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 shutdown_connections 2023-11-24T08:06:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.577+0000 7f1ef164b640 1 -- 172.21.15.64:0/910528727 wait complete. 2023-11-24T08:06:41.599 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:06:41.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:06:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.822+0000 7f497286e640 1 Processor -- start 2023-11-24T08:06:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.822+0000 7f497286e640 1 -- start start 2023-11-24T08:06:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.824+0000 7f497286e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f496c0cf050 0x7f496c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.824+0000 7f497286e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496c140900 con 0x7f496c13dfd0 2023-11-24T08:06:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.824+0000 7f497286e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496c15fa70 con 0x7f496c0cd7b0 2023-11-24T08:06:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.824+0000 7f497286e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496c0cfa40 con 0x7f496c0cf050 2023-11-24T08:06:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.824+0000 7f497186c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f496c13dfd0 0x7f496c15be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45736/0 (socket says 172.21.15.64:45736) 2023-11-24T08:06:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.824+0000 7f497186c640 1 -- 172.21.15.64:0/1055923737 learned_addr learned my addr 172.21.15.64:0/1055923737 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.824+0000 7f497106b640 1 --2- 172.21.15.64:0/1055923737 >> v2:172.21.15.64:3300/0 conn(0x7f496c0cf050 0x7f496c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.825+0000 7f497086a640 1 -- 172.21.15.64:0/1055923737 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956539771 0 0) 0x7f496c140900 con 0x7f496c13dfd0 2023-11-24T08:06:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.825+0000 7f497086a640 1 -- 172.21.15.64:0/1055923737 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4958003520 con 0x7f496c13dfd0 2023-11-24T08:06:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.825+0000 7f497106b640 1 -- 172.21.15.64:0/1055923737 >> v1:172.21.15.91:6789/0 conn(0x7f496c0cd7b0 legacy=0x7f496c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.825+0000 7f497106b640 1 -- 172.21.15.64:0/1055923737 >> v1:172.21.15.64:6789/0 conn(0x7f496c13dfd0 legacy=0x7f496c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.825+0000 7f497106b640 1 -- 172.21.15.64:0/1055923737 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c15ee60 con 0x7f496c0cf050 2023-11-24T08:06:41.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.826+0000 7f497106b640 1 --2- 172.21.15.64:0/1055923737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c0cf050 0x7f496c0cf500 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f495c0097a0 tx=0x7f495c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3548de9ab0fa63c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:41.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.826+0000 7f497086a640 1 -- 172.21.15.64:0/1055923737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c030610 con 0x7f496c0cf050 2023-11-24T08:06:41.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.826+0000 7f497086a640 1 -- 172.21.15.64:0/1055923737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495c030770 con 0x7f496c0cf050 2023-11-24T08:06:41.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.826+0000 7f497086a640 1 -- 172.21.15.64:0/1055923737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c030b00 con 0x7f496c0cf050 2023-11-24T08:06:41.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.827+0000 7f497286e640 1 -- 172.21.15.64:0/1055923737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c0cf050 msgr2=0x7f496c0cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.827+0000 7f497286e640 1 --2- 172.21.15.64:0/1055923737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c0cf050 0x7f496c0cf500 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f495c0097a0 tx=0x7f495c02b960 comp rx=0 tx=0).stop 2023-11-24T08:06:41.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.827+0000 7f497286e640 1 -- 172.21.15.64:0/1055923737 shutdown_connections 2023-11-24T08:06:41.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.827+0000 7f497286e640 1 --2- 172.21.15.64:0/1055923737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c0cf050 0x7f496c0cf500 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.827+0000 7f497286e640 1 -- 172.21.15.64:0/1055923737 >> 172.21.15.64:0/1055923737 conn(0x7f496c06a860 msgr2=0x7f496c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:41.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.827+0000 7f497286e640 1 -- 172.21.15.64:0/1055923737 shutdown_connections 2023-11-24T08:06:41.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.827+0000 7f497286e640 1 -- 172.21.15.64:0/1055923737 wait complete. 2023-11-24T08:06:41.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.829+0000 7f497286e640 1 Processor -- start 2023-11-24T08:06:41.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.829+0000 7f497286e640 1 -- start start 2023-11-24T08:06:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.830+0000 7f497286e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f496c0cd7b0 0x7f496c0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.830+0000 7f497286e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f496c0cf050 0x7f496c13cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.830+0000 7f497286e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c13dfd0 0x7f496c0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.830+0000 7f497286e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c0b3640 con 0x7f496c13dfd0 2023-11-24T08:06:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.830+0000 7f497286e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c0b3360 con 0x7f496c0cf050 2023-11-24T08:06:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.830+0000 7f497286e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496c0b0950 con 0x7f496c0cd7b0 2023-11-24T08:06:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c13dfd0 0x7f496c0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c13dfd0 0x7f496c0b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52612/0 (socket says 172.21.15.64:52612) 2023-11-24T08:06:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 -- 172.21.15.64:0/2095163640 learned_addr learned my addr 172.21.15.64:0/2095163640 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497186c640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f496c0cd7b0 0x7f496c0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497106b640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f496c0cf050 0x7f496c13cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 -- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f496c0cd7b0 msgr2=0x7f496c0b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f496c0cd7b0 0x7f496c0b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 -- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f496c0cf050 msgr2=0x7f496c13cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f496c0cf050 0x7f496c13cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 -- 172.21.15.64:0/2095163640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c009400 con 0x7f496c13dfd0 2023-11-24T08:06:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497106b640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f496c0cf050 0x7f496c13cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:41.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.831+0000 7f497206d640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c13dfd0 0x7f496c0b1430 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f496400c7a0 tx=0x7f496400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.832+0000 7f49627fc640 1 -- 172.21.15.64:0/2095163640 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964003b70 con 0x7f496c13dfd0 2023-11-24T08:06:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.832+0000 7f49627fc640 1 -- 172.21.15.64:0/2095163640 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4964003cf0 con 0x7f496c13dfd0 2023-11-24T08:06:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.832+0000 7f49627fc640 1 -- 172.21.15.64:0/2095163640 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964003ea0 con 0x7f496c13dfd0 2023-11-24T08:06:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.832+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496c0b0c30 con 0x7f496c13dfd0 2023-11-24T08:06:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.832+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c0b3030 con 0x7f496c13dfd0 2023-11-24T08:06:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.833+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496c0b1150 con 0x7f496c13dfd0 2023-11-24T08:06:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.834+0000 7f49627fc640 1 -- 172.21.15.64:0/2095163640 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f496400aa90 con 0x7f496c13dfd0 2023-11-24T08:06:41.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.835+0000 7f49627fc640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f495006b0b0 0x7f495006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:41.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.835+0000 7f49627fc640 1 -- 172.21.15.64:0/2095163640 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4964089960 con 0x7f496c13dfd0 2023-11-24T08:06:41.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.836+0000 7f497186c640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f495006b0b0 0x7f495006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:41.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.837+0000 7f497186c640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f495006b0b0 0x7f495006d570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f49680011e0 tx=0x7f4968009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:41.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:41.839+0000 7f49627fc640 1 -- 172.21.15.64:0/2095163640 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f496c0b1150 con 0x7f496c13dfd0 2023-11-24T08:06:42.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.075+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f496c0b1150 con 0x7f496c13dfd0 2023-11-24T08:06:42.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.081+0000 7f49627fc640 1 -- 172.21.15.64:0/2095163640 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f496c0b1150 con 0x7f496c13dfd0 2023-11-24T08:06:42.083 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:42.084 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/618780376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":618780376},{"type":"v1","addr":"172.21.15.64:6835","nonce":618780376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1212789084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1212789084},{"type":"v1","addr":"172.21.15.91:6836","nonce":1212789084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1895191218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1895191218},{"type":"v1","addr":"172.21.15.91:6837","nonce":1895191218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:04.548725+0000","modified":"2023-11-24T08:06:07.601727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/1271535459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1271535459},{"type":"v1","addr":"172.21.15.64:6834","nonce":1271535459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:06:42.084 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2023-11-24T08:06:42.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f495006b0b0 msgr2=0x7f495006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f495006b0b0 0x7f495006d570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f49680011e0 tx=0x7f4968009040 comp rx=0 tx=0).stop 2023-11-24T08:06:42.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c13dfd0 msgr2=0x7f496c0b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c13dfd0 0x7f496c0b1430 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f496400c7a0 tx=0x7f496400cc60 comp rx=0 tx=0).stop 2023-11-24T08:06:42.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 shutdown_connections 2023-11-24T08:06:42.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f495006b0b0 0x7f495006d570 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c13dfd0 0x7f496c0b1430 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f496c0cf050 0x7f496c13cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 --2- 172.21.15.64:0/2095163640 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f496c0cd7b0 0x7f496c0b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 >> 172.21.15.64:0/2095163640 conn(0x7f496c06a860 msgr2=0x7f496c13ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:42.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 shutdown_connections 2023-11-24T08:06:42.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.084+0000 7f497286e640 1 -- 172.21.15.64:0/2095163640 wait complete. 2023-11-24T08:06:42.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:06:42.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.334+0000 7efd1bfff640 1 Processor -- start 2023-11-24T08:06:42.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.334+0000 7efd1bfff640 1 -- start start 2023-11-24T08:06:42.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.336+0000 7efd1bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd1c0cf760 0x7efd1c15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.336+0000 7efd1bfff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c071700 con 0x7efd1c143570 2023-11-24T08:06:42.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.336+0000 7efd1bfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c15b410 con 0x7efd1c143950 2023-11-24T08:06:42.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.336+0000 7efd1bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c15b130 con 0x7efd1c0cf760 2023-11-24T08:06:42.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.336+0000 7efd1affd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efd1c143570 0x7efd1c0cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45738/0 (socket says 172.21.15.64:45738) 2023-11-24T08:06:42.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.336+0000 7efd1affd640 1 -- 172.21.15.64:0/1285013765 learned_addr learned my addr 172.21.15.64:0/1285013765 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd1a7fc640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd1c0cf760 0x7efd1c15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983931362 0 0) 0x7efd0c001110 con 0x7efd1c143570 2023-11-24T08:06:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd04003520 con 0x7efd1c143570 2023-11-24T08:06:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 64751489 0 0) 0x7efd04003520 con 0x7efd1c143570 2023-11-24T08:06:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd1c0cf760 msgr2=0x7efd1c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd1c0cf760 0x7efd1c15a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 >> v1:172.21.15.91:6789/0 conn(0x7efd1c143950 legacy=0x7efd1c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c15dbd0 con 0x7efd1c143570 2023-11-24T08:06:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7efd0c001800 con 0x7efd1c143570 2023-11-24T08:06:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 >> v1:172.21.15.64:6789/0 conn(0x7efd1c143570 legacy=0x7efd1c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd04003a10 0x7efd04003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd1c0cf760 0x7efd040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd04004d70 0x7efd040051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c15b130 con 0x7efd04003a10 2023-11-24T08:06:42.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0400de30 con 0x7efd04004d70 2023-11-24T08:06:42.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.337+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0400dff0 con 0x7efd1c0cf760 2023-11-24T08:06:42.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.338+0000 7efd1affd640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd04003a10 0x7efd04003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.338+0000 7efd1a7fc640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd1c0cf760 0x7efd040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.338+0000 7efd1b7fe640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd04004d70 0x7efd040051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.338+0000 7efd1affd640 1 -- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd1c0cf760 msgr2=0x7efd040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.338+0000 7efd1affd640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd1c0cf760 0x7efd040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.338+0000 7efd1affd640 1 -- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd04004d70 msgr2=0x7efd040051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.338+0000 7efd1affd640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd04004d70 0x7efd040051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.338+0000 7efd1affd640 1 -- 172.21.15.64:0/1285013765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04003520 con 0x7efd04003a10 2023-11-24T08:06:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.339+0000 7efd1affd640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd04003a10 0x7efd04003e30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efd0c007fc0 tx=0x7efd040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608b5a0cb75e1863 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.339+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c005ad0 con 0x7efd04003a10 2023-11-24T08:06:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.339+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c005c50 con 0x7efd04003a10 2023-11-24T08:06:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.339+0000 7efd19ffb640 1 -- 172.21.15.64:0/1285013765 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c02c720 con 0x7efd04003a10 2023-11-24T08:06:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.340+0000 7efd1bfff640 1 -- 172.21.15.64:0/1285013765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd04003a10 msgr2=0x7efd04003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.340+0000 7efd1bfff640 1 --2- 172.21.15.64:0/1285013765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd04003a10 0x7efd04003e30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efd0c007fc0 tx=0x7efd040057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.340+0000 7efd1b7fe640 1 -- 172.21.15.64:0/1285013765 reap_dead start 2023-11-24T08:06:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.340+0000 7efd1bfff640 1 -- 172.21.15.64:0/1285013765 shutdown_connections 2023-11-24T08:06:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.340+0000 7efd1bfff640 1 -- 172.21.15.64:0/1285013765 >> 172.21.15.64:0/1285013765 conn(0x7efd1c06a860 msgr2=0x7efd1c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.340+0000 7efd1bfff640 1 -- 172.21.15.64:0/1285013765 shutdown_connections 2023-11-24T08:06:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.340+0000 7efd1bfff640 1 -- 172.21.15.64:0/1285013765 wait complete. 2023-11-24T08:06:42.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.342+0000 7efd1bfff640 1 Processor -- start 2023-11-24T08:06:42.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.342+0000 7efd1bfff640 1 -- start start 2023-11-24T08:06:42.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd04003a10 0x7efd1c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0b13b0 0x7efd1c141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd1c1400b0 0x7efd1c140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c13fc20 con 0x7efd1c1400b0 2023-11-24T08:06:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c159db0 con 0x7efd1c0b13b0 2023-11-24T08:06:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c13f010 con 0x7efd04003a10 2023-11-24T08:06:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd1c1400b0 0x7efd1c140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd1c1400b0 0x7efd1c140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52628/0 (socket says 172.21.15.64:52628) 2023-11-24T08:06:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.343+0000 7efd1b7fe640 1 -- 172.21.15.64:0/1275171236 learned_addr learned my addr 172.21.15.64:0/1275171236 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1affd640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd04003a10 0x7efd1c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1a7fc640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0b13b0 0x7efd1c141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1b7fe640 1 -- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd04003a10 msgr2=0x7efd1c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1b7fe640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd04003a10 0x7efd1c159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1b7fe640 1 -- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0b13b0 msgr2=0x7efd1c141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1b7fe640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0b13b0 0x7efd1c141c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1b7fe640 1 -- 172.21.15.64:0/1275171236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c13f2f0 con 0x7efd1c1400b0 2023-11-24T08:06:42.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1a7fc640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0b13b0 0x7efd1c141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:42.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd1b7fe640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd1c1400b0 0x7efd1c140530 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0b3b30 tx=0x7efd1400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.344+0000 7efd037fe640 1 -- 172.21.15.64:0/1275171236 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14003b70 con 0x7efd1c1400b0 2023-11-24T08:06:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.345+0000 7efd037fe640 1 -- 172.21.15.64:0/1275171236 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd14003cd0 con 0x7efd1c1400b0 2023-11-24T08:06:42.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.345+0000 7efd037fe640 1 -- 172.21.15.64:0/1275171236 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd140052b0 con 0x7efd1c1400b0 2023-11-24T08:06:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.345+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c13f5d0 con 0x7efd1c1400b0 2023-11-24T08:06:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.345+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c142180 con 0x7efd1c1400b0 2023-11-24T08:06:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.346+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c04e840 con 0x7efd1c1400b0 2023-11-24T08:06:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.347+0000 7efd037fe640 1 -- 172.21.15.64:0/1275171236 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7efd14005410 con 0x7efd1c1400b0 2023-11-24T08:06:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.348+0000 7efd037fe640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd0806b0b0 0x7efd0806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.348+0000 7efd037fe640 1 -- 172.21.15.64:0/1275171236 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efd140895e0 con 0x7efd1c1400b0 2023-11-24T08:06:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.348+0000 7efd1affd640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd0806b0b0 0x7efd0806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.349+0000 7efd1affd640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd0806b0b0 0x7efd0806d570 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd0c034000 tx=0x7efd0c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.352+0000 7efd037fe640 1 -- 172.21.15.64:0/1275171236 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd140541f0 con 0x7efd1c1400b0 2023-11-24T08:06:42.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.582+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd1c0b3910 con 0x7efd1c1400b0 2023-11-24T08:06:42.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.583+0000 7efd037fe640 1 -- 172.21.15.64:0/1275171236 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4910 (secure 0 0 0) 0x7efd14057810 con 0x7efd1c1400b0 2023-11-24T08:06:42.585 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:42.585 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/618780376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":618780376},{"type":"v1","addr":"172.21.15.64:6835","nonce":618780376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1212789084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1212789084},{"type":"v1","addr":"172.21.15.91:6836","nonce":1212789084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1895191218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1895191218},{"type":"v1","addr":"172.21.15.91:6837","nonce":1895191218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:04.548725+0000","modified":"2023-11-24T08:06:07.601727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5724":{"gid":5724,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/1271535459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1271535459},{"type":"v1","addr":"172.21.15.64:6834","nonce":1271535459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:06:42.586 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2023-11-24T08:06:42.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd0806b0b0 msgr2=0x7efd0806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd0806b0b0 0x7efd0806d570 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd0c034000 tx=0x7efd0c009030 comp rx=0 tx=0).stop 2023-11-24T08:06:42.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd1c1400b0 msgr2=0x7efd1c140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd1c1400b0 0x7efd1c140530 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0b3b30 tx=0x7efd1400dc00 comp rx=0 tx=0).stop 2023-11-24T08:06:42.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 shutdown_connections 2023-11-24T08:06:42.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd0806b0b0 0x7efd0806d570 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd1c1400b0 0x7efd1c140530 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd1c0b13b0 0x7efd1c141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 --2- 172.21.15.64:0/1275171236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd04003a10 0x7efd1c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 >> 172.21.15.64:0/1275171236 conn(0x7efd1c06a860 msgr2=0x7efd1c13afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:42.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 shutdown_connections 2023-11-24T08:06:42.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.586+0000 7efd1bfff640 1 -- 172.21.15.64:0/1275171236 wait complete. 2023-11-24T08:06:42.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:42.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.830+0000 7f639772c640 1 Processor -- start 2023-11-24T08:06:42.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.830+0000 7f639772c640 1 -- start start 2023-11-24T08:06:42.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.831+0000 7f639772c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f639006d520 0x7f639006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.831+0000 7f639772c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f63900abff0 0x7f639015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.831+0000 7f639772c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639015c360 con 0x7f63900ad720 2023-11-24T08:06:42.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f639772c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63900b41a0 con 0x7f63900abff0 2023-11-24T08:06:42.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f639772c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639006de40 con 0x7f639006d520 2023-11-24T08:06:42.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f6395cc9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f63900abff0 0x7f639015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f6395cc9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f63900abff0 0x7f639015f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52640/0 (socket says 172.21.15.64:52640) 2023-11-24T08:06:42.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f6395cc9640 1 -- 172.21.15.64:0/1345548124 learned_addr learned my addr 172.21.15.64:0/1345548124 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:42.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f63954c8640 1 --2- 172.21.15.64:0/1345548124 >> v2:172.21.15.91:3300/0 conn(0x7f639006d520 0x7f639006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f6395cc9640 1 -- 172.21.15.64:0/1345548124 >> v2:172.21.15.91:3300/0 conn(0x7f639006d520 msgr2=0x7f639006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:42.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f6395cc9640 1 --2- 172.21.15.64:0/1345548124 >> v2:172.21.15.91:3300/0 conn(0x7f639006d520 0x7f639006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f6395cc9640 1 -- 172.21.15.64:0/1345548124 >> v1:172.21.15.91:6789/0 conn(0x7f63900ad720 legacy=0x7f639015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:42.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f6395cc9640 1 -- 172.21.15.64:0/1345548124 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639015ba30 con 0x7f63900abff0 2023-11-24T08:06:42.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.832+0000 7f63954c8640 1 --2- 172.21.15.64:0/1345548124 >> v2:172.21.15.91:3300/0 conn(0x7f639006d520 0x7f639006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:42.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.833+0000 7f6395cc9640 1 --2- 172.21.15.64:0/1345548124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f639015f3e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f639006c070 tx=0x7f638802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949f35966e73e4ff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:42.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f6387fff640 1 -- 172.21.15.64:0/1345548124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638802ca80 con 0x7f63900abff0 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f6387fff640 1 -- 172.21.15.64:0/1345548124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638802cbe0 con 0x7f63900abff0 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f6387fff640 1 -- 172.21.15.64:0/1345548124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880343f0 con 0x7f63900abff0 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f639772c640 1 -- 172.21.15.64:0/1345548124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 msgr2=0x7f639015f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f639772c640 1 --2- 172.21.15.64:0/1345548124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f639015f3e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f639006c070 tx=0x7f638802bee0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f639772c640 1 -- 172.21.15.64:0/1345548124 shutdown_connections 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f639772c640 1 --2- 172.21.15.64:0/1345548124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f639015f3e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f639772c640 1 --2- 172.21.15.64:0/1345548124 >> v2:172.21.15.91:3300/0 conn(0x7f639006d520 0x7f639006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f639772c640 1 -- 172.21.15.64:0/1345548124 >> 172.21.15.64:0/1345548124 conn(0x7f63900b0d80 msgr2=0x7f63900b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f639772c640 1 -- 172.21.15.64:0/1345548124 shutdown_connections 2023-11-24T08:06:42.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.834+0000 7f639772c640 1 -- 172.21.15.64:0/1345548124 wait complete. 2023-11-24T08:06:42.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.836+0000 7f639772c640 1 Processor -- start 2023-11-24T08:06:42.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.836+0000 7f639772c640 1 -- start start 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.837+0000 7f639772c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f639006d520 0x7f639014ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.837+0000 7f639772c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f63900708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.837+0000 7f639772c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63900ad720 0x7f63900c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.837+0000 7f639772c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390071d10 con 0x7f63900abff0 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.837+0000 7f639772c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390070df0 con 0x7f639006d520 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.837+0000 7f639772c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6390072130 con 0x7f63900ad720 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.837+0000 7f6394cc7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f63900708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6394cc7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f63900708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52646/0 (socket says 172.21.15.64:52646) 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6394cc7640 1 -- 172.21.15.64:0/4216572605 learned_addr learned my addr 172.21.15.64:0/4216572605 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6395cc9640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63900ad720 0x7f63900c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6394cc7640 1 -- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63900ad720 msgr2=0x7f63900c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6394cc7640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63900ad720 0x7f63900c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6394cc7640 1 -- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f639006d520 msgr2=0x7f639014ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6394cc7640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f639006d520 0x7f639014ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6394cc7640 1 -- 172.21.15.64:0/4216572605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388009b30 con 0x7f63900abff0 2023-11-24T08:06:42.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6394cc7640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f63900708b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6380009fd0 tx=0x7f638000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:42.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6385ffb640 1 -- 172.21.15.64:0/4216572605 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380011830 con 0x7f63900abff0 2023-11-24T08:06:42.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6385ffb640 1 -- 172.21.15.64:0/4216572605 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6380011990 con 0x7f63900abff0 2023-11-24T08:06:42.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.838+0000 7f6385ffb640 1 -- 172.21.15.64:0/4216572605 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380011d20 con 0x7f63900abff0 2023-11-24T08:06:42.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.839+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63900ca040 con 0x7f63900abff0 2023-11-24T08:06:42.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.839+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6390071370 con 0x7f63900abff0 2023-11-24T08:06:42.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.840+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6390071f80 con 0x7f63900abff0 2023-11-24T08:06:42.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.841+0000 7f6385ffb640 1 -- 172.21.15.64:0/4216572605 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f638001a400 con 0x7f63900abff0 2023-11-24T08:06:42.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.842+0000 7f6385ffb640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f637406b080 0x7f637406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:42.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.842+0000 7f6385ffb640 1 -- 172.21.15.64:0/4216572605 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6380089c40 con 0x7f63900abff0 2023-11-24T08:06:42.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.843+0000 7f63954c8640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f637406b080 0x7f637406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:42.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.844+0000 7f63954c8640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f637406b080 0x7f637406d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:42.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:42.847+0000 7f6385ffb640 1 -- 172.21.15.64:0/4216572605 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6380054890 con 0x7f63900abff0 2023-11-24T08:06:43.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.044+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6390074770 con 0x7f63900abff0 2023-11-24T08:06:43.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.047+0000 7f6385ffb640 1 -- 172.21.15.64:0/4216572605 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f6380057eb0 con 0x7f63900abff0 2023-11-24T08:06:43.049 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:43.049 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":48,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:38.040815+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:02.427519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:03.560696+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.9","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f637406b080 msgr2=0x7f637406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f637406b080 0x7f637406d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c008040 comp rx=0 tx=0).stop 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 msgr2=0x7f63900708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f63900708b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6380009fd0 tx=0x7f638000b870 comp rx=0 tx=0).stop 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 shutdown_connections 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f637406b080 0x7f637406d540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f63900ad720 0x7f63900c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63900abff0 0x7f63900708b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 --2- 172.21.15.64:0/4216572605 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f639006d520 0x7f639014ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 >> 172.21.15.64:0/4216572605 conn(0x7f63900b0d80 msgr2=0x7f639015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:43.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 shutdown_connections 2023-11-24T08:06:43.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.050+0000 7f639772c640 1 -- 172.21.15.64:0/4216572605 wait complete. 2023-11-24T08:06:43.072 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:06:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.290+0000 7f4a87fff640 1 Processor -- start 2023-11-24T08:06:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.291+0000 7f4a87fff640 1 -- start start 2023-11-24T08:06:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.292+0000 7f4a87fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4a881416f0 0x7f4a8815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.292+0000 7f4a87fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a88141e80 0x7f4a8815f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.292+0000 7f4a87fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a88070bf0 con 0x7f4a8806cb20 2023-11-24T08:06:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.292+0000 7f4a87fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8806db30 con 0x7f4a881416f0 2023-11-24T08:06:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.292+0000 7f4a87fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8806d850 con 0x7f4a88141e80 2023-11-24T08:06:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a877fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4a8806cb20 0x7f4a8806cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45748/0 (socket says 172.21.15.64:45748) 2023-11-24T08:06:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a877fe640 1 -- 172.21.15.64:0/407053278 learned_addr learned my addr 172.21.15.64:0/407053278 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a7ffff640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a88141e80 0x7f4a8815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a86ffd640 1 --2- 172.21.15.64:0/407053278 >> v2:172.21.15.91:3300/0 conn(0x7f4a881416f0 0x7f4a8815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592898730 0 0) 0x7f4a88070bf0 con 0x7f4a8806cb20 2023-11-24T08:06:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a74003520 con 0x7f4a8806cb20 2023-11-24T08:06:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3234534211 0 0) 0x7f4a74003520 con 0x7f4a8806cb20 2023-11-24T08:06:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a88141e80 msgr2=0x7f4a8815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a88141e80 0x7f4a8815f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 >> v2:172.21.15.91:3300/0 conn(0x7f4a881416f0 msgr2=0x7f4a8815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 --2- 172.21.15.64:0/407053278 >> v2:172.21.15.91:3300/0 conn(0x7f4a881416f0 0x7f4a8815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880705a0 con 0x7f4a8806cb20 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.293+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f4a800019f0 con 0x7f4a8806cb20 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 >> v1:172.21.15.64:6789/0 conn(0x7f4a8806cb20 legacy=0x7f4a8806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a867fc640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a740039e0 0x7f4a74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a867fc640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a88141e80 0x7f4a740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a867fc640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8806db30 con 0x7f4a74004d00 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8806d850 con 0x7f4a740039e0 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7400ba60 con 0x7f4a88141e80 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a877fe640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a7ffff640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a88141e80 0x7f4a740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a86ffd640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a740039e0 0x7f4a74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a877fe640 1 -- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a88141e80 msgr2=0x7f4a740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a877fe640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a88141e80 0x7f4a740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a877fe640 1 -- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a740039e0 msgr2=0x7f4a74003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a877fe640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a740039e0 0x7f4a74003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.294+0000 7f4a877fe640 1 -- 172.21.15.64:0/407053278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a74003520 con 0x7f4a74004d00 2023-11-24T08:06:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.295+0000 7f4a877fe640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a74005180 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000a7b0 tx=0x7f4a740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350a3f51935dbbc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80034630 con 0x7f4a74004d00 2023-11-24T08:06:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8002f070 con 0x7f4a74004d00 2023-11-24T08:06:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a867fc640 1 -- 172.21.15.64:0/407053278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a800349f0 con 0x7f4a74004d00 2023-11-24T08:06:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a87fff640 1 -- 172.21.15.64:0/407053278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 msgr2=0x7f4a74005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a87fff640 1 --2- 172.21.15.64:0/407053278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a74005180 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000a7b0 tx=0x7f4a740057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a877fe640 1 -- 172.21.15.64:0/407053278 reap_dead start 2023-11-24T08:06:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a87fff640 1 -- 172.21.15.64:0/407053278 shutdown_connections 2023-11-24T08:06:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a87fff640 1 -- 172.21.15.64:0/407053278 >> 172.21.15.64:0/407053278 conn(0x7f4a880b0840 msgr2=0x7f4a880b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a87fff640 1 -- 172.21.15.64:0/407053278 shutdown_connections 2023-11-24T08:06:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.296+0000 7f4a87fff640 1 -- 172.21.15.64:0/407053278 wait complete. 2023-11-24T08:06:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.298+0000 7f4a87fff640 1 Processor -- start 2023-11-24T08:06:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.298+0000 7f4a87fff640 1 -- start start 2023-11-24T08:06:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.299+0000 7f4a87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a881f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.299+0000 7f4a87fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a881f6e20 0x7f4a880c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.299+0000 7f4a87fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a880c8980 0x7f4a880afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.299+0000 7f4a87fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881f72a0 con 0x7f4a74004d00 2023-11-24T08:06:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.299+0000 7f4a87fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880c9cd0 con 0x7f4a881f6e20 2023-11-24T08:06:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.299+0000 7f4a87fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880c8e00 con 0x7f4a880c8980 2023-11-24T08:06:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a881f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a881f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52662/0 (socket says 172.21.15.64:52662) 2023-11-24T08:06:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 -- 172.21.15.64:0/3115709384 learned_addr learned my addr 172.21.15.64:0/3115709384 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 -- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a880c8980 msgr2=0x7f4a880afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a880c8980 0x7f4a880afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 -- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a881f6e20 msgr2=0x7f4a880c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a881f6e20 0x7f4a880c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 -- 172.21.15.64:0/3115709384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880c90c0 con 0x7f4a74004d00 2023-11-24T08:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.300+0000 7f4a86ffd640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a881f68e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800f5a0 tx=0x7f4a7800fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.301+0000 7f4a7f7fe640 1 -- 172.21.15.64:0/3115709384 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78014870 con 0x7f4a74004d00 2023-11-24T08:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.301+0000 7f4a7f7fe640 1 -- 172.21.15.64:0/3115709384 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a780149d0 con 0x7f4a74004d00 2023-11-24T08:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.301+0000 7f4a7f7fe640 1 -- 172.21.15.64:0/3115709384 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78014d40 con 0x7f4a74004d00 2023-11-24T08:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.301+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a880c93a0 con 0x7f4a74004d00 2023-11-24T08:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.301+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a880c9960 con 0x7f4a74004d00 2023-11-24T08:06:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.302+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a880c9b20 con 0x7f4a74004d00 2023-11-24T08:06:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.303+0000 7f4a7f7fe640 1 -- 172.21.15.64:0/3115709384 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f4a78015410 con 0x7f4a74004d00 2023-11-24T08:06:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.303+0000 7f4a7f7fe640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4a6806b0b0 0x7f4a6806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.304+0000 7f4a7f7fe640 1 -- 172.21.15.64:0/3115709384 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4a7808a3b0 con 0x7f4a74004d00 2023-11-24T08:06:43.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.304+0000 7f4a7ffff640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4a6806b0b0 0x7f4a6806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.305+0000 7f4a7ffff640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4a6806b0b0 0x7f4a6806d570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4a8806c0a0 tx=0x7f4a70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.308+0000 7f4a7f7fe640 1 -- 172.21.15.64:0/3115709384 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a780550a0 con 0x7f4a74004d00 2023-11-24T08:06:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.528+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4a8804e840 con 0x7f4a74004d00 2023-11-24T08:06:43.531 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-11-24T08:06:43.529+0000 7fb13dee4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:06:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.533+0000 7f4a7f7fe640 1 -- 172.21.15.64:0/3115709384 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4a780586c0 con 0x7f4a74004d00 2023-11-24T08:06:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:06:43.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.535+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4a6806b0b0 msgr2=0x7f4a6806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.535+0000 7f4a87fff640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4a6806b0b0 0x7f4a6806d570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4a8806c0a0 tx=0x7f4a70008040 comp rx=0 tx=0).stop 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 msgr2=0x7f4a881f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a881f68e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800f5a0 tx=0x7f4a7800fa60 comp rx=0 tx=0).stop 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 shutdown_connections 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4a6806b0b0 0x7f4a6806d570 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4a880c8980 0x7f4a880afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4a881f6e20 0x7f4a880c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 --2- 172.21.15.64:0/3115709384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a74004d00 0x7f4a881f68e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 >> 172.21.15.64:0/3115709384 conn(0x7f4a880b0840 msgr2=0x7f4a880b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:43.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.536+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 shutdown_connections 2023-11-24T08:06:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.537+0000 7f4a87fff640 1 -- 172.21.15.64:0/3115709384 wait complete. 2023-11-24T08:06:43.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:06:43.620 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:06:43.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.752+0000 7f415e490640 1 Processor -- start 2023-11-24T08:06:43.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.752+0000 7f415e490640 1 -- start start 2023-11-24T08:06:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.754+0000 7f415e490640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415815b350 0x7f41580abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.754+0000 7f415e490640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f415809cb30 0x7f41580b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.754+0000 7f415e490640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415815b730 con 0x7f41580adb60 2023-11-24T08:06:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.754+0000 7f415e490640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41580ac740 con 0x7f415809cb30 2023-11-24T08:06:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.754+0000 7f415e490640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41580ac460 con 0x7f415815b350 2023-11-24T08:06:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.755+0000 7f4157fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415815b350 0x7f41580abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.755+0000 7f4157fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415815b350 0x7f41580abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:38048/0 (socket says 172.21.15.64:38048) 2023-11-24T08:06:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.755+0000 7f4157fff640 1 -- 172.21.15.64:0/3555943647 learned_addr learned my addr 172.21.15.64:0/3555943647 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.755+0000 7f414ffff640 1 --2- 172.21.15.64:0/3555943647 >> v2:172.21.15.91:3300/0 conn(0x7f415809cb30 0x7f41580b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.756+0000 7f4157fff640 1 -- 172.21.15.64:0/3555943647 >> v2:172.21.15.91:3300/0 conn(0x7f415809cb30 msgr2=0x7f41580b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:43.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.756+0000 7f4157fff640 1 --2- 172.21.15.64:0/3555943647 >> v2:172.21.15.91:3300/0 conn(0x7f415809cb30 0x7f41580b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.756+0000 7f4157fff640 1 -- 172.21.15.64:0/3555943647 >> v1:172.21.15.91:6789/0 conn(0x7f41580adb60 legacy=0x7f41580adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.756+0000 7f4157fff640 1 -- 172.21.15.64:0/3555943647 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580b2a50 con 0x7f415815b350 2023-11-24T08:06:43.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.757+0000 7f4157fff640 1 --2- 172.21.15.64:0/3555943647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415815b350 0x7f41580abe80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4148009870 tx=0x7f414802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9283af87bd1f6a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.757+0000 7f41577fe640 1 -- 172.21.15.64:0/3555943647 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148030840 con 0x7f415815b350 2023-11-24T08:06:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.757+0000 7f41577fe640 1 -- 172.21.15.64:0/3555943647 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41480309a0 con 0x7f415815b350 2023-11-24T08:06:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.757+0000 7f41577fe640 1 -- 172.21.15.64:0/3555943647 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148030d30 con 0x7f415815b350 2023-11-24T08:06:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.758+0000 7f415e490640 1 -- 172.21.15.64:0/3555943647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415815b350 msgr2=0x7f41580abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.758+0000 7f415e490640 1 --2- 172.21.15.64:0/3555943647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415815b350 0x7f41580abe80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4148009870 tx=0x7f414802b980 comp rx=0 tx=0).stop 2023-11-24T08:06:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.758+0000 7f415e490640 1 -- 172.21.15.64:0/3555943647 shutdown_connections 2023-11-24T08:06:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.758+0000 7f415e490640 1 --2- 172.21.15.64:0/3555943647 >> v2:172.21.15.91:3300/0 conn(0x7f415809cb30 0x7f41580b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.758+0000 7f415e490640 1 --2- 172.21.15.64:0/3555943647 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f415815b350 0x7f41580abe80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.758+0000 7f415e490640 1 -- 172.21.15.64:0/3555943647 >> 172.21.15.64:0/3555943647 conn(0x7f41580b0ba0 msgr2=0x7f41580b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.758+0000 7f415e490640 1 -- 172.21.15.64:0/3555943647 shutdown_connections 2023-11-24T08:06:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.758+0000 7f415e490640 1 -- 172.21.15.64:0/3555943647 wait complete. 2023-11-24T08:06:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.760+0000 7f415e490640 1 Processor -- start 2023-11-24T08:06:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.761+0000 7f415e490640 1 -- start start 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f415e490640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415809cb30 0x7f415813ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f415e490640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41580adb60 0x7f41581550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f415e490640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415814d340 0x7f41581555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f415e490640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415813bfc0 con 0x7f415809cb30 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f415e490640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415814d870 con 0x7f415814d340 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f415e490640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415814db50 con 0x7f41580adb60 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f4157fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415809cb30 0x7f415813ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f4157fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415809cb30 0x7f415813ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52684/0 (socket says 172.21.15.64:52684) 2023-11-24T08:06:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f4157fff640 1 -- 172.21.15.64:0/741105737 learned_addr learned my addr 172.21.15.64:0/741105737 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f415ca2d640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415814d340 0x7f41581555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.762+0000 7f414ffff640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41580adb60 0x7f41581550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f4157fff640 1 -- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41580adb60 msgr2=0x7f41581550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f4157fff640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41580adb60 0x7f41581550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f4157fff640 1 -- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415814d340 msgr2=0x7f41581555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f4157fff640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415814d340 0x7f41581555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f4157fff640 1 -- 172.21.15.64:0/741105737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41480094d0 con 0x7f415809cb30 2023-11-24T08:06:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f4157fff640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415809cb30 0x7f415813ba80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4148000c00 tx=0x7f4148004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f41557fa640 1 -- 172.21.15.64:0/741105737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148047070 con 0x7f415809cb30 2023-11-24T08:06:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f41557fa640 1 -- 172.21.15.64:0/741105737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148039a90 con 0x7f415809cb30 2023-11-24T08:06:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415814de30 con 0x7f415809cb30 2023-11-24T08:06:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.763+0000 7f41557fa640 1 -- 172.21.15.64:0/741105737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148039e40 con 0x7f415809cb30 2023-11-24T08:06:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.764+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41581532e0 con 0x7f415809cb30 2023-11-24T08:06:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.764+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4158153130 con 0x7f415809cb30 2023-11-24T08:06:43.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.766+0000 7f41557fa640 1 -- 172.21.15.64:0/741105737 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f414804c410 con 0x7f415809cb30 2023-11-24T08:06:43.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.767+0000 7f41557fa640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f413806b080 0x7f413806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:43.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.767+0000 7f41557fa640 1 -- 172.21.15.64:0/741105737 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f414803e070 con 0x7f415809cb30 2023-11-24T08:06:43.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.767+0000 7f414ffff640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f413806b080 0x7f413806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.768+0000 7f414ffff640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f413806b080 0x7f413806d540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f41581f4760 tx=0x7f4140008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:43.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:43.771+0000 7f41557fa640 1 -- 172.21.15.64:0/741105737 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4148076d50 con 0x7f415809cb30 2023-11-24T08:06:44.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.000+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f41580b3a70 con 0x7f415809cb30 2023-11-24T08:06:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.537+0000 7f41557fa640 1 -- 172.21.15.64:0/741105737 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4148000e60 con 0x7f415809cb30 2023-11-24T08:06:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f413806b080 msgr2=0x7f413806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f413806b080 0x7f413806d540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f41581f4760 tx=0x7f4140008040 comp rx=0 tx=0).stop 2023-11-24T08:06:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415809cb30 msgr2=0x7f415813ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415809cb30 0x7f415813ba80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4148000c00 tx=0x7f4148004730 comp rx=0 tx=0).stop 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 shutdown_connections 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f413806b080 0x7f413806d540 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f415814d340 0x7f41581555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f41580adb60 0x7f41581550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 --2- 172.21.15.64:0/741105737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415809cb30 0x7f415813ba80 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.540+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 >> 172.21.15.64:0/741105737 conn(0x7f41580b0ba0 msgr2=0x7f41580b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.541+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 shutdown_connections 2023-11-24T08:06:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.541+0000 7f415e490640 1 -- 172.21.15.64:0/741105737 wait complete. 2023-11-24T08:06:44.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:06:44.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.797+0000 7f8b5c923640 1 Processor -- start 2023-11-24T08:06:44.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.797+0000 7f8b5c923640 1 -- start start 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b5c923640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b581439e0 0x7f8b580cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b5c923640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8b580cf8c0 0x7f8b5806e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b5c923640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b5806eae0 0x7f8b5815da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b5c923640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5806ef70 con 0x7f8b581439e0 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b5c923640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b580b2f00 con 0x7f8b580cf8c0 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b5c923640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580b31e0 con 0x7f8b5806eae0 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b56d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b581439e0 0x7f8b580cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b56d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b581439e0 0x7f8b580cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52698/0 (socket says 172.21.15.64:52698) 2023-11-24T08:06:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.799+0000 7f8b56d9d640 1 -- 172.21.15.64:0/669092381 learned_addr learned my addr 172.21.15.64:0/669092381 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.800+0000 7f8b56d9d640 1 -- 172.21.15.64:0/669092381 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b5806eae0 msgr2=0x7f8b5815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.800+0000 7f8b56d9d640 1 --2- 172.21.15.64:0/669092381 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b5806eae0 0x7f8b5815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.800+0000 7f8b56d9d640 1 -- 172.21.15.64:0/669092381 >> v2:172.21.15.91:3300/0 conn(0x7f8b580cf8c0 msgr2=0x7f8b5806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.800+0000 7f8b56d9d640 1 --2- 172.21.15.64:0/669092381 >> v2:172.21.15.91:3300/0 conn(0x7f8b580cf8c0 0x7f8b5806e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.800+0000 7f8b56d9d640 1 -- 172.21.15.64:0/669092381 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580b34c0 con 0x7f8b581439e0 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.801+0000 7f8b56d9d640 1 --2- 172.21.15.64:0/669092381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b581439e0 0x7f8b580cf380 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8b580ce170 tx=0x7f8b4802b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb35f2db38e259c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.801+0000 7f8b5659c640 1 -- 172.21.15.64:0/669092381 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b480317f0 con 0x7f8b581439e0 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.801+0000 7f8b5659c640 1 -- 172.21.15.64:0/669092381 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48031950 con 0x7f8b581439e0 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.801+0000 7f8b5659c640 1 -- 172.21.15.64:0/669092381 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48031ce0 con 0x7f8b581439e0 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 -- 172.21.15.64:0/669092381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b581439e0 msgr2=0x7f8b580cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 --2- 172.21.15.64:0/669092381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b581439e0 0x7f8b580cf380 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8b580ce170 tx=0x7f8b4802b890 comp rx=0 tx=0).stop 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 -- 172.21.15.64:0/669092381 shutdown_connections 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 --2- 172.21.15.64:0/669092381 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b5806eae0 0x7f8b5815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 --2- 172.21.15.64:0/669092381 >> v2:172.21.15.91:3300/0 conn(0x7f8b580cf8c0 0x7f8b5806e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 --2- 172.21.15.64:0/669092381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b581439e0 0x7f8b580cf380 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 -- 172.21.15.64:0/669092381 >> 172.21.15.64:0/669092381 conn(0x7f8b580712b0 msgr2=0x7f8b58071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:44.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 -- 172.21.15.64:0/669092381 shutdown_connections 2023-11-24T08:06:44.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.802+0000 7f8b5c923640 1 -- 172.21.15.64:0/669092381 wait complete. 2023-11-24T08:06:44.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.804+0000 7f8b5c923640 1 Processor -- start 2023-11-24T08:06:44.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.804+0000 7f8b5c923640 1 -- start start 2023-11-24T08:06:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b5c923640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5806eae0 0x7f8b58152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b5c923640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b580cf8c0 0x7f8b58142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b5c923640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b581439e0 0x7f8b580b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b5c923640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5813fc10 con 0x7f8b5806eae0 2023-11-24T08:06:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b5c923640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58151fc0 con 0x7f8b580cf8c0 2023-11-24T08:06:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b5c923640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5813fdf0 con 0x7f8b581439e0 2023-11-24T08:06:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b56d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5806eae0 0x7f8b58152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b56d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5806eae0 0x7f8b58152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52700/0 (socket says 172.21.15.64:52700) 2023-11-24T08:06:44.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.805+0000 7f8b56d9d640 1 -- 172.21.15.64:0/3462895304 learned_addr learned my addr 172.21.15.64:0/3462895304 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:44.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b56d9d640 1 -- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b581439e0 msgr2=0x7f8b580b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b56d9d640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b581439e0 0x7f8b580b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b56d9d640 1 -- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b580cf8c0 msgr2=0x7f8b58142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b56d9d640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b580cf8c0 0x7f8b58142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b56d9d640 1 -- 172.21.15.64:0/3462895304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480094d0 con 0x7f8b5806eae0 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b56d9d640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5806eae0 0x7f8b58152a10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8b48000c00 tx=0x7f8b480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b4ffff640 1 -- 172.21.15.64:0/3462895304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48032950 con 0x7f8b5806eae0 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b4ffff640 1 -- 172.21.15.64:0/3462895304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48032ab0 con 0x7f8b5806eae0 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b581400d0 con 0x7f8b5806eae0 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b4ffff640 1 -- 172.21.15.64:0/3462895304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48032e60 con 0x7f8b5806eae0 2023-11-24T08:06:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.806+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b581524f0 con 0x7f8b5806eae0 2023-11-24T08:06:44.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.808+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b581526e0 con 0x7f8b5806eae0 2023-11-24T08:06:44.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.809+0000 7f8b4ffff640 1 -- 172.21.15.64:0/3462895304 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8b480453f0 con 0x7f8b5806eae0 2023-11-24T08:06:44.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.809+0000 7f8b4ffff640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b3806b080 0x7f8b3806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:44.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.810+0000 7f8b4ffff640 1 -- 172.21.15.64:0/3462895304 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f8b480ac0d0 con 0x7f8b5806eae0 2023-11-24T08:06:44.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.810+0000 7f8b4e59c640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b3806b080 0x7f8b3806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:44.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.811+0000 7f8b4e59c640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b3806b080 0x7f8b3806d540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8b58130e50 tx=0x7f8b40006c90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:44.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:44.814+0000 7f8b4ffff640 1 -- 172.21.15.64:0/3462895304 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b48076c50 con 0x7f8b5806eae0 2023-11-24T08:06:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.008+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b5804e840 con 0x7f8b5806eae0 2023-11-24T08:06:45.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:06:45.451 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:06:45.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.549+0000 7f8b4ffff640 1 -- 172.21.15.64:0/3462895304 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8b4807a270 con 0x7f8b5806eae0 2023-11-24T08:06:45.551 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:06:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.552+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b3806b080 msgr2=0x7f8b3806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.552+0000 7f8b5c923640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b3806b080 0x7f8b3806d540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8b58130e50 tx=0x7f8b40006c90 comp rx=0 tx=0).stop 2023-11-24T08:06:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.552+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5806eae0 msgr2=0x7f8b58152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.552+0000 7f8b5c923640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5806eae0 0x7f8b58152a10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8b48000c00 tx=0x7f8b480046c0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.554+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 shutdown_connections 2023-11-24T08:06:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.554+0000 7f8b5c923640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b3806b080 0x7f8b3806d540 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.554+0000 7f8b5c923640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b581439e0 0x7f8b580b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.554+0000 7f8b5c923640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b580cf8c0 0x7f8b58142b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.554+0000 7f8b5c923640 1 --2- 172.21.15.64:0/3462895304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5806eae0 0x7f8b58152a10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.554+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 >> 172.21.15.64:0/3462895304 conn(0x7f8b580712b0 msgr2=0x7f8b580b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.554+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 shutdown_connections 2023-11-24T08:06:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.554+0000 7f8b5c923640 1 -- 172.21.15.64:0/3462895304 wait complete. 2023-11-24T08:06:45.584 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:06:45.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.844+0000 7f6c3b190640 1 Processor -- start 2023-11-24T08:06:45.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.844+0000 7f6c3b190640 1 -- start start 2023-11-24T08:06:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.845+0000 7f6c3b190640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3406cad0 0x7f6c34162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:45.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.846+0000 7f6c3b190640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c34162fa0 con 0x7f6c34162220 2023-11-24T08:06:45.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.846+0000 7f6c3b190640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c340706a0 con 0x7f6c34162600 2023-11-24T08:06:45.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.846+0000 7f6c3b190640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34070c60 con 0x7f6c3406cad0 2023-11-24T08:06:45.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.846+0000 7f6c38f2c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6c34162220 0x7f6c340b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45754/0 (socket says 172.21.15.64:45754) 2023-11-24T08:06:45.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.846+0000 7f6c38f2c640 1 -- 172.21.15.64:0/3766273423 learned_addr learned my addr 172.21.15.64:0/3766273423 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.846+0000 7f6c3972d640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3406cad0 0x7f6c34162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.846+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467362411 0 0) 0x7f6c34162fa0 con 0x7f6c34162220 2023-11-24T08:06:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.846+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c24003520 con 0x7f6c34162220 2023-11-24T08:06:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1136589140 0 0) 0x7f6c24003520 con 0x7f6c34162220 2023-11-24T08:06:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3406cad0 msgr2=0x7f6c34162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3406cad0 0x7f6c34162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 >> v1:172.21.15.91:6789/0 conn(0x7f6c34162600 legacy=0x7f6c3406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340722b0 con 0x7f6c34162220 2023-11-24T08:06:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f6c28001670 con 0x7f6c34162220 2023-11-24T08:06:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 >> v1:172.21.15.64:6789/0 conn(0x7f6c34162220 legacy=0x7f6c340b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:45.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3406cad0 0x7f6c240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c24004d00 0x7f6c24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34070c60 con 0x7f6c240039a0 2023-11-24T08:06:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2400ba30 con 0x7f6c24004d00 2023-11-24T08:06:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2400bbf0 con 0x7f6c3406cad0 2023-11-24T08:06:45.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.847+0000 7f6c38f2c640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.848+0000 7f6c33fff640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3406cad0 0x7f6c240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.848+0000 7f6c3972d640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c24004d00 0x7f6c24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.848+0000 7f6c38f2c640 1 -- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3406cad0 msgr2=0x7f6c240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.848+0000 7f6c38f2c640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3406cad0 0x7f6c240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.848+0000 7f6c38f2c640 1 -- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c24004d00 msgr2=0x7f6c24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.848+0000 7f6c38f2c640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c24004d00 0x7f6c24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.848+0000 7f6c38f2c640 1 -- 172.21.15.64:0/3766273423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24003520 con 0x7f6c240039a0 2023-11-24T08:06:45.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.848+0000 7f6c3972d640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c24004d00 0x7f6c24005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:45.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.849+0000 7f6c38f2c640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c24003dc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c210 tx=0x7f6c240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b63b1706e9f405c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:45.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.849+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28005ad0 con 0x7f6c240039a0 2023-11-24T08:06:45.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.849+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28005c30 con 0x7f6c240039a0 2023-11-24T08:06:45.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.849+0000 7f6c337fe640 1 -- 172.21.15.64:0/3766273423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2802c890 con 0x7f6c240039a0 2023-11-24T08:06:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.850+0000 7f6c3b190640 1 -- 172.21.15.64:0/3766273423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 msgr2=0x7f6c24003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.850+0000 7f6c3b190640 1 --2- 172.21.15.64:0/3766273423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c24003dc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c210 tx=0x7f6c240057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.850+0000 7f6c3972d640 1 -- 172.21.15.64:0/3766273423 reap_dead start 2023-11-24T08:06:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.850+0000 7f6c3b190640 1 -- 172.21.15.64:0/3766273423 shutdown_connections 2023-11-24T08:06:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.850+0000 7f6c3b190640 1 -- 172.21.15.64:0/3766273423 >> 172.21.15.64:0/3766273423 conn(0x7f6c340b0ba0 msgr2=0x7f6c340b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:45.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.850+0000 7f6c3b190640 1 -- 172.21.15.64:0/3766273423 shutdown_connections 2023-11-24T08:06:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.850+0000 7f6c3b190640 1 -- 172.21.15.64:0/3766273423 wait complete. 2023-11-24T08:06:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.852+0000 7f6c3b190640 1 Processor -- start 2023-11-24T08:06:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.852+0000 7f6c3b190640 1 -- start start 2023-11-24T08:06:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c3b190640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c34153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c3b190640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c340ad610 0x7f6c340ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c3b190640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3415b680 0x7f6c3415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:45.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c3b190640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340adfd0 con 0x7f6c240039a0 2023-11-24T08:06:45.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c3b190640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3415c040 con 0x7f6c340ad610 2023-11-24T08:06:45.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c3b190640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3409b390 con 0x7f6c3415b680 2023-11-24T08:06:45.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c38f2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c34153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:45.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c38f2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c34153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52708/0 (socket says 172.21.15.64:52708) 2023-11-24T08:06:45.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.853+0000 7f6c38f2c640 1 -- 172.21.15.64:0/1989484330 learned_addr learned my addr 172.21.15.64:0/1989484330 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c3972d640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3415b680 0x7f6c3415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c33fff640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c340ad610 0x7f6c340ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c38f2c640 1 -- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3415b680 msgr2=0x7f6c3415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c38f2c640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3415b680 0x7f6c3415bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c38f2c640 1 -- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c340ad610 msgr2=0x7f6c340ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c38f2c640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c340ad610 0x7f6c340ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c38f2c640 1 -- 172.21.15.64:0/1989484330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3409b5e0 con 0x7f6c240039a0 2023-11-24T08:06:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c3972d640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3415b680 0x7f6c3415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.854+0000 7f6c38f2c640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c34153280 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c210 tx=0x7f6c28001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:45.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.855+0000 7f6c317fa640 1 -- 172.21.15.64:0/1989484330 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2803c040 con 0x7f6c240039a0 2023-11-24T08:06:45.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.855+0000 7f6c317fa640 1 -- 172.21.15.64:0/1989484330 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2802cca0 con 0x7f6c240039a0 2023-11-24T08:06:45.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.855+0000 7f6c317fa640 1 -- 172.21.15.64:0/1989484330 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28007290 con 0x7f6c240039a0 2023-11-24T08:06:45.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.855+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3409b8c0 con 0x7f6c240039a0 2023-11-24T08:06:45.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.855+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3409be80 con 0x7f6c240039a0 2023-11-24T08:06:45.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.856+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3409d200 con 0x7f6c240039a0 2023-11-24T08:06:45.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.857+0000 7f6c317fa640 1 -- 172.21.15.64:0/1989484330 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6c28032070 con 0x7f6c240039a0 2023-11-24T08:06:45.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.858+0000 7f6c317fa640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1806b0b0 0x7f6c1806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:45.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.858+0000 7f6c317fa640 1 -- 172.21.15.64:0/1989484330 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f6c280aab10 con 0x7f6c240039a0 2023-11-24T08:06:45.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.859+0000 7f6c33fff640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1806b0b0 0x7f6c1806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:45.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.860+0000 7f6c33fff640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1806b0b0 0x7f6c1806d570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6c20007fd0 tx=0x7f6c20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:45.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:45.862+0000 7f6c317fa640 1 -- 172.21.15.64:0/1989484330 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c28075830 con 0x7f6c240039a0 2023-11-24T08:06:46.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:46.066+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6c3404e920 con 0x7f6c240039a0 2023-11-24T08:06:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.012+0000 7f6c317fa640 1 -- 172.21.15.64:0/1989484330 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6c28046030 con 0x7f6c240039a0 2023-11-24T08:06:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-11-24T08:06:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.017+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1806b0b0 msgr2=0x7f6c1806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.017+0000 7f6c3b190640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1806b0b0 0x7f6c1806d570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6c20007fd0 tx=0x7f6c20009040 comp rx=0 tx=0).stop 2023-11-24T08:06:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.018+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 msgr2=0x7f6c34153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.018+0000 7f6c3b190640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c34153280 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c210 tx=0x7f6c28001a00 comp rx=0 tx=0).stop 2023-11-24T08:06:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.019+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 shutdown_connections 2023-11-24T08:06:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.019+0000 7f6c3b190640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c1806b0b0 0x7f6c1806d570 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6c20007fd0 tx=0x7f6c20009040 comp rx=0 tx=0).stop 2023-11-24T08:06:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.019+0000 7f6c3b190640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c3415b680 0x7f6c3415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.019+0000 7f6c3b190640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c340ad610 0x7f6c340ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.019+0000 7f6c3b190640 1 --2- 172.21.15.64:0/1989484330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240039a0 0x7f6c34153280 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.019+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 >> 172.21.15.64:0/1989484330 conn(0x7f6c340b0ba0 msgr2=0x7f6c340af430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.022+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 shutdown_connections 2023-11-24T08:06:47.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.022+0000 7f6c3b190640 1 -- 172.21.15.64:0/1989484330 wait complete. 2023-11-24T08:06:47.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:06:47.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.257+0000 7fce47690640 1 Processor -- start 2023-11-24T08:06:47.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.257+0000 7fce47690640 1 -- start start 2023-11-24T08:06:47.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.258+0000 7fce47690640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fce400ae270 0x7fce400abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.258+0000 7fce47690640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4015b730 con 0x7fce4009cb30 2023-11-24T08:06:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.258+0000 7fce47690640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce400ac740 con 0x7fce4015b350 2023-11-24T08:06:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.258+0000 7fce47690640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce400ac460 con 0x7fce400ae270 2023-11-24T08:06:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.259+0000 7fce44c2b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fce4009cb30 0x7fce400adb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55964/0 (socket says 172.21.15.64:55964) 2023-11-24T08:06:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.259+0000 7fce44c2b640 1 -- 172.21.15.64:0/1639517013 learned_addr learned my addr 172.21.15.64:0/1639517013 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.259+0000 7fce45c2d640 1 --2- 172.21.15.64:0/1639517013 >> v2:172.21.15.91:3300/0 conn(0x7fce400ae270 0x7fce400abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.259+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610928335 0 0) 0x7fce30001110 con 0x7fce4009cb30 2023-11-24T08:06:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.259+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce2c003520 con 0x7fce4009cb30 2023-11-24T08:06:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3690537398 0 0) 0x7fce2c003520 con 0x7fce4009cb30 2023-11-24T08:06:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 >> v2:172.21.15.91:3300/0 conn(0x7fce400ae270 msgr2=0x7fce400abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 --2- 172.21.15.64:0/1639517013 >> v2:172.21.15.91:3300/0 conn(0x7fce400ae270 0x7fce400abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 >> v1:172.21.15.91:6789/0 conn(0x7fce4015b350 legacy=0x7fce400b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce400b2a50 con 0x7fce4009cb30 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fce30001800 con 0x7fce4009cb30 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 >> v1:172.21.15.64:6789/0 conn(0x7fce4009cb30 legacy=0x7fce400adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce2c003970 0x7fce2c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c0047c0 0x7fce2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce2c005180 0x7fce2c00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce400ac460 con 0x7fce2c005180 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0104c0 con 0x7fce2c0047c0 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c010680 con 0x7fce2c003970 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce45c2d640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce2c005180 0x7fce2c00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce44c2b640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c0047c0 0x7fce2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.260+0000 7fce4542c640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce2c003970 0x7fce2c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.261+0000 7fce45c2d640 1 -- 172.21.15.64:0/1639517013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce2c003970 msgr2=0x7fce2c003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.261+0000 7fce45c2d640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce2c003970 0x7fce2c003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.261+0000 7fce45c2d640 1 -- 172.21.15.64:0/1639517013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c0047c0 msgr2=0x7fce2c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.261+0000 7fce45c2d640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c0047c0 0x7fce2c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.261+0000 7fce45c2d640 1 -- 172.21.15.64:0/1639517013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c003520 con 0x7fce2c005180 2023-11-24T08:06:47.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.262+0000 7fce45c2d640 1 -- 172.21.15.64:0/1639517013 reap_dead start 2023-11-24T08:06:47.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.262+0000 7fce45c2d640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce2c005180 0x7fce2c00fd90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fce38007330 tx=0x7fce2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3963623471fbfed7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:47.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.262+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3802ccd0 con 0x7fce2c005180 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.262+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3802ce30 con 0x7fce2c005180 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.262+0000 7fce37fff640 1 -- 172.21.15.64:0/1639517013 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38003bd0 con 0x7fce2c005180 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.263+0000 7fce47690640 1 -- 172.21.15.64:0/1639517013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce2c005180 msgr2=0x7fce2c00fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.263+0000 7fce47690640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce2c005180 0x7fce2c00fd90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fce38007330 tx=0x7fce2c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.263+0000 7fce47690640 1 -- 172.21.15.64:0/1639517013 shutdown_connections 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.263+0000 7fce47690640 1 --2- 172.21.15.64:0/1639517013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce2c005180 0x7fce2c00fd90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.263+0000 7fce47690640 1 -- 172.21.15.64:0/1639517013 >> 172.21.15.64:0/1639517013 conn(0x7fce400b0ba0 msgr2=0x7fce400b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.263+0000 7fce47690640 1 -- 172.21.15.64:0/1639517013 shutdown_connections 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.263+0000 7fce47690640 1 -- 172.21.15.64:0/1639517013 wait complete. 2023-11-24T08:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.265+0000 7fce47690640 1 Processor -- start 2023-11-24T08:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.265+0000 7fce47690640 1 -- start start 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce47690640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c005180 0x7fce4013f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce47690640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce4013f6a0 0x7fce40099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce47690640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce40099f40 0x7fce401f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce47690640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4009b4b0 con 0x7fce4013f6a0 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce47690640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4009a410 con 0x7fce2c005180 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce47690640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4009a6f0 con 0x7fce40099f40 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce44c2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce4013f6a0 0x7fce40099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce44c2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce4013f6a0 0x7fce40099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36478/0 (socket says 172.21.15.64:36478) 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.266+0000 7fce44c2b640 1 -- 172.21.15.64:0/3356229668 learned_addr learned my addr 172.21.15.64:0/3356229668 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce45c2d640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce40099f40 0x7fce401f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce4542c640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c005180 0x7fce4013f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce44c2b640 1 -- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce40099f40 msgr2=0x7fce401f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce44c2b640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce40099f40 0x7fce401f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce44c2b640 1 -- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c005180 msgr2=0x7fce4013f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce44c2b640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c005180 0x7fce4013f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce44c2b640 1 -- 172.21.15.64:0/3356229668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4009a9d0 con 0x7fce4013f6a0 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce45c2d640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce40099f40 0x7fce401f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce44c2b640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce4013f6a0 0x7fce40099a00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e830 tx=0x7fce3000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce35ffb640 1 -- 172.21.15.64:0/3356229668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3000c8d0 con 0x7fce4013f6a0 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce35ffb640 1 -- 172.21.15.64:0/3356229668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3000ca50 con 0x7fce4013f6a0 2023-11-24T08:06:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4009acb0 con 0x7fce4013f6a0 2023-11-24T08:06:47.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.267+0000 7fce35ffb640 1 -- 172.21.15.64:0/3356229668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3000ce00 con 0x7fce4013f6a0 2023-11-24T08:06:47.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.268+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4009bfb0 con 0x7fce4013f6a0 2023-11-24T08:06:47.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.269+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4009c140 con 0x7fce4013f6a0 2023-11-24T08:06:47.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.270+0000 7fce35ffb640 1 -- 172.21.15.64:0/3356229668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fce3001d460 con 0x7fce4013f6a0 2023-11-24T08:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.271+0000 7fce35ffb640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce2406b080 0x7fce2406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.271+0000 7fce35ffb640 1 -- 172.21.15.64:0/3356229668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fce30089420 con 0x7fce4013f6a0 2023-11-24T08:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.271+0000 7fce4542c640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce2406b080 0x7fce2406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.272+0000 7fce4542c640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce2406b080 0x7fce2406d540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce3c007fd0 tx=0x7fce3c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.275+0000 7fce35ffb640 1 -- 172.21.15.64:0/3356229668 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce30054540 con 0x7fce4013f6a0 2023-11-24T08:06:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.472+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fce4015b730 con 0x7fce4013f6a0 2023-11-24T08:06:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.477+0000 7fce35ffb640 1 -- 172.21.15.64:0/3356229668 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fce4015b730 con 0x7fce4013f6a0 2023-11-24T08:06:47.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.480+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce2406b080 msgr2=0x7fce2406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.480+0000 7fce47690640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce2406b080 0x7fce2406d540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce3c007fd0 tx=0x7fce3c009040 comp rx=0 tx=0).stop 2023-11-24T08:06:47.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.480+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce4013f6a0 msgr2=0x7fce40099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.480+0000 7fce47690640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce4013f6a0 0x7fce40099a00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e830 tx=0x7fce3000ecf0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.481+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 shutdown_connections 2023-11-24T08:06:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.481+0000 7fce47690640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce2406b080 0x7fce2406d540 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.481+0000 7fce47690640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce40099f40 0x7fce401f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.481+0000 7fce47690640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce4013f6a0 0x7fce40099a00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.481+0000 7fce47690640 1 --2- 172.21.15.64:0/3356229668 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce2c005180 0x7fce4013f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.481+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 >> 172.21.15.64:0/3356229668 conn(0x7fce400b0ba0 msgr2=0x7fce400af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.481+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 shutdown_connections 2023-11-24T08:06:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.481+0000 7fce47690640 1 -- 172.21.15.64:0/3356229668 wait complete. 2023-11-24T08:06:47.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T08:06:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.699+0000 7f4b6b59e640 1 Processor -- start 2023-11-24T08:06:47.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.700+0000 7f4b6b59e640 1 -- start start 2023-11-24T08:06:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.701+0000 7f4b6b59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f4b6c15d5b0 0x7f4b6c0d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.701+0000 7f4b6b59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b6c0d14a0 0x7f4b6c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.701+0000 7f4b6b59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6c0cf0c0 con 0x7f4b6c0cf7a0 2023-11-24T08:06:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.701+0000 7f4b6b59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6c0b01b0 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.701+0000 7f4b6b59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6c15d990 con 0x7f4b6c15d5b0 2023-11-24T08:06:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b6ad9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4b6c0cf7a0 0x7f4b6c0cfc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55968/0 (socket says 172.21.15.64:55968) 2023-11-24T08:06:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b6ad9d640 1 -- 172.21.15.64:0/4278924442 learned_addr learned my addr 172.21.15.64:0/4278924442 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b69d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b6c0d14a0 0x7f4b6c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b6a59c640 1 --2- 172.21.15.64:0/4278924442 >> v2:172.21.15.91:3300/0 conn(0x7f4b6c15d5b0 0x7f4b6c0d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b69d9b640 1 -- 172.21.15.64:0/4278924442 >> v2:172.21.15.91:3300/0 conn(0x7f4b6c15d5b0 msgr2=0x7f4b6c0d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b69d9b640 1 --2- 172.21.15.64:0/4278924442 >> v2:172.21.15.91:3300/0 conn(0x7f4b6c15d5b0 0x7f4b6c0d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b69d9b640 1 -- 172.21.15.64:0/4278924442 >> v1:172.21.15.64:6789/0 conn(0x7f4b6c0cf7a0 legacy=0x7f4b6c0cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b69d9b640 1 -- 172.21.15.64:0/4278924442 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c0d1e40 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b6a59c640 1 --2- 172.21.15.64:0/4278924442 >> v2:172.21.15.91:3300/0 conn(0x7f4b6c15d5b0 0x7f4b6c0d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.702+0000 7f4b6959a640 1 -- 172.21.15.64:0/4278924442 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519370770 0 0) 0x7f4b6c0cf0c0 con 0x7f4b6c0cf7a0 2023-11-24T08:06:47.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.703+0000 7f4b69d9b640 1 --2- 172.21.15.64:0/4278924442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0d1900 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c08c9e0 tx=0x7f4b5402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f23dcfb9a8aad5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:47.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.704+0000 7f4b6959a640 1 -- 172.21.15.64:0/4278924442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54031830 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.704+0000 7f4b6959a640 1 -- 172.21.15.64:0/4278924442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b54031990 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.704+0000 7f4b6959a640 1 -- 172.21.15.64:0/4278924442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54031d20 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.704+0000 7f4b6b59e640 1 -- 172.21.15.64:0/4278924442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 msgr2=0x7f4b6c0d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.704+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/4278924442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0d1900 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c08c9e0 tx=0x7f4b5402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.705+0000 7f4b6b59e640 1 -- 172.21.15.64:0/4278924442 shutdown_connections 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.705+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/4278924442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0d1900 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.705+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/4278924442 >> v2:172.21.15.91:3300/0 conn(0x7f4b6c15d5b0 0x7f4b6c0d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.705+0000 7f4b6b59e640 1 -- 172.21.15.64:0/4278924442 >> 172.21.15.64:0/4278924442 conn(0x7f4b6c09c3b0 msgr2=0x7f4b6c0d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.705+0000 7f4b6b59e640 1 -- 172.21.15.64:0/4278924442 shutdown_connections 2023-11-24T08:06:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.705+0000 7f4b6b59e640 1 -- 172.21.15.64:0/4278924442 wait complete. 2023-11-24T08:06:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.706+0000 7f4b6b59e640 1 Processor -- start 2023-11-24T08:06:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.707+0000 7f4b6b59e640 1 -- start start 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6b59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b6c0cf7a0 0x7f4b6c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6b59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b6c15d5b0 0x7f4b6c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0b7f10 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6b59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c160a30 con 0x7f4b6c0cf7a0 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6b59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0c7390 con 0x7f4b6c15d5b0 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b69d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b69d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0b79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36494/0 (socket says 172.21.15.64:36494) 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b69d9b640 1 -- 172.21.15.64:0/1149917170 learned_addr learned my addr 172.21.15.64:0/1149917170 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6a59c640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b6c0cf7a0 0x7f4b6c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6ad9d640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b6c15d5b0 0x7f4b6c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b69d9b640 1 -- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b6c15d5b0 msgr2=0x7f4b6c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b69d9b640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b6c15d5b0 0x7f4b6c0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b69d9b640 1 -- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b6c0cf7a0 msgr2=0x7f4b6c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b69d9b640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b6c0cf7a0 0x7f4b6c0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b69d9b640 1 -- 172.21.15.64:0/1149917170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54009400 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.708+0000 7f4b6ad9d640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b6c15d5b0 0x7f4b6c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.709+0000 7f4b69d9b640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0b79d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000c00 tx=0x7f4b540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.709+0000 7f4b62ffd640 1 -- 172.21.15.64:0/1149917170 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5403aa60 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.709+0000 7f4b62ffd640 1 -- 172.21.15.64:0/1149917170 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5403abe0 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.709+0000 7f4b62ffd640 1 -- 172.21.15.64:0/1149917170 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540443f0 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.709+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c160750 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.709+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c0c7020 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.710+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c0c7500 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.711+0000 7f4b62ffd640 1 -- 172.21.15.64:0/1149917170 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f4b5404d450 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.712+0000 7f4b62ffd640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4b4c06b080 0x7f4b4c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.712+0000 7f4b62ffd640 1 -- 172.21.15.64:0/1149917170 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4b540abac0 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.713+0000 7f4b6a59c640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4b4c06b080 0x7f4b4c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.714+0000 7f4b6a59c640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4b4c06b080 0x7f4b4c06d540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0075b0 tx=0x7f4b5c007540 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.717+0000 7f4b62ffd640 1 -- 172.21.15.64:0/1149917170 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b54076a00 con 0x7f4b6c0d14a0 2023-11-24T08:06:47.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:47.984+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f4b6c0c6b40 con 0x7f4b6c0d14a0 2023-11-24T08:06:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.564+0000 7f4b62ffd640 1 -- 172.21.15.64:0/1149917170 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v174) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4b540abe60 con 0x7f4b6c0d14a0 2023-11-24T08:06:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.567+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4b4c06b080 msgr2=0x7f4b4c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.567+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4b4c06b080 0x7f4b4c06d540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0075b0 tx=0x7f4b5c007540 comp rx=0 tx=0).stop 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.567+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 msgr2=0x7f4b6c0b79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.567+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0b79d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000c00 tx=0x7f4b540046c0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.568+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 shutdown_connections 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.568+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4b4c06b080 0x7f4b4c06d540 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.568+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4b6c15d5b0 0x7f4b6c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.568+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c0d14a0 0x7f4b6c0b79d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.568+0000 7f4b6b59e640 1 --2- 172.21.15.64:0/1149917170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4b6c0cf7a0 0x7f4b6c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.568+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 >> 172.21.15.64:0/1149917170 conn(0x7f4b6c09c3b0 msgr2=0x7f4b6c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.568+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 shutdown_connections 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.568+0000 7f4b6b59e640 1 -- 172.21.15.64:0/1149917170 wait complete. 2023-11-24T08:06:48.590 INFO:tasks.cephfs_test_runner:Validate cleaning of stale subvolume snapshot metadata. ... ok 2023-11-24T08:06:48.590 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:06:48.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:48.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.821+0000 7fb349d2e640 1 Processor -- start 2023-11-24T08:06:48.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.821+0000 7fb349d2e640 1 -- start start 2023-11-24T08:06:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.823+0000 7fb349d2e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb34413dfd0 0x7fb34415be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.823+0000 7fb349d2e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3440cf050 0x7fb3440cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.823+0000 7fb349d2e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb34406c130 0x7fb34406c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.823+0000 7fb349d2e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3440cfa10 con 0x7fb34413dfd0 2023-11-24T08:06:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.823+0000 7fb349d2e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34415fa70 con 0x7fb34406c130 2023-11-24T08:06:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.823+0000 7fb349d2e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34415ee60 con 0x7fb3440cf050 2023-11-24T08:06:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.823+0000 7fb348d2c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb34413dfd0 0x7fb34415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb348d2c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb34413dfd0 0x7fb34415be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36506/0 (socket says 172.21.15.64:36506) 2023-11-24T08:06:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb348d2c640 1 -- 172.21.15.64:0/880528817 learned_addr learned my addr 172.21.15.64:0/880528817 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb34952d640 1 --2- 172.21.15.64:0/880528817 >> v2:172.21.15.91:3300/0 conn(0x7fb34406c130 0x7fb34406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb343fff640 1 --2- 172.21.15.64:0/880528817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3440cf050 0x7fb3440cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb348d2c640 1 -- 172.21.15.64:0/880528817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3440cf050 msgr2=0x7fb3440cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb348d2c640 1 --2- 172.21.15.64:0/880528817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3440cf050 0x7fb3440cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb348d2c640 1 -- 172.21.15.64:0/880528817 >> v2:172.21.15.91:3300/0 conn(0x7fb34406c130 msgr2=0x7fb34406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb348d2c640 1 --2- 172.21.15.64:0/880528817 >> v2:172.21.15.91:3300/0 conn(0x7fb34406c130 0x7fb34406c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.824+0000 7fb348d2c640 1 -- 172.21.15.64:0/880528817 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34415f140 con 0x7fb34413dfd0 2023-11-24T08:06:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.826+0000 7fb348d2c640 1 --2- 172.21.15.64:0/880528817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb34415be70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb34406d520 tx=0x7fb33802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f070864d15b07c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb3437fe640 1 -- 172.21.15.64:0/880528817 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338031870 con 0x7fb34413dfd0 2023-11-24T08:06:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb3437fe640 1 -- 172.21.15.64:0/880528817 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3380319d0 con 0x7fb34413dfd0 2023-11-24T08:06:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb3437fe640 1 -- 172.21.15.64:0/880528817 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338031d60 con 0x7fb34413dfd0 2023-11-24T08:06:48.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 -- 172.21.15.64:0/880528817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 msgr2=0x7fb34415be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:48.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 --2- 172.21.15.64:0/880528817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb34415be70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb34406d520 tx=0x7fb33802b910 comp rx=0 tx=0).stop 2023-11-24T08:06:48.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 -- 172.21.15.64:0/880528817 shutdown_connections 2023-11-24T08:06:48.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 --2- 172.21.15.64:0/880528817 >> v2:172.21.15.91:3300/0 conn(0x7fb34406c130 0x7fb34406c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 --2- 172.21.15.64:0/880528817 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb3440cf050 0x7fb3440cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 --2- 172.21.15.64:0/880528817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb34415be70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 -- 172.21.15.64:0/880528817 >> 172.21.15.64:0/880528817 conn(0x7fb34406a860 msgr2=0x7fb34406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 -- 172.21.15.64:0/880528817 shutdown_connections 2023-11-24T08:06:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.827+0000 7fb349d2e640 1 -- 172.21.15.64:0/880528817 wait complete. 2023-11-24T08:06:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.829+0000 7fb349d2e640 1 Processor -- start 2023-11-24T08:06:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.829+0000 7fb349d2e640 1 -- start start 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.830+0000 7fb349d2e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb34406c130 0x7fb344158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.830+0000 7fb349d2e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3440cf050 0x7fb344141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.830+0000 7fb349d2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb3441572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.830+0000 7fb349d2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440b1fd0 con 0x7fb34413dfd0 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.830+0000 7fb349d2e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3441424b0 con 0x7fb3440cf050 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.830+0000 7fb349d2e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440b19c0 con 0x7fb34406c130 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb34952d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb3441572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb34952d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb3441572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36512/0 (socket says 172.21.15.64:36512) 2023-11-24T08:06:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb34952d640 1 -- 172.21.15.64:0/1917320531 learned_addr learned my addr 172.21.15.64:0/1917320531 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:48.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb348d2c640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb34406c130 0x7fb344158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:48.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb343fff640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3440cf050 0x7fb344141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:48.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb34952d640 1 -- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb34406c130 msgr2=0x7fb344158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:48.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb34952d640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb34406c130 0x7fb344158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb34952d640 1 -- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3440cf050 msgr2=0x7fb344141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:48.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb34952d640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3440cf050 0x7fb344141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:48.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.831+0000 7fb34952d640 1 -- 172.21.15.64:0/1917320531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380094d0 con 0x7fb34413dfd0 2023-11-24T08:06:48.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.832+0000 7fb34952d640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb3441572c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00efc0 tx=0x7fb33c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:48.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.832+0000 7fb3417fa640 1 -- 172.21.15.64:0/1917320531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c014480 con 0x7fb34413dfd0 2023-11-24T08:06:48.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.832+0000 7fb3417fa640 1 -- 172.21.15.64:0/1917320531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c00f040 con 0x7fb34413dfd0 2023-11-24T08:06:48.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.832+0000 7fb3417fa640 1 -- 172.21.15.64:0/1917320531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c014820 con 0x7fb34413dfd0 2023-11-24T08:06:48.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.832+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3441421d0 con 0x7fb34413dfd0 2023-11-24T08:06:48.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.832+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb344158980 con 0x7fb34413dfd0 2023-11-24T08:06:48.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.833+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3440b2180 con 0x7fb34413dfd0 2023-11-24T08:06:48.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.834+0000 7fb3417fa640 1 -- 172.21.15.64:0/1917320531 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fb33c014980 con 0x7fb34413dfd0 2023-11-24T08:06:48.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.835+0000 7fb3417fa640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb32806b080 0x7fb32806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:48.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.835+0000 7fb3417fa640 1 -- 172.21.15.64:0/1917320531 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb33c08a7a0 con 0x7fb34413dfd0 2023-11-24T08:06:48.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.836+0000 7fb348d2c640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb32806b080 0x7fb32806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:48.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.837+0000 7fb348d2c640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb32806b080 0x7fb32806d540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb338036000 tx=0x7fb33803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:48.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:48.840+0000 7fb3417fa640 1 -- 172.21.15.64:0/1917320531 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3440b2180 con 0x7fb34413dfd0 2023-11-24T08:06:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.034+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb34404e840 con 0x7fb34413dfd0 2023-11-24T08:06:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.036+0000 7fb3417fa640 1 -- 172.21.15.64:0/1917320531 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+12938 (secure 0 0 0) 0x7fb33c0556d0 con 0x7fb34413dfd0 2023-11-24T08:06:49.038 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:49.038 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":52,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:47.009394+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6834/1271535459":"2023-11-25T08:06:43.530220+0000","172.21.15.64:6832/1271535459":"2023-11-25T08:06:43.530220+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb32806b080 msgr2=0x7fb32806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb32806b080 0x7fb32806d540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb338036000 tx=0x7fb33803c040 comp rx=0 tx=0).stop 2023-11-24T08:06:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 msgr2=0x7fb3441572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb3441572c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00efc0 tx=0x7fb33c00c490 comp rx=0 tx=0).stop 2023-11-24T08:06:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 shutdown_connections 2023-11-24T08:06:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb32806b080 0x7fb32806d540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34413dfd0 0x7fb3441572c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3440cf050 0x7fb344141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 --2- 172.21.15.64:0/1917320531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb34406c130 0x7fb344158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 >> 172.21.15.64:0/1917320531 conn(0x7fb34406a860 msgr2=0x7fb34413f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 shutdown_connections 2023-11-24T08:06:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.039+0000 7fb349d2e640 1 -- 172.21.15.64:0/1917320531 wait complete. 2023-11-24T08:06:49.060 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:06:49.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:06:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.289+0000 7f618c583640 1 Processor -- start 2023-11-24T08:06:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.289+0000 7f618c583640 1 -- start start 2023-11-24T08:06:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f618c583640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f618409cb30 0x7f61840abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f618c583640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61840adb60 0x7f61840adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f618c583640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618415b730 con 0x7f618415b350 2023-11-24T08:06:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f618c583640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61840ac740 con 0x7f61840adb60 2023-11-24T08:06:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f618c583640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61840ac460 con 0x7f618409cb30 2023-11-24T08:06:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f618ab20640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61840adb60 0x7f61840adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f618ab20640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61840adb60 0x7f61840adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36528/0 (socket says 172.21.15.64:36528) 2023-11-24T08:06:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f618ab20640 1 -- 172.21.15.64:0/2482396404 learned_addr learned my addr 172.21.15.64:0/2482396404 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.291+0000 7f6189b1e640 1 --2- 172.21.15.64:0/2482396404 >> v2:172.21.15.91:3300/0 conn(0x7f618409cb30 0x7f61840abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.292+0000 7f618ab20640 1 -- 172.21.15.64:0/2482396404 >> v2:172.21.15.91:3300/0 conn(0x7f618409cb30 msgr2=0x7f61840abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.292+0000 7f618ab20640 1 --2- 172.21.15.64:0/2482396404 >> v2:172.21.15.91:3300/0 conn(0x7f618409cb30 0x7f61840abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.292+0000 7f618ab20640 1 -- 172.21.15.64:0/2482396404 >> v1:172.21.15.91:6789/0 conn(0x7f618415b350 legacy=0x7f61840b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.292+0000 7f618ab20640 1 -- 172.21.15.64:0/2482396404 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61840b2a50 con 0x7f61840adb60 2023-11-24T08:06:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.293+0000 7f618ab20640 1 --2- 172.21.15.64:0/2482396404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61840adf70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f61840d0060 tx=0x7f617c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721b4b6f737f9a05 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.293+0000 7f618931d640 1 -- 172.21.15.64:0/2482396404 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c02ca80 con 0x7f61840adb60 2023-11-24T08:06:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.293+0000 7f618931d640 1 -- 172.21.15.64:0/2482396404 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c02cbe0 con 0x7f61840adb60 2023-11-24T08:06:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.293+0000 7f618931d640 1 -- 172.21.15.64:0/2482396404 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c0343f0 con 0x7f61840adb60 2023-11-24T08:06:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.294+0000 7f618c583640 1 -- 172.21.15.64:0/2482396404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 msgr2=0x7f61840adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.294+0000 7f618c583640 1 --2- 172.21.15.64:0/2482396404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61840adf70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f61840d0060 tx=0x7f617c02bee0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.294+0000 7f618c583640 1 -- 172.21.15.64:0/2482396404 shutdown_connections 2023-11-24T08:06:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.294+0000 7f618c583640 1 --2- 172.21.15.64:0/2482396404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61840adf70 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.294+0000 7f618c583640 1 --2- 172.21.15.64:0/2482396404 >> v2:172.21.15.91:3300/0 conn(0x7f618409cb30 0x7f61840abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.294+0000 7f618c583640 1 -- 172.21.15.64:0/2482396404 >> 172.21.15.64:0/2482396404 conn(0x7f61840b0ba0 msgr2=0x7f61840b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.294+0000 7f618c583640 1 -- 172.21.15.64:0/2482396404 shutdown_connections 2023-11-24T08:06:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.294+0000 7f618c583640 1 -- 172.21.15.64:0/2482396404 wait complete. 2023-11-24T08:06:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.296+0000 7f618c583640 1 Processor -- start 2023-11-24T08:06:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.296+0000 7f618c583640 1 -- start start 2023-11-24T08:06:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.297+0000 7f618c583640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f618409cb30 0x7f6184140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.297+0000 7f618c583640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61841e5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.297+0000 7f618c583640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f618415b350 0x7f618414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.297+0000 7f618c583640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618414c020 con 0x7f61840adb60 2023-11-24T08:06:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.297+0000 7f618c583640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6184071dc0 con 0x7f618415b350 2023-11-24T08:06:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.297+0000 7f618c583640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6184140fa0 con 0x7f618409cb30 2023-11-24T08:06:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f6189b1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61841e5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f6189b1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61841e5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36534/0 (socket says 172.21.15.64:36534) 2023-11-24T08:06:49.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f6189b1e640 1 -- 172.21.15.64:0/271828358 learned_addr learned my addr 172.21.15.64:0/271828358 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:49.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f618ab20640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f618415b350 0x7f618414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f618a31f640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f618409cb30 0x7f6184140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f6189b1e640 1 -- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f618409cb30 msgr2=0x7f6184140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f6189b1e640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f618409cb30 0x7f6184140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f6189b1e640 1 -- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f618415b350 msgr2=0x7f618414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f6189b1e640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f618415b350 0x7f618414e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f6189b1e640 1 -- 172.21.15.64:0/271828358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c009b30 con 0x7f61840adb60 2023-11-24T08:06:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f618ab20640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f618415b350 0x7f618414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.298+0000 7f618a31f640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f618409cb30 0x7f6184140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.299+0000 7f6189b1e640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61841e5390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f617400d5f0 tx=0x7f617400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.299+0000 7f617affd640 1 -- 172.21.15.64:0/271828358 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61740125f0 con 0x7f61840adb60 2023-11-24T08:06:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.299+0000 7f617affd640 1 -- 172.21.15.64:0/271828358 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174012750 con 0x7f61840adb60 2023-11-24T08:06:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.299+0000 7f617affd640 1 -- 172.21.15.64:0/271828358 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174012b00 con 0x7f61840adb60 2023-11-24T08:06:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.299+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61840711b0 con 0x7f61840adb60 2023-11-24T08:06:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.299+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6184071770 con 0x7f61840adb60 2023-11-24T08:06:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.300+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6184071f30 con 0x7f61840adb60 2023-11-24T08:06:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.301+0000 7f617affd640 1 -- 172.21.15.64:0/271828358 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6174012c60 con 0x7f61840adb60 2023-11-24T08:06:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.303+0000 7f617affd640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616806b0b0 0x7f616806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.303+0000 7f617affd640 1 -- 172.21.15.64:0/271828358 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6174089c50 con 0x7f61840adb60 2023-11-24T08:06:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.303+0000 7f618a31f640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616806b0b0 0x7f616806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.304+0000 7f618a31f640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616806b0b0 0x7f616806d570 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6180003eb0 tx=0x7f6180009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.307+0000 7f617affd640 1 -- 172.21.15.64:0/271828358 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6174054c00 con 0x7f61840adb60 2023-11-24T08:06:49.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.504+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f618409d200 con 0x7f61840adb60 2023-11-24T08:06:49.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.505+0000 7f617affd640 1 -- 172.21.15.64:0/271828358 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6174058220 con 0x7f61840adb60 2023-11-24T08:06:49.507 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616806b0b0 msgr2=0x7f616806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616806b0b0 0x7f616806d570 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6180003eb0 tx=0x7f6180009040 comp rx=0 tx=0).stop 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 msgr2=0x7f61841e5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61841e5390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f617400d5f0 tx=0x7f617400dab0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 shutdown_connections 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616806b0b0 0x7f616806d570 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f618415b350 0x7f618414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61840adb60 0x7f61841e5390 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 --2- 172.21.15.64:0/271828358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f618409cb30 0x7f6184140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 >> 172.21.15.64:0/271828358 conn(0x7f61840b0ba0 msgr2=0x7f61840af250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:49.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 shutdown_connections 2023-11-24T08:06:49.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.508+0000 7f618c583640 1 -- 172.21.15.64:0/271828358 wait complete. 2023-11-24T08:06:49.530 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T08:06:49.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.751+0000 7fbdd2471640 1 Processor -- start 2023-11-24T08:06:49.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.752+0000 7fbdd2471640 1 -- start start 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.753+0000 7fbdd2471640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbdcc0d18f0 0x7fbdcc0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.753+0000 7fbdd2471640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0cf0b0 con 0x7fbdcc0d1490 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.753+0000 7fbdd2471640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0b01a0 con 0x7fbdcc15d5a0 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.753+0000 7fbdd2471640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15d980 con 0x7fbdcc0d18f0 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.754+0000 7fbdd0c6e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbdcc0d1490 0x7fbdcc0b0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55978/0 (socket says 172.21.15.64:55978) 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.754+0000 7fbdd0c6e640 1 -- 172.21.15.64:0/2334780610 learned_addr learned my addr 172.21.15.64:0/2334780610 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.754+0000 7fbdd1c70640 1 --2- 172.21.15.64:0/2334780610 >> v2:172.21.15.91:3300/0 conn(0x7fbdcc0d18f0 0x7fbdcc0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.754+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747952072 0 0) 0x7fbdcc0cf0b0 con 0x7fbdcc0d1490 2023-11-24T08:06:49.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.754+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb4003520 con 0x7fbdcc0d1490 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3713318955 0 0) 0x7fbdb4003520 con 0x7fbdcc0d1490 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 >> v2:172.21.15.91:3300/0 conn(0x7fbdcc0d18f0 msgr2=0x7fbdcc0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 --2- 172.21.15.64:0/2334780610 >> v2:172.21.15.91:3300/0 conn(0x7fbdcc0d18f0 0x7fbdcc0b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 >> v1:172.21.15.91:6789/0 conn(0x7fbdcc15d5a0 legacy=0x7fbdcc0d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc0ceaf0 con 0x7fbdcc0d1490 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fbdc0001670 con 0x7fbdcc0d1490 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 >> v1:172.21.15.64:6789/0 conn(0x7fbdcc0d1490 legacy=0x7fbdcc0b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdb4003a30 0x7fbdb4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb40047c0 0x7fbdb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdb4005180 0x7fbdb400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc15d980 con 0x7fbdb4003a30 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb400c120 con 0x7fbdb4005180 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb400c2e0 con 0x7fbdb40047c0 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdd146f640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdb4003a30 0x7fbdb4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.755+0000 7fbdd0c6e640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb40047c0 0x7fbdb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd1c70640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdb4005180 0x7fbdb400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd146f640 1 -- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb40047c0 msgr2=0x7fbdb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd146f640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb40047c0 0x7fbdb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd146f640 1 -- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdb4005180 msgr2=0x7fbdb400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd146f640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdb4005180 0x7fbdb400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd146f640 1 -- 172.21.15.64:0/2334780610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4003520 con 0x7fbdb4003a30 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd0c6e640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb40047c0 0x7fbdb4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:49.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd1c70640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdb4005180 0x7fbdb400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:49.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.756+0000 7fbdd1c70640 1 -- 172.21.15.64:0/2334780610 reap_dead start 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.757+0000 7fbdd146f640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdb4003a30 0x7fbdb4003e30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0044b0 tx=0x7fbdb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8ae6a6e8319609 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.757+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc030400 con 0x7fbdb4003a30 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.757+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc030560 con 0x7fbdb4003a30 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.757+0000 7fbdcbfff640 1 -- 172.21.15.64:0/2334780610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc030950 con 0x7fbdb4003a30 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.757+0000 7fbdd2471640 1 -- 172.21.15.64:0/2334780610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdb4003a30 msgr2=0x7fbdb4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.757+0000 7fbdd2471640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdb4003a30 0x7fbdb4003e30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0044b0 tx=0x7fbdb40057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.758+0000 7fbdd2471640 1 -- 172.21.15.64:0/2334780610 shutdown_connections 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.758+0000 7fbdd2471640 1 --2- 172.21.15.64:0/2334780610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdb4003a30 0x7fbdb4003e30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.758+0000 7fbdd2471640 1 -- 172.21.15.64:0/2334780610 >> 172.21.15.64:0/2334780610 conn(0x7fbdcc09c3a0 msgr2=0x7fbdcc0d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.758+0000 7fbdd2471640 1 -- 172.21.15.64:0/2334780610 shutdown_connections 2023-11-24T08:06:49.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.758+0000 7fbdd2471640 1 -- 172.21.15.64:0/2334780610 wait complete. 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.760+0000 7fbdd2471640 1 Processor -- start 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.760+0000 7fbdd2471640 1 -- start start 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd2471640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb4003a30 0x7fbdcc14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd2471640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdcc150090 0x7fbdcc0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd2471640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdcc0c93e0 0x7fbdcc0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd2471640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c9890 con 0x7fbdcc150090 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd2471640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc15f880 con 0x7fbdcc0c93e0 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd2471640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc160400 con 0x7fbdb4003a30 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd0c6e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdcc150090 0x7fbdcc0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd0c6e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdcc150090 0x7fbdcc0c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36544/0 (socket says 172.21.15.64:36544) 2023-11-24T08:06:49.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd0c6e640 1 -- 172.21.15.64:0/1732912480 learned_addr learned my addr 172.21.15.64:0/1732912480 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:49.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd146f640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb4003a30 0x7fbdcc14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd1c70640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdcc0c93e0 0x7fbdcc0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd0c6e640 1 -- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb4003a30 msgr2=0x7fbdcc14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd0c6e640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb4003a30 0x7fbdcc14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd0c6e640 1 -- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdcc0c93e0 msgr2=0x7fbdcc0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.761+0000 7fbdd0c6e640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdcc0c93e0 0x7fbdcc0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.762+0000 7fbdd0c6e640 1 -- 172.21.15.64:0/1732912480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc15fad0 con 0x7fbdcc150090 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.762+0000 7fbdd1c70640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdcc0c93e0 0x7fbdcc0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.762+0000 7fbdd0c6e640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdcc150090 0x7fbdcc0c8ea0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000d780 tx=0x7fbdc000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.762+0000 7fbdc9ffb640 1 -- 172.21.15.64:0/1732912480 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00074e0 con 0x7fbdcc150090 2023-11-24T08:06:49.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.762+0000 7fbdc9ffb640 1 -- 172.21.15.64:0/1732912480 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc0007640 con 0x7fbdcc150090 2023-11-24T08:06:49.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.762+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdcc15fdb0 con 0x7fbdcc150090 2023-11-24T08:06:49.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.762+0000 7fbdc9ffb640 1 -- 172.21.15.64:0/1732912480 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0004a00 con 0x7fbdcc150090 2023-11-24T08:06:49.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.762+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdcc14fd90 con 0x7fbdcc150090 2023-11-24T08:06:49.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.763+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdcc04e840 con 0x7fbdcc150090 2023-11-24T08:06:49.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.764+0000 7fbdc9ffb640 1 -- 172.21.15.64:0/1732912480 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fbdc0004b60 con 0x7fbdcc150090 2023-11-24T08:06:49.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.765+0000 7fbdc9ffb640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdb806b0b0 0x7fbdb806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:49.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.765+0000 7fbdc9ffb640 1 -- 172.21.15.64:0/1732912480 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbdc0089130 con 0x7fbdcc150090 2023-11-24T08:06:49.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.766+0000 7fbdd146f640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdb806b0b0 0x7fbdb806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:49.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.767+0000 7fbdd146f640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdb806b0b0 0x7fbdb806d570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc036000 tx=0x7fbdbc03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:49.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:49.769+0000 7fbdc9ffb640 1 -- 172.21.15.64:0/1732912480 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbdc0054060 con 0x7fbdcc150090 2023-11-24T08:06:50.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.043+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7fbdcc160540 con 0x7fbdcc150090 2023-11-24T08:06:50.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.579+0000 7fbdc9ffb640 1 -- 172.21.15.64:0/1732912480 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbdc0057680 con 0x7fbdcc150090 2023-11-24T08:06:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdb806b0b0 msgr2=0x7fbdb806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdb806b0b0 0x7fbdb806d570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc036000 tx=0x7fbdbc03b040 comp rx=0 tx=0).stop 2023-11-24T08:06:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdcc150090 msgr2=0x7fbdcc0c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdcc150090 0x7fbdcc0c8ea0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000d780 tx=0x7fbdc000dc40 comp rx=0 tx=0).stop 2023-11-24T08:06:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 shutdown_connections 2023-11-24T08:06:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdb806b0b0 0x7fbdb806d570 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbdcc0c93e0 0x7fbdcc0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdcc150090 0x7fbdcc0c8ea0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 --2- 172.21.15.64:0/1732912480 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbdb4003a30 0x7fbdcc14f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 >> 172.21.15.64:0/1732912480 conn(0x7fbdcc09c3a0 msgr2=0x7fbdcc15c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 shutdown_connections 2023-11-24T08:06:50.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.582+0000 7fbdd2471640 1 -- 172.21.15.64:0/1732912480 wait complete. 2023-11-24T08:06:50.606 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:06:50.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.827+0000 7f1e4937c640 1 Processor -- start 2023-11-24T08:06:50.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.827+0000 7f1e4937c640 1 -- start start 2023-11-24T08:06:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.829+0000 7f1e4937c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e4406dc30 0x7f1e440ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.829+0000 7f1e4937c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1e44070490 0x7f1e44070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.829+0000 7f1e4937c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4406d7a0 con 0x7f1e440acd70 2023-11-24T08:06:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.829+0000 7f1e4937c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4415a5b0 con 0x7f1e4406dc30 2023-11-24T08:06:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.829+0000 7f1e4937c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440ae9d0 con 0x7f1e44070490 2023-11-24T08:06:50.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.830+0000 7f1e427fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e4406dc30 0x7f1e440ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:50.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.830+0000 7f1e427fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e4406dc30 0x7f1e440ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36556/0 (socket says 172.21.15.64:36556) 2023-11-24T08:06:50.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.830+0000 7f1e427fc640 1 -- 172.21.15.64:0/51885529 learned_addr learned my addr 172.21.15.64:0/51885529 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:50.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.830+0000 7f1e437fe640 1 --2- 172.21.15.64:0/51885529 >> v2:172.21.15.91:3300/0 conn(0x7f1e44070490 0x7f1e44070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.830+0000 7f1e427fc640 1 -- 172.21.15.64:0/51885529 >> v2:172.21.15.91:3300/0 conn(0x7f1e44070490 msgr2=0x7f1e44070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.830+0000 7f1e427fc640 1 --2- 172.21.15.64:0/51885529 >> v2:172.21.15.91:3300/0 conn(0x7f1e44070490 0x7f1e44070900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.830+0000 7f1e427fc640 1 -- 172.21.15.64:0/51885529 >> v1:172.21.15.91:6789/0 conn(0x7f1e440acd70 legacy=0x7f1e440ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.830+0000 7f1e427fc640 1 -- 172.21.15.64:0/51885529 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4406ce70 con 0x7f1e4406dc30 2023-11-24T08:06:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.832+0000 7f1e427fc640 1 --2- 172.21.15.64:0/51885529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e440ae490 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1e44067e70 tx=0x7f1e3802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e048a26602e918 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.832+0000 7f1e41ffb640 1 -- 172.21.15.64:0/51885529 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38030860 con 0x7f1e4406dc30 2023-11-24T08:06:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.832+0000 7f1e41ffb640 1 -- 172.21.15.64:0/51885529 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e380309c0 con 0x7f1e4406dc30 2023-11-24T08:06:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.832+0000 7f1e41ffb640 1 -- 172.21.15.64:0/51885529 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38030d50 con 0x7f1e4406dc30 2023-11-24T08:06:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.833+0000 7f1e4937c640 1 -- 172.21.15.64:0/51885529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 msgr2=0x7f1e440ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.833+0000 7f1e4937c640 1 --2- 172.21.15.64:0/51885529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e440ae490 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1e44067e70 tx=0x7f1e3802b9a0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.833+0000 7f1e4937c640 1 -- 172.21.15.64:0/51885529 shutdown_connections 2023-11-24T08:06:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.833+0000 7f1e4937c640 1 --2- 172.21.15.64:0/51885529 >> v2:172.21.15.91:3300/0 conn(0x7f1e44070490 0x7f1e44070900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.833+0000 7f1e4937c640 1 --2- 172.21.15.64:0/51885529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e440ae490 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.833+0000 7f1e4937c640 1 -- 172.21.15.64:0/51885529 >> 172.21.15.64:0/51885529 conn(0x7f1e440b0ba0 msgr2=0x7f1e440b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.833+0000 7f1e4937c640 1 -- 172.21.15.64:0/51885529 shutdown_connections 2023-11-24T08:06:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.833+0000 7f1e4937c640 1 -- 172.21.15.64:0/51885529 wait complete. 2023-11-24T08:06:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.835+0000 7f1e4937c640 1 Processor -- start 2023-11-24T08:06:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.835+0000 7f1e4937c640 1 -- start start 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e4937c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e441f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e4937c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e44070490 0x7f1e440cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e4937c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e440acd70 0x7f1e44153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e4937c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440ce350 con 0x7f1e4406dc30 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e4937c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44156ad0 con 0x7f1e440acd70 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e4937c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44154060 con 0x7f1e44070490 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e42ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e441f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e42ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e441f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36560/0 (socket says 172.21.15.64:36560) 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.836+0000 7f1e42ffd640 1 -- 172.21.15.64:0/2317732140 learned_addr learned my addr 172.21.15.64:0/2317732140 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e427fc640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e44070490 0x7f1e440cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e437fe640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e440acd70 0x7f1e44153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e42ffd640 1 -- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e44070490 msgr2=0x7f1e440cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e42ffd640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e44070490 0x7f1e440cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e42ffd640 1 -- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e440acd70 msgr2=0x7f1e44153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e42ffd640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e440acd70 0x7f1e44153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e42ffd640 1 -- 172.21.15.64:0/2317732140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380094d0 con 0x7f1e4406dc30 2023-11-24T08:06:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e437fe640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e440acd70 0x7f1e44153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.837+0000 7f1e42ffd640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e441f34f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c450 tx=0x7f1e3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.838+0000 7f1e2b7fe640 1 -- 172.21.15.64:0/2317732140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340052d0 con 0x7f1e4406dc30 2023-11-24T08:06:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.838+0000 7f1e2b7fe640 1 -- 172.21.15.64:0/2317732140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e34005430 con 0x7f1e4406dc30 2023-11-24T08:06:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.838+0000 7f1e2b7fe640 1 -- 172.21.15.64:0/2317732140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34012870 con 0x7f1e4406dc30 2023-11-24T08:06:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.838+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e44156d70 con 0x7f1e4406dc30 2023-11-24T08:06:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.838+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e44157330 con 0x7f1e4406dc30 2023-11-24T08:06:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.839+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4409d200 con 0x7f1e4406dc30 2023-11-24T08:06:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.840+0000 7f1e2b7fe640 1 -- 172.21.15.64:0/2317732140 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f1e340129d0 con 0x7f1e4406dc30 2023-11-24T08:06:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.841+0000 7f1e2b7fe640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e3006b080 0x7f1e3006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.841+0000 7f1e2b7fe640 1 -- 172.21.15.64:0/2317732140 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e34089280 con 0x7f1e4406dc30 2023-11-24T08:06:50.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.842+0000 7f1e427fc640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e3006b080 0x7f1e3006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.843+0000 7f1e427fc640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e3006b080 0x7f1e3006d540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1e38036000 tx=0x7f1e3803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:50.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:50.845+0000 7f1e2b7fe640 1 -- 172.21.15.64:0/2317732140 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e340542d0 con 0x7f1e4406dc30 2023-11-24T08:06:51.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.038+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1e4404e920 con 0x7f1e4406dc30 2023-11-24T08:06:51.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.043+0000 7f1e2b7fe640 1 -- 172.21.15.64:0/2317732140 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1e340578f0 con 0x7f1e4406dc30 2023-11-24T08:06:51.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e3006b080 msgr2=0x7f1e3006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:51.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e3006b080 0x7f1e3006d540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1e38036000 tx=0x7f1e3803b040 comp rx=0 tx=0).stop 2023-11-24T08:06:51.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 msgr2=0x7f1e441f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e441f34f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c450 tx=0x7f1e3400c910 comp rx=0 tx=0).stop 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 shutdown_connections 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e3006b080 0x7f1e3006d540 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e440acd70 0x7f1e44153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e44070490 0x7f1e440cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 --2- 172.21.15.64:0/2317732140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e4406dc30 0x7f1e441f34f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.046+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 >> 172.21.15.64:0/2317732140 conn(0x7f1e440b0ba0 msgr2=0x7f1e4415c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.047+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 shutdown_connections 2023-11-24T08:06:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.047+0000 7f1e4937c640 1 -- 172.21.15.64:0/2317732140 wait complete. 2023-11-24T08:06:51.067 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:06:51.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:06:51.187 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:06:51.315 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:06:51.315 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:06:51.315 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:06:51.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:06:51.379 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:06:51.379 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:06:51.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:06:51.437 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:06:51.437 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:06:51.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:06:51.499 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:06:51.500 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:06:51.557 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:06:51.558 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:06:51.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:06:51.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.790+0000 7f72e838a640 1 Processor -- start 2023-11-24T08:06:51.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.790+0000 7f72e838a640 1 -- start start 2023-11-24T08:06:51.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.792+0000 7f72e838a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e0153f10 0x7f72e015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:51.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.792+0000 7f72e838a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f72e006ddb0 0x7f72e006e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:51.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.792+0000 7f72e838a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e00ad6b0 con 0x7f72e015b050 2023-11-24T08:06:51.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.792+0000 7f72e838a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e006e7a0 con 0x7f72e006ddb0 2023-11-24T08:06:51.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.792+0000 7f72e838a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00b2860 con 0x7f72e0153f10 2023-11-24T08:06:51.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.793+0000 7f72e6927640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f72e006ddb0 0x7f72e006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.793+0000 7f72e6126640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e0153f10 0x7f72e015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.793+0000 7f72e6126640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e0153f10 0x7f72e015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:58968/0 (socket says 172.21.15.64:58968) 2023-11-24T08:06:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.793+0000 7f72e6126640 1 -- 172.21.15.64:0/3317013437 learned_addr learned my addr 172.21.15.64:0/3317013437 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.793+0000 7f72e6126640 1 -- 172.21.15.64:0/3317013437 >> v2:172.21.15.91:3300/0 conn(0x7f72e006ddb0 msgr2=0x7f72e006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.793+0000 7f72e6126640 1 --2- 172.21.15.64:0/3317013437 >> v2:172.21.15.91:3300/0 conn(0x7f72e006ddb0 0x7f72e006e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.793+0000 7f72e6126640 1 -- 172.21.15.64:0/3317013437 >> v1:172.21.15.91:6789/0 conn(0x7f72e015b050 legacy=0x7f72e00b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.794+0000 7f72e6126640 1 -- 172.21.15.64:0/3317013437 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e00b2b40 con 0x7f72e0153f10 2023-11-24T08:06:51.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.795+0000 7f72e6126640 1 --2- 172.21.15.64:0/3317013437 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e0153f10 0x7f72e015aaa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00aa10 tx=0x7f72dc02ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52dc61e0b9cbc9a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:51.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.795+0000 7f72e5124640 1 -- 172.21.15.64:0/3317013437 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc02dc10 con 0x7f72e0153f10 2023-11-24T08:06:51.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.795+0000 7f72e5124640 1 -- 172.21.15.64:0/3317013437 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc02dd70 con 0x7f72e0153f10 2023-11-24T08:06:51.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.795+0000 7f72e5124640 1 -- 172.21.15.64:0/3317013437 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc035660 con 0x7f72e0153f10 2023-11-24T08:06:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.796+0000 7f72e838a640 1 -- 172.21.15.64:0/3317013437 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e0153f10 msgr2=0x7f72e015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.796+0000 7f72e838a640 1 --2- 172.21.15.64:0/3317013437 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e0153f10 0x7f72e015aaa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00aa10 tx=0x7f72dc02ce60 comp rx=0 tx=0).stop 2023-11-24T08:06:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.796+0000 7f72e838a640 1 -- 172.21.15.64:0/3317013437 shutdown_connections 2023-11-24T08:06:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.796+0000 7f72e838a640 1 --2- 172.21.15.64:0/3317013437 >> v2:172.21.15.91:3300/0 conn(0x7f72e006ddb0 0x7f72e006e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.796+0000 7f72e838a640 1 --2- 172.21.15.64:0/3317013437 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e0153f10 0x7f72e015aaa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.796+0000 7f72e838a640 1 -- 172.21.15.64:0/3317013437 >> 172.21.15.64:0/3317013437 conn(0x7f72e00b0d80 msgr2=0x7f72e00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.796+0000 7f72e838a640 1 -- 172.21.15.64:0/3317013437 shutdown_connections 2023-11-24T08:06:51.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.796+0000 7f72e838a640 1 -- 172.21.15.64:0/3317013437 wait complete. 2023-11-24T08:06:51.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.798+0000 7f72e838a640 1 Processor -- start 2023-11-24T08:06:51.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.798+0000 7f72e838a640 1 -- start start 2023-11-24T08:06:51.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e838a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72e006ddb0 0x7f72e01471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e838a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e0153f10 0x7f72e0148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e838a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e015b050 0x7f72e0153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e838a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e01f3190 con 0x7f72e0153f10 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e838a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00cc160 con 0x7f72e006ddb0 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e838a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00cb550 con 0x7f72e015b050 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e5925640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e0153f10 0x7f72e0148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e5925640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e0153f10 0x7f72e0148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36574/0 (socket says 172.21.15.64:36574) 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.799+0000 7f72e5925640 1 -- 172.21.15.64:0/3173940049 learned_addr learned my addr 172.21.15.64:0/3173940049 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72e6126640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72e006ddb0 0x7f72e01471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72e6927640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e015b050 0x7f72e0153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72e5925640 1 -- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e015b050 msgr2=0x7f72e0153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:51.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72e5925640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e015b050 0x7f72e0153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72e5925640 1 -- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72e006ddb0 msgr2=0x7f72e01471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72e5925640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72e006ddb0 0x7f72e01471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72e5925640 1 -- 172.21.15.64:0/3173940049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc00a670 con 0x7f72e0153f10 2023-11-24T08:06:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72e5925640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e0153f10 0x7f72e0148ef0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f72d000c400 tx=0x7f72d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.800+0000 7f72d6ffd640 1 -- 172.21.15.64:0/3173940049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d00052d0 con 0x7f72e0153f10 2023-11-24T08:06:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.801+0000 7f72d6ffd640 1 -- 172.21.15.64:0/3173940049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0005430 con 0x7f72e0153f10 2023-11-24T08:06:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.801+0000 7f72d6ffd640 1 -- 172.21.15.64:0/3173940049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0003830 con 0x7f72e0153f10 2023-11-24T08:06:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.801+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e00cb830 con 0x7f72e0153f10 2023-11-24T08:06:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.801+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e00cbdf0 con 0x7f72e0153f10 2023-11-24T08:06:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.802+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e0074770 con 0x7f72e0153f10 2023-11-24T08:06:51.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.803+0000 7f72d6ffd640 1 -- 172.21.15.64:0/3173940049 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f72d0003990 con 0x7f72e0153f10 2023-11-24T08:06:51.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.805+0000 7f72d6ffd640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f72c406b080 0x7f72c406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:51.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.805+0000 7f72d6ffd640 1 -- 172.21.15.64:0/3173940049 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f72d0088c70 con 0x7f72e0153f10 2023-11-24T08:06:51.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.805+0000 7f72e6126640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f72c406b080 0x7f72c406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.806+0000 7f72e6126640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f72c406b080 0x7f72c406d540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f72dc037000 tx=0x7f72dc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:51.810+0000 7f72d6ffd640 1 -- 172.21.15.64:0/3173940049 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72d0053cf0 con 0x7f72e0153f10 2023-11-24T08:06:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.040+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72e00b3340 con 0x7f72e0153f10 2023-11-24T08:06:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.041+0000 7f72d6ffd640 1 -- 172.21.15.64:0/3173940049 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f72d0017090 con 0x7f72e0153f10 2023-11-24T08:06:52.043 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:52.043 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/618780376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":618780376},{"type":"v1","addr":"172.21.15.64:6835","nonce":618780376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/1212789084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":1212789084},{"type":"v1","addr":"172.21.15.91:6836","nonce":1212789084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/1895191218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":1895191218},{"type":"v1","addr":"172.21.15.91:6837","nonce":1895191218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3137908916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3137908916},{"type":"v1","addr":"172.21.15.64:6834","nonce":3137908916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2023-11-24T08:06:52.043 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.043+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f72c406b080 msgr2=0x7f72c406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.043+0000 7f72e838a640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f72c406b080 0x7f72c406d540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f72dc037000 tx=0x7f72dc008040 comp rx=0 tx=0).stop 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.043+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e0153f10 msgr2=0x7f72e0148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.043+0000 7f72e838a640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e0153f10 0x7f72e0148ef0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f72d000c400 tx=0x7f72d000c8c0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.044+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 shutdown_connections 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.044+0000 7f72e838a640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f72c406b080 0x7f72c406d540 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.044+0000 7f72e838a640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f72e015b050 0x7f72e0153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.044+0000 7f72e838a640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e0153f10 0x7f72e0148ef0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.044+0000 7f72e838a640 1 --2- 172.21.15.64:0/3173940049 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f72e006ddb0 0x7f72e01471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.044+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 >> 172.21.15.64:0/3173940049 conn(0x7f72e00b0d80 msgr2=0x7f72e015a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:52.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.044+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 shutdown_connections 2023-11-24T08:06:52.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.044+0000 7f72e838a640 1 -- 172.21.15.64:0/3173940049 wait complete. 2023-11-24T08:06:52.066 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:06:52.066 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:06:52.066 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:06:52.066 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:06:52.067 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:06:52.067 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:06:52.067 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:06:52.067 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:06:52.067 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:06:52.067 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:06:52.067 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:06:52.068 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:06:52.068 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:06:52.068 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:06:52.068 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:06:52.068 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:06:52.087 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:06:52.087 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:06:52.088 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:06:52.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:06:52.089 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:06:52.089 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:06:52.090 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:06:52.090 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:06:52.136 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:06:52.137 DEBUG:teuthology.parallel:result is None 2023-11-24T08:06:52.139 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:06:52.139 DEBUG:teuthology.parallel:result is None 2023-11-24T08:06:52.143 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:06:52.144 DEBUG:teuthology.parallel:result is None 2023-11-24T08:06:52.146 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:06:52.146 DEBUG:teuthology.parallel:result is None 2023-11-24T08:06:52.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:06:52.301 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T08:06:52.301 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:06:52.301 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2023-11-24T08:06:52.310 INFO:tasks.ceph.mds.d.smithi091.stdout:starting mds.d at 2023-11-24T08:06:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.342+0000 7f8df4fc9640 1 Processor -- start 2023-11-24T08:06:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.342+0000 7f8df4fc9640 1 -- start start 2023-11-24T08:06:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8df4fc9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8df00cbcb0 0x7f8df006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8df4fc9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df00b2080 con 0x7f8df00ce880 2023-11-24T08:06:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8df4fc9640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df00b2360 con 0x7f8df015b340 2023-11-24T08:06:52.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8df4fc9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00b2640 con 0x7f8df00cbcb0 2023-11-24T08:06:52.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8deed9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8df00ce880 0x7f8df00b3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56000/0 (socket says 172.21.15.64:56000) 2023-11-24T08:06:52.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8deed9d640 1 -- 172.21.15.64:0/628299753 learned_addr learned my addr 172.21.15.64:0/628299753 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:52.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8de7fff640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8df00cbcb0 0x7f8df006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385588639 0 0) 0x7f8df00b2080 con 0x7f8df00ce880 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.344+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc003520 con 0x7f8df00ce880 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090826604 0 0) 0x7f8df00b2360 con 0x7f8df015b340 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8df00b2080 con 0x7f8df015b340 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2210142950 0 0) 0x7f8ddc003520 con 0x7f8df00ce880 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8df00cbcb0 msgr2=0x7f8df006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8df00cbcb0 0x7f8df006d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 >> v1:172.21.15.91:6789/0 conn(0x7f8df015b340 legacy=0x7f8df00abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df00b2920 con 0x7f8df00ce880 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f8de80019f0 con 0x7f8df00ce880 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 >> v1:172.21.15.64:6789/0 conn(0x7f8df00ce880 legacy=0x7f8df00b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ddc003b30 0x7f8ddc003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ddc0047c0 0x7f8ddc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc005180 0x7f8ddc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00b2640 con 0x7f8ddc003b30 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc00bfa0 con 0x7f8ddc0047c0 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc00c160 con 0x7f8ddc005180 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8dee59c640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ddc003b30 0x7f8ddc003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8de7fff640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ddc0047c0 0x7f8ddc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.345+0000 7f8deed9d640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc005180 0x7f8ddc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.346+0000 7f8dee59c640 1 -- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc005180 msgr2=0x7f8ddc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.346+0000 7f8dee59c640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc005180 0x7f8ddc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.346+0000 7f8dee59c640 1 -- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ddc0047c0 msgr2=0x7f8ddc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.346+0000 7f8dee59c640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ddc0047c0 0x7f8ddc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.346+0000 7f8dee59c640 1 -- 172.21.15.64:0/628299753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc003520 con 0x7f8ddc003b30 2023-11-24T08:06:52.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.347+0000 7f8dee59c640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ddc003b30 0x7f8ddc003f30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8de00018d0 tx=0x7f8ddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb42c5cc2c88546c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.347+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de00052d0 con 0x7f8ddc003b30 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.347+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0005430 con 0x7f8ddc003b30 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.347+0000 7f8dedd9b640 1 -- 172.21.15.64:0/628299753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003810 con 0x7f8ddc003b30 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.348+0000 7f8df4fc9640 1 -- 172.21.15.64:0/628299753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ddc003b30 msgr2=0x7f8ddc003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.348+0000 7f8df4fc9640 1 --2- 172.21.15.64:0/628299753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ddc003b30 0x7f8ddc003f30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8de00018d0 tx=0x7f8ddc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.348+0000 7f8deed9d640 1 -- 172.21.15.64:0/628299753 reap_dead start 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.348+0000 7f8df4fc9640 1 -- 172.21.15.64:0/628299753 shutdown_connections 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.348+0000 7f8df4fc9640 1 -- 172.21.15.64:0/628299753 >> 172.21.15.64:0/628299753 conn(0x7f8df00b0d80 msgr2=0x7f8df00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.348+0000 7f8df4fc9640 1 -- 172.21.15.64:0/628299753 shutdown_connections 2023-11-24T08:06:52.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.348+0000 7f8df4fc9640 1 -- 172.21.15.64:0/628299753 wait complete. 2023-11-24T08:06:52.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.350+0000 7f8df4fc9640 1 Processor -- start 2023-11-24T08:06:52.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.350+0000 7f8df4fc9640 1 -- start start 2023-11-24T08:06:52.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8df4fc9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc003b30 0x7f8df00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8df4fc9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8df00b03e0 0x7f8df014b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8df4fc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df00cc530 0x7f8df0143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8df4fc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00af9d0 con 0x7f8df00cc530 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8df4fc9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00b0860 con 0x7f8df00b03e0 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8df4fc9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df01499b0 con 0x7f8ddc003b30 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8deed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df00cc530 0x7f8df0143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8deed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df00cc530 0x7f8df0143070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36628/0 (socket says 172.21.15.64:36628) 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8deed9d640 1 -- 172.21.15.64:0/3794588276 learned_addr learned my addr 172.21.15.64:0/3794588276 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8dee59c640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc003b30 0x7f8df00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8deed9d640 1 -- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc003b30 msgr2=0x7f8df00723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8deed9d640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc003b30 0x7f8df00723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8deed9d640 1 -- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8df00b03e0 msgr2=0x7f8df014b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8deed9d640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8df00b03e0 0x7f8df014b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.351+0000 7f8deed9d640 1 -- 172.21.15.64:0/3794588276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df014bc70 con 0x7f8df00cc530 2023-11-24T08:06:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.352+0000 7f8deed9d640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df00cc530 0x7f8df0143070 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8de8002230 tx=0x7f8de800a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:52.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.352+0000 7f8de6ffd640 1 -- 172.21.15.64:0/3794588276 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800ac30 con 0x7f8df00cc530 2023-11-24T08:06:52.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.352+0000 7f8de6ffd640 1 -- 172.21.15.64:0/3794588276 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de800add0 con 0x7f8df00cc530 2023-11-24T08:06:52.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.352+0000 7f8de6ffd640 1 -- 172.21.15.64:0/3794588276 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8013630 con 0x7f8df00cc530 2023-11-24T08:06:52.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.352+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df014bf50 con 0x7f8df00cc530 2023-11-24T08:06:52.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.352+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df00afec0 con 0x7f8df00cc530 2023-11-24T08:06:52.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.353+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df014c110 con 0x7f8df00cc530 2023-11-24T08:06:52.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.354+0000 7f8de6ffd640 1 -- 172.21.15.64:0/3794588276 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8de8013790 con 0x7f8df00cc530 2023-11-24T08:06:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.355+0000 7f8de6ffd640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dd006b080 0x7f8dd006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.355+0000 7f8de6ffd640 1 -- 172.21.15.64:0/3794588276 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8de8089f70 con 0x7f8df00cc530 2023-11-24T08:06:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.356+0000 7f8dee59c640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dd006b080 0x7f8dd006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.356+0000 7f8dee59c640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dd006b080 0x7f8dd006d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8de0034000 tx=0x7f8de0031000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:52.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.360+0000 7f8de6ffd640 1 -- 172.21.15.64:0/3794588276 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8de8054f20 con 0x7f8df00cc530 2023-11-24T08:06:52.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.565+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8df00b00b0 con 0x7f8df00cc530 2023-11-24T08:06:52.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.603+0000 7f8de6ffd640 1 -- 172.21.15.64:0/3794588276 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8de8058540 con 0x7f8df00cc530 2023-11-24T08:06:52.605 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-11-24T08:06:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dd006b080 msgr2=0x7f8dd006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dd006b080 0x7f8dd006d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8de0034000 tx=0x7f8de0031000 comp rx=0 tx=0).stop 2023-11-24T08:06:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df00cc530 msgr2=0x7f8df0143070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df00cc530 0x7f8df0143070 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8de8002230 tx=0x7f8de800a600 comp rx=0 tx=0).stop 2023-11-24T08:06:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 shutdown_connections 2023-11-24T08:06:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dd006b080 0x7f8dd006d540 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df00cc530 0x7f8df0143070 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8df00b03e0 0x7f8df014b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 --2- 172.21.15.64:0/3794588276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ddc003b30 0x7f8df00723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 >> 172.21.15.64:0/3794588276 conn(0x7f8df00b0d80 msgr2=0x7f8df00cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.606+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 shutdown_connections 2023-11-24T08:06:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.607+0000 7f8df4fc9640 1 -- 172.21.15.64:0/3794588276 wait complete. 2023-11-24T08:06:52.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:06:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.842+0000 7fc437106640 1 Processor -- start 2023-11-24T08:06:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.842+0000 7fc437106640 1 -- start start 2023-11-24T08:06:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.844+0000 7fc437106640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc4301407e0 0x7fc43006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.844+0000 7fc437106640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc430143b30 0x7fc43015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.844+0000 7fc437106640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc43015e200 0x7fc4300700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.844+0000 7fc437106640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43015e690 con 0x7fc430143b30 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.844+0000 7fc437106640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc430140bc0 con 0x7fc4301407e0 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.844+0000 7fc437106640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4300b1be0 con 0x7fc43015e200 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc435903640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc430143b30 0x7fc43015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc435903640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc430143b30 0x7fc43015dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36640/0 (socket says 172.21.15.64:36640) 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc435903640 1 -- 172.21.15.64:0/333576555 learned_addr learned my addr 172.21.15.64:0/333576555 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc436905640 1 --2- 172.21.15.64:0/333576555 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc43015e200 0x7fc4300700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc436104640 1 --2- 172.21.15.64:0/333576555 >> v2:172.21.15.91:3300/0 conn(0x7fc4301407e0 0x7fc43006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc435903640 1 -- 172.21.15.64:0/333576555 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc43015e200 msgr2=0x7fc4300700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc435903640 1 --2- 172.21.15.64:0/333576555 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc43015e200 0x7fc4300700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc435903640 1 -- 172.21.15.64:0/333576555 >> v2:172.21.15.91:3300/0 conn(0x7fc4301407e0 msgr2=0x7fc43006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc435903640 1 --2- 172.21.15.64:0/333576555 >> v2:172.21.15.91:3300/0 conn(0x7fc4301407e0 0x7fc43006c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.845+0000 7fc435903640 1 -- 172.21.15.64:0/333576555 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4300b1ec0 con 0x7fc430143b30 2023-11-24T08:06:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.846+0000 7fc435903640 1 --2- 172.21.15.64:0/333576555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc430143b30 0x7fc43015dcc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc43006b550 tx=0x7fc42002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910840252537da01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:52.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.847+0000 7fc435102640 1 -- 172.21.15.64:0/333576555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420031870 con 0x7fc430143b30 2023-11-24T08:06:52.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.847+0000 7fc435102640 1 -- 172.21.15.64:0/333576555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4200319d0 con 0x7fc430143b30 2023-11-24T08:06:52.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.847+0000 7fc435102640 1 -- 172.21.15.64:0/333576555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420031d60 con 0x7fc430143b30 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.847+0000 7fc437106640 1 -- 172.21.15.64:0/333576555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc430143b30 msgr2=0x7fc43015dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.847+0000 7fc437106640 1 --2- 172.21.15.64:0/333576555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc430143b30 0x7fc43015dcc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc43006b550 tx=0x7fc42002b910 comp rx=0 tx=0).stop 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.848+0000 7fc437106640 1 -- 172.21.15.64:0/333576555 shutdown_connections 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.848+0000 7fc437106640 1 --2- 172.21.15.64:0/333576555 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc43015e200 0x7fc4300700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.848+0000 7fc437106640 1 --2- 172.21.15.64:0/333576555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc430143b30 0x7fc43015dcc0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.848+0000 7fc437106640 1 --2- 172.21.15.64:0/333576555 >> v2:172.21.15.91:3300/0 conn(0x7fc4301407e0 0x7fc43006c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.848+0000 7fc437106640 1 -- 172.21.15.64:0/333576555 >> 172.21.15.64:0/333576555 conn(0x7fc4300b0840 msgr2=0x7fc4300b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.848+0000 7fc437106640 1 -- 172.21.15.64:0/333576555 shutdown_connections 2023-11-24T08:06:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.848+0000 7fc437106640 1 -- 172.21.15.64:0/333576555 wait complete. 2023-11-24T08:06:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.850+0000 7fc437106640 1 Processor -- start 2023-11-24T08:06:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.850+0000 7fc437106640 1 -- start start 2023-11-24T08:06:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc437106640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4301407e0 0x7fc43013f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc437106640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc430143b30 0x7fc4301f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc437106640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc43015e200 0x7fc4300af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc437106640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4300cebe0 con 0x7fc43015e200 2023-11-24T08:06:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc437106640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4300cca90 con 0x7fc4301407e0 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc437106640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4300cee80 con 0x7fc430143b30 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436905640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc43015e200 0x7fc4300af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436905640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc43015e200 0x7fc4300af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36644/0 (socket says 172.21.15.64:36644) 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436905640 1 -- 172.21.15.64:0/3782644276 learned_addr learned my addr 172.21.15.64:0/3782644276 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436104640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4301407e0 0x7fc43013f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc435903640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc430143b30 0x7fc4301f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436905640 1 -- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc430143b30 msgr2=0x7fc4301f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436905640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc430143b30 0x7fc4301f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436905640 1 -- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4301407e0 msgr2=0x7fc43013f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436905640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4301407e0 0x7fc43013f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.851+0000 7fc436905640 1 -- 172.21.15.64:0/3782644276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200094d0 con 0x7fc43015e200 2023-11-24T08:06:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.852+0000 7fc436905640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc43015e200 0x7fc4300af3e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc42800e7f0 tx=0x7fc42800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.852+0000 7fc426ffd640 1 -- 172.21.15.64:0/3782644276 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc428009510 con 0x7fc43015e200 2023-11-24T08:06:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.852+0000 7fc426ffd640 1 -- 172.21.15.64:0/3782644276 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42800ee40 con 0x7fc43015e200 2023-11-24T08:06:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.852+0000 7fc426ffd640 1 -- 172.21.15.64:0/3782644276 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42800cb80 con 0x7fc43015e200 2023-11-24T08:06:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.852+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4300cf160 con 0x7fc43015e200 2023-11-24T08:06:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.852+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4301f6140 con 0x7fc43015e200 2023-11-24T08:06:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.853+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4300cf2f0 con 0x7fc43015e200 2023-11-24T08:06:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.854+0000 7fc426ffd640 1 -- 172.21.15.64:0/3782644276 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc42800cce0 con 0x7fc43015e200 2023-11-24T08:06:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.855+0000 7fc426ffd640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc41406b080 0x7fc41406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.856+0000 7fc426ffd640 1 -- 172.21.15.64:0/3782644276 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4280897b0 con 0x7fc43015e200 2023-11-24T08:06:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.856+0000 7fc436104640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc41406b080 0x7fc41406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.857+0000 7fc436104640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc41406b080 0x7fc41406d540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0074a0 tx=0x7fc42c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:52.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:52.860+0000 7fc426ffd640 1 -- 172.21.15.64:0/3782644276 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc428054840 con 0x7fc43015e200 2023-11-24T08:06:53.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.122+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc43004e840 con 0x7fc43015e200 2023-11-24T08:06:53.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.123+0000 7fc426ffd640 1 -- 172.21.15.64:0/3782644276 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc428057e60 con 0x7fc43015e200 2023-11-24T08:06:53.125 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:53.125 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:06:53.125 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-11-24T08:06:53.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.125 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-11-24T08:06:53.125 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllV/IJxAAMEWwBM8H9uSLw5Bd93bScA==", 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBl35hoEBAADvW80b1cRhhgBRkYgNaftg==", 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.126 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBlZ1rGORAAjjOrPcL663guE4Q9MKA+6g==", 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:53.127 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlqV+1IRAAaLczF6+eDIuMM6X5Jgni/A==", 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCuWGBlsrGpIBAAVTJlhEtf7rPd1J4wUZRWTg==", 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCxWGBl+u/pLBAAwh85K8evI69aY9Nij5dBRQ==", 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.129 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC1WGBl1kGECxAAtoQ9UI319UpAU7OwZehzew==", 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.130 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC4WGBlSlE2JRAAjPT1020nD4Dnrw1hrB2bKQ==", 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-11-24T08:06:53.131 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC/WGBlrRSGKRAAaiWZZvvd7O8PuipXF1oAUQ==", 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDDWGBls+mkBxAAJZXdXFq1+8QC5dBE14+YJA==", 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:53.132 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDGWGBl71+WIBAA/NNCMwpi0ycNJvXZFHKuIA==", 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJWGBl11BPKxAAnXhK4EqlIyvbHCpHzPCDUg==", 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA==", 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlsyr3DBAASDVvIWUkg06i7YARod5spw==", 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:06:53.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlhsnfABAA5sZEDm1Uyin3dIOEoMX+gg==", 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.136 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlT9iOFxAAiupeG8QnYzYA8pJYftfxNA==", 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCnWGBlKY7iFBAAmkZ+pYWFmZ7aPH06BiE0+w==", 2023-11-24T08:06:53.137 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllqymGhAA+wKOSJeuBn60R4gZJ3edoA==", 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:06:53.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBloKAEDhAAFSBzeCcgrTtmsfXrxKqCRQ==", 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T08:06:53.139 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:06:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc41406b080 msgr2=0x7fc41406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc41406b080 0x7fc41406d540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0074a0 tx=0x7fc42c00b040 comp rx=0 tx=0).stop 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc43015e200 msgr2=0x7fc4300af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc43015e200 0x7fc4300af3e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc42800e7f0 tx=0x7fc42800ebf0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 shutdown_connections 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc41406b080 0x7fc41406d540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc43015e200 0x7fc4300af3e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc430143b30 0x7fc4301f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 --2- 172.21.15.64:0/3782644276 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc4301407e0 0x7fc43013f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 >> 172.21.15.64:0/3782644276 conn(0x7fc4300b0840 msgr2=0x7fc4300b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 shutdown_connections 2023-11-24T08:06:53.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.126+0000 7fc437106640 1 -- 172.21.15.64:0/3782644276 wait complete. 2023-11-24T08:06:53.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.372+0000 7f258b59e640 1 Processor -- start 2023-11-24T08:06:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.373+0000 7f258b59e640 1 -- start start 2023-11-24T08:06:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.374+0000 7f258b59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f258c143570 0x7f258c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.374+0000 7f258b59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c071700 con 0x7f258c143950 2023-11-24T08:06:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.375+0000 7f258b59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c15b410 con 0x7f258c15e6b0 2023-11-24T08:06:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.375+0000 7f258b59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c15b130 con 0x7f258c143570 2023-11-24T08:06:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.375+0000 7f2583fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f258c143950 0x7f258c15a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56014/0 (socket says 172.21.15.64:56014) 2023-11-24T08:06:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.375+0000 7f2583fff640 1 -- 172.21.15.64:0/220364484 learned_addr learned my addr 172.21.15.64:0/220364484 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.375+0000 7f258a59c640 1 --2- 172.21.15.64:0/220364484 >> v2:172.21.15.91:3300/0 conn(0x7f258c143570 0x7f258c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.375+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696492889 0 0) 0x7f2574001110 con 0x7f258c143950 2023-11-24T08:06:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.375+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2578005730 con 0x7f258c143950 2023-11-24T08:06:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458765199 0 0) 0x7f2578005730 con 0x7f258c143950 2023-11-24T08:06:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 >> v2:172.21.15.91:3300/0 conn(0x7f258c143570 msgr2=0x7f258c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 --2- 172.21.15.64:0/220364484 >> v2:172.21.15.91:3300/0 conn(0x7f258c143570 0x7f258c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 >> v1:172.21.15.91:6789/0 conn(0x7f258c15e6b0 legacy=0x7f258c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c15dbd0 con 0x7f258c143950 2023-11-24T08:06:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2574001800 con 0x7f258c143950 2023-11-24T08:06:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 >> v1:172.21.15.64:6789/0 conn(0x7f258c143950 legacy=0x7f258c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2578005c40 0x7f2578006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780069d0 0x7f2578006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25780074b0 0x7f257800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c15b130 con 0x7f25780074b0 2023-11-24T08:06:53.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257800e380 con 0x7f25780069d0 2023-11-24T08:06:53.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257800e540 con 0x7f2578005c40 2023-11-24T08:06:53.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.376+0000 7f258ad9d640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25780074b0 0x7f257800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.377+0000 7f258a59c640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2578005c40 0x7f2578006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.377+0000 7f2583fff640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780069d0 0x7f2578006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.377+0000 7f258ad9d640 1 -- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2578005c40 msgr2=0x7f2578006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:53.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.377+0000 7f258ad9d640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2578005c40 0x7f2578006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.377+0000 7f258ad9d640 1 -- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780069d0 msgr2=0x7f2578006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:53.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.377+0000 7f258ad9d640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780069d0 0x7f2578006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.377+0000 7f258ad9d640 1 -- 172.21.15.64:0/220364484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578005730 con 0x7f25780074b0 2023-11-24T08:06:53.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.377+0000 7f2583fff640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780069d0 0x7f2578006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:53.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.378+0000 7f258ad9d640 1 -- 172.21.15.64:0/220364484 reap_dead start 2023-11-24T08:06:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.378+0000 7f258ad9d640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25780074b0 0x7f257800db90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2584004480 tx=0x7f2578007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56061824fc1f4888 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.378+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584030400 con 0x7f25780074b0 2023-11-24T08:06:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.378+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584030560 con 0x7f25780074b0 2023-11-24T08:06:53.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.378+0000 7f2589d9b640 1 -- 172.21.15.64:0/220364484 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840308f0 con 0x7f25780074b0 2023-11-24T08:06:53.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.379+0000 7f258b59e640 1 -- 172.21.15.64:0/220364484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25780074b0 msgr2=0x7f257800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.379+0000 7f258b59e640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25780074b0 0x7f257800db90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2584004480 tx=0x7f2578007a00 comp rx=0 tx=0).stop 2023-11-24T08:06:53.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.379+0000 7f258b59e640 1 -- 172.21.15.64:0/220364484 shutdown_connections 2023-11-24T08:06:53.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.379+0000 7f258b59e640 1 --2- 172.21.15.64:0/220364484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25780074b0 0x7f257800db90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.379+0000 7f258b59e640 1 -- 172.21.15.64:0/220364484 >> 172.21.15.64:0/220364484 conn(0x7f258c06a860 msgr2=0x7f258c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:53.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.379+0000 7f258b59e640 1 -- 172.21.15.64:0/220364484 shutdown_connections 2023-11-24T08:06:53.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.379+0000 7f258b59e640 1 -- 172.21.15.64:0/220364484 wait complete. 2023-11-24T08:06:53.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.381+0000 7f258b59e640 1 Processor -- start 2023-11-24T08:06:53.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.381+0000 7f258b59e640 1 -- start start 2023-11-24T08:06:53.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f258b59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780074b0 0x7f258c13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f258b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f258c1596e0 0x7f258c0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f258b59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258c0b2130 0x7f258c139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f258b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0cb680 con 0x7f258c1596e0 2023-11-24T08:06:53.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f258b59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c159f10 con 0x7f25780074b0 2023-11-24T08:06:53.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f258b59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0c9e40 con 0x7f258c0b2130 2023-11-24T08:06:53.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f2583fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f258c1596e0 0x7f258c0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f2583fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f258c1596e0 0x7f258c0cb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36656/0 (socket says 172.21.15.64:36656) 2023-11-24T08:06:53.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.382+0000 7f2583fff640 1 -- 172.21.15.64:0/2226437590 learned_addr learned my addr 172.21.15.64:0/2226437590 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:53.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f258a59c640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780074b0 0x7f258c13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f258ad9d640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258c0b2130 0x7f258c139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2583fff640 1 -- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258c0b2130 msgr2=0x7f258c139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2583fff640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258c0b2130 0x7f258c139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2583fff640 1 -- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780074b0 msgr2=0x7f258c13e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2583fff640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780074b0 0x7f258c13e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2583fff640 1 -- 172.21.15.64:0/2226437590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c0ca0e0 con 0x7f258c1596e0 2023-11-24T08:06:53.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f258ad9d640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258c0b2130 0x7f258c139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:53.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2583fff640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f258c1596e0 0x7f258c0cb140 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f257400d770 tx=0x7f257400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:53.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2582ffd640 1 -- 172.21.15.64:0/2226437590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25740074e0 con 0x7f258c1596e0 2023-11-24T08:06:53.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2582ffd640 1 -- 172.21.15.64:0/2226437590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574007640 con 0x7f258c1596e0 2023-11-24T08:06:53.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c0ca380 con 0x7f258c1596e0 2023-11-24T08:06:53.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.383+0000 7f2582ffd640 1 -- 172.21.15.64:0/2226437590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574004a00 con 0x7f258c1596e0 2023-11-24T08:06:53.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.384+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c0ca940 con 0x7f258c1596e0 2023-11-24T08:06:53.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.385+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c04e840 con 0x7f258c1596e0 2023-11-24T08:06:53.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.386+0000 7f2582ffd640 1 -- 172.21.15.64:0/2226437590 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2574004b60 con 0x7f258c1596e0 2023-11-24T08:06:53.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.388+0000 7f2582ffd640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06b080 0x7f256c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.388+0000 7f2582ffd640 1 -- 172.21.15.64:0/2226437590 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2574088ec0 con 0x7f258c1596e0 2023-11-24T08:06:53.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.388+0000 7f258a59c640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06b080 0x7f256c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.389+0000 7f258a59c640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06b080 0x7f256c06d540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f257c009990 tx=0x7f257c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:53.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.392+0000 7f2582ffd640 1 -- 172.21.15.64:0/2226437590 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2574053ee0 con 0x7f258c1596e0 2023-11-24T08:06:53.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.585+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f258c071700 con 0x7f258c1596e0 2023-11-24T08:06:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.587+0000 7f2582ffd640 1 -- 172.21.15.64:0/2226437590 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12809 (secure 0 0 0) 0x7f258c071700 con 0x7f258c1596e0 2023-11-24T08:06:53.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:53.589 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":56,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:52.615087+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:53.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06b080 msgr2=0x7f256c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06b080 0x7f256c06d540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f257c009990 tx=0x7f257c008040 comp rx=0 tx=0).stop 2023-11-24T08:06:53.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f258c1596e0 msgr2=0x7f258c0cb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f258c1596e0 0x7f258c0cb140 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f257400d770 tx=0x7f257400dc30 comp rx=0 tx=0).stop 2023-11-24T08:06:53.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 shutdown_connections 2023-11-24T08:06:53.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f256c06b080 0x7f256c06d540 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f258c0b2130 0x7f258c139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f258c1596e0 0x7f258c0cb140 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 --2- 172.21.15.64:0/2226437590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f25780074b0 0x7f258c13e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 >> 172.21.15.64:0/2226437590 conn(0x7f258c06a860 msgr2=0x7f258c0ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:53.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 shutdown_connections 2023-11-24T08:06:53.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.590+0000 7f258b59e640 1 -- 172.21.15.64:0/2226437590 wait complete. 2023-11-24T08:06:53.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:06:53.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:06:53.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.841+0000 7f5459bac640 1 Processor -- start 2023-11-24T08:06:53.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.841+0000 7f5459bac640 1 -- start start 2023-11-24T08:06:53.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.843+0000 7f5459bac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5454144770 0x7f545415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.843+0000 7f5459bac640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f545415f0a0 0x7f54540ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.843+0000 7f5459bac640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545415f930 con 0x7f54540b35a0 2023-11-24T08:06:53.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.843+0000 7f5459bac640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54540b43b0 con 0x7f545415f0a0 2023-11-24T08:06:53.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.843+0000 7f5459bac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545415f530 con 0x7f5454144770 2023-11-24T08:06:53.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.844+0000 7f54593ab640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f545415f0a0 0x7f54540ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.844+0000 7f5458baa640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5454144770 0x7f545415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.844+0000 7f54593ab640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f545415f0a0 0x7f54540ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:54558/0 (socket says 172.21.15.64:54558) 2023-11-24T08:06:53.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.844+0000 7f54593ab640 1 -- 172.21.15.64:0/203160603 learned_addr learned my addr 172.21.15.64:0/203160603 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:53.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.844+0000 7f5458baa640 1 -- 172.21.15.64:0/203160603 >> v2:172.21.15.91:3300/0 conn(0x7f545415f0a0 msgr2=0x7f54540ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:53.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.844+0000 7f5458baa640 1 --2- 172.21.15.64:0/203160603 >> v2:172.21.15.91:3300/0 conn(0x7f545415f0a0 0x7f54540ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.844+0000 7f5458baa640 1 -- 172.21.15.64:0/203160603 >> v1:172.21.15.91:6789/0 conn(0x7f54540b35a0 legacy=0x7f545406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.844+0000 7f5458baa640 1 -- 172.21.15.64:0/203160603 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540ce9f0 con 0x7f5454144770 2023-11-24T08:06:53.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.845+0000 7f54593ab640 1 --2- 172.21.15.64:0/203160603 >> v2:172.21.15.91:3300/0 conn(0x7f545415f0a0 0x7f54540ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:06:53.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.846+0000 7f5458baa640 1 --2- 172.21.15.64:0/203160603 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5454144770 0x7f545415eb60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f54480097d0 tx=0x7f544802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=334de336afc9492d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:53.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.846+0000 7f54537fe640 1 -- 172.21.15.64:0/203160603 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544802d040 con 0x7f5454144770 2023-11-24T08:06:53.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.846+0000 7f54537fe640 1 -- 172.21.15.64:0/203160603 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448031810 con 0x7f5454144770 2023-11-24T08:06:53.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.846+0000 7f54537fe640 1 -- 172.21.15.64:0/203160603 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448031ba0 con 0x7f5454144770 2023-11-24T08:06:53.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.847+0000 7f5459bac640 1 -- 172.21.15.64:0/203160603 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5454144770 msgr2=0x7f545415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.847+0000 7f5459bac640 1 --2- 172.21.15.64:0/203160603 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5454144770 0x7f545415eb60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f54480097d0 tx=0x7f544802b8b0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.847+0000 7f5459bac640 1 -- 172.21.15.64:0/203160603 shutdown_connections 2023-11-24T08:06:53.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.847+0000 7f5459bac640 1 --2- 172.21.15.64:0/203160603 >> v2:172.21.15.91:3300/0 conn(0x7f545415f0a0 0x7f54540ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.847+0000 7f5459bac640 1 --2- 172.21.15.64:0/203160603 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5454144770 0x7f545415eb60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.847+0000 7f5459bac640 1 -- 172.21.15.64:0/203160603 >> 172.21.15.64:0/203160603 conn(0x7f54540cec10 msgr2=0x7f54540cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:53.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.847+0000 7f5459bac640 1 -- 172.21.15.64:0/203160603 shutdown_connections 2023-11-24T08:06:53.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.847+0000 7f5459bac640 1 -- 172.21.15.64:0/203160603 wait complete. 2023-11-24T08:06:53.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.849+0000 7f5459bac640 1 Processor -- start 2023-11-24T08:06:53.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.850+0000 7f5459bac640 1 -- start start 2023-11-24T08:06:53.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5459bac640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54540b35a0 0x7f54540c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5459bac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5454144770 0x7f54540cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5459bac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f545415f0a0 0x7f545413a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5459bac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540c4e70 con 0x7f5454144770 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5459bac640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540c6c40 con 0x7f54540b35a0 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5459bac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5454159c50 con 0x7f545415f0a0 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5453fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5454144770 0x7f54540cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5453fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5454144770 0x7f54540cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36660/0 (socket says 172.21.15.64:36660) 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5453fff640 1 -- 172.21.15.64:0/2858513376 learned_addr learned my addr 172.21.15.64:0/2858513376 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.851+0000 7f5458baa640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54540b35a0 0x7f54540c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f54593ab640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f545415f0a0 0x7f545413a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f5453fff640 1 -- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f545415f0a0 msgr2=0x7f545413a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f5453fff640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f545415f0a0 0x7f545413a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f5453fff640 1 -- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54540b35a0 msgr2=0x7f54540c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:53.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f5453fff640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54540b35a0 0x7f54540c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:53.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f5453fff640 1 -- 172.21.15.64:0/2858513376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c009b30 con 0x7f5454144770 2023-11-24T08:06:53.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f5453fff640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5454144770 0x7f54540cdc00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f544000c3c0 tx=0x7f544000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:53.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f54517fa640 1 -- 172.21.15.64:0/2858513376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54400052d0 con 0x7f5454144770 2023-11-24T08:06:53.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f54517fa640 1 -- 172.21.15.64:0/2858513376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5440005450 con 0x7f5454144770 2023-11-24T08:06:53.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.852+0000 7f54517fa640 1 -- 172.21.15.64:0/2858513376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440007260 con 0x7f5454144770 2023-11-24T08:06:53.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.853+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5448009400 con 0x7f5454144770 2023-11-24T08:06:53.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.853+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54540c41f0 con 0x7f5454144770 2023-11-24T08:06:53.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.854+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545413b0d0 con 0x7f5454144770 2023-11-24T08:06:53.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.855+0000 7f54517fa640 1 -- 172.21.15.64:0/2858513376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f54400073c0 con 0x7f5454144770 2023-11-24T08:06:53.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.856+0000 7f54517fa640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f543806b080 0x7f543806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:53.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.856+0000 7f54517fa640 1 -- 172.21.15.64:0/2858513376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5440088af0 con 0x7f5454144770 2023-11-24T08:06:53.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.857+0000 7f5458baa640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f543806b080 0x7f543806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:53.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.858+0000 7f5458baa640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f543806b080 0x7f543806d540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5448036000 tx=0x7f544803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:53.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:53.861+0000 7f54517fa640 1 -- 172.21.15.64:0/2858513376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5440053b30 con 0x7f5454144770 2023-11-24T08:06:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.055+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f545404e840 con 0x7f5454144770 2023-11-24T08:06:54.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.056+0000 7f54517fa640 1 -- 172.21.15.64:0/2858513376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5440057150 con 0x7f5454144770 2023-11-24T08:06:54.058 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:06:54.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f543806b080 msgr2=0x7f543806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f543806b080 0x7f543806d540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5448036000 tx=0x7f544803c040 comp rx=0 tx=0).stop 2023-11-24T08:06:54.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5454144770 msgr2=0x7f54540cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5454144770 0x7f54540cdc00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f544000c3c0 tx=0x7f544000c880 comp rx=0 tx=0).stop 2023-11-24T08:06:54.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 shutdown_connections 2023-11-24T08:06:54.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f543806b080 0x7f543806d540 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f545415f0a0 0x7f545413a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5454144770 0x7f54540cdc00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 --2- 172.21.15.64:0/2858513376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f54540b35a0 0x7f54540c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 >> 172.21.15.64:0/2858513376 conn(0x7f54540cec10 msgr2=0x7f545415d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:54.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 shutdown_connections 2023-11-24T08:06:54.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.059+0000 7f5459bac640 1 -- 172.21.15.64:0/2858513376 wait complete. 2023-11-24T08:06:54.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:06:54.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.299+0000 7f75cb021640 1 Processor -- start 2023-11-24T08:06:54.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.299+0000 7f75cb021640 1 -- start start 2023-11-24T08:06:54.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.301+0000 7f75cb021640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f75c4143570 0x7f75c40cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.301+0000 7f75cb021640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c4071700 con 0x7f75c415e6b0 2023-11-24T08:06:54.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.301+0000 7f75cb021640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c415b410 con 0x7f75c4143950 2023-11-24T08:06:54.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.301+0000 7f75cb021640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c415b130 con 0x7f75c4143570 2023-11-24T08:06:54.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.301+0000 7f75ca820640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f75c415e6b0 0x7f75c40b1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56018/0 (socket says 172.21.15.64:56018) 2023-11-24T08:06:54.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.301+0000 7f75ca01f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f75c4143570 0x7f75c40cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.301+0000 7f75ca820640 1 -- 172.21.15.64:0/710189336 learned_addr learned my addr 172.21.15.64:0/710189336 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:54.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.302+0000 7f75c901d640 1 -- 172.21.15.64:0/710189336 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643701016 0 0) 0x7f75bc001110 con 0x7f75c415e6b0 2023-11-24T08:06:54.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.302+0000 7f75c901d640 1 -- 172.21.15.64:0/710189336 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b0005730 con 0x7f75c415e6b0 2023-11-24T08:06:54.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.302+0000 7f75ca01f640 1 -- 172.21.15.64:0/710189336 >> v1:172.21.15.91:6789/0 conn(0x7f75c4143950 legacy=0x7f75c415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:54.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.302+0000 7f75ca01f640 1 -- 172.21.15.64:0/710189336 >> v1:172.21.15.64:6789/0 conn(0x7f75c415e6b0 legacy=0x7f75c40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.302+0000 7f75ca01f640 1 -- 172.21.15.64:0/710189336 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c415dbd0 con 0x7f75c4143570 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.303+0000 7f75ca01f640 1 --2- 172.21.15.64:0/710189336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143570 0x7f75c40cf050 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009d90 tx=0x7f75c002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21948535b51987b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.303+0000 7f75c901d640 1 -- 172.21.15.64:0/710189336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c002cbd0 con 0x7f75c4143570 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.303+0000 7f75c901d640 1 -- 172.21.15.64:0/710189336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c002cd30 con 0x7f75c4143570 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.303+0000 7f75c901d640 1 -- 172.21.15.64:0/710189336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c002cea0 con 0x7f75c4143570 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.304+0000 7f75cb021640 1 -- 172.21.15.64:0/710189336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143570 msgr2=0x7f75c40cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.304+0000 7f75cb021640 1 --2- 172.21.15.64:0/710189336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143570 0x7f75c40cf050 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009d90 tx=0x7f75c002be20 comp rx=0 tx=0).stop 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.304+0000 7f75cb021640 1 -- 172.21.15.64:0/710189336 shutdown_connections 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.304+0000 7f75cb021640 1 --2- 172.21.15.64:0/710189336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143570 0x7f75c40cf050 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.304+0000 7f75cb021640 1 -- 172.21.15.64:0/710189336 >> 172.21.15.64:0/710189336 conn(0x7f75c406a860 msgr2=0x7f75c406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.304+0000 7f75cb021640 1 -- 172.21.15.64:0/710189336 shutdown_connections 2023-11-24T08:06:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.304+0000 7f75cb021640 1 -- 172.21.15.64:0/710189336 wait complete. 2023-11-24T08:06:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.306+0000 7f75cb021640 1 Processor -- start 2023-11-24T08:06:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.306+0000 7f75cb021640 1 -- start start 2023-11-24T08:06:54.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75cb021640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75c4143570 0x7f75c41404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75cb021640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143950 0x7f75c41f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75cb021640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75c415e6b0 0x7f75c40c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75cb021640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40c94c0 con 0x7f75c4143950 2023-11-24T08:06:54.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75cb021640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40c8710 con 0x7f75c4143570 2023-11-24T08:06:54.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75cb021640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40cb500 con 0x7f75c415e6b0 2023-11-24T08:06:54.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75c981e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143950 0x7f75c41f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75c981e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143950 0x7f75c41f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36668/0 (socket says 172.21.15.64:36668) 2023-11-24T08:06:54.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75c981e640 1 -- 172.21.15.64:0/2249565542 learned_addr learned my addr 172.21.15.64:0/2249565542 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:54.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75ca01f640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75c4143570 0x7f75c41404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.307+0000 7f75c981e640 1 -- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75c415e6b0 msgr2=0x7f75c40c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75ca820640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75c415e6b0 0x7f75c40c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75c981e640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75c415e6b0 0x7f75c40c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75c981e640 1 -- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75c4143570 msgr2=0x7f75c41404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75c981e640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75c4143570 0x7f75c41404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75c981e640 1 -- 172.21.15.64:0/2249565542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c0009a70 con 0x7f75c4143950 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75c981e640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143950 0x7f75c41f6660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f75b400c400 tx=0x7f75b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75baffd640 1 -- 172.21.15.64:0/2249565542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4011790 con 0x7f75c4143950 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75baffd640 1 -- 172.21.15.64:0/2249565542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b4011930 con 0x7f75c4143950 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c40ca960 con 0x7f75c4143950 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75baffd640 1 -- 172.21.15.64:0/2249565542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4011ce0 con 0x7f75c4143950 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.308+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c40cabd0 con 0x7f75c4143950 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.310+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c404e840 con 0x7f75c4143950 2023-11-24T08:06:54.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.311+0000 7f75baffd640 1 -- 172.21.15.64:0/2249565542 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f75b401c460 con 0x7f75c4143950 2023-11-24T08:06:54.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.312+0000 7f75baffd640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f75a806b080 0x7f75a806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.312+0000 7f75baffd640 1 -- 172.21.15.64:0/2249565542 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f75b40894c0 con 0x7f75c4143950 2023-11-24T08:06:54.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.313+0000 7f75ca01f640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f75a806b080 0x7f75a806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.314+0000 7f75ca01f640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f75a806b080 0x7f75a806d540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f75c0036000 tx=0x7f75c0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:54.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.317+0000 7f75baffd640 1 -- 172.21.15.64:0/2249565542 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75b40546c0 con 0x7f75c4143950 2023-11-24T08:06:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.519+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f75c40b3910 con 0x7f75c4143950 2023-11-24T08:06:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.520+0000 7f75baffd640 1 -- 172.21.15.64:0/2249565542 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+65 (secure 0 0 0) 0x7f75b4057ce0 con 0x7f75c4143950 2023-11-24T08:06:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:54.522 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:06:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:06:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-11-24T08:06:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-11-24T08:06:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:06:54.522 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:06:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f75a806b080 msgr2=0x7f75a806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f75a806b080 0x7f75a806d540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f75c0036000 tx=0x7f75c0030000 comp rx=0 tx=0).stop 2023-11-24T08:06:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143950 msgr2=0x7f75c41f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143950 0x7f75c41f6660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f75b400c400 tx=0x7f75b400c8c0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 shutdown_connections 2023-11-24T08:06:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f75a806b080 0x7f75a806d540 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75c415e6b0 0x7f75c40c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75c4143950 0x7f75c41f6660 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 --2- 172.21.15.64:0/2249565542 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75c4143570 0x7f75c41404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 >> 172.21.15.64:0/2249565542 conn(0x7f75c406a860 msgr2=0x7f75c406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 shutdown_connections 2023-11-24T08:06:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.523+0000 7f75cb021640 1 -- 172.21.15.64:0/2249565542 wait complete. 2023-11-24T08:06:54.545 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:06:54.546 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:06:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.770+0000 7f5c4947e640 1 Processor -- start 2023-11-24T08:06:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.770+0000 7f5c4947e640 1 -- start start 2023-11-24T08:06:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c4947e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c4415d3b0 0x7f5c440ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c4947e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c4947e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c440d2010 con 0x7f5c440d1bc0 2023-11-24T08:06:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c4947e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c440c6bf0 con 0x7f5c4415d3b0 2023-11-24T08:06:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c4947e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440d1400 con 0x7f5c440d0fa0 2023-11-24T08:06:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c437fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c43fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5c4415d3b0 0x7f5c440ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c437fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:59098/0 (socket says 172.21.15.64:59098) 2023-11-24T08:06:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.772+0000 7f5c437fe640 1 -- 172.21.15.64:0/43701129 learned_addr learned my addr 172.21.15.64:0/43701129 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.773+0000 7f5c42ffd640 1 -- 172.21.15.64:0/43701129 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564532116 0 0) 0x7f5c3c001110 con 0x7f5c440d1bc0 2023-11-24T08:06:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.773+0000 7f5c42ffd640 1 -- 172.21.15.64:0/43701129 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c2c003520 con 0x7f5c440d1bc0 2023-11-24T08:06:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.773+0000 7f5c437fe640 1 -- 172.21.15.64:0/43701129 >> v2:172.21.15.91:3300/0 conn(0x7f5c4415d3b0 msgr2=0x7f5c440ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.773+0000 7f5c437fe640 1 --2- 172.21.15.64:0/43701129 >> v2:172.21.15.91:3300/0 conn(0x7f5c4415d3b0 0x7f5c440ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.773+0000 7f5c437fe640 1 -- 172.21.15.64:0/43701129 >> v1:172.21.15.91:6789/0 conn(0x7f5c440d1bc0 legacy=0x7f5c440b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.773+0000 7f5c437fe640 1 -- 172.21.15.64:0/43701129 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440cedc0 con 0x7f5c440d0fa0 2023-11-24T08:06:54.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.775+0000 7f5c437fe640 1 --2- 172.21.15.64:0/43701129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440d1680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009870 tx=0x7f5c3802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa8ad2a875f0f156 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:54.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.775+0000 7f5c42ffd640 1 -- 172.21.15.64:0/43701129 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38031640 con 0x7f5c440d0fa0 2023-11-24T08:06:54.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.775+0000 7f5c42ffd640 1 -- 172.21.15.64:0/43701129 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c380317a0 con 0x7f5c440d0fa0 2023-11-24T08:06:54.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.775+0000 7f5c42ffd640 1 -- 172.21.15.64:0/43701129 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38031b30 con 0x7f5c440d0fa0 2023-11-24T08:06:54.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.776+0000 7f5c4947e640 1 -- 172.21.15.64:0/43701129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 msgr2=0x7f5c440d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.776+0000 7f5c4947e640 1 --2- 172.21.15.64:0/43701129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440d1680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009870 tx=0x7f5c3802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.776+0000 7f5c4947e640 1 -- 172.21.15.64:0/43701129 shutdown_connections 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.776+0000 7f5c4947e640 1 --2- 172.21.15.64:0/43701129 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440d1680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.776+0000 7f5c4947e640 1 --2- 172.21.15.64:0/43701129 >> v2:172.21.15.91:3300/0 conn(0x7f5c4415d3b0 0x7f5c440ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.776+0000 7f5c4947e640 1 -- 172.21.15.64:0/43701129 >> 172.21.15.64:0/43701129 conn(0x7f5c440d30a0 msgr2=0x7f5c440d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.776+0000 7f5c4947e640 1 -- 172.21.15.64:0/43701129 shutdown_connections 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.776+0000 7f5c4947e640 1 -- 172.21.15.64:0/43701129 wait complete. 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.778+0000 7f5c4947e640 1 Processor -- start 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.778+0000 7f5c4947e640 1 -- start start 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.779+0000 7f5c4947e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.779+0000 7f5c4947e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c440d1bc0 0x7f5c440a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.779+0000 7f5c4947e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c4415d3b0 0x7f5c440a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.779+0000 7f5c4947e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4409cde0 con 0x7f5c4415d3b0 2023-11-24T08:06:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.779+0000 7f5c4947e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4409d0c0 con 0x7f5c440d1bc0 2023-11-24T08:06:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.779+0000 7f5c4947e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4409d3a0 con 0x7f5c440d0fa0 2023-11-24T08:06:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c4415d3b0 0x7f5c440a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c4415d3b0 0x7f5c440a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36684/0 (socket says 172.21.15.64:36684) 2023-11-24T08:06:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 -- 172.21.15.64:0/1606007501 learned_addr learned my addr 172.21.15.64:0/1606007501 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:54.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c437fe640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c440d1bc0 0x7f5c440a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c43fff640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 -- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 msgr2=0x7f5c440b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 -- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c440d1bc0 msgr2=0x7f5c440a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:54.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c440d1bc0 0x7f5c440a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 -- 172.21.15.64:0/1606007501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380094d0 con 0x7f5c4415d3b0 2023-11-24T08:06:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.780+0000 7f5c48c7d640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c4415d3b0 0x7f5c440a08d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c007fd0 tx=0x7f5c3c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:54.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.781+0000 7f5c40ff9640 1 -- 172.21.15.64:0/1606007501 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0052d0 con 0x7f5c4415d3b0 2023-11-24T08:06:54.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.781+0000 7f5c40ff9640 1 -- 172.21.15.64:0/1606007501 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c005450 con 0x7f5c4415d3b0 2023-11-24T08:06:54.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.781+0000 7f5c40ff9640 1 -- 172.21.15.64:0/1606007501 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c003830 con 0x7f5c4415d3b0 2023-11-24T08:06:54.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.781+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4409d680 con 0x7f5c4415d3b0 2023-11-24T08:06:54.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.781+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4409dc40 con 0x7f5c4415d3b0 2023-11-24T08:06:54.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.782+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4409de30 con 0x7f5c4415d3b0 2023-11-24T08:06:54.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.783+0000 7f5c40ff9640 1 -- 172.21.15.64:0/1606007501 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5c3c003a50 con 0x7f5c4415d3b0 2023-11-24T08:06:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.784+0000 7f5c40ff9640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5c3006b080 0x7f5c3006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.784+0000 7f5c40ff9640 1 -- 172.21.15.64:0/1606007501 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5c3c088a40 con 0x7f5c4415d3b0 2023-11-24T08:06:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.785+0000 7f5c43fff640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5c3006b080 0x7f5c3006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.786+0000 7f5c43fff640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5c3006b080 0x7f5c3006d540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5c340073f0 tx=0x7f5c3400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.788+0000 7f5c40ff9640 1 -- 172.21.15.64:0/1606007501 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c3c053ba0 con 0x7f5c4415d3b0 2023-11-24T08:06:54.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:54.983+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5c440d2010 con 0x7f5c4415d3b0 2023-11-24T08:06:55.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.617+0000 7f5c40ff9640 1 -- 172.21.15.64:0/1606007501 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5c440d2010 con 0x7f5c4415d3b0 2023-11-24T08:06:55.619 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-11-24T08:06:55.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.620+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5c3006b080 msgr2=0x7f5c3006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:55.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.620+0000 7f5c4947e640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5c3006b080 0x7f5c3006d540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5c340073f0 tx=0x7f5c3400b040 comp rx=0 tx=0).stop 2023-11-24T08:06:55.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.620+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c4415d3b0 msgr2=0x7f5c440a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:55.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.620+0000 7f5c4947e640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c4415d3b0 0x7f5c440a08d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c007fd0 tx=0x7f5c3c00c820 comp rx=0 tx=0).stop 2023-11-24T08:06:55.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.621+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 shutdown_connections 2023-11-24T08:06:55.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.621+0000 7f5c4947e640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5c3006b080 0x7f5c3006d540 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.621+0000 7f5c4947e640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c4415d3b0 0x7f5c440a08d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.621+0000 7f5c4947e640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5c440d1bc0 0x7f5c440a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.621+0000 7f5c4947e640 1 --2- 172.21.15.64:0/1606007501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5c440d0fa0 0x7f5c440b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.621+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 >> 172.21.15.64:0/1606007501 conn(0x7f5c440d30a0 msgr2=0x7f5c4406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:55.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.621+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 shutdown_connections 2023-11-24T08:06:55.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.621+0000 7f5c4947e640 1 -- 172.21.15.64:0/1606007501 wait complete. 2023-11-24T08:06:55.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:06:55.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.870+0000 7f6c7a64b640 1 Processor -- start 2023-11-24T08:06:55.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.870+0000 7f6c7a64b640 1 -- start start 2023-11-24T08:06:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c7a64b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c741435d0 0x7f6c74071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c7a64b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6c741439b0 0x7f6c740b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c7a64b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c740b3f70 0x7f6c74070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c7a64b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c740b43e0 con 0x7f6c741435d0 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c7a64b640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c74070d40 con 0x7f6c741439b0 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c7a64b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c74071020 con 0x7f6c740b3f70 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c79649640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c741435d0 0x7f6c74071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c79649640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c741435d0 0x7f6c74071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36686/0 (socket says 172.21.15.64:36686) 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.872+0000 7f6c79649640 1 -- 172.21.15.64:0/2558233524 learned_addr learned my addr 172.21.15.64:0/2558233524 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.873+0000 7f6c79e4a640 1 --2- 172.21.15.64:0/2558233524 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c740b3f70 0x7f6c74070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.873+0000 7f6c78e48640 1 --2- 172.21.15.64:0/2558233524 >> v2:172.21.15.91:3300/0 conn(0x7f6c741439b0 0x7f6c740b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.873+0000 7f6c79649640 1 -- 172.21.15.64:0/2558233524 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c740b3f70 msgr2=0x7f6c74070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.873+0000 7f6c79649640 1 --2- 172.21.15.64:0/2558233524 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c740b3f70 0x7f6c74070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.873+0000 7f6c79649640 1 -- 172.21.15.64:0/2558233524 >> v2:172.21.15.91:3300/0 conn(0x7f6c741439b0 msgr2=0x7f6c740b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:55.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.873+0000 7f6c79649640 1 --2- 172.21.15.64:0/2558233524 >> v2:172.21.15.91:3300/0 conn(0x7f6c741439b0 0x7f6c740b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.873+0000 7f6c79649640 1 -- 172.21.15.64:0/2558233524 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c74071300 con 0x7f6c741435d0 2023-11-24T08:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.874+0000 7f6c79649640 1 --2- 172.21.15.64:0/2558233524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c741435d0 0x7f6c74071de0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6c740cbe80 tx=0x7f6c7002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990a9094250c30f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.874+0000 7f6c6bfff640 1 -- 172.21.15.64:0/2558233524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7002ee90 con 0x7f6c741435d0 2023-11-24T08:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.874+0000 7f6c6bfff640 1 -- 172.21.15.64:0/2558233524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c70036440 con 0x7f6c741435d0 2023-11-24T08:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.874+0000 7f6c6bfff640 1 -- 172.21.15.64:0/2558233524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700367d0 con 0x7f6c741435d0 2023-11-24T08:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 -- 172.21.15.64:0/2558233524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c741435d0 msgr2=0x7f6c74071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/2558233524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c741435d0 0x7f6c74071de0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6c740cbe80 tx=0x7f6c7002e0e0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 -- 172.21.15.64:0/2558233524 shutdown_connections 2023-11-24T08:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/2558233524 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c740b3f70 0x7f6c74070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/2558233524 >> v2:172.21.15.91:3300/0 conn(0x7f6c741439b0 0x7f6c740b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/2558233524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c741435d0 0x7f6c74071de0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 -- 172.21.15.64:0/2558233524 >> 172.21.15.64:0/2558233524 conn(0x7f6c740cec10 msgr2=0x7f6c740cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 -- 172.21.15.64:0/2558233524 shutdown_connections 2023-11-24T08:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.875+0000 7f6c7a64b640 1 -- 172.21.15.64:0/2558233524 wait complete. 2023-11-24T08:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.877+0000 7f6c7a64b640 1 Processor -- start 2023-11-24T08:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.877+0000 7f6c7a64b640 1 -- start start 2023-11-24T08:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c7a64b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740b3f70 0x7f6c7413b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c7a64b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c741435d0 0x7f6c7414e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c7a64b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c741439b0 0x7f6c740ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c7a64b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7413c030 con 0x7f6c740b3f70 2023-11-24T08:06:55.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c7a64b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7413c310 con 0x7f6c741439b0 2023-11-24T08:06:55.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c7a64b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7413c5f0 con 0x7f6c741435d0 2023-11-24T08:06:55.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c79649640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740b3f70 0x7f6c7413b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:55.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c79649640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740b3f70 0x7f6c7413b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36694/0 (socket says 172.21.15.64:36694) 2023-11-24T08:06:55.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.878+0000 7f6c79649640 1 -- 172.21.15.64:0/4048735327 learned_addr learned my addr 172.21.15.64:0/4048735327 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:55.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c79649640 1 -- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c741435d0 msgr2=0x7f6c7414e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c79e4a640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c741439b0 0x7f6c740ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c78e48640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c741435d0 0x7f6c7414e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c79649640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c741435d0 0x7f6c7414e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c79649640 1 -- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c741439b0 msgr2=0x7f6c740ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c79649640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c741439b0 0x7f6c740ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c79649640 1 -- 172.21.15.64:0/4048735327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7000b770 con 0x7f6c740b3f70 2023-11-24T08:06:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c79649640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740b3f70 0x7f6c7413b730 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6c7002e120 tx=0x7f6c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:55.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.879+0000 7f6c69ffb640 1 -- 172.21.15.64:0/4048735327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70034850 con 0x7f6c740b3f70 2023-11-24T08:06:55.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.880+0000 7f6c69ffb640 1 -- 172.21.15.64:0/4048735327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c700349b0 con 0x7f6c740b3f70 2023-11-24T08:06:55.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.880+0000 7f6c69ffb640 1 -- 172.21.15.64:0/4048735327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70034d60 con 0x7f6c740b3f70 2023-11-24T08:06:55.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.880+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7413c8d0 con 0x7f6c740b3f70 2023-11-24T08:06:55.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.880+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7413bc70 con 0x7f6c740b3f70 2023-11-24T08:06:55.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.881+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7413be00 con 0x7f6c740b3f70 2023-11-24T08:06:55.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.882+0000 7f6c69ffb640 1 -- 172.21.15.64:0/4048735327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6c700403e0 con 0x7f6c740b3f70 2023-11-24T08:06:55.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.883+0000 7f6c69ffb640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c5806b080 0x7f6c5806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:55.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.883+0000 7f6c69ffb640 1 -- 172.21.15.64:0/4048735327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6c700abff0 con 0x7f6c740b3f70 2023-11-24T08:06:55.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.884+0000 7f6c78e48640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c5806b080 0x7f6c5806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.885+0000 7f6c78e48640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c5806b080 0x7f6c5806d540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c64006fd0 tx=0x7f6c64008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:55.887+0000 7f6c69ffb640 1 -- 172.21.15.64:0/4048735327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c70076f20 con 0x7f6c740b3f70 2023-11-24T08:06:56.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.087+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6c7413c730 con 0x7f6c740b3f70 2023-11-24T08:06:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.624+0000 7f6c69ffb640 1 -- 172.21.15.64:0/4048735327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6c7413c730 con 0x7f6c740b3f70 2023-11-24T08:06:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-11-24T08:06:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.627+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c5806b080 msgr2=0x7f6c5806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.627+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c5806b080 0x7f6c5806d540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c64006fd0 tx=0x7f6c64008040 comp rx=0 tx=0).stop 2023-11-24T08:06:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740b3f70 msgr2=0x7f6c7413b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740b3f70 0x7f6c7413b730 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6c7002e120 tx=0x7f6c700046c0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 shutdown_connections 2023-11-24T08:06:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6c5806b080 0x7f6c5806d540 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6c741439b0 0x7f6c740ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6c741435d0 0x7f6c7414e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 --2- 172.21.15.64:0/4048735327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740b3f70 0x7f6c7413b730 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 >> 172.21.15.64:0/4048735327 conn(0x7f6c740cec10 msgr2=0x7f6c740b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 shutdown_connections 2023-11-24T08:06:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.628+0000 7f6c7a64b640 1 -- 172.21.15.64:0/4048735327 wait complete. 2023-11-24T08:06:56.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:06:56.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.856+0000 7fd5d4246640 1 Processor -- start 2023-11-24T08:06:56.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.856+0000 7fd5d4246640 1 -- start start 2023-11-24T08:06:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.858+0000 7fd5d4246640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd5cc09cb30 0x7fd5cc0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.858+0000 7fd5d4246640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5cc15b730 con 0x7fd5cc15b350 2023-11-24T08:06:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.858+0000 7fd5d4246640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5cc0ac740 con 0x7fd5cc0adb60 2023-11-24T08:06:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.858+0000 7fd5d4246640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5cc0ac460 con 0x7fd5cc09cb30 2023-11-24T08:06:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.858+0000 7fd5d1fe2640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd5cc15b350 0x7fd5cc0b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56022/0 (socket says 172.21.15.64:56022) 2023-11-24T08:06:56.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.858+0000 7fd5d1fe2640 1 -- 172.21.15.64:0/4003895274 learned_addr learned my addr 172.21.15.64:0/4003895274 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.859+0000 7fd5d17e1640 1 --2- 172.21.15.64:0/4003895274 >> v2:172.21.15.91:3300/0 conn(0x7fd5cc09cb30 0x7fd5cc0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.859+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865792470 0 0) 0x7fd5c8001490 con 0x7fd5cc15b350 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.859+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5b8003520 con 0x7fd5cc15b350 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.859+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1142030779 0 0) 0x7fd5b8003520 con 0x7fd5cc15b350 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.859+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 >> v2:172.21.15.91:3300/0 conn(0x7fd5cc09cb30 msgr2=0x7fd5cc0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.859+0000 7fd5d0fe0640 1 --2- 172.21.15.64:0/4003895274 >> v2:172.21.15.91:3300/0 conn(0x7fd5cc09cb30 0x7fd5cc0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.859+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 >> v1:172.21.15.91:6789/0 conn(0x7fd5cc0adb60 legacy=0x7fd5cc0adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc0b2a50 con 0x7fd5cc15b350 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fd5c8001b80 con 0x7fd5cc15b350 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 >> v1:172.21.15.64:6789/0 conn(0x7fd5cc15b350 legacy=0x7fd5cc0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd5b80047c0 0x7fd5b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:56.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5b8005180 0x7fd5b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:56.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc0ac460 con 0x7fd5b8003970 2023-11-24T08:06:56.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b800bfd0 con 0x7fd5b8005180 2023-11-24T08:06:56.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b800c190 con 0x7fd5b80047c0 2023-11-24T08:06:56.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d17e1640 1 --2- 172.21.15.64:0/4003895274 >> v2:172.21.15.91:3300/0 conn(0x7fd5cc09cb30 0x7fd5cc0abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:06:56.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d1fe2640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:56.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d27e3640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5b8005180 0x7fd5b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:56.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d1fe2640 1 -- 172.21.15.64:0/4003895274 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd5b80047c0 msgr2=0x7fd5b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:56.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d1fe2640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd5b80047c0 0x7fd5b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d1fe2640 1 -- 172.21.15.64:0/4003895274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5b8005180 msgr2=0x7fd5b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:56.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.860+0000 7fd5d1fe2640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5b8005180 0x7fd5b800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.861+0000 7fd5d1fe2640 1 -- 172.21.15.64:0/4003895274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc0b2a50 con 0x7fd5b8003970 2023-11-24T08:06:56.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.861+0000 7fd5d27e3640 1 -- 172.21.15.64:0/4003895274 reap_dead start 2023-11-24T08:06:56.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.862+0000 7fd5d1fe2640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5b8003d90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800a8f0 tx=0x7fd5b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33433f9bbafdeefa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:56.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.862+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80072d0 con 0x7fd5b8003970 2023-11-24T08:06:56.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.862+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c802f070 con 0x7fd5b8003970 2023-11-24T08:06:56.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.862+0000 7fd5d0fe0640 1 -- 172.21.15.64:0/4003895274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8007690 con 0x7fd5b8003970 2023-11-24T08:06:56.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.862+0000 7fd5d4246640 1 -- 172.21.15.64:0/4003895274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 msgr2=0x7fd5b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:56.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.862+0000 7fd5d4246640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5b8003d90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800a8f0 tx=0x7fd5b80057f0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.863+0000 7fd5d4246640 1 -- 172.21.15.64:0/4003895274 shutdown_connections 2023-11-24T08:06:56.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.863+0000 7fd5d4246640 1 --2- 172.21.15.64:0/4003895274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5b8003d90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.863+0000 7fd5d4246640 1 -- 172.21.15.64:0/4003895274 >> 172.21.15.64:0/4003895274 conn(0x7fd5cc0b0ba0 msgr2=0x7fd5cc0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:56.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.863+0000 7fd5d4246640 1 -- 172.21.15.64:0/4003895274 shutdown_connections 2023-11-24T08:06:56.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.863+0000 7fd5d4246640 1 -- 172.21.15.64:0/4003895274 wait complete. 2023-11-24T08:06:56.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.864+0000 7fd5d4246640 1 Processor -- start 2023-11-24T08:06:56.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.865+0000 7fd5d4246640 1 -- start start 2023-11-24T08:06:56.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d4246640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5cc1f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:56.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d4246640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd5cc09b460 0x7fd5cc070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:56.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d4246640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5cc0717a0 0x7fd5cc071be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:56.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d4246640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc1f36d0 con 0x7fd5b8003970 2023-11-24T08:06:56.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d4246640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc149390 con 0x7fd5cc0717a0 2023-11-24T08:06:56.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d4246640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc072120 con 0x7fd5cc09b460 2023-11-24T08:06:56.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d1fe2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5cc1f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d1fe2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5cc1f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36702/0 (socket says 172.21.15.64:36702) 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d1fe2640 1 -- 172.21.15.64:0/3175434644 learned_addr learned my addr 172.21.15.64:0/3175434644 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d17e1640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd5cc09b460 0x7fd5cc070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d27e3640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5cc0717a0 0x7fd5cc071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d1fe2640 1 -- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd5cc09b460 msgr2=0x7fd5cc070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d1fe2640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd5cc09b460 0x7fd5cc070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d1fe2640 1 -- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5cc0717a0 msgr2=0x7fd5cc071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d1fe2640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5cc0717a0 0x7fd5cc071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d1fe2640 1 -- 172.21.15.64:0/3175434644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc09b8f0 con 0x7fd5b8003970 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.866+0000 7fd5d27e3640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5cc0717a0 0x7fd5cc071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:56.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.867+0000 7fd5d1fe2640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5cc1f3190 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd5c802cc30 tx=0x7fd5c8001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:56.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.867+0000 7fd5c27fc640 1 -- 172.21.15.64:0/3175434644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8004d60 con 0x7fd5b8003970 2023-11-24T08:06:56.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.867+0000 7fd5c27fc640 1 -- 172.21.15.64:0/3175434644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c803f470 con 0x7fd5b8003970 2023-11-24T08:06:56.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.867+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5cc09bbd0 con 0x7fd5b8003970 2023-11-24T08:06:56.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.867+0000 7fd5c27fc640 1 -- 172.21.15.64:0/3175434644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c803f800 con 0x7fd5b8003970 2023-11-24T08:06:56.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.867+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc09c190 con 0x7fd5b8003970 2023-11-24T08:06:56.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.869+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5cc09d200 con 0x7fd5b8003970 2023-11-24T08:06:56.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.870+0000 7fd5c27fc640 1 -- 172.21.15.64:0/3175434644 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd5c8031030 con 0x7fd5b8003970 2023-11-24T08:06:56.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.871+0000 7fd5c27fc640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd5b006b0d0 0x7fd5b006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:56.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.871+0000 7fd5c27fc640 1 -- 172.21.15.64:0/3175434644 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd5c80aac50 con 0x7fd5b8003970 2023-11-24T08:06:56.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.871+0000 7fd5d17e1640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd5b006b0d0 0x7fd5b006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:56.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.872+0000 7fd5d17e1640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd5b006b0d0 0x7fd5b006d590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0674f0 tx=0x7fd5bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:56.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:56.876+0000 7fd5c27fc640 1 -- 172.21.15.64:0/3175434644 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5c8075b30 con 0x7fd5b8003970 2023-11-24T08:06:57.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:57.110+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd5cc04e920 con 0x7fd5b8003970 2023-11-24T08:06:58.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.005+0000 7fd5c27fc640 1 -- 172.21.15.64:0/3175434644 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd5c8079150 con 0x7fd5b8003970 2023-11-24T08:06:58.008 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 8 and data pool 9 2023-11-24T08:06:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd5b006b0d0 msgr2=0x7fd5b006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd5b006b0d0 0x7fd5b006d590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0674f0 tx=0x7fd5bc008040 comp rx=0 tx=0).stop 2023-11-24T08:06:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 msgr2=0x7fd5cc1f3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5cc1f3190 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd5c802cc30 tx=0x7fd5c8001360 comp rx=0 tx=0).stop 2023-11-24T08:06:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 shutdown_connections 2023-11-24T08:06:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd5b006b0d0 0x7fd5b006d590 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd5cc0717a0 0x7fd5cc071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd5cc09b460 0x7fd5cc070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 --2- 172.21.15.64:0/3175434644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b8003970 0x7fd5cc1f3190 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 >> 172.21.15.64:0/3175434644 conn(0x7fd5cc0b0ba0 msgr2=0x7fd5cc15a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:58.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 shutdown_connections 2023-11-24T08:06:58.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.009+0000 7fd5d4246640 1 -- 172.21.15.64:0/3175434644 wait complete. 2023-11-24T08:06:58.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:58.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.233+0000 7f2ae759e640 1 Processor -- start 2023-11-24T08:06:58.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.233+0000 7f2ae759e640 1 -- start start 2023-11-24T08:06:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.235+0000 7f2ae759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ae80cd320 0x7f2ae815d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.235+0000 7f2ae759e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2ae815de50 0x7f2ae8142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.235+0000 7f2ae759e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae815e910 con 0x7f2ae80cd700 2023-11-24T08:06:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.235+0000 7f2ae759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae806bdf0 con 0x7f2ae80cd320 2023-11-24T08:06:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.235+0000 7f2ae759e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae815e370 con 0x7f2ae815de50 2023-11-24T08:06:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.235+0000 7f2ae659c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ae80cd320 0x7f2ae815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.236+0000 7f2ae659c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ae80cd320 0x7f2ae815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59942/0 (socket says 172.21.15.64:59942) 2023-11-24T08:06:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.236+0000 7f2ae659c640 1 -- 172.21.15.64:0/609571172 learned_addr learned my addr 172.21.15.64:0/609571172 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.236+0000 7f2ae5d9b640 1 --2- 172.21.15.64:0/609571172 >> v2:172.21.15.91:3300/0 conn(0x7f2ae815de50 0x7f2ae8142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.236+0000 7f2ae659c640 1 -- 172.21.15.64:0/609571172 >> v2:172.21.15.91:3300/0 conn(0x7f2ae815de50 msgr2=0x7f2ae8142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.236+0000 7f2ae659c640 1 --2- 172.21.15.64:0/609571172 >> v2:172.21.15.91:3300/0 conn(0x7f2ae815de50 0x7f2ae8142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.236+0000 7f2ae659c640 1 -- 172.21.15.64:0/609571172 >> v1:172.21.15.91:6789/0 conn(0x7f2ae80cd700 legacy=0x7f2ae806dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.236+0000 7f2ae659c640 1 -- 172.21.15.64:0/609571172 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae815e630 con 0x7f2ae80cd320 2023-11-24T08:06:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.236+0000 7f2ae5d9b640 1 --2- 172.21.15.64:0/609571172 >> v2:172.21.15.91:3300/0 conn(0x7f2ae815de50 0x7f2ae8142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.237+0000 7f2ae659c640 1 --2- 172.21.15.64:0/609571172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd320 0x7f2ae815d910 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80d0400 tx=0x7f2ad802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c749ed9ceff9634b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae559a640 1 -- 172.21.15.64:0/609571172 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad802c9f0 con 0x7f2ae80cd320 2023-11-24T08:06:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae559a640 1 -- 172.21.15.64:0/609571172 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad802cb50 con 0x7f2ae80cd320 2023-11-24T08:06:58.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae559a640 1 -- 172.21.15.64:0/609571172 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80343f0 con 0x7f2ae80cd320 2023-11-24T08:06:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae759e640 1 -- 172.21.15.64:0/609571172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd320 msgr2=0x7f2ae815d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae759e640 1 --2- 172.21.15.64:0/609571172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd320 0x7f2ae815d910 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80d0400 tx=0x7f2ad802be50 comp rx=0 tx=0).stop 2023-11-24T08:06:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae759e640 1 -- 172.21.15.64:0/609571172 shutdown_connections 2023-11-24T08:06:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae759e640 1 --2- 172.21.15.64:0/609571172 >> v2:172.21.15.91:3300/0 conn(0x7f2ae815de50 0x7f2ae8142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae759e640 1 --2- 172.21.15.64:0/609571172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd320 0x7f2ae815d910 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae759e640 1 -- 172.21.15.64:0/609571172 >> 172.21.15.64:0/609571172 conn(0x7f2ae80cbf60 msgr2=0x7f2ae80cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.238+0000 7f2ae759e640 1 -- 172.21.15.64:0/609571172 shutdown_connections 2023-11-24T08:06:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.239+0000 7f2ae759e640 1 -- 172.21.15.64:0/609571172 wait complete. 2023-11-24T08:06:58.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.240+0000 7f2ae759e640 1 Processor -- start 2023-11-24T08:06:58.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.240+0000 7f2ae759e640 1 -- start start 2023-11-24T08:06:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.241+0000 7f2ae759e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ae80cd320 0x7f2ae80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.241+0000 7f2ae759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd700 0x7f2ae80b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2ae815de50 0x7f2ae815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8131c80 con 0x7f2ae80cd700 2023-11-24T08:06:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae759e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8131f60 con 0x7f2ae80cd320 2023-11-24T08:06:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8132240 con 0x7f2ae815de50 2023-11-24T08:06:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd700 0x7f2ae80b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd700 0x7f2ae80b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59952/0 (socket says 172.21.15.64:59952) 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae5d9b640 1 -- 172.21.15.64:0/2714013030 learned_addr learned my addr 172.21.15.64:0/2714013030 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae659c640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ae80cd320 0x7f2ae80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae6d9d640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2ae815de50 0x7f2ae815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae5d9b640 1 -- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2ae815de50 msgr2=0x7f2ae815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae5d9b640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2ae815de50 0x7f2ae815a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae5d9b640 1 -- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ae80cd320 msgr2=0x7f2ae80b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae5d9b640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ae80cd320 0x7f2ae80b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae5d9b640 1 -- 172.21.15.64:0/2714013030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad8009b30 con 0x7f2ae80cd700 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.242+0000 7f2ae659c640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ae80cd320 0x7f2ae80b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.243+0000 7f2ae5d9b640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd700 0x7f2ae80b2940 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00c7f0 tx=0x7f2adc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.243+0000 7f2ad6ffd640 1 -- 172.21.15.64:0/2714013030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc007820 con 0x7f2ae80cd700 2023-11-24T08:06:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.243+0000 7f2ad6ffd640 1 -- 172.21.15.64:0/2714013030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc007980 con 0x7f2ae80cd700 2023-11-24T08:06:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.243+0000 7f2ad6ffd640 1 -- 172.21.15.64:0/2714013030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc007cf0 con 0x7f2ae80cd700 2023-11-24T08:06:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.243+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae8132520 con 0x7f2ae80cd700 2023-11-24T08:06:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.243+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae80b2e80 con 0x7f2ae80cd700 2023-11-24T08:06:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.244+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae8132740 con 0x7f2ae80cd700 2023-11-24T08:06:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.246+0000 7f2ad6ffd640 1 -- 172.21.15.64:0/2714013030 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2adc0113f0 con 0x7f2ae80cd700 2023-11-24T08:06:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.247+0000 7f2ad6ffd640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ac806b080 0x7f2ac806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.247+0000 7f2ad6ffd640 1 -- 172.21.15.64:0/2714013030 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2adc089e60 con 0x7f2ae80cd700 2023-11-24T08:06:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.248+0000 7f2ae659c640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ac806b080 0x7f2ac806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.249+0000 7f2ae659c640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ac806b080 0x7f2ac806d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8035000 tx=0x7f2ad8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.251+0000 7f2ad6ffd640 1 -- 172.21.15.64:0/2714013030 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2adc054aa0 con 0x7f2ae80cd700 2023-11-24T08:06:58.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.444+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ae8156d70 con 0x7f2ae80cd700 2023-11-24T08:06:58.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.446+0000 7f2ad6ffd640 1 -- 172.21.15.64:0/2714013030 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16530 (secure 0 0 0) 0x7f2adc01f090 con 0x7f2ae80cd700 2023-11-24T08:06:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:58.449 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":59,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:57.111758+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:55.038686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:56.142053+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ac806b080 msgr2=0x7f2ac806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ac806b080 0x7f2ac806d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8035000 tx=0x7f2ad8030000 comp rx=0 tx=0).stop 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd700 msgr2=0x7f2ae80b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd700 0x7f2ae80b2940 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00c7f0 tx=0x7f2adc00ccb0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 shutdown_connections 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ac806b080 0x7f2ac806d540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2ae815de50 0x7f2ae815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae80cd700 0x7f2ae80b2940 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 --2- 172.21.15.64:0/2714013030 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2ae80cd320 0x7f2ae80b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 >> 172.21.15.64:0/2714013030 conn(0x7f2ae80cbf60 msgr2=0x7f2ae815c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:58.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 shutdown_connections 2023-11-24T08:06:58.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.449+0000 7f2ae759e640 1 -- 172.21.15.64:0/2714013030 wait complete. 2023-11-24T08:06:58.469 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:06:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.691+0000 7f99be728640 1 Processor -- start 2023-11-24T08:06:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.691+0000 7f99be728640 1 -- start start 2023-11-24T08:06:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.693+0000 7f99be728640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f99b806be70 0x7f99b80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.693+0000 7f99be728640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b815d840 0x7f99b80cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.693+0000 7f99be728640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b80cbf60 con 0x7f99b815d460 2023-11-24T08:06:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.693+0000 7f99be728640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b80cc520 con 0x7f99b806be70 2023-11-24T08:06:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.693+0000 7f99be728640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b80d03b0 con 0x7f99b815d840 2023-11-24T08:06:58.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.693+0000 7f99bcf25640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f99b806be70 0x7f99b80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.694+0000 7f99bdf27640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b815d840 0x7f99b80cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.694+0000 7f99bd726640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f99b815d460 0x7f99b806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:54266/0 (socket says 172.21.15.64:54266) 2023-11-24T08:06:58.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.694+0000 7f99bd726640 1 -- 172.21.15.64:0/2033301960 learned_addr learned my addr 172.21.15.64:0/2033301960 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:06:58.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.694+0000 7f99affff640 1 -- 172.21.15.64:0/2033301960 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199256386 0 0) 0x7f99b80cbf60 con 0x7f99b815d460 2023-11-24T08:06:58.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.694+0000 7f99affff640 1 -- 172.21.15.64:0/2033301960 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a4003520 con 0x7f99b815d460 2023-11-24T08:06:58.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.694+0000 7f99bcf25640 1 -- 172.21.15.64:0/2033301960 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b815d840 msgr2=0x7f99b80cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:58.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.695+0000 7f99bcf25640 1 --2- 172.21.15.64:0/2033301960 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b815d840 0x7f99b80cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.695+0000 7f99bcf25640 1 -- 172.21.15.64:0/2033301960 >> v1:172.21.15.91:6789/0 conn(0x7f99b815d460 legacy=0x7f99b806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.695+0000 7f99bcf25640 1 -- 172.21.15.64:0/2033301960 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b806c3c0 con 0x7f99b806be70 2023-11-24T08:06:58.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.697+0000 7f99bcf25640 1 --2- 172.21.15.64:0/2033301960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b806be70 0x7f99b80b2100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99a80097a0 tx=0x7f99a802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf7569dbad557d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:58.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99affff640 1 -- 172.21.15.64:0/2033301960 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8031830 con 0x7f99b806be70 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99affff640 1 -- 172.21.15.64:0/2033301960 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a8031990 con 0x7f99b806be70 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99affff640 1 -- 172.21.15.64:0/2033301960 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8031d20 con 0x7f99b806be70 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99be728640 1 -- 172.21.15.64:0/2033301960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b806be70 msgr2=0x7f99b80b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99be728640 1 --2- 172.21.15.64:0/2033301960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b806be70 0x7f99b80b2100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99a80097a0 tx=0x7f99a802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99be728640 1 -- 172.21.15.64:0/2033301960 shutdown_connections 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99be728640 1 --2- 172.21.15.64:0/2033301960 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b815d840 0x7f99b80cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99be728640 1 --2- 172.21.15.64:0/2033301960 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b806be70 0x7f99b80b2100 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.698+0000 7f99be728640 1 -- 172.21.15.64:0/2033301960 >> 172.21.15.64:0/2033301960 conn(0x7f99b80d0f00 msgr2=0x7f99b80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.699+0000 7f99be728640 1 -- 172.21.15.64:0/2033301960 shutdown_connections 2023-11-24T08:06:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.699+0000 7f99be728640 1 -- 172.21.15.64:0/2033301960 wait complete. 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.700+0000 7f99be728640 1 Processor -- start 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.700+0000 7f99be728640 1 -- start start 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.701+0000 7f99be728640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b806be70 0x7f99b815f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.701+0000 7f99be728640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b815d460 0x7f99b815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.701+0000 7f99be728640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b815d840 0x7f99b8155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.701+0000 7f99be728640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8160900 con 0x7f99b815d840 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.701+0000 7f99be728640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b81f2bd0 con 0x7f99b815d460 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.701+0000 7f99be728640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b815fc80 con 0x7f99b806be70 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bdf27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b815d840 0x7f99b8155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bdf27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b815d840 0x7f99b8155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59954/0 (socket says 172.21.15.64:59954) 2023-11-24T08:06:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bdf27640 1 -- 172.21.15.64:0/977478802 learned_addr learned my addr 172.21.15.64:0/977478802 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bd726640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b806be70 0x7f99b815f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bcf25640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b815d460 0x7f99b815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bdf27640 1 -- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b806be70 msgr2=0x7f99b815f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bdf27640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b806be70 0x7f99b815f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bdf27640 1 -- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b815d460 msgr2=0x7f99b815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.702+0000 7f99bdf27640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b815d460 0x7f99b815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.703+0000 7f99bdf27640 1 -- 172.21.15.64:0/977478802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8009400 con 0x7f99b815d840 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.703+0000 7f99bd726640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b806be70 0x7f99b815f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.703+0000 7f99bdf27640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b815d840 0x7f99b8155430 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f99b000efc0 tx=0x7f99b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.703+0000 7f99adffb640 1 -- 172.21.15.64:0/977478802 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0014480 con 0x7f99b815d840 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.703+0000 7f99adffb640 1 -- 172.21.15.64:0/977478802 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b000f040 con 0x7f99b815d840 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.703+0000 7f99adffb640 1 -- 172.21.15.64:0/977478802 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0014820 con 0x7f99b815d840 2023-11-24T08:06:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.703+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b815ff60 con 0x7f99b815d840 2023-11-24T08:06:58.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.703+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b8160520 con 0x7f99b815d840 2023-11-24T08:06:58.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.704+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b804e840 con 0x7f99b815d840 2023-11-24T08:06:58.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.706+0000 7f99adffb640 1 -- 172.21.15.64:0/977478802 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f99b0014980 con 0x7f99b815d840 2023-11-24T08:06:58.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.707+0000 7f99adffb640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999c06b080 0x7f999c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:58.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.707+0000 7f99adffb640 1 -- 172.21.15.64:0/977478802 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99b008aa90 con 0x7f99b815d840 2023-11-24T08:06:58.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.707+0000 7f99bd726640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999c06b080 0x7f999c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:58.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.708+0000 7f99bd726640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999c06b080 0x7f999c06d540 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f99b40011e0 tx=0x7f99b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:58.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.711+0000 7f99adffb640 1 -- 172.21.15.64:0/977478802 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99b0055630 con 0x7f99b815d840 2023-11-24T08:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.908+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99b806e170 con 0x7f99b815d840 2023-11-24T08:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.909+0000 7f99adffb640 1 -- 172.21.15.64:0/977478802 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f99b0018090 con 0x7f99b815d840 2023-11-24T08:06:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:06:58.912 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:58.644237+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:55.038686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:56.142053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:06:58.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999c06b080 msgr2=0x7f999c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999c06b080 0x7f999c06d540 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f99b40011e0 tx=0x7f99b4009040 comp rx=0 tx=0).stop 2023-11-24T08:06:58.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b815d840 msgr2=0x7f99b8155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b815d840 0x7f99b8155430 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f99b000efc0 tx=0x7f99b000c490 comp rx=0 tx=0).stop 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 shutdown_connections 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f999c06b080 0x7f999c06d540 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b815d840 0x7f99b8155430 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99b815d460 0x7f99b815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 --2- 172.21.15.64:0/977478802 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99b806be70 0x7f99b815f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 >> 172.21.15.64:0/977478802 conn(0x7f99b80d0f00 msgr2=0x7f99b80cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.912+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 shutdown_connections 2023-11-24T08:06:58.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:58.913+0000 7f99be728640 1 -- 172.21.15.64:0/977478802 wait complete. 2023-11-24T08:06:58.934 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:06:59.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.158+0000 7f3f3da4b640 1 Processor -- start 2023-11-24T08:06:59.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.158+0000 7f3f3da4b640 1 -- start start 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f3da4b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f3815f270 0x7f3f380cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f3da4b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3f3806d9e0 0x7f3f380cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f3da4b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3815f930 con 0x7f3f38144770 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f3da4b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f380b43b0 con 0x7f3f3815f270 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f3da4b640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f380b3b10 con 0x7f3f3806d9e0 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f37fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f3815f270 0x7f3f380cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f37fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f3815f270 0x7f3f380cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59966/0 (socket says 172.21.15.64:59966) 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f37fff640 1 -- 172.21.15.64:0/3515838477 learned_addr learned my addr 172.21.15.64:0/3515838477 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.160+0000 7f3f3ca49640 1 --1- 172.21.15.64:0/3515838477 >> v1:172.21.15.64:6789/0 conn(0x7f3f38144770 0x7f3f3815eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59558/0 (socket says 172.21.15.64:59558) 2023-11-24T08:06:59.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.161+0000 7f3f3d24a640 1 --2- 172.21.15.64:0/3515838477 >> v2:172.21.15.91:3300/0 conn(0x7f3f3806d9e0 0x7f3f380cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:59.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.161+0000 7f3f37fff640 1 -- 172.21.15.64:0/3515838477 >> v2:172.21.15.91:3300/0 conn(0x7f3f3806d9e0 msgr2=0x7f3f380cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:06:59.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.161+0000 7f3f37fff640 1 --2- 172.21.15.64:0/3515838477 >> v2:172.21.15.91:3300/0 conn(0x7f3f3806d9e0 0x7f3f380cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:59.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.161+0000 7f3f37fff640 1 -- 172.21.15.64:0/3515838477 >> v1:172.21.15.64:6789/0 conn(0x7f3f38144770 legacy=0x7f3f3815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:59.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.161+0000 7f3f37fff640 1 -- 172.21.15.64:0/3515838477 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f380b3df0 con 0x7f3f3815f270 2023-11-24T08:06:59.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.162+0000 7f3f37fff640 1 --2- 172.21.15.64:0/3515838477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3815f270 0x7f3f380cd4a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3f38144eb0 tx=0x7f3f2402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefc6af9dfb8b1f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:59.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f377fe640 1 -- 172.21.15.64:0/3515838477 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24030860 con 0x7f3f3815f270 2023-11-24T08:06:59.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f377fe640 1 -- 172.21.15.64:0/3515838477 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f240309c0 con 0x7f3f3815f270 2023-11-24T08:06:59.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f377fe640 1 -- 172.21.15.64:0/3515838477 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24030d50 con 0x7f3f3815f270 2023-11-24T08:06:59.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f3da4b640 1 -- 172.21.15.64:0/3515838477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3815f270 msgr2=0x7f3f380cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:59.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/3515838477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3815f270 0x7f3f380cd4a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3f38144eb0 tx=0x7f3f2402b9a0 comp rx=0 tx=0).stop 2023-11-24T08:06:59.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f3da4b640 1 -- 172.21.15.64:0/3515838477 shutdown_connections 2023-11-24T08:06:59.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/3515838477 >> v2:172.21.15.91:3300/0 conn(0x7f3f3806d9e0 0x7f3f380cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:59.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/3515838477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3815f270 0x7f3f380cd4a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:59.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.163+0000 7f3f3da4b640 1 -- 172.21.15.64:0/3515838477 >> 172.21.15.64:0/3515838477 conn(0x7f3f380cec10 msgr2=0x7f3f380cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:06:59.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.164+0000 7f3f3da4b640 1 -- 172.21.15.64:0/3515838477 shutdown_connections 2023-11-24T08:06:59.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.164+0000 7f3f3da4b640 1 -- 172.21.15.64:0/3515838477 wait complete. 2023-11-24T08:06:59.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.165+0000 7f3f3da4b640 1 Processor -- start 2023-11-24T08:06:59.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.165+0000 7f3f3da4b640 1 -- start start 2023-11-24T08:06:59.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.166+0000 7f3f3da4b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3806d9e0 0x7f3f380cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:59.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.166+0000 7f3f3da4b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f38144770 0x7f3f3815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:59.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.166+0000 7f3f3da4b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3815f270 0x7f3f3813fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:59.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.166+0000 7f3f3da4b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3806d010 con 0x7f3f3806d9e0 2023-11-24T08:06:59.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.166+0000 7f3f3da4b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3815e440 con 0x7f3f38144770 2023-11-24T08:06:59.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.166+0000 7f3f3da4b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3815e160 con 0x7f3f3815f270 2023-11-24T08:06:59.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3ca49640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3806d9e0 0x7f3f380cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:59.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3ca49640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3806d9e0 0x7f3f380cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59980/0 (socket says 172.21.15.64:59980) 2023-11-24T08:06:59.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3ca49640 1 -- 172.21.15.64:0/2479135322 learned_addr learned my addr 172.21.15.64:0/2479135322 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:06:59.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f37fff640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f38144770 0x7f3f3815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:59.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3d24a640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3815f270 0x7f3f3813fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:59.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3ca49640 1 -- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3815f270 msgr2=0x7f3f3813fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3ca49640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3815f270 0x7f3f3813fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3ca49640 1 -- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f38144770 msgr2=0x7f3f3815d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3ca49640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f38144770 0x7f3f3815d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f3ca49640 1 -- 172.21.15.64:0/2479135322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240094d0 con 0x7f3f3806d9e0 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.167+0000 7f3f37fff640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f38144770 0x7f3f3815d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.168+0000 7f3f3ca49640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3806d9e0 0x7f3f380cb8c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c4d0 tx=0x7f3f2c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.168+0000 7f3f357fa640 1 -- 172.21.15.64:0/2479135322 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c007480 con 0x7f3f3806d9e0 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.168+0000 7f3f357fa640 1 -- 172.21.15.64:0/2479135322 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c0075e0 con 0x7f3f3806d9e0 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.168+0000 7f3f357fa640 1 -- 172.21.15.64:0/2479135322 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0049e0 con 0x7f3f3806d9e0 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.168+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3806c950 con 0x7f3f3806d9e0 2023-11-24T08:06:59.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.168+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3815de50 con 0x7f3f3806d9e0 2023-11-24T08:06:59.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.169+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3806d150 con 0x7f3f3806d9e0 2023-11-24T08:06:59.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.170+0000 7f3f357fa640 1 -- 172.21.15.64:0/2479135322 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3f2c004b40 con 0x7f3f3806d9e0 2023-11-24T08:06:59.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.171+0000 7f3f357fa640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1c06b080 0x7f3f1c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:06:59.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.171+0000 7f3f357fa640 1 -- 172.21.15.64:0/2479135322 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f2c088b20 con 0x7f3f3806d9e0 2023-11-24T08:06:59.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.172+0000 7f3f37fff640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1c06b080 0x7f3f1c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:06:59.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.173+0000 7f3f37fff640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1c06b080 0x7f3f1c06d540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3f24036000 tx=0x7f3f2403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:06:59.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.176+0000 7f3f357fa640 1 -- 172.21.15.64:0/2479135322 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f3806d150 con 0x7f3f3806d9e0 2023-11-24T08:06:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:06:59.409+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3f3806d150 con 0x7f3f3806d9e0 2023-11-24T08:07:00.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.024+0000 7f3f357fa640 1 -- 172.21.15.64:0/2479135322 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3f3806d150 con 0x7f3f3806d9e0 2023-11-24T08:07:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1c06b080 msgr2=0x7f3f1c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1c06b080 0x7f3f1c06d540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3f24036000 tx=0x7f3f2403b040 comp rx=0 tx=0).stop 2023-11-24T08:07:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3806d9e0 msgr2=0x7f3f380cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3806d9e0 0x7f3f380cb8c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c4d0 tx=0x7f3f2c00c990 comp rx=0 tx=0).stop 2023-11-24T08:07:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 shutdown_connections 2023-11-24T08:07:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f1c06b080 0x7f3f1c06d540 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f3815f270 0x7f3f3813fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:00.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f38144770 0x7f3f3815d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:00.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 --2- 172.21.15.64:0/2479135322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3806d9e0 0x7f3f380cb8c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:00.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 >> 172.21.15.64:0/2479135322 conn(0x7f3f380cec10 msgr2=0x7f3f3815eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:00.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 shutdown_connections 2023-11-24T08:07:00.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.029+0000 7f3f3da4b640 1 -- 172.21.15.64:0/2479135322 wait complete. 2023-11-24T08:07:00.053 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:07:00.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:07:00.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.288+0000 7f34f5bac640 1 Processor -- start 2023-11-24T08:07:00.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.288+0000 7f34f5bac640 1 -- start start 2023-11-24T08:07:00.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.289+0000 7f34f5bac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f00c8930 0x7f34f00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.289+0000 7f34f5bac640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00c6bf0 con 0x7f34f015d3b0 2023-11-24T08:07:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.289+0000 7f34f5bac640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00cedc0 con 0x7f34f00d1d90 2023-11-24T08:07:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.289+0000 7f34f5bac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00c6e40 con 0x7f34f00c8930 2023-11-24T08:07:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.290+0000 7f34f4baa640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f34f015d3b0 0x7f34f00d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59562/0 (socket says 172.21.15.64:59562) 2023-11-24T08:07:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.290+0000 7f34f4baa640 1 -- 172.21.15.64:0/274965828 learned_addr learned my addr 172.21.15.64:0/274965828 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.290+0000 7f34f53ab640 1 --2- 172.21.15.64:0/274965828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f00c8930 0x7f34f00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.291+0000 7f34ef7fe640 1 -- 172.21.15.64:0/274965828 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229030783 0 0) 0x7f34f00c6bf0 con 0x7f34f015d3b0 2023-11-24T08:07:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.291+0000 7f34ef7fe640 1 -- 172.21.15.64:0/274965828 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34d8003520 con 0x7f34f015d3b0 2023-11-24T08:07:00.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.291+0000 7f34ef7fe640 1 -- 172.21.15.64:0/274965828 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747952862 0 0) 0x7f34f00cedc0 con 0x7f34f00d1d90 2023-11-24T08:07:00.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.291+0000 7f34f53ab640 1 -- 172.21.15.64:0/274965828 >> v1:172.21.15.91:6789/0 conn(0x7f34f00d1d90 legacy=0x7f34f00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:00.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.291+0000 7f34f53ab640 1 -- 172.21.15.64:0/274965828 >> v1:172.21.15.64:6789/0 conn(0x7f34f015d3b0 legacy=0x7f34f00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:00.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.291+0000 7f34f53ab640 1 -- 172.21.15.64:0/274965828 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00c7120 con 0x7f34f00c8930 2023-11-24T08:07:00.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.292+0000 7f34f53ab640 1 --2- 172.21.15.64:0/274965828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f00c8930 0x7f34f00ce6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009dc0 tx=0x7f34e802bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc99d8d97e736e24 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.293+0000 7f34ef7fe640 1 -- 172.21.15.64:0/274965828 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00015f0 con 0x7f34f00c8930 2023-11-24T08:07:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.293+0000 7f34ef7fe640 1 -- 172.21.15.64:0/274965828 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e802c7b0 con 0x7f34f00c8930 2023-11-24T08:07:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.293+0000 7f34ef7fe640 1 -- 172.21.15.64:0/274965828 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e802cb40 con 0x7f34f00c8930 2023-11-24T08:07:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.294+0000 7f34f5bac640 1 -- 172.21.15.64:0/274965828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f00c8930 msgr2=0x7f34f00ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.294+0000 7f34f5bac640 1 --2- 172.21.15.64:0/274965828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f00c8930 0x7f34f00ce6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009dc0 tx=0x7f34e802bc80 comp rx=0 tx=0).stop 2023-11-24T08:07:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.294+0000 7f34f5bac640 1 -- 172.21.15.64:0/274965828 shutdown_connections 2023-11-24T08:07:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.294+0000 7f34f5bac640 1 --2- 172.21.15.64:0/274965828 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f00c8930 0x7f34f00ce6c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.294+0000 7f34f5bac640 1 -- 172.21.15.64:0/274965828 >> 172.21.15.64:0/274965828 conn(0x7f34f00d30a0 msgr2=0x7f34f00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.294+0000 7f34f5bac640 1 -- 172.21.15.64:0/274965828 shutdown_connections 2023-11-24T08:07:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.294+0000 7f34f5bac640 1 -- 172.21.15.64:0/274965828 wait complete. 2023-11-24T08:07:00.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.296+0000 7f34f5bac640 1 Processor -- start 2023-11-24T08:07:00.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.296+0000 7f34f5bac640 1 -- start start 2023-11-24T08:07:00.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34f5bac640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00c8930 0x7f34f009e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:00.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34f5bac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f00d1d90 0x7f34f014c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:00.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34f5bac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f015d3b0 0x7f34f014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:00.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34f5bac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f014b7c0 con 0x7f34f00d1d90 2023-11-24T08:07:00.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34f5bac640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f014baa0 con 0x7f34f00c8930 2023-11-24T08:07:00.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34f5bac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f014bd80 con 0x7f34f015d3b0 2023-11-24T08:07:00.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f00d1d90 0x7f34f014c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:00.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f00d1d90 0x7f34f014c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59988/0 (socket says 172.21.15.64:59988) 2023-11-24T08:07:00.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.297+0000 7f34effff640 1 -- 172.21.15.64:0/1555818098 learned_addr learned my addr 172.21.15.64:0/1555818098 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:00.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34f4baa640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00c8930 0x7f34f009e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:00.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34effff640 1 -- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f015d3b0 msgr2=0x7f34f014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:00.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34f53ab640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f015d3b0 0x7f34f014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:00.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34effff640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f015d3b0 0x7f34f014cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:00.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34effff640 1 -- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00c8930 msgr2=0x7f34f009e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:00.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34effff640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00c8930 0x7f34f009e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:00.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34effff640 1 -- 172.21.15.64:0/1555818098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e8009a70 con 0x7f34f00d1d90 2023-11-24T08:07:00.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34f4baa640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00c8930 0x7f34f009e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:00.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34effff640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f00d1d90 0x7f34f014c860 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f34e400c550 tx=0x7f34e400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:00.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.298+0000 7f34ed7fa640 1 -- 172.21.15.64:0/1555818098 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4005480 con 0x7f34f00d1d90 2023-11-24T08:07:00.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.299+0000 7f34ed7fa640 1 -- 172.21.15.64:0/1555818098 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e40035f0 con 0x7f34f00d1d90 2023-11-24T08:07:00.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.299+0000 7f34ed7fa640 1 -- 172.21.15.64:0/1555818098 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e40039a0 con 0x7f34f00d1d90 2023-11-24T08:07:00.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.299+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f014c060 con 0x7f34f00d1d90 2023-11-24T08:07:00.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.299+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f015ab40 con 0x7f34f00d1d90 2023-11-24T08:07:00.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.300+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f014c580 con 0x7f34f00d1d90 2023-11-24T08:07:00.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.301+0000 7f34ed7fa640 1 -- 172.21.15.64:0/1555818098 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f34e4003b00 con 0x7f34f00d1d90 2023-11-24T08:07:00.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.302+0000 7f34ed7fa640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f34dc06b080 0x7f34dc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:00.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.303+0000 7f34ed7fa640 1 -- 172.21.15.64:0/1555818098 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34e4089730 con 0x7f34f00d1d90 2023-11-24T08:07:00.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.303+0000 7f34f4baa640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f34dc06b080 0x7f34dc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:00.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.304+0000 7f34f4baa640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f34dc06b080 0x7f34dc06d540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34e0007aa0 tx=0x7f34e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:00.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.307+0000 7f34ed7fa640 1 -- 172.21.15.64:0/1555818098 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34f014c580 con 0x7f34f00d1d90 2023-11-24T08:07:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:00.538+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f34f014c580 con 0x7f34f00d1d90 2023-11-24T08:07:01.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.036+0000 7f34ed7fa640 1 -- 172.21.15.64:0/1555818098 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f34f014c580 con 0x7f34f00d1d90 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.040+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f34dc06b080 msgr2=0x7f34dc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.040+0000 7f34f5bac640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f34dc06b080 0x7f34dc06d540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34e0007aa0 tx=0x7f34e0009040 comp rx=0 tx=0).stop 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.040+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f00d1d90 msgr2=0x7f34f014c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.040+0000 7f34f5bac640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f00d1d90 0x7f34f014c860 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f34e400c550 tx=0x7f34e400ca10 comp rx=0 tx=0).stop 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.041+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 shutdown_connections 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.041+0000 7f34f5bac640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f34dc06b080 0x7f34dc06d540 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.041+0000 7f34f5bac640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f34f015d3b0 0x7f34f014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.041+0000 7f34f5bac640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f00d1d90 0x7f34f014c860 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.041+0000 7f34f5bac640 1 --2- 172.21.15.64:0/1555818098 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f34f00c8930 0x7f34f009e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.041+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 >> 172.21.15.64:0/1555818098 conn(0x7f34f00d30a0 msgr2=0x7f34f00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.041+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 shutdown_connections 2023-11-24T08:07:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.041+0000 7f34f5bac640 1 -- 172.21.15.64:0/1555818098 wait complete. 2023-11-24T08:07:01.068 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:07:01.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:01.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.294+0000 7f6de9ac2640 1 Processor -- start 2023-11-24T08:07:01.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.294+0000 7f6de9ac2640 1 -- start start 2023-11-24T08:07:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.296+0000 7f6de9ac2640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4070780 0x7f6de406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.296+0000 7f6de9ac2640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de4071cd0 con 0x7f6de4070cd0 2023-11-24T08:07:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.296+0000 7f6de9ac2640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de40b2120 con 0x7f6de4160970 2023-11-24T08:07:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.296+0000 7f6de9ac2640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40b2400 con 0x7f6de4070780 2023-11-24T08:07:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.296+0000 7f6de3fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6de4070cd0 0x7f6de40acbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59576/0 (socket says 172.21.15.64:59576) 2023-11-24T08:07:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.296+0000 7f6de3fff640 1 -- 172.21.15.64:0/940734567 learned_addr learned my addr 172.21.15.64:0/940734567 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.296+0000 7f6ddbfff640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4070780 0x7f6de406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037846367 0 0) 0x7f6de4071cd0 con 0x7f6de4070cd0 2023-11-24T08:07:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd0003520 con 0x7f6de4070cd0 2023-11-24T08:07:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1993952759 0 0) 0x7f6dd0003520 con 0x7f6de4070cd0 2023-11-24T08:07:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4070780 msgr2=0x7f6de406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4070780 0x7f6de406d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 >> v1:172.21.15.91:6789/0 conn(0x7f6de4160970 legacy=0x7f6de40b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de40b26e0 con 0x7f6de4070cd0 2023-11-24T08:07:01.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f6ddc0019f0 con 0x7f6de4070cd0 2023-11-24T08:07:01.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 >> v1:172.21.15.64:6789/0 conn(0x7f6de4070cd0 legacy=0x7f6de40acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4070780 0x7f6dd00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0003ee0 0x7f6dd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0004d00 0x7f6dd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40b2400 con 0x7f6dd0004d00 2023-11-24T08:07:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd000bac0 con 0x7f6dd0003ee0 2023-11-24T08:07:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.297+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd000bc80 con 0x7f6de4070780 2023-11-24T08:07:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.298+0000 7f6de3fff640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0004d00 0x7f6dd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.298+0000 7f6de37fe640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4070780 0x7f6dd00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.298+0000 7f6ddbfff640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0003ee0 0x7f6dd00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.298+0000 7f6de3fff640 1 -- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4070780 msgr2=0x7f6dd00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.298+0000 7f6de3fff640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4070780 0x7f6dd00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.298+0000 7f6de3fff640 1 -- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0003ee0 msgr2=0x7f6dd00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:01.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.298+0000 7f6de3fff640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0003ee0 0x7f6dd00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.298+0000 7f6de3fff640 1 -- 172.21.15.64:0/940734567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0003520 con 0x7f6dd0004d00 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.299+0000 7f6de3fff640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0004d00 0x7f6dd0005180 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00a810 tx=0x7f6dd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e71d2268b711528 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.299+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc034630 con 0x7f6dd0004d00 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.299+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc02f070 con 0x7f6dd0004d00 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.299+0000 7f6de2ffd640 1 -- 172.21.15.64:0/940734567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc034a10 con 0x7f6dd0004d00 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.300+0000 7f6de9ac2640 1 -- 172.21.15.64:0/940734567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0004d00 msgr2=0x7f6dd0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.300+0000 7f6de9ac2640 1 --2- 172.21.15.64:0/940734567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0004d00 0x7f6dd0005180 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00a810 tx=0x7f6dd00057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.300+0000 7f6de3fff640 1 -- 172.21.15.64:0/940734567 reap_dead start 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.300+0000 7f6de9ac2640 1 -- 172.21.15.64:0/940734567 shutdown_connections 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.300+0000 7f6de9ac2640 1 -- 172.21.15.64:0/940734567 >> 172.21.15.64:0/940734567 conn(0x7f6de40b0d80 msgr2=0x7f6de40b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:01.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.300+0000 7f6de9ac2640 1 -- 172.21.15.64:0/940734567 shutdown_connections 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.300+0000 7f6de9ac2640 1 -- 172.21.15.64:0/940734567 wait complete. 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.302+0000 7f6de9ac2640 1 Processor -- start 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.302+0000 7f6de9ac2640 1 -- start start 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.303+0000 7f6de9ac2640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0004d00 0x7f6de40acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.303+0000 7f6de9ac2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de41575b0 0x7f6de4157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.303+0000 7f6de9ac2640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4155190 0x7f6de41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.303+0000 7f6de9ac2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4155b30 con 0x7f6de41575b0 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.303+0000 7f6de9ac2640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de41f2bc0 con 0x7f6dd0004d00 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.303+0000 7f6de9ac2640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de41f2e60 con 0x7f6de4155190 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6ddbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de41575b0 0x7f6de4157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6ddbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de41575b0 0x7f6de4157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60006/0 (socket says 172.21.15.64:60006) 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6ddbfff640 1 -- 172.21.15.64:0/2189500374 learned_addr learned my addr 172.21.15.64:0/2189500374 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:01.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6de3fff640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4155190 0x7f6de41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6de37fe640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0004d00 0x7f6de40acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6ddbfff640 1 -- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4155190 msgr2=0x7f6de41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6ddbfff640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4155190 0x7f6de41555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6ddbfff640 1 -- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0004d00 msgr2=0x7f6de40acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6ddbfff640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0004d00 0x7f6de40acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6ddbfff640 1 -- 172.21.15.64:0/2189500374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de41f3140 con 0x7f6de41575b0 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6de3fff640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4155190 0x7f6de41555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.304+0000 7f6de37fe640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0004d00 0x7f6de40acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.305+0000 7f6ddbfff640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de41575b0 0x7f6de4157a30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc010860 tx=0x7f6dcc010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.305+0000 7f6de0ff9640 1 -- 172.21.15.64:0/2189500374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc009610 con 0x7f6de41575b0 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.305+0000 7f6de0ff9640 1 -- 172.21.15.64:0/2189500374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc00eb60 con 0x7f6de41575b0 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.305+0000 7f6de0ff9640 1 -- 172.21.15.64:0/2189500374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc01c400 con 0x7f6de41575b0 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.305+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de41f3420 con 0x7f6de41575b0 2023-11-24T08:07:01.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.305+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de41f39e0 con 0x7f6de41575b0 2023-11-24T08:07:01.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.306+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4074770 con 0x7f6de41575b0 2023-11-24T08:07:01.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.307+0000 7f6de0ff9640 1 -- 172.21.15.64:0/2189500374 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6dcc01c560 con 0x7f6de41575b0 2023-11-24T08:07:01.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.308+0000 7f6de0ff9640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dc406b080 0x7f6dc406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.308+0000 7f6de0ff9640 1 -- 172.21.15.64:0/2189500374 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6dcc08a060 con 0x7f6de41575b0 2023-11-24T08:07:01.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.309+0000 7f6de37fe640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dc406b080 0x7f6dc406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.310+0000 7f6de37fe640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dc406b080 0x7f6dc406d540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4001780 tx=0x7f6dd4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:01.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.313+0000 7f6de0ff9640 1 -- 172.21.15.64:0/2189500374 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6dcc0bce90 con 0x7f6de41575b0 2023-11-24T08:07:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.547+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6de40ce690 con 0x7f6de41575b0 2023-11-24T08:07:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.548+0000 7f6de0ff9640 1 -- 172.21.15.64:0/2189500374 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f6dcc054dc0 con 0x7f6de41575b0 2023-11-24T08:07:01.550 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:01.550 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2873504597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2873504597},{"type":"v1","addr":"172.21.15.91:6835","nonce":2873504597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3295227147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3295227147},{"type":"v1","addr":"172.21.15.91:6837","nonce":3295227147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2949237565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2949237565},{"type":"v1","addr":"172.21.15.64:6834","nonce":2949237565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:57.112228+0000","modified":"2023-11-24T08:07:00.539822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4200493009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":4200493009},{"type":"v1","addr":"172.21.15.64:6835","nonce":4200493009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:07:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 34 2023-11-24T08:07:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dc406b080 msgr2=0x7f6dc406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dc406b080 0x7f6dc406d540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4001780 tx=0x7f6dd4009040 comp rx=0 tx=0).stop 2023-11-24T08:07:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de41575b0 msgr2=0x7f6de4157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de41575b0 0x7f6de4157a30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc010860 tx=0x7f6dcc010d20 comp rx=0 tx=0).stop 2023-11-24T08:07:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 shutdown_connections 2023-11-24T08:07:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dc406b080 0x7f6dc406d540 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6de4155190 0x7f6de41555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de41575b0 0x7f6de4157a30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 --2- 172.21.15.64:0/2189500374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6dd0004d00 0x7f6de40acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.550+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 >> 172.21.15.64:0/2189500374 conn(0x7f6de40b0d80 msgr2=0x7f6de40adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.551+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 shutdown_connections 2023-11-24T08:07:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.551+0000 7f6de9ac2640 1 -- 172.21.15.64:0/2189500374 wait complete. 2023-11-24T08:07:01.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:01.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.793+0000 7ff75f59e640 1 Processor -- start 2023-11-24T08:07:01.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.793+0000 7ff75f59e640 1 -- start start 2023-11-24T08:07:01.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.795+0000 7ff75f59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff760157770 0x7ff76006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.795+0000 7ff75f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff760161670 con 0x7ff76006d080 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.795+0000 7ff75f59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76015e610 con 0x7ff76015df80 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.795+0000 7ff75f59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76015e410 con 0x7ff760157770 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.795+0000 7ff757fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff76006d080 0x7ff76015d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59590/0 (socket says 172.21.15.64:59590) 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.795+0000 7ff757fff640 1 -- 172.21.15.64:0/3494094052 learned_addr learned my addr 172.21.15.64:0/3494094052 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75e59c640 1 --2- 172.21.15.64:0/3494094052 >> v2:172.21.15.91:3300/0 conn(0x7ff760157770 0x7ff76006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159187524 0 0) 0x7ff760161670 con 0x7ff76006d080 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff74c003520 con 0x7ff76006d080 2023-11-24T08:07:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013447386 0 0) 0x7ff74c003520 con 0x7ff76006d080 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 >> v2:172.21.15.91:3300/0 conn(0x7ff760157770 msgr2=0x7ff76006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 --2- 172.21.15.64:0/3494094052 >> v2:172.21.15.91:3300/0 conn(0x7ff760157770 0x7ff76006cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 >> v1:172.21.15.91:6789/0 conn(0x7ff76015df80 legacy=0x7ff760070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff760160d40 con 0x7ff76006d080 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7ff748001670 con 0x7ff76006d080 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.796+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 >> v1:172.21.15.64:6789/0 conn(0x7ff76006d080 legacy=0x7ff76015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75dd9b640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c0039e0 0x7ff74c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75dd9b640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff74c0047c0 0x7ff74c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75dd9b640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff74c005180 0x7ff74c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76015e410 con 0x7ff74c005180 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c00c0b0 con 0x7ff74c0047c0 2023-11-24T08:07:01.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c00c270 con 0x7ff74c0039e0 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75ed9d640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff74c005180 0x7ff74c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75e59c640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c0039e0 0x7ff74c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff757fff640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff74c0047c0 0x7ff74c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75ed9d640 1 -- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c0039e0 msgr2=0x7ff74c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75ed9d640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c0039e0 0x7ff74c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75ed9d640 1 -- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff74c0047c0 msgr2=0x7ff74c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75ed9d640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff74c0047c0 0x7ff74c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75ed9d640 1 -- 172.21.15.64:0/3494094052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c003520 con 0x7ff74c005180 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff75e59c640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c0039e0 0x7ff74c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.797+0000 7ff757fff640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff74c0047c0 0x7ff74c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.798+0000 7ff75ed9d640 1 -- 172.21.15.64:0/3494094052 reap_dead start 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.799+0000 7ff75ed9d640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff74c005180 0x7ff74c00b980 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff758001e50 tx=0x7ff74c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff01ed97b359d27 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.799+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758003b70 con 0x7ff74c005180 2023-11-24T08:07:01.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.799+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758003cf0 con 0x7ff74c005180 2023-11-24T08:07:01.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.799+0000 7ff75dd9b640 1 -- 172.21.15.64:0/3494094052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758005310 con 0x7ff74c005180 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.799+0000 7ff75f59e640 1 -- 172.21.15.64:0/3494094052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff74c005180 msgr2=0x7ff74c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.799+0000 7ff75f59e640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff74c005180 0x7ff74c00b980 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff758001e50 tx=0x7ff74c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.800+0000 7ff75f59e640 1 -- 172.21.15.64:0/3494094052 shutdown_connections 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.800+0000 7ff75f59e640 1 --2- 172.21.15.64:0/3494094052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff74c005180 0x7ff74c00b980 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.800+0000 7ff75f59e640 1 -- 172.21.15.64:0/3494094052 >> 172.21.15.64:0/3494094052 conn(0x7ff76006a860 msgr2=0x7ff76006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.800+0000 7ff75f59e640 1 -- 172.21.15.64:0/3494094052 shutdown_connections 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.800+0000 7ff75f59e640 1 -- 172.21.15.64:0/3494094052 wait complete. 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.802+0000 7ff75f59e640 1 Processor -- start 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.802+0000 7ff75f59e640 1 -- start start 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff75f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c005180 0x7ff7600cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff75f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76014b080 0x7ff76014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff75f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff76014f230 0x7ff7600cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff75f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76014f6b0 con 0x7ff76014b080 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff75f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600cca30 con 0x7ff76014f230 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff75f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600b3a80 con 0x7ff74c005180 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff757fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76014b080 0x7ff76014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff757fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76014b080 0x7ff76014ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60020/0 (socket says 172.21.15.64:60020) 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff757fff640 1 -- 172.21.15.64:0/2038231499 learned_addr learned my addr 172.21.15.64:0/2038231499 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff757fff640 1 -- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c005180 msgr2=0x7ff7600cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff757fff640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c005180 0x7ff7600cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff757fff640 1 -- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff76014f230 msgr2=0x7ff7600cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.803+0000 7ff75ed9d640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff76014f230 0x7ff7600cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.804+0000 7ff757fff640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff76014f230 0x7ff7600cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:01.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.804+0000 7ff757fff640 1 -- 172.21.15.64:0/2038231499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600b2f00 con 0x7ff76014b080 2023-11-24T08:07:01.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.804+0000 7ff757fff640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76014b080 0x7ff76014ecf0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff74800d800 tx=0x7ff74800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:01.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.804+0000 7ff756ffd640 1 -- 172.21.15.64:0/2038231499 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480074e0 con 0x7ff76014b080 2023-11-24T08:07:01.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.804+0000 7ff756ffd640 1 -- 172.21.15.64:0/2038231499 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748007640 con 0x7ff76014b080 2023-11-24T08:07:01.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.804+0000 7ff756ffd640 1 -- 172.21.15.64:0/2038231499 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480049e0 con 0x7ff76014b080 2023-11-24T08:07:01.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.804+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7600b31e0 con 0x7ff76014b080 2023-11-24T08:07:01.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.804+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7600b37a0 con 0x7ff76014b080 2023-11-24T08:07:01.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.805+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76004e840 con 0x7ff76014b080 2023-11-24T08:07:01.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.807+0000 7ff756ffd640 1 -- 172.21.15.64:0/2038231499 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff748004b40 con 0x7ff76014b080 2023-11-24T08:07:01.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.807+0000 7ff756ffd640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff74006b0b0 0x7ff74006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:01.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.808+0000 7ff756ffd640 1 -- 172.21.15.64:0/2038231499 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7480893e0 con 0x7ff76014b080 2023-11-24T08:07:01.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.808+0000 7ff75e59c640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff74006b0b0 0x7ff74006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:01.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.809+0000 7ff75e59c640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff74006b0b0 0x7ff74006d570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7600b5010 tx=0x7ff750008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:01.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:01.812+0000 7ff756ffd640 1 -- 172.21.15.64:0/2038231499 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff748054010 con 0x7ff76014b080 2023-11-24T08:07:02.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.007+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7600b1930 con 0x7ff76014b080 2023-11-24T08:07:02.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.009+0000 7ff756ffd640 1 -- 172.21.15.64:0/2038231499 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16517 (secure 0 0 0) 0x7ff748057630 con 0x7ff76014b080 2023-11-24T08:07:02.011 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:02.012 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:58.644237+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:55.038686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:56.142053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff74006b0b0 msgr2=0x7ff74006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff74006b0b0 0x7ff74006d570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7600b5010 tx=0x7ff750008040 comp rx=0 tx=0).stop 2023-11-24T08:07:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76014b080 msgr2=0x7ff76014ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76014b080 0x7ff76014ecf0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff74800d800 tx=0x7ff74800dcc0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 shutdown_connections 2023-11-24T08:07:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff74006b0b0 0x7ff74006d570 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff76014f230 0x7ff7600cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76014b080 0x7ff76014ecf0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 --2- 172.21.15.64:0/2038231499 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff74c005180 0x7ff7600cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 >> 172.21.15.64:0/2038231499 conn(0x7ff76006a860 msgr2=0x7ff76006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:02.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 shutdown_connections 2023-11-24T08:07:02.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.012+0000 7ff75f59e640 1 -- 172.21.15.64:0/2038231499 wait complete. 2023-11-24T08:07:02.033 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:07:02.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:07:02.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.255+0000 7f78edb47640 1 Processor -- start 2023-11-24T08:07:02.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.255+0000 7f78edb47640 1 -- start start 2023-11-24T08:07:02.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.257+0000 7f78edb47640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f78e813dfd0 0x7f78e815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.257+0000 7f78edb47640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e80ce7c0 con 0x7f78e80cf050 2023-11-24T08:07:02.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.257+0000 7f78edb47640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e8140900 con 0x7f78e80cf500 2023-11-24T08:07:02.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.257+0000 7f78edb47640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e806cf50 con 0x7f78e813dfd0 2023-11-24T08:07:02.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.257+0000 7f78e7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f78e80cf050 0x7f78e806c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59592/0 (socket says 172.21.15.64:59592) 2023-11-24T08:07:02.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.257+0000 7f78e7fff640 1 -- 172.21.15.64:0/387528541 learned_addr learned my addr 172.21.15.64:0/387528541 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78ecb45640 1 --2- 172.21.15.64:0/387528541 >> v2:172.21.15.91:3300/0 conn(0x7f78e813dfd0 0x7f78e815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012932011 0 0) 0x7f78e80ce7c0 con 0x7f78e80cf050 2023-11-24T08:07:02.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d8003520 con 0x7f78e80cf050 2023-11-24T08:07:02.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 809143798 0 0) 0x7f78d8003520 con 0x7f78e80cf050 2023-11-24T08:07:02.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 >> v2:172.21.15.91:3300/0 conn(0x7f78e813dfd0 msgr2=0x7f78e815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:02.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 --2- 172.21.15.64:0/387528541 >> v2:172.21.15.91:3300/0 conn(0x7f78e813dfd0 0x7f78e815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 >> v1:172.21.15.91:6789/0 conn(0x7f78e80cf500 legacy=0x7f78e806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:02.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e815fa70 con 0x7f78e80cf050 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f78d4001670 con 0x7f78e80cf050 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 >> v1:172.21.15.64:6789/0 conn(0x7f78e80cf050 legacy=0x7f78e806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d8003a30 0x7f78d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d80047c0 0x7f78d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.258+0000 7f78e77fe640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e806cf50 con 0x7f78d8005180 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d800c120 con 0x7f78d8003a30 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d800c2e0 con 0x7f78d80047c0 2023-11-24T08:07:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78ed346640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78ecb45640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d8003a30 0x7f78d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78e7fff640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d80047c0 0x7f78d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78ed346640 1 -- 172.21.15.64:0/387528541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d80047c0 msgr2=0x7f78d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78ed346640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d80047c0 0x7f78d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78ed346640 1 -- 172.21.15.64:0/387528541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d8003a30 msgr2=0x7f78d8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78ed346640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d8003a30 0x7f78d8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.259+0000 7f78ed346640 1 -- 172.21.15.64:0/387528541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8003520 con 0x7f78d8005180 2023-11-24T08:07:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.260+0000 7f78ed346640 1 -- 172.21.15.64:0/387528541 reap_dead start 2023-11-24T08:07:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.260+0000 7f78ed346640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78d800b980 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f78e00044b0 tx=0x7f78d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d70cee1dc8b13 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.261+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0006d50 con 0x7f78d8005180 2023-11-24T08:07:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.261+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e0007020 con 0x7f78d8005180 2023-11-24T08:07:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.261+0000 7f78e77fe640 1 -- 172.21.15.64:0/387528541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00073d0 con 0x7f78d8005180 2023-11-24T08:07:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.261+0000 7f78edb47640 1 -- 172.21.15.64:0/387528541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 msgr2=0x7f78d800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.261+0000 7f78edb47640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78d800b980 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f78e00044b0 tx=0x7f78d80057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.261+0000 7f78edb47640 1 -- 172.21.15.64:0/387528541 shutdown_connections 2023-11-24T08:07:02.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.261+0000 7f78edb47640 1 --2- 172.21.15.64:0/387528541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78d800b980 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.261+0000 7f78edb47640 1 -- 172.21.15.64:0/387528541 >> 172.21.15.64:0/387528541 conn(0x7f78e806a860 msgr2=0x7f78e806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:02.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.262+0000 7f78edb47640 1 -- 172.21.15.64:0/387528541 shutdown_connections 2023-11-24T08:07:02.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.262+0000 7f78edb47640 1 -- 172.21.15.64:0/387528541 wait complete. 2023-11-24T08:07:02.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.263+0000 7f78edb47640 1 Processor -- start 2023-11-24T08:07:02.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.264+0000 7f78edb47640 1 -- start start 2023-11-24T08:07:02.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78edb47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78e81339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78edb47640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78e8157a10 0x7f78e813aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78edb47640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78e813aff0 0x7f78e815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78edb47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e813b470 con 0x7f78d8005180 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78edb47640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e81f64a0 con 0x7f78e813aff0 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78edb47640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e81f6780 con 0x7f78e8157a10 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ecb45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78e81339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ecb45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78e81339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60026/0 (socket says 172.21.15.64:60026) 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ecb45640 1 -- 172.21.15.64:0/652941402 learned_addr learned my addr 172.21.15.64:0/652941402 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ecb45640 1 -- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78e8157a10 msgr2=0x7f78e813aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78e7fff640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78e8157a10 0x7f78e813aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ed346640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78e813aff0 0x7f78e815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ecb45640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78e8157a10 0x7f78e813aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ecb45640 1 -- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78e813aff0 msgr2=0x7f78e815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ecb45640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78e813aff0 0x7f78e815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.265+0000 7f78ecb45640 1 -- 172.21.15.64:0/652941402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e81f6a60 con 0x7f78d8005180 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.266+0000 7f78ecb45640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78e81339e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f78dc011540 tx=0x7f78dc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.266+0000 7f78e57fa640 1 -- 172.21.15.64:0/652941402 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc015930 con 0x7f78d8005180 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.266+0000 7f78e57fa640 1 -- 172.21.15.64:0/652941402 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc015a90 con 0x7f78d8005180 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.266+0000 7f78e57fa640 1 -- 172.21.15.64:0/652941402 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc015e60 con 0x7f78d8005180 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.266+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e81f6d40 con 0x7f78d8005180 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.266+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e8157e90 con 0x7f78d8005180 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.267+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e8071ed0 con 0x7f78d8005180 2023-11-24T08:07:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.268+0000 7f78e57fa640 1 -- 172.21.15.64:0/652941402 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f78dc00f460 con 0x7f78d8005180 2023-11-24T08:07:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.269+0000 7f78e57fa640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78cc06b080 0x7f78cc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.269+0000 7f78e57fa640 1 -- 172.21.15.64:0/652941402 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78dc08a2f0 con 0x7f78d8005180 2023-11-24T08:07:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.270+0000 7f78e7fff640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78cc06b080 0x7f78cc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.271+0000 7f78e7fff640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78cc06b080 0x7f78cc06d540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f78d4001050 tx=0x7f78d4000fc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.273+0000 7f78e57fa640 1 -- 172.21.15.64:0/652941402 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78dc054fa0 con 0x7f78d8005180 2023-11-24T08:07:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.467+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78e815f420 con 0x7f78cc06b080 2023-11-24T08:07:02.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.495+0000 7f78e57fa640 1 -- 172.21.15.64:0/652941402 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286816 (secure 0 0 0) 0x7f78d4001d70 con 0x7f78cc06b080 2023-11-24T08:07:02.501 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:07:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78cc06b080 msgr2=0x7f78cc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78cc06b080 0x7f78cc06d540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f78d4001050 tx=0x7f78d4000fc0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 msgr2=0x7f78e81339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78e81339e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f78dc011540 tx=0x7f78dc011a00 comp rx=0 tx=0).stop 2023-11-24T08:07:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 shutdown_connections 2023-11-24T08:07:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78cc06b080 0x7f78cc06d540 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78e813aff0 0x7f78e815d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78e8157a10 0x7f78e813aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 --2- 172.21.15.64:0/652941402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d8005180 0x7f78e81339e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.500+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 >> 172.21.15.64:0/652941402 conn(0x7f78e806a860 msgr2=0x7f78e806ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.501+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 shutdown_connections 2023-11-24T08:07:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.501+0000 7f78edb47640 1 -- 172.21.15.64:0/652941402 wait complete. 2023-11-24T08:07:02.523 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2023-11-24T08:07:02.082048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239628,"kb_used_data":6780,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735092,"statfs":{"total":773094113280,"available":772848734208,"internally_reserved":0,"allocated":6942720,"data_stored":3569025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002319"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.653277+0000","last_change":"2023-11-24T08:06:57.674621+0000","last_active":"2023-11-24T08:06:58.653277+0000","last_peered":"2023-11-24T08:06:58.653277+0000","last_clean":"2023-11-24T08:06:58.653277+0000","last_became_active":"2023-11-24T08:06:57.674311+0000","last_became_peered":"2023-11-24T08:06:57.674311+0000","last_unstale":"2023-11-24T08:06:58.653277+0000","last_undegraded":"2023-11-24T08:06:58.653277+0000","last_fullsized":"2023-11-24T08:06:58.653277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:48.451930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657164+0000","last_change":"2023-11-24T08:06:57.683332+0000","last_active":"2023-11-24T08:06:58.657164+0000","last_peered":"2023-11-24T08:06:58.657164+0000","last_clean":"2023-11-24T08:06:58.657164+0000","last_became_active":"2023-11-24T08:06:57.683038+0000","last_became_peered":"2023-11-24T08:06:57.683038+0000","last_unstale":"2023-11-24T08:06:58.657164+0000","last_undegraded":"2023-11-24T08:06:58.657164+0000","last_fullsized":"2023-11-24T08:06:58.657164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:55.298367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863986+0000","last_change":"2023-11-24T08:06:57.688141+0000","last_active":"2023-11-24T08:06:58.863986+0000","last_peered":"2023-11-24T08:06:58.863986+0000","last_clean":"2023-11-24T08:06:58.863986+0000","last_became_active":"2023-11-24T08:06:57.685078+0000","last_became_peered":"2023-11-24T08:06:57.685078+0000","last_unstale":"2023-11-24T08:06:58.863986+0000","last_undegraded":"2023-11-24T08:06:58.863986+0000","last_fullsized":"2023-11-24T08:06:58.863986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:34.577120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863973+0000","last_change":"2023-11-24T08:06:57.679172+0000","last_active":"2023-11-24T08:06:58.863973+0000","last_peered":"2023-11-24T08:06:58.863973+0000","last_clean":"2023-11-24T08:06:58.863973+0000","last_became_active":"2023-11-24T08:06:57.678839+0000","last_became_peered":"2023-11-24T08:06:57.678839+0000","last_unstale":"2023-11-24T08:06:58.863973+0000","last_undegraded":"2023-11-24T08:06:58.863973+0000","last_fullsized":"2023-11-24T08:06:58.863973+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:26.674291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.653453+0000","last_change":"2023-11-24T08:06:57.673834+0000","last_active":"2023-11-24T08:06:58.653453+0000","last_peered":"2023-11-24T08:06:58.653453+0000","last_clean":"2023-11-24T08:06:58.653453+0000","last_became_active":"2023-11-24T08:06:57.673549+0000","last_became_peered":"2023-11-24T08:06:57.673549+0000","last_unstale":"2023-11-24T08:06:58.653453+0000","last_undegraded":"2023-11-24T08:06:58.653453+0000","last_fullsized":"2023-11-24T08:06:58.653453+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:27.170334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863438+0000","last_change":"2023-11-24T08:06:57.676622+0000","last_active":"2023-11-24T08:06:58.863438+0000","last_peered":"2023-11-24T08:06:58.863438+0000","last_clean":"2023-11-24T08:06:58.863438+0000","last_became_active":"2023-11-24T08:06:57.676309+0000","last_became_peered":"2023-11-24T08:06:57.676309+0000","last_unstale":"2023-11-24T08:06:58.863438+0000","last_undegraded":"2023-11-24T08:06:58.863438+0000","last_fullsized":"2023-11-24T08:06:58.863438+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:47:33.979558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654867+0000","last_change":"2023-11-24T08:06:57.673771+0000","last_active":"2023-11-24T08:06:58.654867+0000","last_peered":"2023-11-24T08:06:58.654867+0000","last_clean":"2023-11-24T08:06:58.654867+0000","last_became_active":"2023-11-24T08:06:57.673323+0000","last_became_peered":"2023-11-24T08:06:57.673323+0000","last_unstale":"2023-11-24T08:06:58.654867+0000","last_undegraded":"2023-11-24T08:06:58.654867+0000","last_fullsized":"2023-11-24T08:06:58.654867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:34.458263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655524+0000","last_change":"2023-11-24T08:06:57.668015+0000","last_active":"2023-11-24T08:06:58.655524+0000","last_peered":"2023-11-24T08:06:58.655524+0000","last_clean":"2023-11-24T08:06:58.655524+0000","last_became_active":"2023-11-24T08:06:57.667604+0000","last_became_peered":"2023-11-24T08:06:57.667604+0000","last_unstale":"2023-11-24T08:06:58.655524+0000","last_undegraded":"2023-11-24T08:06:58.655524+0000","last_fullsized":"2023-11-24T08:06:58.655524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:38:17.382639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655576+0000","last_change":"2023-11-24T08:06:57.681027+0000","last_active":"2023-11-24T08:06:58.655576+0000","last_peered":"2023-11-24T08:06:58.655576+0000","last_clean":"2023-11-24T08:06:58.655576+0000","last_became_active":"2023-11-24T08:06:57.680745+0000","last_became_peered":"2023-11-24T08:06:57.680745+0000","last_unstale":"2023-11-24T08:06:58.655576+0000","last_undegraded":"2023-11-24T08:06:58.655576+0000","last_fullsized":"2023-11-24T08:06:58.655576+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:41.456025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863375+0000","last_change":"2023-11-24T08:06:57.687573+0000","last_active":"2023-11-24T08:06:58.863375+0000","last_peered":"2023-11-24T08:06:58.863375+0000","last_clean":"2023-11-24T08:06:58.863375+0000","last_became_active":"2023-11-24T08:06:57.686663+0000","last_became_peered":"2023-11-24T08:06:57.686663+0000","last_unstale":"2023-11-24T08:06:58.863375+0000","last_undegraded":"2023-11-24T08:06:58.863375+0000","last_fullsized":"2023-11-24T08:06:58.863375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:13:14.056149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863154+0000","last_change":"2023-11-24T08:06:57.680923+0000","last_active":"2023-11-24T08:06:58.863154+0000","last_peered":"2023-11-24T08:06:58.863154+0000","last_clean":"2023-11-24T08:06:58.863154+0000","last_became_active":"2023-11-24T08:06:57.680220+0000","last_became_peered":"2023-11-24T08:06:57.680220+0000","last_unstale":"2023-11-24T08:06:58.863154+0000","last_undegraded":"2023-11-24T08:06:58.863154+0000","last_fullsized":"2023-11-24T08:06:58.863154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:26.987702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863013+0000","last_change":"2023-11-24T08:06:57.686230+0000","last_active":"2023-11-24T08:06:58.863013+0000","last_peered":"2023-11-24T08:06:58.863013+0000","last_clean":"2023-11-24T08:06:58.863013+0000","last_became_active":"2023-11-24T08:06:57.685937+0000","last_became_peered":"2023-11-24T08:06:57.685937+0000","last_unstale":"2023-11-24T08:06:58.863013+0000","last_undegraded":"2023-11-24T08:06:58.863013+0000","last_fullsized":"2023-11-24T08:06:58.863013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:17.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.866445+0000","last_change":"2023-11-24T08:06:57.688137+0000","last_active":"2023-11-24T08:06:58.866445+0000","last_peered":"2023-11-24T08:06:58.866445+0000","last_clean":"2023-11-24T08:06:58.866445+0000","last_became_active":"2023-11-24T08:06:57.687620+0000","last_became_peered":"2023-11-24T08:06:57.687620+0000","last_unstale":"2023-11-24T08:06:58.866445+0000","last_undegraded":"2023-11-24T08:06:58.866445+0000","last_fullsized":"2023-11-24T08:06:58.866445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:03.110816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654772+0000","last_change":"2023-11-24T08:06:57.672943+0000","last_active":"2023-11-24T08:06:58.654772+0000","last_peered":"2023-11-24T08:06:58.654772+0000","last_clean":"2023-11-24T08:06:58.654772+0000","last_became_active":"2023-11-24T08:06:57.669818+0000","last_became_peered":"2023-11-24T08:06:57.669818+0000","last_unstale":"2023-11-24T08:06:58.654772+0000","last_undegraded":"2023-11-24T08:06:58.654772+0000","last_fullsized":"2023-11-24T08:06:58.654772+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:30.446731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654705+0000","last_change":"2023-11-24T08:06:57.683874+0000","last_active":"2023-11-24T08:06:58.654705+0000","last_peered":"2023-11-24T08:06:58.654705+0000","last_clean":"2023-11-24T08:06:58.654705+0000","last_became_active":"2023-11-24T08:06:57.683597+0000","last_became_peered":"2023-11-24T08:06:57.683597+0000","last_unstale":"2023-11-24T08:06:58.654705+0000","last_undegraded":"2023-11-24T08:06:58.654705+0000","last_fullsized":"2023-11-24T08:06:58.654705+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:33:12.607909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.861784+0000","last_change":"2023-11-24T08:06:57.683906+0000","last_active":"2023-11-24T08:06:58.861784+0000","last_peered":"2023-11-24T08:06:58.861784+0000","last_clean":"2023-11-24T08:06:58.861784+0000","last_became_active":"2023-11-24T08:06:57.683588+0000","last_became_peered":"2023-11-24T08:06:57.683588+0000","last_unstale":"2023-11-24T08:06:58.861784+0000","last_undegraded":"2023-11-24T08:06:58.861784+0000","last_fullsized":"2023-11-24T08:06:58.861784+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:12:33.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656783+0000","last_change":"2023-11-24T08:06:57.684411+0000","last_active":"2023-11-24T08:06:58.656783+0000","last_peered":"2023-11-24T08:06:58.656783+0000","last_clean":"2023-11-24T08:06:58.656783+0000","last_became_active":"2023-11-24T08:06:57.684120+0000","last_became_peered":"2023-11-24T08:06:57.684120+0000","last_unstale":"2023-11-24T08:06:58.656783+0000","last_undegraded":"2023-11-24T08:06:58.656783+0000","last_fullsized":"2023-11-24T08:06:58.656783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:32.191838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864423+0000","last_change":"2023-11-24T08:06:57.684644+0000","last_active":"2023-11-24T08:06:58.864423+0000","last_peered":"2023-11-24T08:06:58.864423+0000","last_clean":"2023-11-24T08:06:58.864423+0000","last_became_active":"2023-11-24T08:06:57.684258+0000","last_became_peered":"2023-11-24T08:06:57.684258+0000","last_unstale":"2023-11-24T08:06:58.864423+0000","last_undegraded":"2023-11-24T08:06:58.864423+0000","last_fullsized":"2023-11-24T08:06:58.864423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:54.650345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.652921+0000","last_change":"2023-11-24T08:06:57.666208+0000","last_active":"2023-11-24T08:06:58.652921+0000","last_peered":"2023-11-24T08:06:58.652921+0000","last_clean":"2023-11-24T08:06:58.652921+0000","last_became_active":"2023-11-24T08:06:57.665742+0000","last_became_peered":"2023-11-24T08:06:57.665742+0000","last_unstale":"2023-11-24T08:06:58.652921+0000","last_undegraded":"2023-11-24T08:06:58.652921+0000","last_fullsized":"2023-11-24T08:06:58.652921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:22.944151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657250+0000","last_change":"2023-11-24T08:06:57.681769+0000","last_active":"2023-11-24T08:06:58.657250+0000","last_peered":"2023-11-24T08:06:58.657250+0000","last_clean":"2023-11-24T08:06:58.657250+0000","last_became_active":"2023-11-24T08:06:57.681532+0000","last_became_peered":"2023-11-24T08:06:57.681532+0000","last_unstale":"2023-11-24T08:06:58.657250+0000","last_undegraded":"2023-11-24T08:06:58.657250+0000","last_fullsized":"2023-11-24T08:06:58.657250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:07.649707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864479+0000","last_change":"2023-11-24T08:06:57.687079+0000","last_active":"2023-11-24T08:06:58.864479+0000","last_peered":"2023-11-24T08:06:58.864479+0000","last_clean":"2023-11-24T08:06:58.864479+0000","last_became_active":"2023-11-24T08:06:57.684687+0000","last_became_peered":"2023-11-24T08:06:57.684687+0000","last_unstale":"2023-11-24T08:06:58.864479+0000","last_undegraded":"2023-11-24T08:06:58.864479+0000","last_fullsized":"2023-11-24T08:06:58.864479+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:08.781566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656091+0000","last_change":"2023-11-24T08:06:57.680674+0000","last_active":"2023-11-24T08:06:58.656091+0000","last_peered":"2023-11-24T08:06:58.656091+0000","last_clean":"2023-11-24T08:06:58.656091+0000","last_became_active":"2023-11-24T08:06:57.680354+0000","last_became_peered":"2023-11-24T08:06:57.680354+0000","last_unstale":"2023-11-24T08:06:58.656091+0000","last_undegraded":"2023-11-24T08:06:58.656091+0000","last_fullsized":"2023-11-24T08:06:58.656091+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:06:20.551336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863662+0000","last_change":"2023-11-24T08:06:57.681440+0000","last_active":"2023-11-24T08:06:58.863662+0000","last_peered":"2023-11-24T08:06:58.863662+0000","last_clean":"2023-11-24T08:06:58.863662+0000","last_became_active":"2023-11-24T08:06:57.680417+0000","last_became_peered":"2023-11-24T08:06:57.680417+0000","last_unstale":"2023-11-24T08:06:58.863662+0000","last_undegraded":"2023-11-24T08:06:58.863662+0000","last_fullsized":"2023-11-24T08:06:58.863662+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:53.404909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865841+0000","last_change":"2023-11-24T08:06:57.678374+0000","last_active":"2023-11-24T08:06:58.865841+0000","last_peered":"2023-11-24T08:06:58.865841+0000","last_clean":"2023-11-24T08:06:58.865841+0000","last_became_active":"2023-11-24T08:06:57.678028+0000","last_became_peered":"2023-11-24T08:06:57.678028+0000","last_unstale":"2023-11-24T08:06:58.865841+0000","last_undegraded":"2023-11-24T08:06:58.865841+0000","last_fullsized":"2023-11-24T08:06:58.865841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:46.577873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.653642+0000","last_change":"2023-11-24T08:06:57.683704+0000","last_active":"2023-11-24T08:06:58.653642+0000","last_peered":"2023-11-24T08:06:58.653642+0000","last_clean":"2023-11-24T08:06:58.653642+0000","last_became_active":"2023-11-24T08:06:57.682465+0000","last_became_peered":"2023-11-24T08:06:57.682465+0000","last_unstale":"2023-11-24T08:06:58.653642+0000","last_undegraded":"2023-11-24T08:06:58.653642+0000","last_fullsized":"2023-11-24T08:06:58.653642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:57.628251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863148+0000","last_change":"2023-11-24T08:06:57.680639+0000","last_active":"2023-11-24T08:06:58.863148+0000","last_peered":"2023-11-24T08:06:58.863148+0000","last_clean":"2023-11-24T08:06:58.863148+0000","last_became_active":"2023-11-24T08:06:57.680373+0000","last_became_peered":"2023-11-24T08:06:57.680373+0000","last_unstale":"2023-11-24T08:06:58.863148+0000","last_undegraded":"2023-11-24T08:06:58.863148+0000","last_fullsized":"2023-11-24T08:06:58.863148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:39:40.474356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864522+0000","last_change":"2023-11-24T08:06:57.686822+0000","last_active":"2023-11-24T08:06:58.864522+0000","last_peered":"2023-11-24T08:06:58.864522+0000","last_clean":"2023-11-24T08:06:58.864522+0000","last_became_active":"2023-11-24T08:06:57.684940+0000","last_became_peered":"2023-11-24T08:06:57.684940+0000","last_unstale":"2023-11-24T08:06:58.864522+0000","last_undegraded":"2023-11-24T08:06:58.864522+0000","last_fullsized":"2023-11-24T08:06:58.864522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:35.713574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.652572+0000","last_change":"2023-11-24T08:06:57.674601+0000","last_active":"2023-11-24T08:06:58.652572+0000","last_peered":"2023-11-24T08:06:58.652572+0000","last_clean":"2023-11-24T08:06:58.652572+0000","last_became_active":"2023-11-24T08:06:57.674290+0000","last_became_peered":"2023-11-24T08:06:57.674290+0000","last_unstale":"2023-11-24T08:06:58.652572+0000","last_undegraded":"2023-11-24T08:06:58.652572+0000","last_fullsized":"2023-11-24T08:06:58.652572+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:31:52.096168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656486+0000","last_change":"2023-11-24T08:06:57.672514+0000","last_active":"2023-11-24T08:06:58.656486+0000","last_peered":"2023-11-24T08:06:58.656486+0000","last_clean":"2023-11-24T08:06:58.656486+0000","last_became_active":"2023-11-24T08:06:57.672202+0000","last_became_peered":"2023-11-24T08:06:57.672202+0000","last_unstale":"2023-11-24T08:06:58.656486+0000","last_undegraded":"2023-11-24T08:06:58.656486+0000","last_fullsized":"2023-11-24T08:06:58.656486+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:39:04.407449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865439+0000","last_change":"2023-11-24T08:06:57.684670+0000","last_active":"2023-11-24T08:06:58.865439+0000","last_peered":"2023-11-24T08:06:58.865439+0000","last_clean":"2023-11-24T08:06:58.865439+0000","last_became_active":"2023-11-24T08:06:57.683051+0000","last_became_peered":"2023-11-24T08:06:57.683051+0000","last_unstale":"2023-11-24T08:06:58.865439+0000","last_undegraded":"2023-11-24T08:06:58.865439+0000","last_fullsized":"2023-11-24T08:06:58.865439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:44.326725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.652141+0000","last_change":"2023-11-24T08:06:57.672693+0000","last_active":"2023-11-24T08:06:58.652141+0000","last_peered":"2023-11-24T08:06:58.652141+0000","last_clean":"2023-11-24T08:06:58.652141+0000","last_became_active":"2023-11-24T08:06:57.670540+0000","last_became_peered":"2023-11-24T08:06:57.670540+0000","last_unstale":"2023-11-24T08:06:58.652141+0000","last_undegraded":"2023-11-24T08:06:58.652141+0000","last_fullsized":"2023-11-24T08:06:58.652141+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:38.102009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862483+0000","last_change":"2023-11-24T08:06:57.687837+0000","last_active":"2023-11-24T08:06:58.862483+0000","last_peered":"2023-11-24T08:06:58.862483+0000","last_clean":"2023-11-24T08:06:58.862483+0000","last_became_active":"2023-11-24T08:06:57.687574+0000","last_became_peered":"2023-11-24T08:06:57.687574+0000","last_unstale":"2023-11-24T08:06:58.862483+0000","last_undegraded":"2023-11-24T08:06:58.862483+0000","last_fullsized":"2023-11-24T08:06:58.862483+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:34.286354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863702+0000","last_change":"2023-11-24T08:06:57.685785+0000","last_active":"2023-11-24T08:06:58.863702+0000","last_peered":"2023-11-24T08:06:58.863702+0000","last_clean":"2023-11-24T08:06:58.863702+0000","last_became_active":"2023-11-24T08:06:57.684938+0000","last_became_peered":"2023-11-24T08:06:57.684938+0000","last_unstale":"2023-11-24T08:06:58.863702+0000","last_undegraded":"2023-11-24T08:06:58.863702+0000","last_fullsized":"2023-11-24T08:06:58.863702+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:19.723929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864393+0000","last_change":"2023-11-24T08:06:56.664279+0000","last_active":"2023-11-24T08:06:58.864393+0000","last_peered":"2023-11-24T08:06:58.864393+0000","last_clean":"2023-11-24T08:06:58.864393+0000","last_became_active":"2023-11-24T08:06:56.663597+0000","last_became_peered":"2023-11-24T08:06:56.663597+0000","last_unstale":"2023-11-24T08:06:58.864393+0000","last_undegraded":"2023-11-24T08:06:58.864393+0000","last_fullsized":"2023-11-24T08:06:58.864393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:43.312572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863004+0000","last_change":"2023-11-24T08:06:57.680929+0000","last_active":"2023-11-24T08:06:58.863004+0000","last_peered":"2023-11-24T08:06:58.863004+0000","last_clean":"2023-11-24T08:06:58.863004+0000","last_became_active":"2023-11-24T08:06:57.680621+0000","last_became_peered":"2023-11-24T08:06:57.680621+0000","last_unstale":"2023-11-24T08:06:58.863004+0000","last_undegraded":"2023-11-24T08:06:58.863004+0000","last_fullsized":"2023-11-24T08:06:58.863004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:59.119252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657439+0000","last_change":"2023-11-24T08:06:56.659707+0000","last_active":"2023-11-24T08:06:58.657439+0000","last_peered":"2023-11-24T08:06:58.657439+0000","last_clean":"2023-11-24T08:06:58.657439+0000","last_became_active":"2023-11-24T08:06:56.659454+0000","last_became_peered":"2023-11-24T08:06:56.659454+0000","last_unstale":"2023-11-24T08:06:58.657439+0000","last_undegraded":"2023-11-24T08:06:58.657439+0000","last_fullsized":"2023-11-24T08:06:58.657439+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:51:37.057433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864478+0000","last_change":"2023-11-24T08:06:57.690172+0000","last_active":"2023-11-24T08:06:58.864478+0000","last_peered":"2023-11-24T08:06:58.864478+0000","last_clean":"2023-11-24T08:06:58.864478+0000","last_became_active":"2023-11-24T08:06:57.689878+0000","last_became_peered":"2023-11-24T08:06:57.689878+0000","last_unstale":"2023-11-24T08:06:58.864478+0000","last_undegraded":"2023-11-24T08:06:58.864478+0000","last_fullsized":"2023-11-24T08:06:58.864478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:26.741577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864907+0000","last_change":"2023-11-24T08:06:56.655031+0000","last_active":"2023-11-24T08:06:58.864907+0000","last_peered":"2023-11-24T08:06:58.864907+0000","last_clean":"2023-11-24T08:06:58.864907+0000","last_became_active":"2023-11-24T08:06:56.654763+0000","last_became_peered":"2023-11-24T08:06:56.654763+0000","last_unstale":"2023-11-24T08:06:58.864907+0000","last_undegraded":"2023-11-24T08:06:58.864907+0000","last_fullsized":"2023-11-24T08:06:58.864907+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:51:37.057433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864860+0000","last_change":"2023-11-24T08:06:57.682108+0000","last_active":"2023-11-24T08:06:58.864860+0000","last_peered":"2023-11-24T08:06:58.864860+0000","last_clean":"2023-11-24T08:06:58.864860+0000","last_became_active":"2023-11-24T08:06:57.681687+0000","last_became_peered":"2023-11-24T08:06:57.681687+0000","last_unstale":"2023-11-24T08:06:58.864860+0000","last_undegraded":"2023-11-24T08:06:58.864860+0000","last_fullsized":"2023-11-24T08:06:58.864860+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:04.581597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863466+0000","last_change":"2023-11-24T08:06:56.663844+0000","last_active":"2023-11-24T08:06:58.863466+0000","last_peered":"2023-11-24T08:06:58.863466+0000","last_clean":"2023-11-24T08:06:58.863466+0000","last_became_active":"2023-11-24T08:06:56.663463+0000","last_became_peered":"2023-11-24T08:06:56.663463+0000","last_unstale":"2023-11-24T08:06:58.863466+0000","last_undegraded":"2023-11-24T08:06:58.863466+0000","last_fullsized":"2023-11-24T08:06:58.863466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:37:48.366130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656347+0000","last_change":"2023-11-24T08:06:57.677823+0000","last_active":"2023-11-24T08:06:58.656347+0000","last_peered":"2023-11-24T08:06:58.656347+0000","last_clean":"2023-11-24T08:06:58.656347+0000","last_became_active":"2023-11-24T08:06:57.677558+0000","last_became_peered":"2023-11-24T08:06:57.677558+0000","last_unstale":"2023-11-24T08:06:58.656347+0000","last_undegraded":"2023-11-24T08:06:58.656347+0000","last_fullsized":"2023-11-24T08:06:58.656347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:26:52.615885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654707+0000","last_change":"2023-11-24T08:06:56.663489+0000","last_active":"2023-11-24T08:06:58.654707+0000","last_peered":"2023-11-24T08:06:58.654707+0000","last_clean":"2023-11-24T08:06:58.654707+0000","last_became_active":"2023-11-24T08:06:56.660710+0000","last_became_peered":"2023-11-24T08:06:56.660710+0000","last_unstale":"2023-11-24T08:06:58.654707+0000","last_undegraded":"2023-11-24T08:06:58.654707+0000","last_fullsized":"2023-11-24T08:06:58.654707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:07:41.041665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656048+0000","last_change":"2023-11-24T08:06:57.681172+0000","last_active":"2023-11-24T08:06:58.656048+0000","last_peered":"2023-11-24T08:06:58.656048+0000","last_clean":"2023-11-24T08:06:58.656048+0000","last_became_active":"2023-11-24T08:06:57.680901+0000","last_became_peered":"2023-11-24T08:06:57.680901+0000","last_unstale":"2023-11-24T08:06:58.656048+0000","last_undegraded":"2023-11-24T08:06:58.656048+0000","last_fullsized":"2023-11-24T08:06:58.656048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:04.581597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863148+0000","last_change":"2023-11-24T08:06:56.652955+0000","last_active":"2023-11-24T08:06:58.863148+0000","last_peered":"2023-11-24T08:06:58.863148+0000","last_clean":"2023-11-24T08:06:58.863148+0000","last_became_active":"2023-11-24T08:06:56.652182+0000","last_became_peered":"2023-11-24T08:06:56.652182+0000","last_unstale":"2023-11-24T08:06:58.863148+0000","last_undegraded":"2023-11-24T08:06:58.863148+0000","last_fullsized":"2023-11-24T08:06:58.863148+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:18.679075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865816+0000","last_change":"2023-11-24T08:06:57.680536+0000","last_active":"2023-11-24T08:06:58.865816+0000","last_peered":"2023-11-24T08:06:58.865816+0000","last_clean":"2023-11-24T08:06:58.865816+0000","last_became_active":"2023-11-24T08:06:57.680137+0000","last_became_peered":"2023-11-24T08:06:57.680137+0000","last_unstale":"2023-11-24T08:06:58.865816+0000","last_undegraded":"2023-11-24T08:06:58.865816+0000","last_fullsized":"2023-11-24T08:06:58.865816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:16.062829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862430+0000","last_change":"2023-11-24T08:06:56.666202+0000","last_active":"2023-11-24T08:06:58.862430+0000","last_peered":"2023-11-24T08:06:58.862430+0000","last_clean":"2023-11-24T08:06:58.862430+0000","last_became_active":"2023-11-24T08:06:56.665818+0000","last_became_peered":"2023-11-24T08:06:56.665818+0000","last_unstale":"2023-11-24T08:06:58.862430+0000","last_undegraded":"2023-11-24T08:06:58.862430+0000","last_fullsized":"2023-11-24T08:06:58.862430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:45.563720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655904+0000","last_change":"2023-11-24T08:06:57.684827+0000","last_active":"2023-11-24T08:06:58.655904+0000","last_peered":"2023-11-24T08:06:58.655904+0000","last_clean":"2023-11-24T08:06:58.655904+0000","last_became_active":"2023-11-24T08:06:57.684287+0000","last_became_peered":"2023-11-24T08:06:57.684287+0000","last_unstale":"2023-11-24T08:06:58.655904+0000","last_undegraded":"2023-11-24T08:06:58.655904+0000","last_fullsized":"2023-11-24T08:06:58.655904+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:51.211107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654364+0000","last_change":"2023-11-24T08:06:56.664389+0000","last_active":"2023-11-24T08:06:58.654364+0000","last_peered":"2023-11-24T08:06:58.654364+0000","last_clean":"2023-11-24T08:06:58.654364+0000","last_became_active":"2023-11-24T08:06:56.662305+0000","last_became_peered":"2023-11-24T08:06:56.662305+0000","last_unstale":"2023-11-24T08:06:58.654364+0000","last_undegraded":"2023-11-24T08:06:58.654364+0000","last_fullsized":"2023-11-24T08:06:58.654364+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:15.048677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655610+0000","last_change":"2023-11-24T08:06:57.683358+0000","last_active":"2023-11-24T08:06:58.655610+0000","last_peered":"2023-11-24T08:06:58.655610+0000","last_clean":"2023-11-24T08:06:58.655610+0000","last_became_active":"2023-11-24T08:06:57.682944+0000","last_became_peered":"2023-11-24T08:06:57.682944+0000","last_unstale":"2023-11-24T08:06:58.655610+0000","last_undegraded":"2023-11-24T08:06:58.655610+0000","last_fullsized":"2023-11-24T08:06:58.655610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:30:58.023042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864648+0000","last_change":"2023-11-24T08:06:56.657043+0000","last_active":"2023-11-24T08:06:58.864648+0000","last_peered":"2023-11-24T08:06:58.864648+0000","last_clean":"2023-11-24T08:06:58.864648+0000","last_became_active":"2023-11-24T08:06:56.651593+0000","last_became_peered":"2023-11-24T08:06:56.651593+0000","last_unstale":"2023-11-24T08:06:58.864648+0000","last_undegraded":"2023-11-24T08:06:58.864648+0000","last_fullsized":"2023-11-24T08:06:58.864648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:31.061959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865208+0000","last_change":"2023-11-24T08:06:57.691550+0000","last_active":"2023-11-24T08:06:58.865208+0000","last_peered":"2023-11-24T08:06:58.865208+0000","last_clean":"2023-11-24T08:06:58.865208+0000","last_became_active":"2023-11-24T08:06:57.691124+0000","last_became_peered":"2023-11-24T08:06:57.691124+0000","last_unstale":"2023-11-24T08:06:58.865208+0000","last_undegraded":"2023-11-24T08:06:58.865208+0000","last_fullsized":"2023-11-24T08:06:58.865208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:13.606639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657471+0000","last_change":"2023-11-24T08:06:56.655928+0000","last_active":"2023-11-24T08:06:58.657471+0000","last_peered":"2023-11-24T08:06:58.657471+0000","last_clean":"2023-11-24T08:06:58.657471+0000","last_became_active":"2023-11-24T08:06:56.655584+0000","last_became_peered":"2023-11-24T08:06:56.655584+0000","last_unstale":"2023-11-24T08:06:58.657471+0000","last_undegraded":"2023-11-24T08:06:58.657471+0000","last_fullsized":"2023-11-24T08:06:58.657471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:43.549985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862464+0000","last_change":"2023-11-24T08:06:57.684155+0000","last_active":"2023-11-24T08:06:58.862464+0000","last_peered":"2023-11-24T08:06:58.862464+0000","last_clean":"2023-11-24T08:06:58.862464+0000","last_became_active":"2023-11-24T08:06:57.679755+0000","last_became_peered":"2023-11-24T08:06:57.679755+0000","last_unstale":"2023-11-24T08:06:58.862464+0000","last_undegraded":"2023-11-24T08:06:58.862464+0000","last_fullsized":"2023-11-24T08:06:58.862464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:54.423943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864390+0000","last_change":"2023-11-24T08:06:56.659865+0000","last_active":"2023-11-24T08:06:58.864390+0000","last_peered":"2023-11-24T08:06:58.864390+0000","last_clean":"2023-11-24T08:06:58.864390+0000","last_became_active":"2023-11-24T08:06:56.659553+0000","last_became_peered":"2023-11-24T08:06:56.659553+0000","last_unstale":"2023-11-24T08:06:58.864390+0000","last_undegraded":"2023-11-24T08:06:58.864390+0000","last_fullsized":"2023-11-24T08:06:58.864390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:35.180063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864132+0000","last_change":"2023-11-24T08:06:57.691601+0000","last_active":"2023-11-24T08:06:58.864132+0000","last_peered":"2023-11-24T08:06:58.864132+0000","last_clean":"2023-11-24T08:06:58.864132+0000","last_became_active":"2023-11-24T08:06:57.691006+0000","last_became_peered":"2023-11-24T08:06:57.691006+0000","last_unstale":"2023-11-24T08:06:58.864132+0000","last_undegraded":"2023-11-24T08:06:58.864132+0000","last_fullsized":"2023-11-24T08:06:58.864132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:13:54.849451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655463+0000","last_change":"2023-11-24T08:06:56.661990+0000","last_active":"2023-11-24T08:06:58.655463+0000","last_peered":"2023-11-24T08:06:58.655463+0000","last_clean":"2023-11-24T08:06:58.655463+0000","last_became_active":"2023-11-24T08:06:56.661658+0000","last_became_peered":"2023-11-24T08:06:56.661658+0000","last_unstale":"2023-11-24T08:06:58.655463+0000","last_undegraded":"2023-11-24T08:06:58.655463+0000","last_fullsized":"2023-11-24T08:06:58.655463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:02.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862517+0000","last_change":"2023-11-24T08:06:57.693003+0000","last_active":"2023-11-24T08:06:58.862517+0000","last_peered":"2023-11-24T08:06:58.862517+0000","last_clean":"2023-11-24T08:06:58.862517+0000","last_became_active":"2023-11-24T08:06:57.692666+0000","last_became_peered":"2023-11-24T08:06:57.692666+0000","last_unstale":"2023-11-24T08:06:58.862517+0000","last_undegraded":"2023-11-24T08:06:58.862517+0000","last_fullsized":"2023-11-24T08:06:58.862517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:41.456025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656225+0000","last_change":"2023-11-24T08:06:56.655748+0000","last_active":"2023-11-24T08:06:58.656225+0000","last_peered":"2023-11-24T08:06:58.656225+0000","last_clean":"2023-11-24T08:06:58.656225+0000","last_became_active":"2023-11-24T08:06:56.655470+0000","last_became_peered":"2023-11-24T08:06:56.655470+0000","last_unstale":"2023-11-24T08:06:58.656225+0000","last_undegraded":"2023-11-24T08:06:58.656225+0000","last_fullsized":"2023-11-24T08:06:58.656225+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:12:32.037053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864835+0000","last_change":"2023-11-24T08:06:57.683864+0000","last_active":"2023-11-24T08:06:58.864835+0000","last_peered":"2023-11-24T08:06:58.864835+0000","last_clean":"2023-11-24T08:06:58.864835+0000","last_became_active":"2023-11-24T08:06:57.683480+0000","last_became_peered":"2023-11-24T08:06:57.683480+0000","last_unstale":"2023-11-24T08:06:58.864835+0000","last_undegraded":"2023-11-24T08:06:58.864835+0000","last_fullsized":"2023-11-24T08:06:58.864835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:24.560556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865849+0000","last_change":"2023-11-24T08:06:56.657014+0000","last_active":"2023-11-24T08:06:58.865849+0000","last_peered":"2023-11-24T08:06:58.865849+0000","last_clean":"2023-11-24T08:06:58.865849+0000","last_became_active":"2023-11-24T08:06:56.656654+0000","last_became_peered":"2023-11-24T08:06:56.656654+0000","last_unstale":"2023-11-24T08:06:58.865849+0000","last_undegraded":"2023-11-24T08:06:58.865849+0000","last_fullsized":"2023-11-24T08:06:58.865849+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:15.370260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654372+0000","last_change":"2023-11-24T08:06:57.677347+0000","last_active":"2023-11-24T08:06:58.654372+0000","last_peered":"2023-11-24T08:06:58.654372+0000","last_clean":"2023-11-24T08:06:58.654372+0000","last_became_active":"2023-11-24T08:06:57.677048+0000","last_became_peered":"2023-11-24T08:06:57.677048+0000","last_unstale":"2023-11-24T08:06:58.654372+0000","last_undegraded":"2023-11-24T08:06:58.654372+0000","last_fullsized":"2023-11-24T08:06:58.654372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:37:49.380283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862595+0000","last_change":"2023-11-24T08:06:56.665018+0000","last_active":"2023-11-24T08:06:58.862595+0000","last_peered":"2023-11-24T08:06:58.862595+0000","last_clean":"2023-11-24T08:06:58.862595+0000","last_became_active":"2023-11-24T08:06:56.664737+0000","last_became_peered":"2023-11-24T08:06:56.664737+0000","last_unstale":"2023-11-24T08:06:58.862595+0000","last_undegraded":"2023-11-24T08:06:58.862595+0000","last_fullsized":"2023-11-24T08:06:58.862595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:56.532423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863621+0000","last_change":"2023-11-24T08:06:57.681653+0000","last_active":"2023-11-24T08:06:58.863621+0000","last_peered":"2023-11-24T08:06:58.863621+0000","last_clean":"2023-11-24T08:06:58.863621+0000","last_became_active":"2023-11-24T08:06:57.680759+0000","last_became_peered":"2023-11-24T08:06:57.680759+0000","last_unstale":"2023-11-24T08:06:58.863621+0000","last_undegraded":"2023-11-24T08:06:58.863621+0000","last_fullsized":"2023-11-24T08:06:58.863621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:56:24.479991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.866719+0000","last_change":"2023-11-24T08:06:56.661302+0000","last_active":"2023-11-24T08:06:58.866719+0000","last_peered":"2023-11-24T08:06:58.866719+0000","last_clean":"2023-11-24T08:06:58.866719+0000","last_became_active":"2023-11-24T08:06:56.661006+0000","last_became_peered":"2023-11-24T08:06:56.661006+0000","last_unstale":"2023-11-24T08:06:58.866719+0000","last_undegraded":"2023-11-24T08:06:58.866719+0000","last_fullsized":"2023-11-24T08:06:58.866719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:56.532423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864763+0000","last_change":"2023-11-24T08:06:57.681801+0000","last_active":"2023-11-24T08:06:58.864763+0000","last_peered":"2023-11-24T08:06:58.864763+0000","last_clean":"2023-11-24T08:06:58.864763+0000","last_became_active":"2023-11-24T08:06:57.681496+0000","last_became_peered":"2023-11-24T08:06:57.681496+0000","last_unstale":"2023-11-24T08:06:58.864763+0000","last_undegraded":"2023-11-24T08:06:58.864763+0000","last_fullsized":"2023-11-24T08:06:58.864763+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:19.693227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657041+0000","last_change":"2023-11-24T08:06:56.657212+0000","last_active":"2023-11-24T08:06:58.657041+0000","last_peered":"2023-11-24T08:06:58.657041+0000","last_clean":"2023-11-24T08:06:58.657041+0000","last_became_active":"2023-11-24T08:06:56.655754+0000","last_became_peered":"2023-11-24T08:06:56.655754+0000","last_unstale":"2023-11-24T08:06:58.657041+0000","last_undegraded":"2023-11-24T08:06:58.657041+0000","last_fullsized":"2023-11-24T08:06:58.657041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:47:32.965405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865190+0000","last_change":"2023-11-24T08:06:57.691660+0000","last_active":"2023-11-24T08:06:58.865190+0000","last_peered":"2023-11-24T08:06:58.865190+0000","last_clean":"2023-11-24T08:06:58.865190+0000","last_became_active":"2023-11-24T08:06:57.691145+0000","last_became_peered":"2023-11-24T08:06:57.691145+0000","last_unstale":"2023-11-24T08:06:58.865190+0000","last_undegraded":"2023-11-24T08:06:58.865190+0000","last_fullsized":"2023-11-24T08:06:58.865190+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:47.509181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863949+0000","last_change":"2023-11-24T08:06:56.652436+0000","last_active":"2023-11-24T08:06:58.863949+0000","last_peered":"2023-11-24T08:06:58.863949+0000","last_clean":"2023-11-24T08:06:58.863949+0000","last_became_active":"2023-11-24T08:06:56.649633+0000","last_became_peered":"2023-11-24T08:06:56.649633+0000","last_unstale":"2023-11-24T08:06:58.863949+0000","last_undegraded":"2023-11-24T08:06:58.863949+0000","last_fullsized":"2023-11-24T08:06:58.863949+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:55.024911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657000+0000","last_change":"2023-11-24T08:06:57.668846+0000","last_active":"2023-11-24T08:06:58.657000+0000","last_peered":"2023-11-24T08:06:58.657000+0000","last_clean":"2023-11-24T08:06:58.657000+0000","last_became_active":"2023-11-24T08:06:57.668509+0000","last_became_peered":"2023-11-24T08:06:57.668509+0000","last_unstale":"2023-11-24T08:06:58.657000+0000","last_undegraded":"2023-11-24T08:06:58.657000+0000","last_fullsized":"2023-11-24T08:06:58.657000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:55.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863083+0000","last_change":"2023-11-24T08:06:56.662000+0000","last_active":"2023-11-24T08:06:58.863083+0000","last_peered":"2023-11-24T08:06:58.863083+0000","last_clean":"2023-11-24T08:06:58.863083+0000","last_became_active":"2023-11-24T08:06:56.661686+0000","last_became_peered":"2023-11-24T08:06:56.661686+0000","last_unstale":"2023-11-24T08:06:58.863083+0000","last_undegraded":"2023-11-24T08:06:58.863083+0000","last_fullsized":"2023-11-24T08:06:58.863083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:33.444110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864288+0000","last_change":"2023-11-24T08:06:57.691344+0000","last_active":"2023-11-24T08:06:58.864288+0000","last_peered":"2023-11-24T08:06:58.864288+0000","last_clean":"2023-11-24T08:06:58.864288+0000","last_became_active":"2023-11-24T08:06:57.691023+0000","last_became_peered":"2023-11-24T08:06:57.691023+0000","last_unstale":"2023-11-24T08:06:58.864288+0000","last_undegraded":"2023-11-24T08:06:58.864288+0000","last_fullsized":"2023-11-24T08:06:58.864288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:10.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864456+0000","last_change":"2023-11-24T08:06:56.660339+0000","last_active":"2023-11-24T08:06:58.864456+0000","last_peered":"2023-11-24T08:06:58.864456+0000","last_clean":"2023-11-24T08:06:58.864456+0000","last_became_active":"2023-11-24T08:06:56.660069+0000","last_became_peered":"2023-11-24T08:06:56.660069+0000","last_unstale":"2023-11-24T08:06:58.864456+0000","last_undegraded":"2023-11-24T08:06:58.864456+0000","last_fullsized":"2023-11-24T08:06:58.864456+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:23.546403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862902+0000","last_change":"2023-11-24T08:06:57.692531+0000","last_active":"2023-11-24T08:06:58.862902+0000","last_peered":"2023-11-24T08:06:58.862902+0000","last_clean":"2023-11-24T08:06:58.862902+0000","last_became_active":"2023-11-24T08:06:57.691273+0000","last_became_peered":"2023-11-24T08:06:57.691273+0000","last_unstale":"2023-11-24T08:06:58.862902+0000","last_undegraded":"2023-11-24T08:06:58.862902+0000","last_fullsized":"2023-11-24T08:06:58.862902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:30:58.023042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656680+0000","last_change":"2023-11-24T08:06:56.657959+0000","last_active":"2023-11-24T08:06:58.656680+0000","last_peered":"2023-11-24T08:06:58.656680+0000","last_clean":"2023-11-24T08:06:58.656680+0000","last_became_active":"2023-11-24T08:06:56.657664+0000","last_became_peered":"2023-11-24T08:06:56.657664+0000","last_unstale":"2023-11-24T08:06:58.656680+0000","last_undegraded":"2023-11-24T08:06:58.656680+0000","last_fullsized":"2023-11-24T08:06:58.656680+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:33.272202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863932+0000","last_change":"2023-11-24T08:06:57.686742+0000","last_active":"2023-11-24T08:06:58.863932+0000","last_peered":"2023-11-24T08:06:58.863932+0000","last_clean":"2023-11-24T08:06:58.863932+0000","last_became_active":"2023-11-24T08:06:57.684969+0000","last_became_peered":"2023-11-24T08:06:57.684969+0000","last_unstale":"2023-11-24T08:06:58.863932+0000","last_undegraded":"2023-11-24T08:06:58.863932+0000","last_fullsized":"2023-11-24T08:06:58.863932+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:38:17.382639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862597+0000","last_change":"2023-11-24T08:06:56.644142+0000","last_active":"2023-11-24T08:06:58.862597+0000","last_peered":"2023-11-24T08:06:58.862597+0000","last_clean":"2023-11-24T08:06:58.862597+0000","last_became_active":"2023-11-24T08:06:56.643821+0000","last_became_peered":"2023-11-24T08:06:56.643821+0000","last_unstale":"2023-11-24T08:06:58.862597+0000","last_undegraded":"2023-11-24T08:06:58.862597+0000","last_fullsized":"2023-11-24T08:06:58.862597+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:22:33.775265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862928+0000","last_change":"2023-11-24T08:06:57.684651+0000","last_active":"2023-11-24T08:06:58.862928+0000","last_peered":"2023-11-24T08:06:58.862928+0000","last_clean":"2023-11-24T08:06:58.862928+0000","last_became_active":"2023-11-24T08:06:57.684332+0000","last_became_peered":"2023-11-24T08:06:57.684332+0000","last_unstale":"2023-11-24T08:06:58.862928+0000","last_undegraded":"2023-11-24T08:06:58.862928+0000","last_fullsized":"2023-11-24T08:06:58.862928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:44.564138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862311+0000","last_change":"2023-11-24T08:06:56.660500+0000","last_active":"2023-11-24T08:06:58.862311+0000","last_peered":"2023-11-24T08:06:58.862311+0000","last_clean":"2023-11-24T08:06:58.862311+0000","last_became_active":"2023-11-24T08:06:56.660210+0000","last_became_peered":"2023-11-24T08:06:56.660210+0000","last_unstale":"2023-11-24T08:06:58.862311+0000","last_undegraded":"2023-11-24T08:06:58.862311+0000","last_fullsized":"2023-11-24T08:06:58.862311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:02.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863439+0000","last_change":"2023-11-24T08:06:57.678815+0000","last_active":"2023-11-24T08:06:58.863439+0000","last_peered":"2023-11-24T08:06:58.863439+0000","last_clean":"2023-11-24T08:06:58.863439+0000","last_became_active":"2023-11-24T08:06:57.678451+0000","last_became_peered":"2023-11-24T08:06:57.678451+0000","last_unstale":"2023-11-24T08:06:58.863439+0000","last_undegraded":"2023-11-24T08:06:58.863439+0000","last_fullsized":"2023-11-24T08:06:58.863439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:36.194216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863114+0000","last_change":"2023-11-24T08:06:56.660878+0000","last_active":"2023-11-24T08:06:58.863114+0000","last_peered":"2023-11-24T08:06:58.863114+0000","last_clean":"2023-11-24T08:06:58.863114+0000","last_became_active":"2023-11-24T08:06:56.660244+0000","last_became_peered":"2023-11-24T08:06:56.660244+0000","last_unstale":"2023-11-24T08:06:58.863114+0000","last_undegraded":"2023-11-24T08:06:58.863114+0000","last_fullsized":"2023-11-24T08:06:58.863114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:59:02.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865281+0000","last_change":"2023-11-24T08:06:57.684736+0000","last_active":"2023-11-24T08:06:58.865281+0000","last_peered":"2023-11-24T08:06:58.865281+0000","last_clean":"2023-11-24T08:06:58.865281+0000","last_became_active":"2023-11-24T08:06:57.684467+0000","last_became_peered":"2023-11-24T08:06:57.684467+0000","last_unstale":"2023-11-24T08:06:58.865281+0000","last_undegraded":"2023-11-24T08:06:58.865281+0000","last_fullsized":"2023-11-24T08:06:58.865281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:32.076112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864037+0000","last_change":"2023-11-24T08:06:56.657246+0000","last_active":"2023-11-24T08:06:58.864037+0000","last_peered":"2023-11-24T08:06:58.864037+0000","last_clean":"2023-11-24T08:06:58.864037+0000","last_became_active":"2023-11-24T08:06:56.656732+0000","last_became_peered":"2023-11-24T08:06:56.656732+0000","last_unstale":"2023-11-24T08:06:58.864037+0000","last_undegraded":"2023-11-24T08:06:58.864037+0000","last_fullsized":"2023-11-24T08:06:58.864037+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:12.592486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862822+0000","last_change":"2023-11-24T08:06:57.670421+0000","last_active":"2023-11-24T08:06:58.862822+0000","last_peered":"2023-11-24T08:06:58.862822+0000","last_clean":"2023-11-24T08:06:58.862822+0000","last_became_active":"2023-11-24T08:06:57.670007+0000","last_became_peered":"2023-11-24T08:06:57.670007+0000","last_unstale":"2023-11-24T08:06:58.862822+0000","last_undegraded":"2023-11-24T08:06:58.862822+0000","last_fullsized":"2023-11-24T08:06:58.862822+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:02.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.866399+0000","last_change":"2023-11-24T08:06:56.659180+0000","last_active":"2023-11-24T08:06:58.866399+0000","last_peered":"2023-11-24T08:06:58.866399+0000","last_clean":"2023-11-24T08:06:58.866399+0000","last_became_active":"2023-11-24T08:06:56.656913+0000","last_became_peered":"2023-11-24T08:06:56.656913+0000","last_unstale":"2023-11-24T08:06:58.866399+0000","last_undegraded":"2023-11-24T08:06:58.866399+0000","last_fullsized":"2023-11-24T08:06:58.866399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:45:42.861609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864788+0000","last_change":"2023-11-24T08:06:57.682924+0000","last_active":"2023-11-24T08:06:58.864788+0000","last_peered":"2023-11-24T08:06:58.864788+0000","last_clean":"2023-11-24T08:06:58.864788+0000","last_became_active":"2023-11-24T08:06:57.681434+0000","last_became_peered":"2023-11-24T08:06:57.681434+0000","last_unstale":"2023-11-24T08:06:58.864788+0000","last_undegraded":"2023-11-24T08:06:58.864788+0000","last_fullsized":"2023-11-24T08:06:58.864788+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:51.211107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863860+0000","last_change":"2023-11-24T08:06:56.666196+0000","last_active":"2023-11-24T08:06:58.863860+0000","last_peered":"2023-11-24T08:06:58.863860+0000","last_clean":"2023-11-24T08:06:58.863860+0000","last_became_active":"2023-11-24T08:06:56.665789+0000","last_became_peered":"2023-11-24T08:06:56.665789+0000","last_unstale":"2023-11-24T08:06:58.863860+0000","last_undegraded":"2023-11-24T08:06:58.863860+0000","last_fullsized":"2023-11-24T08:06:58.863860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:33:11.593756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656456+0000","last_change":"2023-11-24T08:06:57.686831+0000","last_active":"2023-11-24T08:06:58.656456+0000","last_peered":"2023-11-24T08:06:58.656456+0000","last_clean":"2023-11-24T08:06:58.656456+0000","last_became_active":"2023-11-24T08:06:57.682593+0000","last_became_peered":"2023-11-24T08:06:57.682593+0000","last_unstale":"2023-11-24T08:06:58.656456+0000","last_undegraded":"2023-11-24T08:06:58.656456+0000","last_fullsized":"2023-11-24T08:06:58.656456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:25.153836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862036+0000","last_change":"2023-11-24T08:06:56.661053+0000","last_active":"2023-11-24T08:06:58.862036+0000","last_peered":"2023-11-24T08:06:58.862036+0000","last_clean":"2023-11-24T08:06:58.862036+0000","last_became_active":"2023-11-24T08:06:56.660773+0000","last_became_peered":"2023-11-24T08:06:56.660773+0000","last_unstale":"2023-11-24T08:06:58.862036+0000","last_undegraded":"2023-11-24T08:06:58.862036+0000","last_fullsized":"2023-11-24T08:06:58.862036+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:05.087900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865193+0000","last_change":"2023-11-24T08:06:57.685706+0000","last_active":"2023-11-24T08:06:58.865193+0000","last_peered":"2023-11-24T08:06:58.865193+0000","last_clean":"2023-11-24T08:06:58.865193+0000","last_became_active":"2023-11-24T08:06:57.685417+0000","last_became_peered":"2023-11-24T08:06:57.685417+0000","last_unstale":"2023-11-24T08:06:58.865193+0000","last_undegraded":"2023-11-24T08:06:58.865193+0000","last_fullsized":"2023-11-24T08:06:58.865193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:56.039063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654870+0000","last_change":"2023-11-24T08:06:56.659791+0000","last_active":"2023-11-24T08:06:58.654870+0000","last_peered":"2023-11-24T08:06:58.654870+0000","last_clean":"2023-11-24T08:06:58.654870+0000","last_became_active":"2023-11-24T08:06:56.659515+0000","last_became_peered":"2023-11-24T08:06:56.659515+0000","last_unstale":"2023-11-24T08:06:58.654870+0000","last_undegraded":"2023-11-24T08:06:58.654870+0000","last_fullsized":"2023-11-24T08:06:58.654870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:28.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862139+0000","last_change":"2023-11-24T08:06:57.686058+0000","last_active":"2023-11-24T08:06:58.862139+0000","last_peered":"2023-11-24T08:06:58.862139+0000","last_clean":"2023-11-24T08:06:58.862139+0000","last_became_active":"2023-11-24T08:06:57.685721+0000","last_became_peered":"2023-11-24T08:06:57.685721+0000","last_unstale":"2023-11-24T08:06:58.862139+0000","last_undegraded":"2023-11-24T08:06:58.862139+0000","last_fullsized":"2023-11-24T08:06:58.862139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:38:52.500159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862225+0000","last_change":"2023-11-24T08:06:56.647691+0000","last_active":"2023-11-24T08:06:58.862225+0000","last_peered":"2023-11-24T08:06:58.862225+0000","last_clean":"2023-11-24T08:06:58.862225+0000","last_became_active":"2023-11-24T08:06:56.647413+0000","last_became_peered":"2023-11-24T08:06:56.647413+0000","last_unstale":"2023-11-24T08:06:58.862225+0000","last_undegraded":"2023-11-24T08:06:58.862225+0000","last_fullsized":"2023-11-24T08:06:58.862225+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:38.336330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655770+0000","last_change":"2023-11-24T08:06:57.678491+0000","last_active":"2023-11-24T08:06:58.655770+0000","last_peered":"2023-11-24T08:06:58.655770+0000","last_clean":"2023-11-24T08:06:58.655770+0000","last_became_active":"2023-11-24T08:06:57.676817+0000","last_became_peered":"2023-11-24T08:06:57.676817+0000","last_unstale":"2023-11-24T08:06:58.655770+0000","last_undegraded":"2023-11-24T08:06:58.655770+0000","last_fullsized":"2023-11-24T08:06:58.655770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:35.713574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655822+0000","last_change":"2023-11-24T08:06:56.662280+0000","last_active":"2023-11-24T08:06:58.655822+0000","last_peered":"2023-11-24T08:06:58.655822+0000","last_clean":"2023-11-24T08:06:58.655822+0000","last_became_active":"2023-11-24T08:06:56.661921+0000","last_became_peered":"2023-11-24T08:06:56.661921+0000","last_unstale":"2023-11-24T08:06:58.655822+0000","last_undegraded":"2023-11-24T08:06:58.655822+0000","last_fullsized":"2023-11-24T08:06:58.655822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:43.312572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655694+0000","last_change":"2023-11-24T08:06:57.679668+0000","last_active":"2023-11-24T08:06:58.655694+0000","last_peered":"2023-11-24T08:06:58.655694+0000","last_clean":"2023-11-24T08:06:58.655694+0000","last_became_active":"2023-11-24T08:06:57.679410+0000","last_became_peered":"2023-11-24T08:06:57.679410+0000","last_unstale":"2023-11-24T08:06:58.655694+0000","last_undegraded":"2023-11-24T08:06:58.655694+0000","last_fullsized":"2023-11-24T08:06:58.655694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:26.741577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.866265+0000","last_change":"2023-11-24T08:06:56.663050+0000","last_active":"2023-11-24T08:06:58.866265+0000","last_peered":"2023-11-24T08:06:58.866265+0000","last_clean":"2023-11-24T08:06:58.866265+0000","last_became_active":"2023-11-24T08:06:56.662780+0000","last_became_peered":"2023-11-24T08:06:56.662780+0000","last_unstale":"2023-11-24T08:06:58.866265+0000","last_undegraded":"2023-11-24T08:06:58.866265+0000","last_fullsized":"2023-11-24T08:06:58.866265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:16:37.636592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T08:06:57.663376+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:06:57.663376+0000","last_peered":"2023-11-24T08:06:57.663376+0000","last_clean":"2023-11-24T08:06:57.663376+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:06:57.663376+0000","last_undegraded":"2023-11-24T08:06:57.663376+0000","last_fullsized":"2023-11-24T08:06:57.663376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1310720,"data_stored":887014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0860000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.484,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2789999999999999,"15min":1.3009999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6639999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.099},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3420000000000001,"15min":1.3839999999999999},"min":{"1min":0.97299999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5569999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.6080000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.151,"15min":1.1479999999999999},"min":{"1min":0.70099999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0840000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.728},{"interface":"front","average":{"1min":1.268,"5min":1.3069999999999999,"15min":1.3160000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.91000000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2829999999999999,"15min":1.321},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.966,"5min":2.524,"15min":2.524},"last":1.8859999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.454,"15min":1.5169999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8939999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.323}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4550000000000001,"15min":1.532},"min":{"1min":0.88400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6419999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.379},"min":{"1min":0.84499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.544}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.581,"15min":1.619},"min":{"1min":1.0229999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.173,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.601,"15min":1.661},"min":{"1min":1.069,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7350000000000001,"5min":2.944,"15min":2.944},"last":1.6890000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.702,"15min":1.7589999999999999},"min":{"1min":1.252,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.649,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.79},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6060000000000001,"15min":1.665},"min":{"1min":1.0509999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.841,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.962}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":296486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.276,"15min":1.3300000000000001},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.113,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.274,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.22}]},{"osd":1,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1539999999999999,"15min":1.159},"min":{"1min":0.68400000000000005,"5min":0.625,"15min":0.625},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2370000000000001,"15min":1.3069999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6830000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.524}]},{"osd":2,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.232,"15min":1.296},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.508,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.419,"15min":1.5069999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.752,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.96799999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.3879999999999999,"15min":1.444},"min":{"1min":0.79200000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.931,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.4690000000000001,"15min":1.498},"min":{"1min":0.94899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.968,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.4630000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.6279999999999999,"15min":1.758},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2120000000000002,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.4390000000000001,"15min":1.472},"min":{"1min":0.86199999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.855,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.552}]},{"osd":5,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6679999999999999,"15min":1.8080000000000001},"min":{"1min":1.0029999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.893,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.806,"15min":1.9490000000000001},"min":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.901,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.609}]},{"osd":7,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6379999999999999,"15min":1.6890000000000001},"min":{"1min":0.75800000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.379,"5min":2.98,"15min":2.98},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.1659999999999999,"5min":2.29,"15min":2.29},"last":1.2789999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":716800,"data_stored":297138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.048},"min":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2949999999999999,"5min":1.764,"15min":1.764},"last":1.046},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0329999999999999,"15min":1.006},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.3240000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6950000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0780000000000001,"15min":1.0509999999999999},"min":{"1min":0.70899999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.458,"5min":1.635,"15min":1.635},"last":0.95099999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2090000000000001,"15min":1.2130000000000001},"min":{"1min":0.77100000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.673,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.085},{"interface":"front","average":{"1min":1.226,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.745,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6259999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.3660000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.427},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.575,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2470000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5669999999999999,"5min":1.889,"15min":1.889},"last":1.2450000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.284,"15min":1.298},"min":{"1min":0.79100000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.871,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.68400000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.766,"5min":2.27,"15min":2.27},"last":1.401}]},{"osd":6,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.448,"15min":1.47},"min":{"1min":0.97799999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7909999999999999,"5min":3.806,"15min":3.806},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.409,"15min":1.4079999999999999},"min":{"1min":0.84699999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0169999999999999,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.278}]},{"osd":7,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5189999999999999,"15min":1.5269999999999999},"min":{"1min":0.63,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.974,"5min":2.923,"15min":2.923},"last":1.167},{"interface":"front","average":{"1min":1.569,"5min":1.54,"15min":1.53},"min":{"1min":1.0640000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.214,"5min":2.859,"15min":2.859},"last":1.4279999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":720896,"data_stored":298098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0409999999999999,"15min":1.042},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4399999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.167},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.1279999999999999},"min":{"1min":0.77900000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.595,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.446}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.079,"15min":1.0840000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.377,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.111,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.79000000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5640000000000001,"5min":1.903,"15min":1.903},"last":1.403}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1890000000000001,"15min":1.2},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.589,"5min":1.823,"15min":1.823},"last":1.099},{"interface":"front","average":{"1min":1.155,"5min":1.2410000000000001,"15min":1.258},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.2969999999999999},"min":{"1min":0.81399999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7330000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3109999999999999,"15min":1.3149999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9530000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.351}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.282,"15min":1.296},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.617,"5min":1.944,"15min":1.944},"last":1.734},{"interface":"front","average":{"1min":1.26,"5min":1.3400000000000001,"15min":1.3580000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":2.012,"15min":2.012},"last":1.6619999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.377,"15min":1.3680000000000001},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4890000000000001,"15min":1.5069999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.927,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.8740000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.4330000000000001,"15min":1.431},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.768,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.554,"15min":1.5589999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.411,"5min":2.722,"15min":2.722},"last":1.827}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":302842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.82099999999999995},"min":{"1min":0.50800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.569,"5min":1.585,"15min":1.585},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.92300000000000004,"15min":0.94799999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4930000000000001,"5min":1.51,"15min":1.51},"last":1.1879999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.075,"15min":1.151},"min":{"1min":0.63100000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.835,"5min":3.246,"15min":3.246},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97399999999999998,"15min":0.96699999999999997},"min":{"1min":0.54900000000000004,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4179999999999999,"5min":1.665,"15min":1.665},"last":1.079}]},{"osd":2,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1399999999999999,"15min":1.1819999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7410000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.008},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1160000000000001,"15min":1.155},"min":{"1min":0.66800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.399,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.95899999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2450000000000001,"15min":1.264},"min":{"1min":0.73299999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.671,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.321},{"interface":"front","average":{"1min":1.292,"5min":1.355,"15min":1.3899999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.665,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.117}]},{"osd":5,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.4199999999999999,"15min":1.4590000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9430000000000001,"5min":2.395,"15min":2.395},"last":1.284},{"interface":"front","average":{"1min":1.304,"5min":1.306,"15min":1.3},"min":{"1min":0.90400000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9399999999999999,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.351}]},{"osd":6,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.421,"15min":1.4319999999999999},"min":{"1min":0.94099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8899999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3979999999999999,"15min":1.4179999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.589}]},{"osd":7,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4219999999999999,"15min":1.444},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.994,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.528,"15min":1.581},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.895,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3919999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":716800,"data_stored":296542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.417,"5min":1.524,"15min":1.524},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":1.016},"min":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5740000000000001,"5min":2.133,"15min":2.133},"last":1.2769999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1419999999999999,"15min":1.2250000000000001},"min":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8280000000000001,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0029999999999999,"15min":1.069},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4550000000000001,"5min":1.998,"15min":1.998},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.321},"min":{"1min":0.73299999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6879999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.625},{"interface":"front","average":{"1min":1.077,"5min":1.147,"15min":1.1950000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7809999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3260000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.754,"5min":0.442,"15min":0.442},"max":{"1min":1.7969999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.4359999999999999,"15min":1.502},"min":{"1min":1.018,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.758,"5min":3.161,"15min":3.161},"last":1.5720000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.345,"15min":1.393},"min":{"1min":0.77000000000000002,"5min":0.623,"15min":0.623},"max":{"1min":1.752,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.355,"15min":1.411},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.96,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.083}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.5169999999999999,"15min":1.593},"min":{"1min":0.96199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8979999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.5649999999999999},"min":{"1min":0.80900000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8520000000000001,"5min":2.702,"15min":2.702},"last":1.409}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.448,"15min":1.514},"min":{"1min":0.78200000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0419999999999998,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.373},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.5660000000000001,"15min":1.6439999999999999},"min":{"1min":0.89300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.952,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.508}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":296576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.878,"15min":0.89200000000000002},"min":{"1min":0.70799999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.228},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0029999999999999,"15min":1.0580000000000001},"min":{"1min":0.56599999999999995,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.397,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.6459999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.03},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.746,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.724},{"interface":"front","average":{"1min":1.081,"5min":1.179,"15min":1.2290000000000001},"min":{"1min":0.76400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.794,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.8959999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.2729999999999999},"min":{"1min":0.72299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.897,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.999},{"interface":"front","average":{"1min":1.091,"5min":1.177,"15min":1.232},"min":{"1min":0.66500000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.6890000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.5620000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4199999999999999,"15min":1.454},"min":{"1min":0.95799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.7909999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4259999999999999,"15min":1.4570000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0089999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.8740000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.452,"15min":1.478},"min":{"1min":0.91000000000000003,"5min":0.871,"15min":0.871},"max":{"1min":1.903,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.855}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.4319999999999999,"15min":1.458},"min":{"1min":0.83099999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.774,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.758},{"interface":"front","average":{"1min":1.373,"5min":1.4430000000000001,"15min":1.4830000000000001},"min":{"1min":0.79200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8420000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.6020000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":0.996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8700000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.55,"5min":1.5680000000000001,"15min":1.595},"min":{"1min":0.96599999999999997,"5min":0.872,"15min":0.872},"max":{"1min":1.8500000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.8280000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1327104,"data_stored":894329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.52500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":1.083},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.97799999999999998,"15min":0.94799999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.4139999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.111,"15min":1.075},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.899,"5min":3.899,"15min":3.899},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1,"15min":0.95999999999999996},"min":{"1min":0.498,"5min":0.443,"15min":0.443},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.0700000000000001,"15min":1.0409999999999999},"min":{"1min":0.627,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.169},{"interface":"front","average":{"1min":1.631,"5min":1.2729999999999999,"15min":1.2270000000000001},"min":{"1min":0.57299999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":6.3570000000000002,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.621}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.3069999999999999,"15min":1.2709999999999999},"min":{"1min":0.92000000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.137},{"interface":"front","average":{"1min":1.7070000000000001,"5min":1.5,"15min":1.482},"min":{"1min":1.022,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":3.399,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.734}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.4399999999999999,"15min":1.3899999999999999},"min":{"1min":0.88100000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":5.9939999999999998,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.77},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":1.026,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.391,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.673}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.248},{"interface":"front","average":{"1min":1.744,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":1.0840000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.879,"5min":3.879,"15min":3.879},"last":1.8879999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.8029999999999999,"5min":1.5660000000000001,"15min":1.548},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":3.4540000000000002,"5min":3.823,"15min":3.823},"last":1.833},{"interface":"front","average":{"1min":1.75,"5min":1.5209999999999999,"15min":1.4970000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.3109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:07:02.528 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:07:02.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.798+0000 7fab95763640 1 Processor -- start 2023-11-24T08:07:02.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.798+0000 7fab95763640 1 -- start start 2023-11-24T08:07:02.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.799+0000 7fab95763640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fab9015d5b0 0x7fab900d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.799+0000 7fab95763640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab900b1650 0x7fab900d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.799+0000 7fab95763640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab900cf0c0 con 0x7fab900d14a0 2023-11-24T08:07:02.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.799+0000 7fab95763640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab900b01b0 con 0x7fab900b1650 2023-11-24T08:07:02.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab95763640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9015d990 con 0x7fab9015d5b0 2023-11-24T08:07:02.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab94f62640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab900b1650 0x7fab900d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab87fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fab900d14a0 0x7fab900cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59600/0 (socket says 172.21.15.64:59600) 2023-11-24T08:07:02.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab87fff640 1 -- 172.21.15.64:0/1006203416 learned_addr learned my addr 172.21.15.64:0/1006203416 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:02.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab8ffff640 1 --2- 172.21.15.64:0/1006203416 >> v2:172.21.15.91:3300/0 conn(0x7fab9015d5b0 0x7fab900d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab94f62640 1 -- 172.21.15.64:0/1006203416 >> v2:172.21.15.91:3300/0 conn(0x7fab9015d5b0 msgr2=0x7fab900d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:02.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab94f62640 1 --2- 172.21.15.64:0/1006203416 >> v2:172.21.15.91:3300/0 conn(0x7fab9015d5b0 0x7fab900d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab94f62640 1 -- 172.21.15.64:0/1006203416 >> v1:172.21.15.64:6789/0 conn(0x7fab900d14a0 legacy=0x7fab900cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab8f7fe640 1 -- 172.21.15.64:0/1006203416 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173847529 0 0) 0x7fab900cf0c0 con 0x7fab900d14a0 2023-11-24T08:07:02.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.800+0000 7fab94f62640 1 -- 172.21.15.64:0/1006203416 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900d1e40 con 0x7fab900b1650 2023-11-24T08:07:02.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.802+0000 7fab94f62640 1 --2- 172.21.15.64:0/1006203416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900d1900 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fab9008c9e0 tx=0x7fab8802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a74450d052ff9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:02.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.802+0000 7fab8f7fe640 1 -- 172.21.15.64:0/1006203416 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8802cc80 con 0x7fab900b1650 2023-11-24T08:07:02.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.802+0000 7fab8f7fe640 1 -- 172.21.15.64:0/1006203416 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8802cde0 con 0x7fab900b1650 2023-11-24T08:07:02.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.802+0000 7fab8f7fe640 1 -- 172.21.15.64:0/1006203416 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88034660 con 0x7fab900b1650 2023-11-24T08:07:02.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.802+0000 7fab95763640 1 -- 172.21.15.64:0/1006203416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 msgr2=0x7fab900d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.802+0000 7fab95763640 1 --2- 172.21.15.64:0/1006203416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900d1900 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fab9008c9e0 tx=0x7fab8802bed0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.803+0000 7fab95763640 1 -- 172.21.15.64:0/1006203416 shutdown_connections 2023-11-24T08:07:02.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.803+0000 7fab95763640 1 --2- 172.21.15.64:0/1006203416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900d1900 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.803+0000 7fab95763640 1 --2- 172.21.15.64:0/1006203416 >> v2:172.21.15.91:3300/0 conn(0x7fab9015d5b0 0x7fab900d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.803+0000 7fab95763640 1 -- 172.21.15.64:0/1006203416 >> 172.21.15.64:0/1006203416 conn(0x7fab9009c3b0 msgr2=0x7fab900d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:02.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.803+0000 7fab95763640 1 -- 172.21.15.64:0/1006203416 shutdown_connections 2023-11-24T08:07:02.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.803+0000 7fab95763640 1 -- 172.21.15.64:0/1006203416 wait complete. 2023-11-24T08:07:02.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.805+0000 7fab95763640 1 Processor -- start 2023-11-24T08:07:02.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.805+0000 7fab95763640 1 -- start start 2023-11-24T08:07:02.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab95763640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab95763640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab900d14a0 0x7fab901f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab95763640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab9015d5b0 0x7fab900b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab95763640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900b7a30 con 0x7fab900b1650 2023-11-24T08:07:02.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab95763640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900b6e20 con 0x7fab900d14a0 2023-11-24T08:07:02.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab95763640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab900b7100 con 0x7fab9015d5b0 2023-11-24T08:07:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab8ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab8ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60034/0 (socket says 172.21.15.64:60034) 2023-11-24T08:07:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab8ffff640 1 -- 172.21.15.64:0/1207737161 learned_addr learned my addr 172.21.15.64:0/1207737161 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab87fff640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab900d14a0 0x7fab901f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab94f62640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab9015d5b0 0x7fab900b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab8ffff640 1 -- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab9015d5b0 msgr2=0x7fab900b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab8ffff640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab9015d5b0 0x7fab900b6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab8ffff640 1 -- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab900d14a0 msgr2=0x7fab901f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:02.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab8ffff640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab900d14a0 0x7fab901f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:02.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.806+0000 7fab8ffff640 1 -- 172.21.15.64:0/1207737161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88009a70 con 0x7fab900b1650 2023-11-24T08:07:02.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.807+0000 7fab8ffff640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900ca140 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fab8000d800 tx=0x7fab8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:02.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.807+0000 7fab8d7fa640 1 -- 172.21.15.64:0/1207737161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8000b6c0 con 0x7fab900b1650 2023-11-24T08:07:02.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.807+0000 7fab8d7fa640 1 -- 172.21.15.64:0/1207737161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8000b820 con 0x7fab900b1650 2023-11-24T08:07:02.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.807+0000 7fab8d7fa640 1 -- 172.21.15.64:0/1207737161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8000bbb0 con 0x7fab900b1650 2023-11-24T08:07:02.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.807+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab900b73e0 con 0x7fab900b1650 2023-11-24T08:07:02.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.807+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab901f2910 con 0x7fab900b1650 2023-11-24T08:07:02.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.808+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab901f2aa0 con 0x7fab900b1650 2023-11-24T08:07:02.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.809+0000 7fab8d7fa640 1 -- 172.21.15.64:0/1207737161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fab8000bd10 con 0x7fab900b1650 2023-11-24T08:07:02.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.811+0000 7fab8d7fa640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab7006b080 0x7fab7006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.811+0000 7fab8d7fa640 1 -- 172.21.15.64:0/1207737161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab80089ff0 con 0x7fab900b1650 2023-11-24T08:07:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.811+0000 7fab87fff640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab7006b080 0x7fab7006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:02.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.812+0000 7fab87fff640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab7006b080 0x7fab7006d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fab9006cae0 tx=0x7fab78009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:02.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:02.816+0000 7fab8d7fa640 1 -- 172.21.15.64:0/1207737161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab901f2aa0 con 0x7fab900b1650 2023-11-24T08:07:03.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.011+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab900afe40 con 0x7fab7006b080 2023-11-24T08:07:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.035+0000 7fab8d7fa640 1 -- 172.21.15.64:0/1207737161 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286816 (secure 0 0 0) 0x7fab780070d0 con 0x7fab7006b080 2023-11-24T08:07:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:07:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab7006b080 msgr2=0x7fab7006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab7006b080 0x7fab7006d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fab9006cae0 tx=0x7fab78009040 comp rx=0 tx=0).stop 2023-11-24T08:07:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 msgr2=0x7fab900ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900ca140 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fab8000d800 tx=0x7fab8000dcc0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 shutdown_connections 2023-11-24T08:07:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fab7006b080 0x7fab7006d540 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fab9015d5b0 0x7fab900b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fab900d14a0 0x7fab901f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 --2- 172.21.15.64:0/1207737161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab900b1650 0x7fab900ca140 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 >> 172.21.15.64:0/1207737161 conn(0x7fab9009c3b0 msgr2=0x7fab9015b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 shutdown_connections 2023-11-24T08:07:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.044+0000 7fab95763640 1 -- 172.21.15.64:0/1207737161 wait complete. 2023-11-24T08:07:03.070 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2023-11-24T08:07:02.082048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239628,"kb_used_data":6780,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735092,"statfs":{"total":773094113280,"available":772848734208,"internally_reserved":0,"allocated":6942720,"data_stored":3569025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002319"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.653277+0000","last_change":"2023-11-24T08:06:57.674621+0000","last_active":"2023-11-24T08:06:58.653277+0000","last_peered":"2023-11-24T08:06:58.653277+0000","last_clean":"2023-11-24T08:06:58.653277+0000","last_became_active":"2023-11-24T08:06:57.674311+0000","last_became_peered":"2023-11-24T08:06:57.674311+0000","last_unstale":"2023-11-24T08:06:58.653277+0000","last_undegraded":"2023-11-24T08:06:58.653277+0000","last_fullsized":"2023-11-24T08:06:58.653277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:48.451930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657164+0000","last_change":"2023-11-24T08:06:57.683332+0000","last_active":"2023-11-24T08:06:58.657164+0000","last_peered":"2023-11-24T08:06:58.657164+0000","last_clean":"2023-11-24T08:06:58.657164+0000","last_became_active":"2023-11-24T08:06:57.683038+0000","last_became_peered":"2023-11-24T08:06:57.683038+0000","last_unstale":"2023-11-24T08:06:58.657164+0000","last_undegraded":"2023-11-24T08:06:58.657164+0000","last_fullsized":"2023-11-24T08:06:58.657164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:55.298367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863986+0000","last_change":"2023-11-24T08:06:57.688141+0000","last_active":"2023-11-24T08:06:58.863986+0000","last_peered":"2023-11-24T08:06:58.863986+0000","last_clean":"2023-11-24T08:06:58.863986+0000","last_became_active":"2023-11-24T08:06:57.685078+0000","last_became_peered":"2023-11-24T08:06:57.685078+0000","last_unstale":"2023-11-24T08:06:58.863986+0000","last_undegraded":"2023-11-24T08:06:58.863986+0000","last_fullsized":"2023-11-24T08:06:58.863986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:34.577120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863973+0000","last_change":"2023-11-24T08:06:57.679172+0000","last_active":"2023-11-24T08:06:58.863973+0000","last_peered":"2023-11-24T08:06:58.863973+0000","last_clean":"2023-11-24T08:06:58.863973+0000","last_became_active":"2023-11-24T08:06:57.678839+0000","last_became_peered":"2023-11-24T08:06:57.678839+0000","last_unstale":"2023-11-24T08:06:58.863973+0000","last_undegraded":"2023-11-24T08:06:58.863973+0000","last_fullsized":"2023-11-24T08:06:58.863973+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:26.674291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.653453+0000","last_change":"2023-11-24T08:06:57.673834+0000","last_active":"2023-11-24T08:06:58.653453+0000","last_peered":"2023-11-24T08:06:58.653453+0000","last_clean":"2023-11-24T08:06:58.653453+0000","last_became_active":"2023-11-24T08:06:57.673549+0000","last_became_peered":"2023-11-24T08:06:57.673549+0000","last_unstale":"2023-11-24T08:06:58.653453+0000","last_undegraded":"2023-11-24T08:06:58.653453+0000","last_fullsized":"2023-11-24T08:06:58.653453+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:27.170334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863438+0000","last_change":"2023-11-24T08:06:57.676622+0000","last_active":"2023-11-24T08:06:58.863438+0000","last_peered":"2023-11-24T08:06:58.863438+0000","last_clean":"2023-11-24T08:06:58.863438+0000","last_became_active":"2023-11-24T08:06:57.676309+0000","last_became_peered":"2023-11-24T08:06:57.676309+0000","last_unstale":"2023-11-24T08:06:58.863438+0000","last_undegraded":"2023-11-24T08:06:58.863438+0000","last_fullsized":"2023-11-24T08:06:58.863438+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:47:33.979558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654867+0000","last_change":"2023-11-24T08:06:57.673771+0000","last_active":"2023-11-24T08:06:58.654867+0000","last_peered":"2023-11-24T08:06:58.654867+0000","last_clean":"2023-11-24T08:06:58.654867+0000","last_became_active":"2023-11-24T08:06:57.673323+0000","last_became_peered":"2023-11-24T08:06:57.673323+0000","last_unstale":"2023-11-24T08:06:58.654867+0000","last_undegraded":"2023-11-24T08:06:58.654867+0000","last_fullsized":"2023-11-24T08:06:58.654867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:34.458263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655524+0000","last_change":"2023-11-24T08:06:57.668015+0000","last_active":"2023-11-24T08:06:58.655524+0000","last_peered":"2023-11-24T08:06:58.655524+0000","last_clean":"2023-11-24T08:06:58.655524+0000","last_became_active":"2023-11-24T08:06:57.667604+0000","last_became_peered":"2023-11-24T08:06:57.667604+0000","last_unstale":"2023-11-24T08:06:58.655524+0000","last_undegraded":"2023-11-24T08:06:58.655524+0000","last_fullsized":"2023-11-24T08:06:58.655524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:38:17.382639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655576+0000","last_change":"2023-11-24T08:06:57.681027+0000","last_active":"2023-11-24T08:06:58.655576+0000","last_peered":"2023-11-24T08:06:58.655576+0000","last_clean":"2023-11-24T08:06:58.655576+0000","last_became_active":"2023-11-24T08:06:57.680745+0000","last_became_peered":"2023-11-24T08:06:57.680745+0000","last_unstale":"2023-11-24T08:06:58.655576+0000","last_undegraded":"2023-11-24T08:06:58.655576+0000","last_fullsized":"2023-11-24T08:06:58.655576+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:41.456025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863375+0000","last_change":"2023-11-24T08:06:57.687573+0000","last_active":"2023-11-24T08:06:58.863375+0000","last_peered":"2023-11-24T08:06:58.863375+0000","last_clean":"2023-11-24T08:06:58.863375+0000","last_became_active":"2023-11-24T08:06:57.686663+0000","last_became_peered":"2023-11-24T08:06:57.686663+0000","last_unstale":"2023-11-24T08:06:58.863375+0000","last_undegraded":"2023-11-24T08:06:58.863375+0000","last_fullsized":"2023-11-24T08:06:58.863375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:13:14.056149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863154+0000","last_change":"2023-11-24T08:06:57.680923+0000","last_active":"2023-11-24T08:06:58.863154+0000","last_peered":"2023-11-24T08:06:58.863154+0000","last_clean":"2023-11-24T08:06:58.863154+0000","last_became_active":"2023-11-24T08:06:57.680220+0000","last_became_peered":"2023-11-24T08:06:57.680220+0000","last_unstale":"2023-11-24T08:06:58.863154+0000","last_undegraded":"2023-11-24T08:06:58.863154+0000","last_fullsized":"2023-11-24T08:06:58.863154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:26.987702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863013+0000","last_change":"2023-11-24T08:06:57.686230+0000","last_active":"2023-11-24T08:06:58.863013+0000","last_peered":"2023-11-24T08:06:58.863013+0000","last_clean":"2023-11-24T08:06:58.863013+0000","last_became_active":"2023-11-24T08:06:57.685937+0000","last_became_peered":"2023-11-24T08:06:57.685937+0000","last_unstale":"2023-11-24T08:06:58.863013+0000","last_undegraded":"2023-11-24T08:06:58.863013+0000","last_fullsized":"2023-11-24T08:06:58.863013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:17.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.866445+0000","last_change":"2023-11-24T08:06:57.688137+0000","last_active":"2023-11-24T08:06:58.866445+0000","last_peered":"2023-11-24T08:06:58.866445+0000","last_clean":"2023-11-24T08:06:58.866445+0000","last_became_active":"2023-11-24T08:06:57.687620+0000","last_became_peered":"2023-11-24T08:06:57.687620+0000","last_unstale":"2023-11-24T08:06:58.866445+0000","last_undegraded":"2023-11-24T08:06:58.866445+0000","last_fullsized":"2023-11-24T08:06:58.866445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:03.110816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654772+0000","last_change":"2023-11-24T08:06:57.672943+0000","last_active":"2023-11-24T08:06:58.654772+0000","last_peered":"2023-11-24T08:06:58.654772+0000","last_clean":"2023-11-24T08:06:58.654772+0000","last_became_active":"2023-11-24T08:06:57.669818+0000","last_became_peered":"2023-11-24T08:06:57.669818+0000","last_unstale":"2023-11-24T08:06:58.654772+0000","last_undegraded":"2023-11-24T08:06:58.654772+0000","last_fullsized":"2023-11-24T08:06:58.654772+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:30.446731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654705+0000","last_change":"2023-11-24T08:06:57.683874+0000","last_active":"2023-11-24T08:06:58.654705+0000","last_peered":"2023-11-24T08:06:58.654705+0000","last_clean":"2023-11-24T08:06:58.654705+0000","last_became_active":"2023-11-24T08:06:57.683597+0000","last_became_peered":"2023-11-24T08:06:57.683597+0000","last_unstale":"2023-11-24T08:06:58.654705+0000","last_undegraded":"2023-11-24T08:06:58.654705+0000","last_fullsized":"2023-11-24T08:06:58.654705+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:33:12.607909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.861784+0000","last_change":"2023-11-24T08:06:57.683906+0000","last_active":"2023-11-24T08:06:58.861784+0000","last_peered":"2023-11-24T08:06:58.861784+0000","last_clean":"2023-11-24T08:06:58.861784+0000","last_became_active":"2023-11-24T08:06:57.683588+0000","last_became_peered":"2023-11-24T08:06:57.683588+0000","last_unstale":"2023-11-24T08:06:58.861784+0000","last_undegraded":"2023-11-24T08:06:58.861784+0000","last_fullsized":"2023-11-24T08:06:58.861784+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:12:33.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656783+0000","last_change":"2023-11-24T08:06:57.684411+0000","last_active":"2023-11-24T08:06:58.656783+0000","last_peered":"2023-11-24T08:06:58.656783+0000","last_clean":"2023-11-24T08:06:58.656783+0000","last_became_active":"2023-11-24T08:06:57.684120+0000","last_became_peered":"2023-11-24T08:06:57.684120+0000","last_unstale":"2023-11-24T08:06:58.656783+0000","last_undegraded":"2023-11-24T08:06:58.656783+0000","last_fullsized":"2023-11-24T08:06:58.656783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:32.191838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864423+0000","last_change":"2023-11-24T08:06:57.684644+0000","last_active":"2023-11-24T08:06:58.864423+0000","last_peered":"2023-11-24T08:06:58.864423+0000","last_clean":"2023-11-24T08:06:58.864423+0000","last_became_active":"2023-11-24T08:06:57.684258+0000","last_became_peered":"2023-11-24T08:06:57.684258+0000","last_unstale":"2023-11-24T08:06:58.864423+0000","last_undegraded":"2023-11-24T08:06:58.864423+0000","last_fullsized":"2023-11-24T08:06:58.864423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:54.650345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.652921+0000","last_change":"2023-11-24T08:06:57.666208+0000","last_active":"2023-11-24T08:06:58.652921+0000","last_peered":"2023-11-24T08:06:58.652921+0000","last_clean":"2023-11-24T08:06:58.652921+0000","last_became_active":"2023-11-24T08:06:57.665742+0000","last_became_peered":"2023-11-24T08:06:57.665742+0000","last_unstale":"2023-11-24T08:06:58.652921+0000","last_undegraded":"2023-11-24T08:06:58.652921+0000","last_fullsized":"2023-11-24T08:06:58.652921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:22.944151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657250+0000","last_change":"2023-11-24T08:06:57.681769+0000","last_active":"2023-11-24T08:06:58.657250+0000","last_peered":"2023-11-24T08:06:58.657250+0000","last_clean":"2023-11-24T08:06:58.657250+0000","last_became_active":"2023-11-24T08:06:57.681532+0000","last_became_peered":"2023-11-24T08:06:57.681532+0000","last_unstale":"2023-11-24T08:06:58.657250+0000","last_undegraded":"2023-11-24T08:06:58.657250+0000","last_fullsized":"2023-11-24T08:06:58.657250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:07.649707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864479+0000","last_change":"2023-11-24T08:06:57.687079+0000","last_active":"2023-11-24T08:06:58.864479+0000","last_peered":"2023-11-24T08:06:58.864479+0000","last_clean":"2023-11-24T08:06:58.864479+0000","last_became_active":"2023-11-24T08:06:57.684687+0000","last_became_peered":"2023-11-24T08:06:57.684687+0000","last_unstale":"2023-11-24T08:06:58.864479+0000","last_undegraded":"2023-11-24T08:06:58.864479+0000","last_fullsized":"2023-11-24T08:06:58.864479+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:03:08.781566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656091+0000","last_change":"2023-11-24T08:06:57.680674+0000","last_active":"2023-11-24T08:06:58.656091+0000","last_peered":"2023-11-24T08:06:58.656091+0000","last_clean":"2023-11-24T08:06:58.656091+0000","last_became_active":"2023-11-24T08:06:57.680354+0000","last_became_peered":"2023-11-24T08:06:57.680354+0000","last_unstale":"2023-11-24T08:06:58.656091+0000","last_undegraded":"2023-11-24T08:06:58.656091+0000","last_fullsized":"2023-11-24T08:06:58.656091+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:06:20.551336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863662+0000","last_change":"2023-11-24T08:06:57.681440+0000","last_active":"2023-11-24T08:06:58.863662+0000","last_peered":"2023-11-24T08:06:58.863662+0000","last_clean":"2023-11-24T08:06:58.863662+0000","last_became_active":"2023-11-24T08:06:57.680417+0000","last_became_peered":"2023-11-24T08:06:57.680417+0000","last_unstale":"2023-11-24T08:06:58.863662+0000","last_undegraded":"2023-11-24T08:06:58.863662+0000","last_fullsized":"2023-11-24T08:06:58.863662+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:53.404909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865841+0000","last_change":"2023-11-24T08:06:57.678374+0000","last_active":"2023-11-24T08:06:58.865841+0000","last_peered":"2023-11-24T08:06:58.865841+0000","last_clean":"2023-11-24T08:06:58.865841+0000","last_became_active":"2023-11-24T08:06:57.678028+0000","last_became_peered":"2023-11-24T08:06:57.678028+0000","last_unstale":"2023-11-24T08:06:58.865841+0000","last_undegraded":"2023-11-24T08:06:58.865841+0000","last_fullsized":"2023-11-24T08:06:58.865841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:46.577873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.653642+0000","last_change":"2023-11-24T08:06:57.683704+0000","last_active":"2023-11-24T08:06:58.653642+0000","last_peered":"2023-11-24T08:06:58.653642+0000","last_clean":"2023-11-24T08:06:58.653642+0000","last_became_active":"2023-11-24T08:06:57.682465+0000","last_became_peered":"2023-11-24T08:06:57.682465+0000","last_unstale":"2023-11-24T08:06:58.653642+0000","last_undegraded":"2023-11-24T08:06:58.653642+0000","last_fullsized":"2023-11-24T08:06:58.653642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:57.628251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863148+0000","last_change":"2023-11-24T08:06:57.680639+0000","last_active":"2023-11-24T08:06:58.863148+0000","last_peered":"2023-11-24T08:06:58.863148+0000","last_clean":"2023-11-24T08:06:58.863148+0000","last_became_active":"2023-11-24T08:06:57.680373+0000","last_became_peered":"2023-11-24T08:06:57.680373+0000","last_unstale":"2023-11-24T08:06:58.863148+0000","last_undegraded":"2023-11-24T08:06:58.863148+0000","last_fullsized":"2023-11-24T08:06:58.863148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:39:40.474356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864522+0000","last_change":"2023-11-24T08:06:57.686822+0000","last_active":"2023-11-24T08:06:58.864522+0000","last_peered":"2023-11-24T08:06:58.864522+0000","last_clean":"2023-11-24T08:06:58.864522+0000","last_became_active":"2023-11-24T08:06:57.684940+0000","last_became_peered":"2023-11-24T08:06:57.684940+0000","last_unstale":"2023-11-24T08:06:58.864522+0000","last_undegraded":"2023-11-24T08:06:58.864522+0000","last_fullsized":"2023-11-24T08:06:58.864522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:35.713574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.652572+0000","last_change":"2023-11-24T08:06:57.674601+0000","last_active":"2023-11-24T08:06:58.652572+0000","last_peered":"2023-11-24T08:06:58.652572+0000","last_clean":"2023-11-24T08:06:58.652572+0000","last_became_active":"2023-11-24T08:06:57.674290+0000","last_became_peered":"2023-11-24T08:06:57.674290+0000","last_unstale":"2023-11-24T08:06:58.652572+0000","last_undegraded":"2023-11-24T08:06:58.652572+0000","last_fullsized":"2023-11-24T08:06:58.652572+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:31:52.096168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656486+0000","last_change":"2023-11-24T08:06:57.672514+0000","last_active":"2023-11-24T08:06:58.656486+0000","last_peered":"2023-11-24T08:06:58.656486+0000","last_clean":"2023-11-24T08:06:58.656486+0000","last_became_active":"2023-11-24T08:06:57.672202+0000","last_became_peered":"2023-11-24T08:06:57.672202+0000","last_unstale":"2023-11-24T08:06:58.656486+0000","last_undegraded":"2023-11-24T08:06:58.656486+0000","last_fullsized":"2023-11-24T08:06:58.656486+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:39:04.407449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865439+0000","last_change":"2023-11-24T08:06:57.684670+0000","last_active":"2023-11-24T08:06:58.865439+0000","last_peered":"2023-11-24T08:06:58.865439+0000","last_clean":"2023-11-24T08:06:58.865439+0000","last_became_active":"2023-11-24T08:06:57.683051+0000","last_became_peered":"2023-11-24T08:06:57.683051+0000","last_unstale":"2023-11-24T08:06:58.865439+0000","last_undegraded":"2023-11-24T08:06:58.865439+0000","last_fullsized":"2023-11-24T08:06:58.865439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:44.326725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.652141+0000","last_change":"2023-11-24T08:06:57.672693+0000","last_active":"2023-11-24T08:06:58.652141+0000","last_peered":"2023-11-24T08:06:58.652141+0000","last_clean":"2023-11-24T08:06:58.652141+0000","last_became_active":"2023-11-24T08:06:57.670540+0000","last_became_peered":"2023-11-24T08:06:57.670540+0000","last_unstale":"2023-11-24T08:06:58.652141+0000","last_undegraded":"2023-11-24T08:06:58.652141+0000","last_fullsized":"2023-11-24T08:06:58.652141+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:38.102009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862483+0000","last_change":"2023-11-24T08:06:57.687837+0000","last_active":"2023-11-24T08:06:58.862483+0000","last_peered":"2023-11-24T08:06:58.862483+0000","last_clean":"2023-11-24T08:06:58.862483+0000","last_became_active":"2023-11-24T08:06:57.687574+0000","last_became_peered":"2023-11-24T08:06:57.687574+0000","last_unstale":"2023-11-24T08:06:58.862483+0000","last_undegraded":"2023-11-24T08:06:58.862483+0000","last_fullsized":"2023-11-24T08:06:58.862483+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:34.286354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863702+0000","last_change":"2023-11-24T08:06:57.685785+0000","last_active":"2023-11-24T08:06:58.863702+0000","last_peered":"2023-11-24T08:06:58.863702+0000","last_clean":"2023-11-24T08:06:58.863702+0000","last_became_active":"2023-11-24T08:06:57.684938+0000","last_became_peered":"2023-11-24T08:06:57.684938+0000","last_unstale":"2023-11-24T08:06:58.863702+0000","last_undegraded":"2023-11-24T08:06:58.863702+0000","last_fullsized":"2023-11-24T08:06:58.863702+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:19.723929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864393+0000","last_change":"2023-11-24T08:06:56.664279+0000","last_active":"2023-11-24T08:06:58.864393+0000","last_peered":"2023-11-24T08:06:58.864393+0000","last_clean":"2023-11-24T08:06:58.864393+0000","last_became_active":"2023-11-24T08:06:56.663597+0000","last_became_peered":"2023-11-24T08:06:56.663597+0000","last_unstale":"2023-11-24T08:06:58.864393+0000","last_undegraded":"2023-11-24T08:06:58.864393+0000","last_fullsized":"2023-11-24T08:06:58.864393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:43.312572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863004+0000","last_change":"2023-11-24T08:06:57.680929+0000","last_active":"2023-11-24T08:06:58.863004+0000","last_peered":"2023-11-24T08:06:58.863004+0000","last_clean":"2023-11-24T08:06:58.863004+0000","last_became_active":"2023-11-24T08:06:57.680621+0000","last_became_peered":"2023-11-24T08:06:57.680621+0000","last_unstale":"2023-11-24T08:06:58.863004+0000","last_undegraded":"2023-11-24T08:06:58.863004+0000","last_fullsized":"2023-11-24T08:06:58.863004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:59.119252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657439+0000","last_change":"2023-11-24T08:06:56.659707+0000","last_active":"2023-11-24T08:06:58.657439+0000","last_peered":"2023-11-24T08:06:58.657439+0000","last_clean":"2023-11-24T08:06:58.657439+0000","last_became_active":"2023-11-24T08:06:56.659454+0000","last_became_peered":"2023-11-24T08:06:56.659454+0000","last_unstale":"2023-11-24T08:06:58.657439+0000","last_undegraded":"2023-11-24T08:06:58.657439+0000","last_fullsized":"2023-11-24T08:06:58.657439+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:51:37.057433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864478+0000","last_change":"2023-11-24T08:06:57.690172+0000","last_active":"2023-11-24T08:06:58.864478+0000","last_peered":"2023-11-24T08:06:58.864478+0000","last_clean":"2023-11-24T08:06:58.864478+0000","last_became_active":"2023-11-24T08:06:57.689878+0000","last_became_peered":"2023-11-24T08:06:57.689878+0000","last_unstale":"2023-11-24T08:06:58.864478+0000","last_undegraded":"2023-11-24T08:06:58.864478+0000","last_fullsized":"2023-11-24T08:06:58.864478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:26.741577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864907+0000","last_change":"2023-11-24T08:06:56.655031+0000","last_active":"2023-11-24T08:06:58.864907+0000","last_peered":"2023-11-24T08:06:58.864907+0000","last_clean":"2023-11-24T08:06:58.864907+0000","last_became_active":"2023-11-24T08:06:56.654763+0000","last_became_peered":"2023-11-24T08:06:56.654763+0000","last_unstale":"2023-11-24T08:06:58.864907+0000","last_undegraded":"2023-11-24T08:06:58.864907+0000","last_fullsized":"2023-11-24T08:06:58.864907+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:51:37.057433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864860+0000","last_change":"2023-11-24T08:06:57.682108+0000","last_active":"2023-11-24T08:06:58.864860+0000","last_peered":"2023-11-24T08:06:58.864860+0000","last_clean":"2023-11-24T08:06:58.864860+0000","last_became_active":"2023-11-24T08:06:57.681687+0000","last_became_peered":"2023-11-24T08:06:57.681687+0000","last_unstale":"2023-11-24T08:06:58.864860+0000","last_undegraded":"2023-11-24T08:06:58.864860+0000","last_fullsized":"2023-11-24T08:06:58.864860+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:04.581597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863466+0000","last_change":"2023-11-24T08:06:56.663844+0000","last_active":"2023-11-24T08:06:58.863466+0000","last_peered":"2023-11-24T08:06:58.863466+0000","last_clean":"2023-11-24T08:06:58.863466+0000","last_became_active":"2023-11-24T08:06:56.663463+0000","last_became_peered":"2023-11-24T08:06:56.663463+0000","last_unstale":"2023-11-24T08:06:58.863466+0000","last_undegraded":"2023-11-24T08:06:58.863466+0000","last_fullsized":"2023-11-24T08:06:58.863466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:37:48.366130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656347+0000","last_change":"2023-11-24T08:06:57.677823+0000","last_active":"2023-11-24T08:06:58.656347+0000","last_peered":"2023-11-24T08:06:58.656347+0000","last_clean":"2023-11-24T08:06:58.656347+0000","last_became_active":"2023-11-24T08:06:57.677558+0000","last_became_peered":"2023-11-24T08:06:57.677558+0000","last_unstale":"2023-11-24T08:06:58.656347+0000","last_undegraded":"2023-11-24T08:06:58.656347+0000","last_fullsized":"2023-11-24T08:06:58.656347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:26:52.615885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654707+0000","last_change":"2023-11-24T08:06:56.663489+0000","last_active":"2023-11-24T08:06:58.654707+0000","last_peered":"2023-11-24T08:06:58.654707+0000","last_clean":"2023-11-24T08:06:58.654707+0000","last_became_active":"2023-11-24T08:06:56.660710+0000","last_became_peered":"2023-11-24T08:06:56.660710+0000","last_unstale":"2023-11-24T08:06:58.654707+0000","last_undegraded":"2023-11-24T08:06:58.654707+0000","last_fullsized":"2023-11-24T08:06:58.654707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:07:41.041665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656048+0000","last_change":"2023-11-24T08:06:57.681172+0000","last_active":"2023-11-24T08:06:58.656048+0000","last_peered":"2023-11-24T08:06:58.656048+0000","last_clean":"2023-11-24T08:06:58.656048+0000","last_became_active":"2023-11-24T08:06:57.680901+0000","last_became_peered":"2023-11-24T08:06:57.680901+0000","last_unstale":"2023-11-24T08:06:58.656048+0000","last_undegraded":"2023-11-24T08:06:58.656048+0000","last_fullsized":"2023-11-24T08:06:58.656048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:04.581597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863148+0000","last_change":"2023-11-24T08:06:56.652955+0000","last_active":"2023-11-24T08:06:58.863148+0000","last_peered":"2023-11-24T08:06:58.863148+0000","last_clean":"2023-11-24T08:06:58.863148+0000","last_became_active":"2023-11-24T08:06:56.652182+0000","last_became_peered":"2023-11-24T08:06:56.652182+0000","last_unstale":"2023-11-24T08:06:58.863148+0000","last_undegraded":"2023-11-24T08:06:58.863148+0000","last_fullsized":"2023-11-24T08:06:58.863148+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:18.679075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865816+0000","last_change":"2023-11-24T08:06:57.680536+0000","last_active":"2023-11-24T08:06:58.865816+0000","last_peered":"2023-11-24T08:06:58.865816+0000","last_clean":"2023-11-24T08:06:58.865816+0000","last_became_active":"2023-11-24T08:06:57.680137+0000","last_became_peered":"2023-11-24T08:06:57.680137+0000","last_unstale":"2023-11-24T08:06:58.865816+0000","last_undegraded":"2023-11-24T08:06:58.865816+0000","last_fullsized":"2023-11-24T08:06:58.865816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:16.062829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862430+0000","last_change":"2023-11-24T08:06:56.666202+0000","last_active":"2023-11-24T08:06:58.862430+0000","last_peered":"2023-11-24T08:06:58.862430+0000","last_clean":"2023-11-24T08:06:58.862430+0000","last_became_active":"2023-11-24T08:06:56.665818+0000","last_became_peered":"2023-11-24T08:06:56.665818+0000","last_unstale":"2023-11-24T08:06:58.862430+0000","last_undegraded":"2023-11-24T08:06:58.862430+0000","last_fullsized":"2023-11-24T08:06:58.862430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:45.563720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655904+0000","last_change":"2023-11-24T08:06:57.684827+0000","last_active":"2023-11-24T08:06:58.655904+0000","last_peered":"2023-11-24T08:06:58.655904+0000","last_clean":"2023-11-24T08:06:58.655904+0000","last_became_active":"2023-11-24T08:06:57.684287+0000","last_became_peered":"2023-11-24T08:06:57.684287+0000","last_unstale":"2023-11-24T08:06:58.655904+0000","last_undegraded":"2023-11-24T08:06:58.655904+0000","last_fullsized":"2023-11-24T08:06:58.655904+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:51.211107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654364+0000","last_change":"2023-11-24T08:06:56.664389+0000","last_active":"2023-11-24T08:06:58.654364+0000","last_peered":"2023-11-24T08:06:58.654364+0000","last_clean":"2023-11-24T08:06:58.654364+0000","last_became_active":"2023-11-24T08:06:56.662305+0000","last_became_peered":"2023-11-24T08:06:56.662305+0000","last_unstale":"2023-11-24T08:06:58.654364+0000","last_undegraded":"2023-11-24T08:06:58.654364+0000","last_fullsized":"2023-11-24T08:06:58.654364+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:15.048677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655610+0000","last_change":"2023-11-24T08:06:57.683358+0000","last_active":"2023-11-24T08:06:58.655610+0000","last_peered":"2023-11-24T08:06:58.655610+0000","last_clean":"2023-11-24T08:06:58.655610+0000","last_became_active":"2023-11-24T08:06:57.682944+0000","last_became_peered":"2023-11-24T08:06:57.682944+0000","last_unstale":"2023-11-24T08:06:58.655610+0000","last_undegraded":"2023-11-24T08:06:58.655610+0000","last_fullsized":"2023-11-24T08:06:58.655610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:30:58.023042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864648+0000","last_change":"2023-11-24T08:06:56.657043+0000","last_active":"2023-11-24T08:06:58.864648+0000","last_peered":"2023-11-24T08:06:58.864648+0000","last_clean":"2023-11-24T08:06:58.864648+0000","last_became_active":"2023-11-24T08:06:56.651593+0000","last_became_peered":"2023-11-24T08:06:56.651593+0000","last_unstale":"2023-11-24T08:06:58.864648+0000","last_undegraded":"2023-11-24T08:06:58.864648+0000","last_fullsized":"2023-11-24T08:06:58.864648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:31.061959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865208+0000","last_change":"2023-11-24T08:06:57.691550+0000","last_active":"2023-11-24T08:06:58.865208+0000","last_peered":"2023-11-24T08:06:58.865208+0000","last_clean":"2023-11-24T08:06:58.865208+0000","last_became_active":"2023-11-24T08:06:57.691124+0000","last_became_peered":"2023-11-24T08:06:57.691124+0000","last_unstale":"2023-11-24T08:06:58.865208+0000","last_undegraded":"2023-11-24T08:06:58.865208+0000","last_fullsized":"2023-11-24T08:06:58.865208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:13.606639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657471+0000","last_change":"2023-11-24T08:06:56.655928+0000","last_active":"2023-11-24T08:06:58.657471+0000","last_peered":"2023-11-24T08:06:58.657471+0000","last_clean":"2023-11-24T08:06:58.657471+0000","last_became_active":"2023-11-24T08:06:56.655584+0000","last_became_peered":"2023-11-24T08:06:56.655584+0000","last_unstale":"2023-11-24T08:06:58.657471+0000","last_undegraded":"2023-11-24T08:06:58.657471+0000","last_fullsized":"2023-11-24T08:06:58.657471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:43.549985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862464+0000","last_change":"2023-11-24T08:06:57.684155+0000","last_active":"2023-11-24T08:06:58.862464+0000","last_peered":"2023-11-24T08:06:58.862464+0000","last_clean":"2023-11-24T08:06:58.862464+0000","last_became_active":"2023-11-24T08:06:57.679755+0000","last_became_peered":"2023-11-24T08:06:57.679755+0000","last_unstale":"2023-11-24T08:06:58.862464+0000","last_undegraded":"2023-11-24T08:06:58.862464+0000","last_fullsized":"2023-11-24T08:06:58.862464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:54.423943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864390+0000","last_change":"2023-11-24T08:06:56.659865+0000","last_active":"2023-11-24T08:06:58.864390+0000","last_peered":"2023-11-24T08:06:58.864390+0000","last_clean":"2023-11-24T08:06:58.864390+0000","last_became_active":"2023-11-24T08:06:56.659553+0000","last_became_peered":"2023-11-24T08:06:56.659553+0000","last_unstale":"2023-11-24T08:06:58.864390+0000","last_undegraded":"2023-11-24T08:06:58.864390+0000","last_fullsized":"2023-11-24T08:06:58.864390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:35.180063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864132+0000","last_change":"2023-11-24T08:06:57.691601+0000","last_active":"2023-11-24T08:06:58.864132+0000","last_peered":"2023-11-24T08:06:58.864132+0000","last_clean":"2023-11-24T08:06:58.864132+0000","last_became_active":"2023-11-24T08:06:57.691006+0000","last_became_peered":"2023-11-24T08:06:57.691006+0000","last_unstale":"2023-11-24T08:06:58.864132+0000","last_undegraded":"2023-11-24T08:06:58.864132+0000","last_fullsized":"2023-11-24T08:06:58.864132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:13:54.849451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655463+0000","last_change":"2023-11-24T08:06:56.661990+0000","last_active":"2023-11-24T08:06:58.655463+0000","last_peered":"2023-11-24T08:06:58.655463+0000","last_clean":"2023-11-24T08:06:58.655463+0000","last_became_active":"2023-11-24T08:06:56.661658+0000","last_became_peered":"2023-11-24T08:06:56.661658+0000","last_unstale":"2023-11-24T08:06:58.655463+0000","last_undegraded":"2023-11-24T08:06:58.655463+0000","last_fullsized":"2023-11-24T08:06:58.655463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:02.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862517+0000","last_change":"2023-11-24T08:06:57.693003+0000","last_active":"2023-11-24T08:06:58.862517+0000","last_peered":"2023-11-24T08:06:58.862517+0000","last_clean":"2023-11-24T08:06:58.862517+0000","last_became_active":"2023-11-24T08:06:57.692666+0000","last_became_peered":"2023-11-24T08:06:57.692666+0000","last_unstale":"2023-11-24T08:06:58.862517+0000","last_undegraded":"2023-11-24T08:06:58.862517+0000","last_fullsized":"2023-11-24T08:06:58.862517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:41.456025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656225+0000","last_change":"2023-11-24T08:06:56.655748+0000","last_active":"2023-11-24T08:06:58.656225+0000","last_peered":"2023-11-24T08:06:58.656225+0000","last_clean":"2023-11-24T08:06:58.656225+0000","last_became_active":"2023-11-24T08:06:56.655470+0000","last_became_peered":"2023-11-24T08:06:56.655470+0000","last_unstale":"2023-11-24T08:06:58.656225+0000","last_undegraded":"2023-11-24T08:06:58.656225+0000","last_fullsized":"2023-11-24T08:06:58.656225+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:12:32.037053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864835+0000","last_change":"2023-11-24T08:06:57.683864+0000","last_active":"2023-11-24T08:06:58.864835+0000","last_peered":"2023-11-24T08:06:58.864835+0000","last_clean":"2023-11-24T08:06:58.864835+0000","last_became_active":"2023-11-24T08:06:57.683480+0000","last_became_peered":"2023-11-24T08:06:57.683480+0000","last_unstale":"2023-11-24T08:06:58.864835+0000","last_undegraded":"2023-11-24T08:06:58.864835+0000","last_fullsized":"2023-11-24T08:06:58.864835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:24.560556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865849+0000","last_change":"2023-11-24T08:06:56.657014+0000","last_active":"2023-11-24T08:06:58.865849+0000","last_peered":"2023-11-24T08:06:58.865849+0000","last_clean":"2023-11-24T08:06:58.865849+0000","last_became_active":"2023-11-24T08:06:56.656654+0000","last_became_peered":"2023-11-24T08:06:56.656654+0000","last_unstale":"2023-11-24T08:06:58.865849+0000","last_undegraded":"2023-11-24T08:06:58.865849+0000","last_fullsized":"2023-11-24T08:06:58.865849+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:15.370260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654372+0000","last_change":"2023-11-24T08:06:57.677347+0000","last_active":"2023-11-24T08:06:58.654372+0000","last_peered":"2023-11-24T08:06:58.654372+0000","last_clean":"2023-11-24T08:06:58.654372+0000","last_became_active":"2023-11-24T08:06:57.677048+0000","last_became_peered":"2023-11-24T08:06:57.677048+0000","last_unstale":"2023-11-24T08:06:58.654372+0000","last_undegraded":"2023-11-24T08:06:58.654372+0000","last_fullsized":"2023-11-24T08:06:58.654372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:37:49.380283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862595+0000","last_change":"2023-11-24T08:06:56.665018+0000","last_active":"2023-11-24T08:06:58.862595+0000","last_peered":"2023-11-24T08:06:58.862595+0000","last_clean":"2023-11-24T08:06:58.862595+0000","last_became_active":"2023-11-24T08:06:56.664737+0000","last_became_peered":"2023-11-24T08:06:56.664737+0000","last_unstale":"2023-11-24T08:06:58.862595+0000","last_undegraded":"2023-11-24T08:06:58.862595+0000","last_fullsized":"2023-11-24T08:06:58.862595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:56.532423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863621+0000","last_change":"2023-11-24T08:06:57.681653+0000","last_active":"2023-11-24T08:06:58.863621+0000","last_peered":"2023-11-24T08:06:58.863621+0000","last_clean":"2023-11-24T08:06:58.863621+0000","last_became_active":"2023-11-24T08:06:57.680759+0000","last_became_peered":"2023-11-24T08:06:57.680759+0000","last_unstale":"2023-11-24T08:06:58.863621+0000","last_undegraded":"2023-11-24T08:06:58.863621+0000","last_fullsized":"2023-11-24T08:06:58.863621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:56:24.479991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.866719+0000","last_change":"2023-11-24T08:06:56.661302+0000","last_active":"2023-11-24T08:06:58.866719+0000","last_peered":"2023-11-24T08:06:58.866719+0000","last_clean":"2023-11-24T08:06:58.866719+0000","last_became_active":"2023-11-24T08:06:56.661006+0000","last_became_peered":"2023-11-24T08:06:56.661006+0000","last_unstale":"2023-11-24T08:06:58.866719+0000","last_undegraded":"2023-11-24T08:06:58.866719+0000","last_fullsized":"2023-11-24T08:06:58.866719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:56.532423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864763+0000","last_change":"2023-11-24T08:06:57.681801+0000","last_active":"2023-11-24T08:06:58.864763+0000","last_peered":"2023-11-24T08:06:58.864763+0000","last_clean":"2023-11-24T08:06:58.864763+0000","last_became_active":"2023-11-24T08:06:57.681496+0000","last_became_peered":"2023-11-24T08:06:57.681496+0000","last_unstale":"2023-11-24T08:06:58.864763+0000","last_undegraded":"2023-11-24T08:06:58.864763+0000","last_fullsized":"2023-11-24T08:06:58.864763+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:19.693227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657041+0000","last_change":"2023-11-24T08:06:56.657212+0000","last_active":"2023-11-24T08:06:58.657041+0000","last_peered":"2023-11-24T08:06:58.657041+0000","last_clean":"2023-11-24T08:06:58.657041+0000","last_became_active":"2023-11-24T08:06:56.655754+0000","last_became_peered":"2023-11-24T08:06:56.655754+0000","last_unstale":"2023-11-24T08:06:58.657041+0000","last_undegraded":"2023-11-24T08:06:58.657041+0000","last_fullsized":"2023-11-24T08:06:58.657041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:47:32.965405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865190+0000","last_change":"2023-11-24T08:06:57.691660+0000","last_active":"2023-11-24T08:06:58.865190+0000","last_peered":"2023-11-24T08:06:58.865190+0000","last_clean":"2023-11-24T08:06:58.865190+0000","last_became_active":"2023-11-24T08:06:57.691145+0000","last_became_peered":"2023-11-24T08:06:57.691145+0000","last_unstale":"2023-11-24T08:06:58.865190+0000","last_undegraded":"2023-11-24T08:06:58.865190+0000","last_fullsized":"2023-11-24T08:06:58.865190+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:47.509181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863949+0000","last_change":"2023-11-24T08:06:56.652436+0000","last_active":"2023-11-24T08:06:58.863949+0000","last_peered":"2023-11-24T08:06:58.863949+0000","last_clean":"2023-11-24T08:06:58.863949+0000","last_became_active":"2023-11-24T08:06:56.649633+0000","last_became_peered":"2023-11-24T08:06:56.649633+0000","last_unstale":"2023-11-24T08:06:58.863949+0000","last_undegraded":"2023-11-24T08:06:58.863949+0000","last_fullsized":"2023-11-24T08:06:58.863949+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:55.024911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.657000+0000","last_change":"2023-11-24T08:06:57.668846+0000","last_active":"2023-11-24T08:06:58.657000+0000","last_peered":"2023-11-24T08:06:58.657000+0000","last_clean":"2023-11-24T08:06:58.657000+0000","last_became_active":"2023-11-24T08:06:57.668509+0000","last_became_peered":"2023-11-24T08:06:57.668509+0000","last_unstale":"2023-11-24T08:06:58.657000+0000","last_undegraded":"2023-11-24T08:06:58.657000+0000","last_fullsized":"2023-11-24T08:06:58.657000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:55.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863083+0000","last_change":"2023-11-24T08:06:56.662000+0000","last_active":"2023-11-24T08:06:58.863083+0000","last_peered":"2023-11-24T08:06:58.863083+0000","last_clean":"2023-11-24T08:06:58.863083+0000","last_became_active":"2023-11-24T08:06:56.661686+0000","last_became_peered":"2023-11-24T08:06:56.661686+0000","last_unstale":"2023-11-24T08:06:58.863083+0000","last_undegraded":"2023-11-24T08:06:58.863083+0000","last_fullsized":"2023-11-24T08:06:58.863083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:33.444110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864288+0000","last_change":"2023-11-24T08:06:57.691344+0000","last_active":"2023-11-24T08:06:58.864288+0000","last_peered":"2023-11-24T08:06:58.864288+0000","last_clean":"2023-11-24T08:06:58.864288+0000","last_became_active":"2023-11-24T08:06:57.691023+0000","last_became_peered":"2023-11-24T08:06:57.691023+0000","last_unstale":"2023-11-24T08:06:58.864288+0000","last_undegraded":"2023-11-24T08:06:58.864288+0000","last_fullsized":"2023-11-24T08:06:58.864288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:10.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864456+0000","last_change":"2023-11-24T08:06:56.660339+0000","last_active":"2023-11-24T08:06:58.864456+0000","last_peered":"2023-11-24T08:06:58.864456+0000","last_clean":"2023-11-24T08:06:58.864456+0000","last_became_active":"2023-11-24T08:06:56.660069+0000","last_became_peered":"2023-11-24T08:06:56.660069+0000","last_unstale":"2023-11-24T08:06:58.864456+0000","last_undegraded":"2023-11-24T08:06:58.864456+0000","last_fullsized":"2023-11-24T08:06:58.864456+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:23.546403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862902+0000","last_change":"2023-11-24T08:06:57.692531+0000","last_active":"2023-11-24T08:06:58.862902+0000","last_peered":"2023-11-24T08:06:58.862902+0000","last_clean":"2023-11-24T08:06:58.862902+0000","last_became_active":"2023-11-24T08:06:57.691273+0000","last_became_peered":"2023-11-24T08:06:57.691273+0000","last_unstale":"2023-11-24T08:06:58.862902+0000","last_undegraded":"2023-11-24T08:06:58.862902+0000","last_fullsized":"2023-11-24T08:06:58.862902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:30:58.023042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656680+0000","last_change":"2023-11-24T08:06:56.657959+0000","last_active":"2023-11-24T08:06:58.656680+0000","last_peered":"2023-11-24T08:06:58.656680+0000","last_clean":"2023-11-24T08:06:58.656680+0000","last_became_active":"2023-11-24T08:06:56.657664+0000","last_became_peered":"2023-11-24T08:06:56.657664+0000","last_unstale":"2023-11-24T08:06:58.656680+0000","last_undegraded":"2023-11-24T08:06:58.656680+0000","last_fullsized":"2023-11-24T08:06:58.656680+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:33.272202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863932+0000","last_change":"2023-11-24T08:06:57.686742+0000","last_active":"2023-11-24T08:06:58.863932+0000","last_peered":"2023-11-24T08:06:58.863932+0000","last_clean":"2023-11-24T08:06:58.863932+0000","last_became_active":"2023-11-24T08:06:57.684969+0000","last_became_peered":"2023-11-24T08:06:57.684969+0000","last_unstale":"2023-11-24T08:06:58.863932+0000","last_undegraded":"2023-11-24T08:06:58.863932+0000","last_fullsized":"2023-11-24T08:06:58.863932+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:38:17.382639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862597+0000","last_change":"2023-11-24T08:06:56.644142+0000","last_active":"2023-11-24T08:06:58.862597+0000","last_peered":"2023-11-24T08:06:58.862597+0000","last_clean":"2023-11-24T08:06:58.862597+0000","last_became_active":"2023-11-24T08:06:56.643821+0000","last_became_peered":"2023-11-24T08:06:56.643821+0000","last_unstale":"2023-11-24T08:06:58.862597+0000","last_undegraded":"2023-11-24T08:06:58.862597+0000","last_fullsized":"2023-11-24T08:06:58.862597+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:22:33.775265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862928+0000","last_change":"2023-11-24T08:06:57.684651+0000","last_active":"2023-11-24T08:06:58.862928+0000","last_peered":"2023-11-24T08:06:58.862928+0000","last_clean":"2023-11-24T08:06:58.862928+0000","last_became_active":"2023-11-24T08:06:57.684332+0000","last_became_peered":"2023-11-24T08:06:57.684332+0000","last_unstale":"2023-11-24T08:06:58.862928+0000","last_undegraded":"2023-11-24T08:06:58.862928+0000","last_fullsized":"2023-11-24T08:06:58.862928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:44.564138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862311+0000","last_change":"2023-11-24T08:06:56.660500+0000","last_active":"2023-11-24T08:06:58.862311+0000","last_peered":"2023-11-24T08:06:58.862311+0000","last_clean":"2023-11-24T08:06:58.862311+0000","last_became_active":"2023-11-24T08:06:56.660210+0000","last_became_peered":"2023-11-24T08:06:56.660210+0000","last_unstale":"2023-11-24T08:06:58.862311+0000","last_undegraded":"2023-11-24T08:06:58.862311+0000","last_fullsized":"2023-11-24T08:06:58.862311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:02.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863439+0000","last_change":"2023-11-24T08:06:57.678815+0000","last_active":"2023-11-24T08:06:58.863439+0000","last_peered":"2023-11-24T08:06:58.863439+0000","last_clean":"2023-11-24T08:06:58.863439+0000","last_became_active":"2023-11-24T08:06:57.678451+0000","last_became_peered":"2023-11-24T08:06:57.678451+0000","last_unstale":"2023-11-24T08:06:58.863439+0000","last_undegraded":"2023-11-24T08:06:58.863439+0000","last_fullsized":"2023-11-24T08:06:58.863439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:36.194216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863114+0000","last_change":"2023-11-24T08:06:56.660878+0000","last_active":"2023-11-24T08:06:58.863114+0000","last_peered":"2023-11-24T08:06:58.863114+0000","last_clean":"2023-11-24T08:06:58.863114+0000","last_became_active":"2023-11-24T08:06:56.660244+0000","last_became_peered":"2023-11-24T08:06:56.660244+0000","last_unstale":"2023-11-24T08:06:58.863114+0000","last_undegraded":"2023-11-24T08:06:58.863114+0000","last_fullsized":"2023-11-24T08:06:58.863114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:59:02.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865281+0000","last_change":"2023-11-24T08:06:57.684736+0000","last_active":"2023-11-24T08:06:58.865281+0000","last_peered":"2023-11-24T08:06:58.865281+0000","last_clean":"2023-11-24T08:06:58.865281+0000","last_became_active":"2023-11-24T08:06:57.684467+0000","last_became_peered":"2023-11-24T08:06:57.684467+0000","last_unstale":"2023-11-24T08:06:58.865281+0000","last_undegraded":"2023-11-24T08:06:58.865281+0000","last_fullsized":"2023-11-24T08:06:58.865281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:32.076112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864037+0000","last_change":"2023-11-24T08:06:56.657246+0000","last_active":"2023-11-24T08:06:58.864037+0000","last_peered":"2023-11-24T08:06:58.864037+0000","last_clean":"2023-11-24T08:06:58.864037+0000","last_became_active":"2023-11-24T08:06:56.656732+0000","last_became_peered":"2023-11-24T08:06:56.656732+0000","last_unstale":"2023-11-24T08:06:58.864037+0000","last_undegraded":"2023-11-24T08:06:58.864037+0000","last_fullsized":"2023-11-24T08:06:58.864037+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:12.592486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862822+0000","last_change":"2023-11-24T08:06:57.670421+0000","last_active":"2023-11-24T08:06:58.862822+0000","last_peered":"2023-11-24T08:06:58.862822+0000","last_clean":"2023-11-24T08:06:58.862822+0000","last_became_active":"2023-11-24T08:06:57.670007+0000","last_became_peered":"2023-11-24T08:06:57.670007+0000","last_unstale":"2023-11-24T08:06:58.862822+0000","last_undegraded":"2023-11-24T08:06:58.862822+0000","last_fullsized":"2023-11-24T08:06:58.862822+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:02.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.866399+0000","last_change":"2023-11-24T08:06:56.659180+0000","last_active":"2023-11-24T08:06:58.866399+0000","last_peered":"2023-11-24T08:06:58.866399+0000","last_clean":"2023-11-24T08:06:58.866399+0000","last_became_active":"2023-11-24T08:06:56.656913+0000","last_became_peered":"2023-11-24T08:06:56.656913+0000","last_unstale":"2023-11-24T08:06:58.866399+0000","last_undegraded":"2023-11-24T08:06:58.866399+0000","last_fullsized":"2023-11-24T08:06:58.866399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:45:42.861609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.864788+0000","last_change":"2023-11-24T08:06:57.682924+0000","last_active":"2023-11-24T08:06:58.864788+0000","last_peered":"2023-11-24T08:06:58.864788+0000","last_clean":"2023-11-24T08:06:58.864788+0000","last_became_active":"2023-11-24T08:06:57.681434+0000","last_became_peered":"2023-11-24T08:06:57.681434+0000","last_unstale":"2023-11-24T08:06:58.864788+0000","last_undegraded":"2023-11-24T08:06:58.864788+0000","last_fullsized":"2023-11-24T08:06:58.864788+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:51.211107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.863860+0000","last_change":"2023-11-24T08:06:56.666196+0000","last_active":"2023-11-24T08:06:58.863860+0000","last_peered":"2023-11-24T08:06:58.863860+0000","last_clean":"2023-11-24T08:06:58.863860+0000","last_became_active":"2023-11-24T08:06:56.665789+0000","last_became_peered":"2023-11-24T08:06:56.665789+0000","last_unstale":"2023-11-24T08:06:58.863860+0000","last_undegraded":"2023-11-24T08:06:58.863860+0000","last_fullsized":"2023-11-24T08:06:58.863860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:33:11.593756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.656456+0000","last_change":"2023-11-24T08:06:57.686831+0000","last_active":"2023-11-24T08:06:58.656456+0000","last_peered":"2023-11-24T08:06:58.656456+0000","last_clean":"2023-11-24T08:06:58.656456+0000","last_became_active":"2023-11-24T08:06:57.682593+0000","last_became_peered":"2023-11-24T08:06:57.682593+0000","last_unstale":"2023-11-24T08:06:58.656456+0000","last_undegraded":"2023-11-24T08:06:58.656456+0000","last_fullsized":"2023-11-24T08:06:58.656456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:25.153836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862036+0000","last_change":"2023-11-24T08:06:56.661053+0000","last_active":"2023-11-24T08:06:58.862036+0000","last_peered":"2023-11-24T08:06:58.862036+0000","last_clean":"2023-11-24T08:06:58.862036+0000","last_became_active":"2023-11-24T08:06:56.660773+0000","last_became_peered":"2023-11-24T08:06:56.660773+0000","last_unstale":"2023-11-24T08:06:58.862036+0000","last_undegraded":"2023-11-24T08:06:58.862036+0000","last_fullsized":"2023-11-24T08:06:58.862036+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:05.087900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.865193+0000","last_change":"2023-11-24T08:06:57.685706+0000","last_active":"2023-11-24T08:06:58.865193+0000","last_peered":"2023-11-24T08:06:58.865193+0000","last_clean":"2023-11-24T08:06:58.865193+0000","last_became_active":"2023-11-24T08:06:57.685417+0000","last_became_peered":"2023-11-24T08:06:57.685417+0000","last_unstale":"2023-11-24T08:06:58.865193+0000","last_undegraded":"2023-11-24T08:06:58.865193+0000","last_fullsized":"2023-11-24T08:06:58.865193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:56.039063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.654870+0000","last_change":"2023-11-24T08:06:56.659791+0000","last_active":"2023-11-24T08:06:58.654870+0000","last_peered":"2023-11-24T08:06:58.654870+0000","last_clean":"2023-11-24T08:06:58.654870+0000","last_became_active":"2023-11-24T08:06:56.659515+0000","last_became_peered":"2023-11-24T08:06:56.659515+0000","last_unstale":"2023-11-24T08:06:58.654870+0000","last_undegraded":"2023-11-24T08:06:58.654870+0000","last_fullsized":"2023-11-24T08:06:58.654870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:28.422591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862139+0000","last_change":"2023-11-24T08:06:57.686058+0000","last_active":"2023-11-24T08:06:58.862139+0000","last_peered":"2023-11-24T08:06:58.862139+0000","last_clean":"2023-11-24T08:06:58.862139+0000","last_became_active":"2023-11-24T08:06:57.685721+0000","last_became_peered":"2023-11-24T08:06:57.685721+0000","last_unstale":"2023-11-24T08:06:58.862139+0000","last_undegraded":"2023-11-24T08:06:58.862139+0000","last_fullsized":"2023-11-24T08:06:58.862139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:38:52.500159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.862225+0000","last_change":"2023-11-24T08:06:56.647691+0000","last_active":"2023-11-24T08:06:58.862225+0000","last_peered":"2023-11-24T08:06:58.862225+0000","last_clean":"2023-11-24T08:06:58.862225+0000","last_became_active":"2023-11-24T08:06:56.647413+0000","last_became_peered":"2023-11-24T08:06:56.647413+0000","last_unstale":"2023-11-24T08:06:58.862225+0000","last_undegraded":"2023-11-24T08:06:58.862225+0000","last_fullsized":"2023-11-24T08:06:58.862225+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:38.336330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655770+0000","last_change":"2023-11-24T08:06:57.678491+0000","last_active":"2023-11-24T08:06:58.655770+0000","last_peered":"2023-11-24T08:06:58.655770+0000","last_clean":"2023-11-24T08:06:58.655770+0000","last_became_active":"2023-11-24T08:06:57.676817+0000","last_became_peered":"2023-11-24T08:06:57.676817+0000","last_unstale":"2023-11-24T08:06:58.655770+0000","last_undegraded":"2023-11-24T08:06:58.655770+0000","last_fullsized":"2023-11-24T08:06:58.655770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:35.713574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655822+0000","last_change":"2023-11-24T08:06:56.662280+0000","last_active":"2023-11-24T08:06:58.655822+0000","last_peered":"2023-11-24T08:06:58.655822+0000","last_clean":"2023-11-24T08:06:58.655822+0000","last_became_active":"2023-11-24T08:06:56.661921+0000","last_became_peered":"2023-11-24T08:06:56.661921+0000","last_unstale":"2023-11-24T08:06:58.655822+0000","last_undegraded":"2023-11-24T08:06:58.655822+0000","last_fullsized":"2023-11-24T08:06:58.655822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:43.312572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.655694+0000","last_change":"2023-11-24T08:06:57.679668+0000","last_active":"2023-11-24T08:06:58.655694+0000","last_peered":"2023-11-24T08:06:58.655694+0000","last_clean":"2023-11-24T08:06:58.655694+0000","last_became_active":"2023-11-24T08:06:57.679410+0000","last_became_peered":"2023-11-24T08:06:57.679410+0000","last_unstale":"2023-11-24T08:06:58.655694+0000","last_undegraded":"2023-11-24T08:06:58.655694+0000","last_fullsized":"2023-11-24T08:06:58.655694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:56.618243+0000","last_clean_scrub_stamp":"2023-11-24T08:06:56.618243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:26.741577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T08:06:58.866265+0000","last_change":"2023-11-24T08:06:56.663050+0000","last_active":"2023-11-24T08:06:58.866265+0000","last_peered":"2023-11-24T08:06:58.866265+0000","last_clean":"2023-11-24T08:06:58.866265+0000","last_became_active":"2023-11-24T08:06:56.662780+0000","last_became_peered":"2023-11-24T08:06:56.662780+0000","last_unstale":"2023-11-24T08:06:58.866265+0000","last_undegraded":"2023-11-24T08:06:58.866265+0000","last_fullsized":"2023-11-24T08:06:58.866265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:06:55.604090+0000","last_clean_scrub_stamp":"2023-11-24T08:06:55.604090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:16:37.636592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T08:06:57.663376+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:06:57.663376+0000","last_peered":"2023-11-24T08:06:57.663376+0000","last_clean":"2023-11-24T08:06:57.663376+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:06:57.663376+0000","last_undegraded":"2023-11-24T08:06:57.663376+0000","last_fullsized":"2023-11-24T08:06:57.663376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1310720,"data_stored":887014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0860000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.484,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2789999999999999,"15min":1.3009999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6639999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.099},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3420000000000001,"15min":1.3839999999999999},"min":{"1min":0.97299999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5569999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.6080000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.151,"15min":1.1479999999999999},"min":{"1min":0.70099999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0840000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.728},{"interface":"front","average":{"1min":1.268,"5min":1.3069999999999999,"15min":1.3160000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.91000000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2829999999999999,"15min":1.321},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.966,"5min":2.524,"15min":2.524},"last":1.8859999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.454,"15min":1.5169999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8939999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.323}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4550000000000001,"15min":1.532},"min":{"1min":0.88400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6419999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.379},"min":{"1min":0.84499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.544}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.581,"15min":1.619},"min":{"1min":1.0229999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.173,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.601,"15min":1.661},"min":{"1min":1.069,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7350000000000001,"5min":2.944,"15min":2.944},"last":1.6890000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.702,"15min":1.7589999999999999},"min":{"1min":1.252,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.649,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.79},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6060000000000001,"15min":1.665},"min":{"1min":1.0509999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.841,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.962}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":296486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.276,"15min":1.3300000000000001},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.113,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.274,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.22}]},{"osd":1,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1539999999999999,"15min":1.159},"min":{"1min":0.68400000000000005,"5min":0.625,"15min":0.625},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2370000000000001,"15min":1.3069999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6830000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.524}]},{"osd":2,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.232,"15min":1.296},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.508,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.419,"15min":1.5069999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.752,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.96799999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.3879999999999999,"15min":1.444},"min":{"1min":0.79200000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.931,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.4690000000000001,"15min":1.498},"min":{"1min":0.94899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.968,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.4630000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.6279999999999999,"15min":1.758},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2120000000000002,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.4390000000000001,"15min":1.472},"min":{"1min":0.86199999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.855,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.552}]},{"osd":5,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6679999999999999,"15min":1.8080000000000001},"min":{"1min":1.0029999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.893,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.806,"15min":1.9490000000000001},"min":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.901,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.609}]},{"osd":7,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6379999999999999,"15min":1.6890000000000001},"min":{"1min":0.75800000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.379,"5min":2.98,"15min":2.98},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.1659999999999999,"5min":2.29,"15min":2.29},"last":1.2789999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":716800,"data_stored":297138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.048},"min":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2949999999999999,"5min":1.764,"15min":1.764},"last":1.046},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0329999999999999,"15min":1.006},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.3240000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6950000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0780000000000001,"15min":1.0509999999999999},"min":{"1min":0.70899999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.458,"5min":1.635,"15min":1.635},"last":0.95099999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2090000000000001,"15min":1.2130000000000001},"min":{"1min":0.77100000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.673,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.085},{"interface":"front","average":{"1min":1.226,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.745,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6259999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.3660000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.427},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.575,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2470000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5669999999999999,"5min":1.889,"15min":1.889},"last":1.2450000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.284,"15min":1.298},"min":{"1min":0.79100000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.871,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.68400000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.766,"5min":2.27,"15min":2.27},"last":1.401}]},{"osd":6,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.448,"15min":1.47},"min":{"1min":0.97799999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7909999999999999,"5min":3.806,"15min":3.806},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.409,"15min":1.4079999999999999},"min":{"1min":0.84699999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0169999999999999,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.278}]},{"osd":7,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5189999999999999,"15min":1.5269999999999999},"min":{"1min":0.63,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.974,"5min":2.923,"15min":2.923},"last":1.167},{"interface":"front","average":{"1min":1.569,"5min":1.54,"15min":1.53},"min":{"1min":1.0640000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.214,"5min":2.859,"15min":2.859},"last":1.4279999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":720896,"data_stored":298098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0409999999999999,"15min":1.042},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4399999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.167},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.1279999999999999},"min":{"1min":0.77900000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.595,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.446}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.079,"15min":1.0840000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.377,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.111,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.79000000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5640000000000001,"5min":1.903,"15min":1.903},"last":1.403}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1890000000000001,"15min":1.2},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.589,"5min":1.823,"15min":1.823},"last":1.099},{"interface":"front","average":{"1min":1.155,"5min":1.2410000000000001,"15min":1.258},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.2969999999999999},"min":{"1min":0.81399999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7330000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3109999999999999,"15min":1.3149999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9530000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.351}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.282,"15min":1.296},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.617,"5min":1.944,"15min":1.944},"last":1.734},{"interface":"front","average":{"1min":1.26,"5min":1.3400000000000001,"15min":1.3580000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":2.012,"15min":2.012},"last":1.6619999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.377,"15min":1.3680000000000001},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4890000000000001,"15min":1.5069999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.927,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.8740000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.4330000000000001,"15min":1.431},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.768,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.554,"15min":1.5589999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.411,"5min":2.722,"15min":2.722},"last":1.827}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":302842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.82099999999999995},"min":{"1min":0.50800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.569,"5min":1.585,"15min":1.585},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.92300000000000004,"15min":0.94799999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4930000000000001,"5min":1.51,"15min":1.51},"last":1.1879999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.075,"15min":1.151},"min":{"1min":0.63100000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.835,"5min":3.246,"15min":3.246},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97399999999999998,"15min":0.96699999999999997},"min":{"1min":0.54900000000000004,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4179999999999999,"5min":1.665,"15min":1.665},"last":1.079}]},{"osd":2,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1399999999999999,"15min":1.1819999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7410000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.008},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1160000000000001,"15min":1.155},"min":{"1min":0.66800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.399,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.95899999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2450000000000001,"15min":1.264},"min":{"1min":0.73299999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.671,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.321},{"interface":"front","average":{"1min":1.292,"5min":1.355,"15min":1.3899999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.665,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.117}]},{"osd":5,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.4199999999999999,"15min":1.4590000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9430000000000001,"5min":2.395,"15min":2.395},"last":1.284},{"interface":"front","average":{"1min":1.304,"5min":1.306,"15min":1.3},"min":{"1min":0.90400000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9399999999999999,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.351}]},{"osd":6,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.421,"15min":1.4319999999999999},"min":{"1min":0.94099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8899999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3979999999999999,"15min":1.4179999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.589}]},{"osd":7,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4219999999999999,"15min":1.444},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.994,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.528,"15min":1.581},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.895,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3919999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":716800,"data_stored":296542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.417,"5min":1.524,"15min":1.524},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":1.016},"min":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5740000000000001,"5min":2.133,"15min":2.133},"last":1.2769999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1419999999999999,"15min":1.2250000000000001},"min":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8280000000000001,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0029999999999999,"15min":1.069},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4550000000000001,"5min":1.998,"15min":1.998},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.321},"min":{"1min":0.73299999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6879999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.625},{"interface":"front","average":{"1min":1.077,"5min":1.147,"15min":1.1950000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7809999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3260000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.754,"5min":0.442,"15min":0.442},"max":{"1min":1.7969999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.4359999999999999,"15min":1.502},"min":{"1min":1.018,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.758,"5min":3.161,"15min":3.161},"last":1.5720000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.345,"15min":1.393},"min":{"1min":0.77000000000000002,"5min":0.623,"15min":0.623},"max":{"1min":1.752,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.355,"15min":1.411},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.96,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.083}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.5169999999999999,"15min":1.593},"min":{"1min":0.96199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8979999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.5649999999999999},"min":{"1min":0.80900000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8520000000000001,"5min":2.702,"15min":2.702},"last":1.409}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.448,"15min":1.514},"min":{"1min":0.78200000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0419999999999998,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.373},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.5660000000000001,"15min":1.6439999999999999},"min":{"1min":0.89300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.952,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.508}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":296576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.878,"15min":0.89200000000000002},"min":{"1min":0.70799999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.228},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0029999999999999,"15min":1.0580000000000001},"min":{"1min":0.56599999999999995,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.397,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.6459999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.03},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.746,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.724},{"interface":"front","average":{"1min":1.081,"5min":1.179,"15min":1.2290000000000001},"min":{"1min":0.76400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.794,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.8959999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.2729999999999999},"min":{"1min":0.72299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.897,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.999},{"interface":"front","average":{"1min":1.091,"5min":1.177,"15min":1.232},"min":{"1min":0.66500000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.6890000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.5620000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4199999999999999,"15min":1.454},"min":{"1min":0.95799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.7909999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4259999999999999,"15min":1.4570000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0089999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.8740000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.452,"15min":1.478},"min":{"1min":0.91000000000000003,"5min":0.871,"15min":0.871},"max":{"1min":1.903,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.855}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.4319999999999999,"15min":1.458},"min":{"1min":0.83099999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.774,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.758},{"interface":"front","average":{"1min":1.373,"5min":1.4430000000000001,"15min":1.4830000000000001},"min":{"1min":0.79200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8420000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.6020000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":0.996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8700000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.55,"5min":1.5680000000000001,"15min":1.595},"min":{"1min":0.96599999999999997,"5min":0.872,"15min":0.872},"max":{"1min":1.8500000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.8280000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1327104,"data_stored":894329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.52500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":1.083},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.97799999999999998,"15min":0.94799999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.4139999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.111,"15min":1.075},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.899,"5min":3.899,"15min":3.899},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1,"15min":0.95999999999999996},"min":{"1min":0.498,"5min":0.443,"15min":0.443},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.0700000000000001,"15min":1.0409999999999999},"min":{"1min":0.627,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.169},{"interface":"front","average":{"1min":1.631,"5min":1.2729999999999999,"15min":1.2270000000000001},"min":{"1min":0.57299999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":6.3570000000000002,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.621}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.3069999999999999,"15min":1.2709999999999999},"min":{"1min":0.92000000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.137},{"interface":"front","average":{"1min":1.7070000000000001,"5min":1.5,"15min":1.482},"min":{"1min":1.022,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":3.399,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.734}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.4399999999999999,"15min":1.3899999999999999},"min":{"1min":0.88100000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":5.9939999999999998,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.77},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":1.026,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.391,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.673}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.248},{"interface":"front","average":{"1min":1.744,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":1.0840000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.879,"5min":3.879,"15min":3.879},"last":1.8879999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.8029999999999999,"5min":1.5660000000000001,"15min":1.548},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":3.4540000000000002,"5min":3.823,"15min":3.823},"last":1.833},{"interface":"front","average":{"1min":1.75,"5min":1.5209999999999999,"15min":1.4970000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.3109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:07:03.075 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:07:03.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:07:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.354+0000 7fdaea3c9640 1 Processor -- start 2023-11-24T08:07:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.354+0000 7fdaea3c9640 1 -- start start 2023-11-24T08:07:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.356+0000 7fdaea3c9640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdae40b36d0 0x7fdae40d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.356+0000 7fdaea3c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae40cef30 con 0x7fdae40d1d90 2023-11-24T08:07:03.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.356+0000 7fdaea3c9640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae40d11a0 con 0x7fdae415d340 2023-11-24T08:07:03.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.356+0000 7fdaea3c9640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae40ce320 con 0x7fdae40b36d0 2023-11-24T08:07:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.356+0000 7fdae8bc6640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdae40d1d90 0x7fdae40b3dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59616/0 (socket says 172.21.15.64:59616) 2023-11-24T08:07:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.356+0000 7fdae8bc6640 1 -- 172.21.15.64:0/1317424590 learned_addr learned my addr 172.21.15.64:0/1317424590 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.356+0000 7fdae9bc8640 1 --2- 172.21.15.64:0/1317424590 >> v2:172.21.15.91:3300/0 conn(0x7fdae40b36d0 0x7fdae40d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323285353 0 0) 0x7fdae40cef30 con 0x7fdae40d1d90 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad0003520 con 0x7fdae40d1d90 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3954579800 0 0) 0x7fdad0003520 con 0x7fdae40d1d90 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 >> v2:172.21.15.91:3300/0 conn(0x7fdae40b36d0 msgr2=0x7fdae40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 --2- 172.21.15.64:0/1317424590 >> v2:172.21.15.91:3300/0 conn(0x7fdae40b36d0 0x7fdae40d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 >> v1:172.21.15.91:6789/0 conn(0x7fdae415d340 legacy=0x7fdae40d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae40ce600 con 0x7fdae40d1d90 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fdad4001670 con 0x7fdae40d1d90 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 >> v1:172.21.15.64:6789/0 conn(0x7fdae40d1d90 legacy=0x7fdae40b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdad0003a30 0x7fdad0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdad0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdad00051d0 0x7fdad000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae40ce320 con 0x7fdad00047c0 2023-11-24T08:07:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad000c100 con 0x7fdad00051d0 2023-11-24T08:07:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.357+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad000c2c0 con 0x7fdad0003a30 2023-11-24T08:07:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae8bc6640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdad0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae93c7640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdad0003a30 0x7fdad0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae9bc8640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdad00051d0 0x7fdad000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae8bc6640 1 -- 172.21.15.64:0/1317424590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdad0003a30 msgr2=0x7fdad0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae8bc6640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdad0003a30 0x7fdad0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae8bc6640 1 -- 172.21.15.64:0/1317424590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdad00051d0 msgr2=0x7fdad000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae8bc6640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdad00051d0 0x7fdad000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae8bc6640 1 -- 172.21.15.64:0/1317424590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad0003520 con 0x7fdad00047c0 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.358+0000 7fdae9bc8640 1 -- 172.21.15.64:0/1317424590 reap_dead start 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.359+0000 7fdae8bc6640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdad0004c40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c0e0 tx=0x7fdad00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daa8bd5f8dba000 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4005ad0 con 0x7fdad00047c0 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad4005c30 con 0x7fdad00047c0 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdadbfff640 1 -- 172.21.15.64:0/1317424590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad402ca00 con 0x7fdad00047c0 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdaea3c9640 1 -- 172.21.15.64:0/1317424590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 msgr2=0x7fdad0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdaea3c9640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdad0004c40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c0e0 tx=0x7fdad00057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdaea3c9640 1 -- 172.21.15.64:0/1317424590 shutdown_connections 2023-11-24T08:07:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdaea3c9640 1 --2- 172.21.15.64:0/1317424590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdad0004c40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdaea3c9640 1 -- 172.21.15.64:0/1317424590 >> 172.21.15.64:0/1317424590 conn(0x7fdae409c370 msgr2=0x7fdae40d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdaea3c9640 1 -- 172.21.15.64:0/1317424590 shutdown_connections 2023-11-24T08:07:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.360+0000 7fdaea3c9640 1 -- 172.21.15.64:0/1317424590 wait complete. 2023-11-24T08:07:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.362+0000 7fdaea3c9640 1 Processor -- start 2023-11-24T08:07:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.362+0000 7fdaea3c9640 1 -- start start 2023-11-24T08:07:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.363+0000 7fdaea3c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdae40cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.363+0000 7fdaea3c9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdae40cc2b0 0x7fdae41452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.363+0000 7fdaea3c9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdae415f650 0x7fdae41538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.363+0000 7fdaea3c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae40cbe20 con 0x7fdad00047c0 2023-11-24T08:07:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.363+0000 7fdaea3c9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4145800 con 0x7fdae415f650 2023-11-24T08:07:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.363+0000 7fdaea3c9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae40cb210 con 0x7fdae40cc2b0 2023-11-24T08:07:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae93c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdae40cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae93c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdae40cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60048/0 (socket says 172.21.15.64:60048) 2023-11-24T08:07:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae93c7640 1 -- 172.21.15.64:0/4263801572 learned_addr learned my addr 172.21.15.64:0/4263801572 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:03.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae8bc6640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdae40cc2b0 0x7fdae41452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae9bc8640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdae415f650 0x7fdae41538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae93c7640 1 -- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdae40cc2b0 msgr2=0x7fdae41452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae93c7640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdae40cc2b0 0x7fdae41452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae93c7640 1 -- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdae415f650 msgr2=0x7fdae41538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae93c7640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdae415f650 0x7fdae41538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae93c7640 1 -- 172.21.15.64:0/4263801572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae40cb4f0 con 0x7fdad00047c0 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.364+0000 7fdae9bc8640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdae415f650 0x7fdae41538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.365+0000 7fdae93c7640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdae40cab80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e8c0 tx=0x7fdae000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.365+0000 7fdad9ffb640 1 -- 172.21.15.64:0/4263801572 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0003b80 con 0x7fdad00047c0 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.365+0000 7fdad9ffb640 1 -- 172.21.15.64:0/4263801572 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0003ce0 con 0x7fdad00047c0 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.365+0000 7fdad9ffb640 1 -- 172.21.15.64:0/4263801572 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00052f0 con 0x7fdad00047c0 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.365+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae40cb7d0 con 0x7fdad00047c0 2023-11-24T08:07:03.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.365+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae415fae0 con 0x7fdad00047c0 2023-11-24T08:07:03.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.366+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae415fc70 con 0x7fdad00047c0 2023-11-24T08:07:03.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.367+0000 7fdad9ffb640 1 -- 172.21.15.64:0/4263801572 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fdae0005450 con 0x7fdad00047c0 2023-11-24T08:07:03.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.368+0000 7fdad9ffb640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdac806b080 0x7fdac806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.368+0000 7fdad9ffb640 1 -- 172.21.15.64:0/4263801572 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdae0089ce0 con 0x7fdad00047c0 2023-11-24T08:07:03.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.369+0000 7fdae8bc6640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdac806b080 0x7fdac806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.370+0000 7fdae8bc6640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdac806b080 0x7fdac806d540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdad4034000 tx=0x7fdad4009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:03.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.373+0000 7fdad9ffb640 1 -- 172.21.15.64:0/4263801572 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdae0054900 con 0x7fdad00047c0 2023-11-24T08:07:03.611 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-11-24T08:07:03.609+0000 7fb0ed34c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:07:03.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.636+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdae4067d70 con 0x7fdad00047c0 2023-11-24T08:07:03.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.641+0000 7fdad9ffb640 1 -- 172.21.15.64:0/4263801572 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdae001a310 con 0x7fdad00047c0 2023-11-24T08:07:03.643 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-11-24T08:07:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA== 2023-11-24T08:07:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow" 2023-11-24T08:07:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow r" 2023-11-24T08:07:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:07:03.644 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-11-24T08:07:03.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.644+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdac806b080 msgr2=0x7fdac806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.644+0000 7fdaea3c9640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdac806b080 0x7fdac806d540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdad4034000 tx=0x7fdad4009030 comp rx=0 tx=0).stop 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.644+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 msgr2=0x7fdae40cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.644+0000 7fdaea3c9640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdae40cab80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e8c0 tx=0x7fdae000ed80 comp rx=0 tx=0).stop 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.645+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 shutdown_connections 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.645+0000 7fdaea3c9640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdac806b080 0x7fdac806d540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.645+0000 7fdaea3c9640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdae415f650 0x7fdae41538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.645+0000 7fdaea3c9640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdae40cc2b0 0x7fdae41452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.645+0000 7fdaea3c9640 1 --2- 172.21.15.64:0/4263801572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdad00047c0 0x7fdae40cab80 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.645+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 >> 172.21.15.64:0/4263801572 conn(0x7fdae409c370 msgr2=0x7fdae40b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:03.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.645+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 shutdown_connections 2023-11-24T08:07:03.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.645+0000 7fdaea3c9640 1 -- 172.21.15.64:0/4263801572 wait complete. 2023-11-24T08:07:03.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:03.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.879+0000 7fa6419a0640 1 Processor -- start 2023-11-24T08:07:03.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.879+0000 7fa6419a0640 1 -- start start 2023-11-24T08:07:03.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.881+0000 7fa6419a0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa63c157770 0x7fa63c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.881+0000 7fa6419a0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c161670 con 0x7fa63c06d080 2023-11-24T08:07:03.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.881+0000 7fa6419a0640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c15e610 con 0x7fa63c15df80 2023-11-24T08:07:03.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.881+0000 7fa6419a0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c15e410 con 0x7fa63c157770 2023-11-24T08:07:03.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.882+0000 7fa64099e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa63c157770 0x7fa63c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.882+0000 7fa63bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa63c06d080 0x7fa63c15d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59622/0 (socket says 172.21.15.64:59622) 2023-11-24T08:07:03.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.882+0000 7fa63bfff640 1 -- 172.21.15.64:0/911684385 learned_addr learned my addr 172.21.15.64:0/911684385 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:03.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.882+0000 7fa64099e640 1 -- 172.21.15.64:0/911684385 >> v1:172.21.15.91:6789/0 conn(0x7fa63c15df80 legacy=0x7fa63c070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:03.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.882+0000 7fa64099e640 1 -- 172.21.15.64:0/911684385 >> v1:172.21.15.64:6789/0 conn(0x7fa63c06d080 legacy=0x7fa63c15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.882+0000 7fa64099e640 1 -- 172.21.15.64:0/911684385 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c160d40 con 0x7fa63c157770 2023-11-24T08:07:03.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.882+0000 7fa63b7fe640 1 -- 172.21.15.64:0/911684385 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522049089 0 0) 0x7fa63c161670 con 0x7fa63c06d080 2023-11-24T08:07:03.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.883+0000 7fa64099e640 1 --2- 172.21.15.64:0/911684385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c157770 0x7fa63c06cb40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa6300097a0 tx=0x7fa63002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a2372d39230221 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:03.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa63b7fe640 1 -- 172.21.15.64:0/911684385 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630031830 con 0x7fa63c157770 2023-11-24T08:07:03.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa63b7fe640 1 -- 172.21.15.64:0/911684385 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630031990 con 0x7fa63c157770 2023-11-24T08:07:03.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa63b7fe640 1 -- 172.21.15.64:0/911684385 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630031d20 con 0x7fa63c157770 2023-11-24T08:07:03.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa6419a0640 1 -- 172.21.15.64:0/911684385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c157770 msgr2=0x7fa63c06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa6419a0640 1 --2- 172.21.15.64:0/911684385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c157770 0x7fa63c06cb40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa6300097a0 tx=0x7fa63002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa6419a0640 1 -- 172.21.15.64:0/911684385 shutdown_connections 2023-11-24T08:07:03.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa6419a0640 1 --2- 172.21.15.64:0/911684385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c157770 0x7fa63c06cb40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa6419a0640 1 -- 172.21.15.64:0/911684385 >> 172.21.15.64:0/911684385 conn(0x7fa63c06a860 msgr2=0x7fa63c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:03.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa6419a0640 1 -- 172.21.15.64:0/911684385 shutdown_connections 2023-11-24T08:07:03.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.884+0000 7fa6419a0640 1 -- 172.21.15.64:0/911684385 wait complete. 2023-11-24T08:07:03.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.886+0000 7fa6419a0640 1 Processor -- start 2023-11-24T08:07:03.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.886+0000 7fa6419a0640 1 -- start start 2023-11-24T08:07:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.887+0000 7fa6419a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c06d080 0x7fa63c14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.887+0000 7fa6419a0640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa63c157770 0x7fa63c15f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.887+0000 7fa6419a0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa63c15df80 0x7fa63c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.887+0000 7fa6419a0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c14acc0 con 0x7fa63c06d080 2023-11-24T08:07:03.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.887+0000 7fa6419a0640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c15be10 con 0x7fa63c157770 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.887+0000 7fa6419a0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c15c0f0 con 0x7fa63c15df80 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64099e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c06d080 0x7fa63c14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64099e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c06d080 0x7fa63c14a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60060/0 (socket says 172.21.15.64:60060) 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64099e640 1 -- 172.21.15.64:0/3388006071 learned_addr learned my addr 172.21.15.64:0/3388006071 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa63bfff640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa63c157770 0x7fa63c15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64119f640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa63c15df80 0x7fa63c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64099e640 1 -- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa63c15df80 msgr2=0x7fa63c15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64099e640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa63c15df80 0x7fa63c15fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64099e640 1 -- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa63c157770 msgr2=0x7fa63c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:03.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64099e640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa63c157770 0x7fa63c15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:03.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.888+0000 7fa64099e640 1 -- 172.21.15.64:0/3388006071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630009400 con 0x7fa63c06d080 2023-11-24T08:07:03.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.889+0000 7fa64099e640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c06d080 0x7fa63c14a780 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa6300098c0 tx=0x7fa6300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:03.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.889+0000 7fa6397fa640 1 -- 172.21.15.64:0/3388006071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63003ba70 con 0x7fa63c06d080 2023-11-24T08:07:03.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.889+0000 7fa6397fa640 1 -- 172.21.15.64:0/3388006071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63003bbd0 con 0x7fa63c06d080 2023-11-24T08:07:03.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.889+0000 7fa6397fa640 1 -- 172.21.15.64:0/3388006071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300453f0 con 0x7fa63c06d080 2023-11-24T08:07:03.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.889+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c15c3d0 con 0x7fa63c06d080 2023-11-24T08:07:03.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.889+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c15c990 con 0x7fa63c06d080 2023-11-24T08:07:03.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.890+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c04e840 con 0x7fa63c06d080 2023-11-24T08:07:03.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.891+0000 7fa6397fa640 1 -- 172.21.15.64:0/3388006071 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa63004e450 con 0x7fa63c06d080 2023-11-24T08:07:03.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.892+0000 7fa6397fa640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa62006b080 0x7fa62006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:03.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.893+0000 7fa6397fa640 1 -- 172.21.15.64:0/3388006071 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6300abec0 con 0x7fa63c06d080 2023-11-24T08:07:03.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.893+0000 7fa63bfff640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa62006b080 0x7fa62006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:03.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.894+0000 7fa63bfff640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa62006b080 0x7fa62006d540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa63c15ce40 tx=0x7fa628007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:03.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:03.897+0000 7fa6397fa640 1 -- 172.21.15.64:0/3388006071 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa630076a60 con 0x7fa63c06d080 2023-11-24T08:07:04.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.130+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa63c0cdd80 con 0x7fa63c06d080 2023-11-24T08:07:04.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.132+0000 7fa6397fa640 1 -- 172.21.15.64:0/3388006071 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fa63007a080 con 0x7fa63c06d080 2023-11-24T08:07:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:04.134 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2873504597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2873504597},{"type":"v1","addr":"172.21.15.91:6835","nonce":2873504597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3295227147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3295227147},{"type":"v1","addr":"172.21.15.91:6837","nonce":3295227147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2949237565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2949237565},{"type":"v1","addr":"172.21.15.64:6834","nonce":2949237565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:57.112228+0000","modified":"2023-11-24T08:07:00.539822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4200493009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":4200493009},{"type":"v1","addr":"172.21.15.64:6835","nonce":4200493009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:07:04.134 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 34 2023-11-24T08:07:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa62006b080 msgr2=0x7fa62006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa62006b080 0x7fa62006d540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa63c15ce40 tx=0x7fa628007c00 comp rx=0 tx=0).stop 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c06d080 msgr2=0x7fa63c14a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c06d080 0x7fa63c14a780 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa6300098c0 tx=0x7fa6300046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 shutdown_connections 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa62006b080 0x7fa62006d540 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa63c15df80 0x7fa63c15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa63c157770 0x7fa63c15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 --2- 172.21.15.64:0/3388006071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa63c06d080 0x7fa63c14a780 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 >> 172.21.15.64:0/3388006071 conn(0x7fa63c06a860 msgr2=0x7fa63c155420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 shutdown_connections 2023-11-24T08:07:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.135+0000 7fa6419a0640 1 -- 172.21.15.64:0/3388006071 wait complete. 2023-11-24T08:07:04.559 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:06:57.112228+0000', 'modified': '2023-11-24T08:07:00.539822+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6240}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6240': {'gid': 6240, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/4200493009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6833', 'nonce': 4200493009}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 4200493009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:07:04.559 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:07:04.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T08:07:04.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.784+0000 7f86affff640 1 Processor -- start 2023-11-24T08:07:04.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.785+0000 7f86affff640 1 -- start start 2023-11-24T08:07:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86affff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f86b01416f0 0x7f86b015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86affff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b006d230 0x7f86b015f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86affff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b0070bf0 con 0x7f86b0141e80 2023-11-24T08:07:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86affff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b006db30 con 0x7f86b01416f0 2023-11-24T08:07:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86affff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b006d660 con 0x7f86b006d230 2023-11-24T08:07:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86a7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f86b0141e80 0x7f86b006cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59628/0 (socket says 172.21.15.64:59628) 2023-11-24T08:07:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86a7fff640 1 -- 172.21.15.64:0/1342268997 learned_addr learned my addr 172.21.15.64:0/1342268997 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86aeffd640 1 --2- 172.21.15.64:0/1342268997 >> v2:172.21.15.91:3300/0 conn(0x7f86b01416f0 0x7f86b015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.787+0000 7f86af7fe640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b006d230 0x7f86b015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909009064 0 0) 0x7f86b0070bf0 con 0x7f86b0141e80 2023-11-24T08:07:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f869c003520 con 0x7f86b0141e80 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 794083986 0 0) 0x7f869c003520 con 0x7f86b0141e80 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b006d230 msgr2=0x7f86b015f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b006d230 0x7f86b015f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 >> v2:172.21.15.91:3300/0 conn(0x7f86b01416f0 msgr2=0x7f86b015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 --2- 172.21.15.64:0/1342268997 >> v2:172.21.15.91:3300/0 conn(0x7f86b01416f0 0x7f86b015ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b006d850 con 0x7f86b0141e80 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f8698001670 con 0x7f86b0141e80 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 >> v1:172.21.15.64:6789/0 conn(0x7f86b0141e80 legacy=0x7f86b006cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f869c0039e0 0x7f869c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b006d230 0x7f869c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c004d00 0x7f869c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b006db30 con 0x7f869c0039e0 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c00b980 con 0x7f869c004d00 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.788+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c00baf0 con 0x7f86b006d230 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.789+0000 7f86aeffd640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f869c0039e0 0x7f869c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.789+0000 7f86a7fff640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b006d230 0x7f869c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.789+0000 7f86af7fe640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c004d00 0x7f869c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.789+0000 7f86aeffd640 1 -- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b006d230 msgr2=0x7f869c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.789+0000 7f86aeffd640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b006d230 0x7f869c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.789+0000 7f86aeffd640 1 -- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c004d00 msgr2=0x7f869c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.789+0000 7f86aeffd640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c004d00 0x7f869c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.789+0000 7f86aeffd640 1 -- 172.21.15.64:0/1342268997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c003520 con 0x7f869c0039e0 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.790+0000 7f86aeffd640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f869c0039e0 0x7f869c003e00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009ac0 tx=0x7f869c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e566878bc7b64d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.790+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a002c710 con 0x7f869c0039e0 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.790+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a002c870 con 0x7f869c0039e0 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.790+0000 7f86ae7fc640 1 -- 172.21.15.64:0/1342268997 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a002cc00 con 0x7f869c0039e0 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.791+0000 7f86affff640 1 -- 172.21.15.64:0/1342268997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f869c0039e0 msgr2=0x7f869c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.791+0000 7f86affff640 1 --2- 172.21.15.64:0/1342268997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f869c0039e0 0x7f869c003e00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009ac0 tx=0x7f869c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.791+0000 7f86af7fe640 1 -- 172.21.15.64:0/1342268997 reap_dead start 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.791+0000 7f86affff640 1 -- 172.21.15.64:0/1342268997 shutdown_connections 2023-11-24T08:07:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.791+0000 7f86affff640 1 -- 172.21.15.64:0/1342268997 >> 172.21.15.64:0/1342268997 conn(0x7f86b00b0840 msgr2=0x7f86b00b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.791+0000 7f86affff640 1 -- 172.21.15.64:0/1342268997 shutdown_connections 2023-11-24T08:07:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.791+0000 7f86affff640 1 -- 172.21.15.64:0/1342268997 wait complete. 2023-11-24T08:07:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.793+0000 7f86affff640 1 Processor -- start 2023-11-24T08:07:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.793+0000 7f86affff640 1 -- start start 2023-11-24T08:07:04.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.794+0000 7f86affff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c0039e0 0x7f86b00b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.794+0000 7f86affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86b013df30 0x7f86b013e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.794+0000 7f86affff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b01f2040 0x7f86b01f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.794+0000 7f86affff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b015ce80 con 0x7f86b013df30 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.794+0000 7f86affff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b013f220 con 0x7f869c0039e0 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.794+0000 7f86affff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b015d0b0 con 0x7f86b01f2040 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.794+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86b013df30 0x7f86b013e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86b013df30 0x7f86b013e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60080/0 (socket says 172.21.15.64:60080) 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a7fff640 1 -- 172.21.15.64:0/847259323 learned_addr learned my addr 172.21.15.64:0/847259323 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86aeffd640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c0039e0 0x7f86b00b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a7fff640 1 -- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b01f2040 msgr2=0x7f86b01f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86af7fe640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b01f2040 0x7f86b01f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a7fff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b01f2040 0x7f86b01f24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a7fff640 1 -- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c0039e0 msgr2=0x7f86b00b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a7fff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c0039e0 0x7f86b00b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a7fff640 1 -- 172.21.15.64:0/847259323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b013ec60 con 0x7f86b013df30 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a7fff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86b013df30 0x7f86b013e3b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f869800d770 tx=0x7f869800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.795+0000 7f86a77fe640 1 -- 172.21.15.64:0/847259323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980072d0 con 0x7f86b013df30 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.796+0000 7f86a77fe640 1 -- 172.21.15.64:0/847259323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698007430 con 0x7f86b013df30 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.796+0000 7f86affff640 1 -- 172.21.15.64:0/847259323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b013ef40 con 0x7f86b013df30 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.796+0000 7f86a77fe640 1 -- 172.21.15.64:0/847259323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980047e0 con 0x7f86b013df30 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.796+0000 7f86affff640 1 -- 172.21.15.64:0/847259323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b013e8f0 con 0x7f86b013df30 2023-11-24T08:07:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.798+0000 7f86a77fe640 1 -- 172.21.15.64:0/847259323 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8698004940 con 0x7f86b013df30 2023-11-24T08:07:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.799+0000 7f86a77fe640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f869006b0b0 0x7f869006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.800+0000 7f86a77fe640 1 -- 172.21.15.64:0/847259323 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8698089770 con 0x7f86b013df30 2023-11-24T08:07:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.800+0000 7f86aeffd640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f869006b0b0 0x7f869006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.800+0000 7f86b7fe2740 1 Processor -- start 2023-11-24T08:07:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.801+0000 7f86b7fe2740 1 -- start start 2023-11-24T08:07:04.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.801+0000 7f86aeffd640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f869006b0b0 0x7f869006d570 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86a0036000 tx=0x7f86a0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c9070 0x55dd8c8c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c827880 con 0x55dd8c8c3d70 2023-11-24T08:07:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c836ae0 con 0x55dd8c8c9070 2023-11-24T08:07:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c8341d0 con 0x55dd8c8c87b0 2023-11-24T08:07:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60094/0 (socket says 172.21.15.64:60094) 2023-11-24T08:07:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86a7fff640 1 -- 172.21.15.64:0/613061602 learned_addr learned my addr 172.21.15.64:0/613061602 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:04.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86aeffd640 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c9070 0x55dd8c8c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86af7fe640 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86a7fff640 1 -- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 msgr2=0x55dd8c90c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86a7fff640 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86a7fff640 1 -- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c9070 msgr2=0x55dd8c8c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86a7fff640 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c9070 0x55dd8c8c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86a7fff640 1 -- 172.21.15.64:0/613061602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd8c7f7860 con 0x55dd8c8c3d70 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.802+0000 7f86af7fe640 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.803+0000 7f86a7fff640 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8c8270 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8698057da0 tx=0x7f8698057b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5af77f9af2011c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.803+0000 7f86a4ff9640 1 -- 172.21.15.64:0/613061602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980547b0 con 0x55dd8c8c3d70 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.803+0000 7f86a4ff9640 1 -- 172.21.15.64:0/613061602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698054910 con 0x55dd8c8c3d70 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.803+0000 7f86a4ff9640 1 -- 172.21.15.64:0/613061602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869805e2c0 con 0x55dd8c8c3d70 2023-11-24T08:07:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 -- 172.21.15.64:0/613061602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c3d70 msgr2=0x55dd8c8c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8c8270 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8698057da0 tx=0x7f8698057b10 comp rx=0 tx=0).stop 2023-11-24T08:07:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 -- 172.21.15.64:0/613061602 shutdown_connections 2023-11-24T08:07:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8c8270 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/613061602 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c9070 0x55dd8c8c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 -- 172.21.15.64:0/613061602 >> 172.21.15.64:0/613061602 conn(0x55dd8c88ac70 msgr2=0x55dd8c90eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 -- 172.21.15.64:0/613061602 shutdown_connections 2023-11-24T08:07:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.804+0000 7f86b7fe2740 1 -- 172.21.15.64:0/613061602 wait complete. 2023-11-24T08:07:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.805+0000 7f86b7fe2740 1 -- start start 2023-11-24T08:07:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.805+0000 7f86b7fe2740 1 Processor -- start 2023-11-24T08:07:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8e94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c9149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c8ee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c82fac0 con 0x55dd8c8c9070 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c7cc2f0 con 0x55dd8c8c3d70 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c7e9c70 con 0x55dd8c8c87b0 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86b7fe2740 10 client.0 _ll_register_callbacks cb 0x55dd8c7ee3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c8ee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c8ee7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60102/0 (socket says 172.21.15.64:60102) 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.806+0000 7f86af7fe640 1 -- 172.21.15.64:0/3803153375 learned_addr learned my addr 172.21.15.64:0/3803153375 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f86aeffd640 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8e94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f86af7fe640 1 -- 172.21.15.64:0/3803153375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 msgr2=0x55dd8c9149f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f86af7fe640 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c9149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f86af7fe640 1 -- 172.21.15.64:0/3803153375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 msgr2=0x55dd8c8e94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f86af7fe640 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8e94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f86af7fe640 1 -- 172.21.15.64:0/3803153375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b013e8f0 con 0x55dd8c8c9070 2023-11-24T08:07:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f86af7fe640 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c8ee7d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f86b0070060 tx=0x7f86a8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1108d1d070153c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f868dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:07:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f868dffb640 1 -- 172.21.15.64:0/3803153375 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8007c70 con 0x55dd8c8c9070 2023-11-24T08:07:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f868dffb640 1 -- 172.21.15.64:0/3803153375 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a8004830 con 0x55dd8c8c9070 2023-11-24T08:07:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.807+0000 7f86b7fe2740 1 -- 172.21.15.64:0/3803153375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd8c828aa0 con 0x55dd8c8c9070 2023-11-24T08:07:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f868dffb640 1 -- 172.21.15.64:0/3803153375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86a8007dd0 con 0x55dd8c8c9070 2023-11-24T08:07:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f86b7fe2740 10 client.6366 fetch_fsmap learned FSMap version 34 2023-11-24T08:07:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f86b7fe2740 1 -- 172.21.15.64:0/3803153375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55dd8c82e140 con 0x55dd8c8c9070 2023-11-24T08:07:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f868dffb640 1 -- 172.21.15.64:0/3803153375 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dd8c828aa0 con 0x55dd8c8c9070 2023-11-24T08:07:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f868dffb640 1 -- 172.21.15.64:0/3803153375 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86a8016410 con 0x55dd8c8c9070 2023-11-24T08:07:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f86b7fe2740 10 client.6366 fetch_fsmap finished waiting for FSMap version 34 2023-11-24T08:07:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f86b7fe2740 10 client.6366 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:07:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f86b7fe2740 20 client.6366 populate_metadata read hostname 'smithi064' 2023-11-24T08:07:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] conn(0x55dd8c912910 0x55dd8c90ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f86b7fe2740 4 client.6366 mds_command: new command op to 6240 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:07:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.808+0000 7f86b7fe2740 1 -- 172.21.15.64:0/3803153375 --> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd8c8287c0 con 0x55dd8c912910 2023-11-24T08:07:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.809+0000 7f86aeffd640 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] conn(0x55dd8c912910 0x55dd8c90ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.809+0000 7f86aeffd640 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] conn(0x55dd8c912910 0x55dd8c90ce90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8276209c28feceb server_cookie=282f9842f8acf8e3 in_seq=0 out_seq=0 2023-11-24T08:07:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.809+0000 7f868dffb640 10 client.6366 ms_handle_connect on v2:172.21.15.64:6833/4200493009 2023-11-24T08:07:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f868dffb640 1 -- 172.21.15.64:0/3803153375 <== mds.0 v2:172.21.15.64:6833/4200493009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd8c8287c0 con 0x55dd8c912910 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f868dffb640 10 client.6366 handle_command_reply: tid=0 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 client.6366 shutdown 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 -- 172.21.15.64:0/3803153375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 msgr2=0x55dd8c8ee7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c8ee7d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f86b0070060 tx=0x7f86a8004700 comp rx=0 tx=0).stop 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 -- 172.21.15.64:0/3803153375 shutdown_connections 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] conn(0x55dd8c912910 0x55dd8c90ce90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c8ee7d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c9149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/3803153375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8e94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f868dffb640 0 client.6366 ms_handle_reset on v2:172.21.15.64:6833/4200493009 2023-11-24T08:07:04.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.810+0000 7f86b7fe2740 1 -- 172.21.15.64:0/3803153375 >> 172.21.15.64:0/3803153375 conn(0x55dd8c88ac70 msgr2=0x55dd8c8c9450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:04.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.811+0000 7f86b7fe2740 1 -- 172.21.15.64:0/3803153375 shutdown_connections 2023-11-24T08:07:04.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.811+0000 7f86b7fe2740 1 -- 172.21.15.64:0/3803153375 wait complete. 2023-11-24T08:07:04.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.811+0000 7f86b7fe2740 20 client.6366 trim_cache size 0 max 16384 2023-11-24T08:07:04.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.828+0000 7f86b7fe2740 1 Processor -- start 2023-11-24T08:07:04.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.828+0000 7f86b7fe2740 1 -- start start 2023-11-24T08:07:04.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.829+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c90ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.829+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c910330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.829+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c910870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.829+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c592f20 con 0x55dd8c8c9070 2023-11-24T08:07:04.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.829+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c90e520 con 0x55dd8c8c3d70 2023-11-24T08:07:04.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.829+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c910db0 con 0x55dd8c8c87b0 2023-11-24T08:07:04.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.829+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c910870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c910870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60118/0 (socket says 172.21.15.64:60118) 2023-11-24T08:07:04.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86af7fe640 1 -- 172.21.15.64:0/492118330 learned_addr learned my addr 172.21.15.64:0/492118330 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:04.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86af7fe640 1 -- 172.21.15.64:0/492118330 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 msgr2=0x55dd8c910330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86af7fe640 1 --2- 172.21.15.64:0/492118330 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c910330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86af7fe640 1 -- 172.21.15.64:0/492118330 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 msgr2=0x55dd8c90ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86af7fe640 1 --2- 172.21.15.64:0/492118330 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c90ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86af7fe640 1 -- 172.21.15.64:0/492118330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd8c7cbb80 con 0x55dd8c8c9070 2023-11-24T08:07:04.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86af7fe640 1 --2- 172.21.15.64:0/492118330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c910870 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f86a8004740 tx=0x7f86a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3774a9b2bcd1e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:04.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.830+0000 7f86a57fa640 1 -- 172.21.15.64:0/492118330 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8009c00 con 0x55dd8c8c9070 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86a57fa640 1 -- 172.21.15.64:0/492118330 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a8011050 con 0x55dd8c8c9070 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86a57fa640 1 -- 172.21.15.64:0/492118330 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8009070 con 0x55dd8c8c9070 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 -- 172.21.15.64:0/492118330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 msgr2=0x55dd8c910870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/492118330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c910870 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f86a8004740 tx=0x7f86a8015000 comp rx=0 tx=0).stop 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 -- 172.21.15.64:0/492118330 shutdown_connections 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/492118330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c910870 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/492118330 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c910330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/492118330 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c90ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 -- 172.21.15.64:0/492118330 >> 172.21.15.64:0/492118330 conn(0x55dd8c88ac70 msgr2=0x55dd8c8e92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:04.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 -- 172.21.15.64:0/492118330 shutdown_connections 2023-11-24T08:07:04.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.831+0000 7f86b7fe2740 1 -- 172.21.15.64:0/492118330 wait complete. 2023-11-24T08:07:04.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.832+0000 7f86b7fe2740 1 -- start start 2023-11-24T08:07:04.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.832+0000 7f86b7fe2740 1 Processor -- start 2023-11-24T08:07:04.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.833+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.833+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.833+0000 7f86b7fe2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c90bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.833+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c979c50 con 0x55dd8c8c9070 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.833+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c8eebf0 con 0x55dd8c8c3d70 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.833+0000 7f86b7fe2740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd8c8e8340 con 0x55dd8c8c87b0 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86b7fe2740 10 client.0 _ll_register_callbacks cb 0x7f86a8009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c90bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c90bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60128/0 (socket says 172.21.15.64:60128) 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86af7fe640 1 -- 172.21.15.64:0/1544593036 learned_addr learned my addr 172.21.15.64:0/1544593036 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86aeffd640 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86a7fff640 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86af7fe640 1 -- 172.21.15.64:0/1544593036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 msgr2=0x55dd8c90ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86af7fe640 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86af7fe640 1 -- 172.21.15.64:0/1544593036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 msgr2=0x55dd8c8ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:04.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86af7fe640 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8ee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.834+0000 7f86af7fe640 1 -- 172.21.15.64:0/1544593036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd8c82e140 con 0x55dd8c8c9070 2023-11-24T08:07:04.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.835+0000 7f86af7fe640 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c90bfb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f86a8016800 tx=0x7f86a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a82a9fbc6f9bd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:04.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.835+0000 7f868dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.835+0000 7f868dffb640 1 -- 172.21.15.64:0/1544593036 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8016be0 con 0x55dd8c8c9070 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.835+0000 7f868dffb640 1 -- 172.21.15.64:0/1544593036 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a8016d40 con 0x55dd8c8c9070 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.835+0000 7f86b7fe2740 1 -- 172.21.15.64:0/1544593036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8698054910 con 0x55dd8c8c9070 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.835+0000 7f868dffb640 1 -- 172.21.15.64:0/1544593036 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86a8007a70 con 0x55dd8c8c9070 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.835+0000 7f86b7fe2740 10 client.6372 fetch_fsmap learned FSMap version 34 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.835+0000 7f86b7fe2740 1 -- 172.21.15.64:0/1544593036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55dd8c7f1170 con 0x55dd8c8c9070 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f868dffb640 1 -- 172.21.15.64:0/1544593036 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698054910 con 0x55dd8c8c9070 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f868dffb640 1 -- 172.21.15.64:0/1544593036 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86a8007bd0 con 0x55dd8c8c9070 2023-11-24T08:07:04.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f86b7fe2740 10 client.6372 fetch_fsmap finished waiting for FSMap version 34 2023-11-24T08:07:04.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f86b7fe2740 10 client.6372 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:07:04.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f86b7fe2740 20 client.6372 populate_metadata read hostname 'smithi064' 2023-11-24T08:07:04.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] conn(0x55dd8c90f0d0 0x55dd8c90f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:04.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f86b7fe2740 4 client.6372 mds_command: new command op to 6240 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:07:04.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f86b7fe2740 1 -- 172.21.15.64:0/1544593036 --> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dd8c5306c0 con 0x55dd8c90f0d0 2023-11-24T08:07:04.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.836+0000 7f86aeffd640 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] conn(0x55dd8c90f0d0 0x55dd8c90f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:04.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f868dffb640 10 client.6372 ms_handle_connect on v2:172.21.15.64:6833/4200493009 2023-11-24T08:07:04.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86aeffd640 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] conn(0x55dd8c90f0d0 0x55dd8c90f4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f88457a591bf3a server_cookie=c19f1e20e46cadfd in_seq=0 out_seq=0 2023-11-24T08:07:04.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f868dffb640 1 -- 172.21.15.64:0/1544593036 <== mds.0 v2:172.21.15.64:6833/4200493009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55dd8c5306c0 con 0x55dd8c90f0d0 2023-11-24T08:07:04.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f868dffb640 10 client.6372 handle_command_reply: tid=0 2023-11-24T08:07:04.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 client.6372 shutdown 2023-11-24T08:07:04.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 -- 172.21.15.64:0/1544593036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 msgr2=0x55dd8c90bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c90bfb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f86a8016800 tx=0x7f86a8004ab0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 -- 172.21.15.64:0/1544593036 shutdown_connections 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.64:6833/4200493009,v1:172.21.15.64:6835/4200493009] conn(0x55dd8c90f0d0 0x55dd8c90f4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd8c8c9070 0x55dd8c90bfb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55dd8c8c87b0 0x55dd8c90ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 --2- 172.21.15.64:0/1544593036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55dd8c8c3d70 0x55dd8c8ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 -- 172.21.15.64:0/1544593036 >> 172.21.15.64:0/1544593036 conn(0x55dd8c88ac70 msgr2=0x55dd8c8f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f868dffb640 0 client.6372 ms_handle_reset on v2:172.21.15.64:6833/4200493009 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 -- 172.21.15.64:0/1544593036 shutdown_connections 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.837+0000 7f86b7fe2740 1 -- 172.21.15.64:0/1544593036 wait complete. 2023-11-24T08:07:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86b7fe2740 20 client.6372 trim_cache size 0 max 16384 2023-11-24T08:07:04.847 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:07:04.847 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918", 2023-11-24T08:07:04.847 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-11-24T08:07:04.847 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 6240, 2023-11-24T08:07:04.847 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-11-24T08:07:04.847 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-11-24T08:07:04.847 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-11-24T08:07:04.847 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 6.8296631100000003, 2023-11-24T08:07:04.848 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 34, 2023-11-24T08:07:04.848 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 60, 2023-11-24T08:07:04.848 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 60, 2023-11-24T08:07:04.848 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 12.532595926000001 2023-11-24T08:07:04.848 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:07:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 -- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f869006b0b0 msgr2=0x7f869006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f869006b0b0 0x7f869006d570 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86a0036000 tx=0x7f86a0030000 comp rx=0 tx=0).stop 2023-11-24T08:07:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 -- 172.21.15.64:0/847259323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86b013df30 msgr2=0x7f86b013e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86b013df30 0x7f86b013e3b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f869800d770 tx=0x7f869800dc30 comp rx=0 tx=0).stop 2023-11-24T08:07:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 -- 172.21.15.64:0/847259323 shutdown_connections 2023-11-24T08:07:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f869006b0b0 0x7f869006d570 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f86b01f2040 0x7f86b01f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86b013df30 0x7f86b013e3b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 --2- 172.21.15.64:0/847259323 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f869c0039e0 0x7f86b00b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 -- 172.21.15.64:0/847259323 >> 172.21.15.64:0/847259323 conn(0x7f86b00b0840 msgr2=0x7f86b00ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 -- 172.21.15.64:0/847259323 shutdown_connections 2023-11-24T08:07:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:04.838+0000 7f86affff640 1 -- 172.21.15.64:0/847259323 wait complete. 2023-11-24T08:07:04.857 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:07:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.072+0000 7efd34a22640 1 Processor -- start 2023-11-24T08:07:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.072+0000 7efd34a22640 1 -- start start 2023-11-24T08:07:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd34a22640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cf050 0x7efd300cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd34a22640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd300cd7b0 0x7efd300cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd34a22640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3015fa70 con 0x7efd3013dfd0 2023-11-24T08:07:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd34a22640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3015c580 con 0x7efd300cd7b0 2023-11-24T08:07:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd34a22640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300cfa10 con 0x7efd300cf050 2023-11-24T08:07:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd2f59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd300cd7b0 0x7efd300cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd2f59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd300cd7b0 0x7efd300cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60134/0 (socket says 172.21.15.64:60134) 2023-11-24T08:07:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd2f59e640 1 -- 172.21.15.64:0/383482007 learned_addr learned my addr 172.21.15.64:0/383482007 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:05.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.074+0000 7efd27fff640 1 --2- 172.21.15.64:0/383482007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cf050 0x7efd300cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:05.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.075+0000 7efd2f59e640 1 -- 172.21.15.64:0/383482007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cf050 msgr2=0x7efd300cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:05.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.075+0000 7efd2f59e640 1 --2- 172.21.15.64:0/383482007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cf050 0x7efd300cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.075+0000 7efd2f59e640 1 -- 172.21.15.64:0/383482007 >> v1:172.21.15.91:6789/0 conn(0x7efd3013dfd0 legacy=0x7efd3015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:05.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.075+0000 7efd2f59e640 1 -- 172.21.15.64:0/383482007 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd30140900 con 0x7efd300cd7b0 2023-11-24T08:07:05.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.076+0000 7efd2f59e640 1 --2- 172.21.15.64:0/383482007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd300cd7b0 0x7efd300cdc00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efd28009e50 tx=0x7efd2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81035ac3026c553f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:05.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.076+0000 7efd2e59c640 1 -- 172.21.15.64:0/383482007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2802ca70 con 0x7efd300cd7b0 2023-11-24T08:07:05.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.076+0000 7efd2e59c640 1 -- 172.21.15.64:0/383482007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2802cbd0 con 0x7efd300cd7b0 2023-11-24T08:07:05.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.076+0000 7efd2e59c640 1 -- 172.21.15.64:0/383482007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd280343f0 con 0x7efd300cd7b0 2023-11-24T08:07:05.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.077+0000 7efd34a22640 1 -- 172.21.15.64:0/383482007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd300cd7b0 msgr2=0x7efd300cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:05.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.077+0000 7efd34a22640 1 --2- 172.21.15.64:0/383482007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd300cd7b0 0x7efd300cdc00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efd28009e50 tx=0x7efd2802bed0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.077+0000 7efd34a22640 1 -- 172.21.15.64:0/383482007 shutdown_connections 2023-11-24T08:07:05.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.077+0000 7efd34a22640 1 --2- 172.21.15.64:0/383482007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd300cd7b0 0x7efd300cdc00 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.077+0000 7efd34a22640 1 --2- 172.21.15.64:0/383482007 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cf050 0x7efd300cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.077+0000 7efd34a22640 1 -- 172.21.15.64:0/383482007 >> 172.21.15.64:0/383482007 conn(0x7efd3006a860 msgr2=0x7efd3006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:05.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.077+0000 7efd34a22640 1 -- 172.21.15.64:0/383482007 shutdown_connections 2023-11-24T08:07:05.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.077+0000 7efd34a22640 1 -- 172.21.15.64:0/383482007 wait complete. 2023-11-24T08:07:05.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.079+0000 7efd34a22640 1 Processor -- start 2023-11-24T08:07:05.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.079+0000 7efd34a22640 1 -- start start 2023-11-24T08:07:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd34a22640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cd7b0 0x7efd301f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd34a22640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd300cf050 0x7efd300b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd34a22640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd3013dfd0 0x7efd301ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd34a22640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b2bb0 con 0x7efd3013dfd0 2023-11-24T08:07:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd34a22640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b2e50 con 0x7efd300cf050 2023-11-24T08:07:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd34a22640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b3130 con 0x7efd300cd7b0 2023-11-24T08:07:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd2f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd3013dfd0 0x7efd301ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:05.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd2f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd3013dfd0 0x7efd301ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60140/0 (socket says 172.21.15.64:60140) 2023-11-24T08:07:05.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.080+0000 7efd2f59e640 1 -- 172.21.15.64:0/2455807426 learned_addr learned my addr 172.21.15.64:0/2455807426 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd27fff640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd300cf050 0x7efd300b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd2f59e640 1 -- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cd7b0 msgr2=0x7efd301f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd2ed9d640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cd7b0 0x7efd301f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd2f59e640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cd7b0 0x7efd301f0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd2f59e640 1 -- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd300cf050 msgr2=0x7efd300b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:05.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd2f59e640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd300cf050 0x7efd300b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd2f59e640 1 -- 172.21.15.64:0/2455807426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28009b30 con 0x7efd3013dfd0 2023-11-24T08:07:05.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd27fff640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd300cf050 0x7efd300b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:05.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd2f59e640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd3013dfd0 0x7efd301ee0c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efd28009e50 tx=0x7efd28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:05.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd277fe640 1 -- 172.21.15.64:0/2455807426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28009510 con 0x7efd3013dfd0 2023-11-24T08:07:05.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd277fe640 1 -- 172.21.15.64:0/2455807426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2803e450 con 0x7efd3013dfd0 2023-11-24T08:07:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd277fe640 1 -- 172.21.15.64:0/2455807426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2803e7e0 con 0x7efd3013dfd0 2023-11-24T08:07:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.081+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd300b3410 con 0x7efd3013dfd0 2023-11-24T08:07:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.082+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd300b39d0 con 0x7efd3013dfd0 2023-11-24T08:07:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.083+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd300b3bc0 con 0x7efd3013dfd0 2023-11-24T08:07:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.084+0000 7efd277fe640 1 -- 172.21.15.64:0/2455807426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7efd2803e940 con 0x7efd3013dfd0 2023-11-24T08:07:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.085+0000 7efd277fe640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd1006b0d0 0x7efd1006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.085+0000 7efd277fe640 1 -- 172.21.15.64:0/2455807426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd280ac6b0 con 0x7efd3013dfd0 2023-11-24T08:07:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.085+0000 7efd2ed9d640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd1006b0d0 0x7efd1006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:05.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.087+0000 7efd2ed9d640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd1006b0d0 0x7efd1006d590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efd2000a750 tx=0x7efd20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.089+0000 7efd277fe640 1 -- 172.21.15.64:0/2455807426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd28077570 con 0x7efd3013dfd0 2023-11-24T08:07:05.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.322+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd3015b8c0 con 0x7efd3013dfd0 2023-11-24T08:07:05.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.323+0000 7efd277fe640 1 -- 172.21.15.64:0/2455807426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd2807ab90 con 0x7efd3013dfd0 2023-11-24T08:07:05.325 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:07:05.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd1006b0d0 msgr2=0x7efd1006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:05.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd1006b0d0 0x7efd1006d590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efd2000a750 tx=0x7efd20008040 comp rx=0 tx=0).stop 2023-11-24T08:07:05.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd3013dfd0 msgr2=0x7efd301ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:05.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd3013dfd0 0x7efd301ee0c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efd28009e50 tx=0x7efd28004a40 comp rx=0 tx=0).stop 2023-11-24T08:07:05.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 shutdown_connections 2023-11-24T08:07:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd1006b0d0 0x7efd1006d590 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd3013dfd0 0x7efd301ee0c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd300cf050 0x7efd300b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 --2- 172.21.15.64:0/2455807426 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd300cd7b0 0x7efd301f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.326+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 >> 172.21.15.64:0/2455807426 conn(0x7efd3006a860 msgr2=0x7efd3013f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.327+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 shutdown_connections 2023-11-24T08:07:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:05.327+0000 7efd34a22640 1 -- 172.21.15.64:0/2455807426 wait complete. 2023-11-24T08:07:05.347 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:07:05.347 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:07:05.347 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T08:07:05.347 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:07:05.348 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:05.348 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:07:05.348 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:07:05.348 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:07:05.348 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:07:05.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:05.348 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T08:07:05.404 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:07:05.404 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:07:05.404 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:05.405 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42157sec preferred_lft 42157sec 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3c83:ffff:fee3:836f/64 scope link 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:05.406 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:07:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::54d4:76ff:fe60:984a/64 scope link 2023-11-24T08:07:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:05.407 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:07:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether ae:e6:20:dc:18:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ace6:20ff:fedc:18d9/64 scope link 2023-11-24T08:07:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:05.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:05.408 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:07:05.503 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T08:07:05.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:05.508 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T08:07:05.572 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T08:07:05.572 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:07:05.572 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:07:05.572 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:07:05.572 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:07:05.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:05.573 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:07:05.573 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:07:05.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:07:05.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:07:05.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:07:05.573 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:07:05.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:05.780 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:07:05.836 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:07:05.836 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:07:05.837 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:07:05.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:05.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-11-24T08:07:05.942 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T08:07:05.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:07:05.942 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:06.004 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:07:06.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:07:06.004 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:06.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:07:06.166 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:06.227 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:07:06.227 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:07:06.227 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:07:06.227 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:07:06.227 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:07:06.228 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:07:06.229 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:07:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:07:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:07:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:06.230 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:06.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:06.231 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:07:06.288 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:07:06.288 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T08:07:06.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:07:06.374 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:06.417 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:07:06.417 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:06.418 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:07:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:07:06.420 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:07:06.420 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:07:06.420 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:07:06.420 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:06.420 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:06.420 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:06.420 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:06.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:06.421 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:07:06.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-11-24T08:07:06.493+0000 7f80dcc8f480 -1 init, newargv = 0x557ee5157900 newargc=15 2023-11-24T08:07:06.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[109190]: starting ceph client 2023-11-24T08:07:06.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[109190]: starting fuse 2023-11-24T08:07:06.510 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T08:07:06.511 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T08:07:06.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:07:06.511 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T08:07:06.511 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T08:07:06.511 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T08:07:06.512 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T08:07:06.512 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:07:06.512 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T08:07:06.512 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T08:07:06.512 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:07:06.513 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:07:06.514 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:07:06.514 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T08:07:06.514 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T08:07:06.514 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T08:07:06.514 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:07:06.514 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:07:06.514 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-11-24T08:07:06.514 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T08:07:06.727 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.109190.asok 2023-11-24T08:07:06.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:07:06.745 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109190.asok 2023-11-24T08:07:06.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:06.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109190.asok status 2023-11-24T08:07:06.989 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:07:06.989 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T08:07:06.989 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:07:06.989 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:07:06.989 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "109190", 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 6387, 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T08:07:06.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 6387 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 86122946 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:07:06.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 86122946 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.6387 192.168.0.1:0/86122946", 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/86122946", 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 34, 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 60, 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T08:07:06.992 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T08:07:06.993 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:07:07.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:07.006 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T08:07:07.006 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:07.062 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T08:07:07.062 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:07.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:07.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:07.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:07:07.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.388+0000 7f2a9bfff640 1 Processor -- start 2023-11-24T08:07:07.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.389+0000 7f2a9bfff640 1 -- start start 2023-11-24T08:07:07.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9bfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2a9c13dfd0 0x7f2a9c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:07.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a9c0cf500 0x7f2a9c06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:07.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9bfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c0ce7c0 con 0x7f2a9c0cf050 2023-11-24T08:07:07.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9bfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c140900 con 0x7f2a9c0cf500 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9bfff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c15fa70 con 0x7f2a9c13dfd0 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a9c0cf500 0x7f2a9c06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a9c0cf500 0x7f2a9c06c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55742/0 (socket says 172.21.15.64:55742) 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9b7fe640 1 -- 172.21.15.64:0/352479294 learned_addr learned my addr 172.21.15.64:0/352479294 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.391+0000 7f2a9affd640 1 --2- 172.21.15.64:0/352479294 >> v2:172.21.15.91:3300/0 conn(0x7f2a9c13dfd0 0x7f2a9c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.392+0000 7f2a9b7fe640 1 -- 172.21.15.64:0/352479294 >> v2:172.21.15.91:3300/0 conn(0x7f2a9c13dfd0 msgr2=0x7f2a9c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.392+0000 7f2a9b7fe640 1 --2- 172.21.15.64:0/352479294 >> v2:172.21.15.91:3300/0 conn(0x7f2a9c13dfd0 0x7f2a9c15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.392+0000 7f2a9b7fe640 1 -- 172.21.15.64:0/352479294 >> v1:172.21.15.91:6789/0 conn(0x7f2a9c0cf050 legacy=0x7f2a9c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:07.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.392+0000 7f2a9b7fe640 1 -- 172.21.15.64:0/352479294 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c15ee60 con 0x7f2a9c0cf500 2023-11-24T08:07:07.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.393+0000 7f2a9b7fe640 1 --2- 172.21.15.64:0/352479294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf500 0x7f2a9c06c840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0cfc10 tx=0x7f2a9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a9bf85c21550b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:07.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.393+0000 7f2a9a7fc640 1 -- 172.21.15.64:0/352479294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94031870 con 0x7f2a9c0cf500 2023-11-24T08:07:07.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.393+0000 7f2a9a7fc640 1 -- 172.21.15.64:0/352479294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a940319d0 con 0x7f2a9c0cf500 2023-11-24T08:07:07.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.393+0000 7f2a9a7fc640 1 -- 172.21.15.64:0/352479294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94031d60 con 0x7f2a9c0cf500 2023-11-24T08:07:07.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.394+0000 7f2a9bfff640 1 -- 172.21.15.64:0/352479294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf500 msgr2=0x7f2a9c06c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:07.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.394+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/352479294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf500 0x7f2a9c06c840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0cfc10 tx=0x7f2a9402b910 comp rx=0 tx=0).stop 2023-11-24T08:07:07.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.394+0000 7f2a9bfff640 1 -- 172.21.15.64:0/352479294 shutdown_connections 2023-11-24T08:07:07.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.394+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/352479294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf500 0x7f2a9c06c840 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.394+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/352479294 >> v2:172.21.15.91:3300/0 conn(0x7f2a9c13dfd0 0x7f2a9c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.394+0000 7f2a9bfff640 1 -- 172.21.15.64:0/352479294 >> 172.21.15.64:0/352479294 conn(0x7f2a9c06a860 msgr2=0x7f2a9c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:07.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.394+0000 7f2a9bfff640 1 -- 172.21.15.64:0/352479294 shutdown_connections 2023-11-24T08:07:07.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.394+0000 7f2a9bfff640 1 -- 172.21.15.64:0/352479294 wait complete. 2023-11-24T08:07:07.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.396+0000 7f2a9bfff640 1 Processor -- start 2023-11-24T08:07:07.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.396+0000 7f2a9bfff640 1 -- start start 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf050 0x7f2a9c070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a9c0cf500 0x7f2a9c158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a9c13dfd0 0x7f2a9c13b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c157fb0 con 0x7f2a9c0cf050 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c071020 con 0x7f2a9c13dfd0 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c158980 con 0x7f2a9c0cf500 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf050 0x7f2a9c070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf050 0x7f2a9c070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55752/0 (socket says 172.21.15.64:55752) 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.397+0000 7f2a9affd640 1 -- 172.21.15.64:0/4244106108 learned_addr learned my addr 172.21.15.64:0/4244106108 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a9affd640 1 -- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a9c0cf500 msgr2=0x7f2a9c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a927fc640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a9c0cf500 0x7f2a9c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a9b7fe640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a9c13dfd0 0x7f2a9c13b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a9affd640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a9c0cf500 0x7f2a9c158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a9affd640 1 -- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a9c13dfd0 msgr2=0x7f2a9c13b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a9affd640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a9c13dfd0 0x7f2a9c13b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a9affd640 1 -- 172.21.15.64:0/4244106108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a940094d0 con 0x7f2a9c0cf050 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a927fc640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a9c0cf500 0x7f2a9c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a9b7fe640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a9c13dfd0 0x7f2a9c13b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:07.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a9affd640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf050 0x7f2a9c070ae0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00d5f0 tx=0x7f2a8c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:07.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.398+0000 7f2a98ff9640 1 -- 172.21.15.64:0/4244106108 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0125f0 con 0x7f2a9c0cf050 2023-11-24T08:07:07.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.399+0000 7f2a98ff9640 1 -- 172.21.15.64:0/4244106108 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c012750 con 0x7f2a9c0cf050 2023-11-24T08:07:07.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.399+0000 7f2a98ff9640 1 -- 172.21.15.64:0/4244106108 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c012b00 con 0x7f2a9c0cf050 2023-11-24T08:07:07.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.399+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c1573a0 con 0x7f2a9c0cf050 2023-11-24T08:07:07.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.399+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c157960 con 0x7f2a9c0cf050 2023-11-24T08:07:07.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.400+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c04e840 con 0x7f2a9c0cf050 2023-11-24T08:07:07.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.401+0000 7f2a98ff9640 1 -- 172.21.15.64:0/4244106108 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2a8c012c60 con 0x7f2a9c0cf050 2023-11-24T08:07:07.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.402+0000 7f2a98ff9640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7c06b0d0 0x7f2a7c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:07.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.402+0000 7f2a98ff9640 1 -- 172.21.15.64:0/4244106108 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a8c089ef0 con 0x7f2a9c0cf050 2023-11-24T08:07:07.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.402+0000 7f2a927fc640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7c06b0d0 0x7f2a7c06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:07.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.403+0000 7f2a927fc640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7c06b0d0 0x7f2a7c06d590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c155860 tx=0x7f2a84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:07.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.407+0000 7f2a98ff9640 1 -- 172.21.15.64:0/4244106108 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a8c054c60 con 0x7f2a9c0cf050 2023-11-24T08:07:07.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.640+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2a9c0ce9a0 con 0x7f2a9c0cf050 2023-11-24T08:07:07.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.641+0000 7f2a98ff9640 1 -- 172.21.15.64:0/4244106108 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2a8c058280 con 0x7f2a9c0cf050 2023-11-24T08:07:07.643 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:07:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7c06b0d0 msgr2=0x7f2a7c06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7c06b0d0 0x7f2a7c06d590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c155860 tx=0x7f2a84009040 comp rx=0 tx=0).stop 2023-11-24T08:07:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf050 msgr2=0x7f2a9c070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf050 0x7f2a9c070ae0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00d5f0 tx=0x7f2a8c00dab0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 shutdown_connections 2023-11-24T08:07:07.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a7c06b0d0 0x7f2a7c06d590 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a9c13dfd0 0x7f2a9c13b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a9c0cf500 0x7f2a9c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 --2- 172.21.15.64:0/4244106108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a9c0cf050 0x7f2a9c070ae0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:07.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 >> 172.21.15.64:0/4244106108 conn(0x7f2a9c06a860 msgr2=0x7f2a9c06ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:07.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 shutdown_connections 2023-11-24T08:07:07.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:07.644+0000 7f2a9bfff640 1 -- 172.21.15.64:0/4244106108 wait complete. 2023-11-24T08:07:07.665 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:07:07.665 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:07:07.665 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2023-11-24T08:07:07.665 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:07:07.665 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:07.665 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:07:07.665 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:07:07.666 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:07:07.666 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:07:07.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:07.666 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2023-11-24T08:07:07.725 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:07.726 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42145sec preferred_lft 42145sec 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:07.727 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::5073:3fff:fef1:71bb/64 scope link 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::2497:37ff:fe0c:852a/64 scope link 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 32:d9:7a:07:6c:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::30d9:7aff:fe07:6ce0/64 scope link 2023-11-24T08:07:07.728 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:07.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:07.729 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:07:07.819 INFO:teuthology.orchestra.run.smithi091.stdout:1 2023-11-24T08:07:07.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:07.823 DEBUG:teuthology.orchestra.run.smithi091:> route 2023-11-24T08:07:07.876 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2023-11-24T08:07:07.876 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:07:07.876 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:07:07.876 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:07:07.877 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:07:07.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:07.877 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:07:07.877 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:07:07.877 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:07:07.877 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:07:07.877 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:07:07.877 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:07:08.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:08.047 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:07:08.096 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:07:08.096 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:07:08.097 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:07:08.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:08.098 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2023-11-24T08:07:08.183 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T08:07:08.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:07:08.184 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:08.236 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:07:08.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:07:08.237 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:08.289 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:07:08.380 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:07:08.436 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:08.437 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:07:08.438 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:08.439 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:08.439 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:08.439 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:08.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:08.440 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:07:08.492 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:07:08.492 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T08:07:08.497 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:07:08.573 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:08.614 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:07:08.614 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:07:08.614 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:07:08.614 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:07:08.615 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:07:08.616 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:07:08.617 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:08.617 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:08.617 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:08.617 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:08.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:08.618 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:07:08.801 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:2023-11-24T08:07:08.797+0000 7f0aae941480 -1 init, newargv = 0x560c3f7839c0 newargc=15 2023-11-24T08:07:08.802 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[97180]: starting ceph client 2023-11-24T08:07:08.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[97180]: starting fuse 2023-11-24T08:07:09.802 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:07:09.907 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:09.962 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:07:09.962 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:07:09.963 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:07:09.963 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:07:09.964 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:07:09.964 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:07:09.965 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:07:09.965 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:07:09.966 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:07:09.966 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:07:09.966 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:09.967 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:09.968 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:09.969 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:09.969 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:09.969 INFO:teuthology.orchestra.run.smithi091.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:09.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:09.970 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:07:10.025 INFO:teuthology.orchestra.run.smithi091.stdout:41 2023-11-24T08:07:10.025 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> import glob 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> import re 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> import os 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> def _find_admin_socket(client_name): 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T08:07:10.026 DEBUG:teuthology.orchestra.run.smithi091:> files = glob.glob(asok_path) 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> # Given a non-glob path, it better be there 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> if "*" not in asok_path: 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> assert(len(files) == 1) 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> return files[0] 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> for f in files: 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:07:10.027 DEBUG:teuthology.orchestra.run.smithi091:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:07:10.031 DEBUG:teuthology.orchestra.run.smithi091:> contents = proc_f.read() 2023-11-24T08:07:10.031 DEBUG:teuthology.orchestra.run.smithi091:> if mountpoint in contents: 2023-11-24T08:07:10.031 DEBUG:teuthology.orchestra.run.smithi091:> return f 2023-11-24T08:07:10.031 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:07:10.031 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:07:10.031 DEBUG:teuthology.orchestra.run.smithi091:> print(_find_admin_socket("client.1")) 2023-11-24T08:07:10.031 DEBUG:teuthology.orchestra.run.smithi091:> ' 2023-11-24T08:07:10.213 INFO:teuthology.orchestra.run.smithi091.stdout:/var/run/ceph/ceph-client.1.97180.asok 2023-11-24T08:07:10.217 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:07:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:07:10.234 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.97180.asok 2023-11-24T08:07:10.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:10.234 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.97180.asok status 2023-11-24T08:07:10.474 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": { 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: "entity_id": "1", 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: "hostname": "smithi091", 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: "pid": "97180", 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: "root": "/" 2023-11-24T08:07:10.475 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:07:10.476 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_count": 0, 2023-11-24T08:07:10.476 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_pinned_count": 0, 2023-11-24T08:07:10.476 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 5141, 2023-11-24T08:07:10.476 INFO:teuthology.orchestra.run.smithi091.stdout: "inst": { 2023-11-24T08:07:10.476 INFO:teuthology.orchestra.run.smithi091.stdout: "name": { 2023-11-24T08:07:10.476 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "client", 2023-11-24T08:07:10.476 INFO:teuthology.orchestra.run.smithi091.stdout: "num": 5141 2023-11-24T08:07:10.476 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 3131530397 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: } 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:07:10.477 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 3131530397 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "inst_str": "client.5141 192.168.0.1:0/3131530397", 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "addr_str": "192.168.0.1:0/3131530397", 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "inode_count": 1, 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_epoch": 34, 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch": 60, 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklisted": false, 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs" 2023-11-24T08:07:10.478 INFO:teuthology.orchestra.run.smithi091.stdout:} 2023-11-24T08:07:10.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:10.489 DEBUG:teuthology.orchestra.run.smithi091:> stat --file-system '--printf=%T 2023-11-24T08:07:10.489 DEBUG:teuthology.orchestra.run.smithi091:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:10.545 INFO:teuthology.orchestra.run.smithi091.stdout:fuseblk 2023-11-24T08:07:10.545 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:10.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:10.546 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:10.645 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:07:10.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.886+0000 7f0346f27640 1 Processor -- start 2023-11-24T08:07:10.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.886+0000 7f0346f27640 1 -- start start 2023-11-24T08:07:10.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.888+0000 7f0346f27640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0340153ea0 0x7f03400ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:10.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.888+0000 7f0346f27640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03400ce650 con 0x7f0340154280 2023-11-24T08:07:10.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.888+0000 7f0346f27640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034015e0a0 con 0x7f0340070850 2023-11-24T08:07:10.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.888+0000 7f0346f27640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034015e380 con 0x7f0340153ea0 2023-11-24T08:07:10.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.889+0000 7f0345724640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0340154280 0x7f034015d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32826/0 (socket says 172.21.15.64:32826) 2023-11-24T08:07:10.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.889+0000 7f0345724640 1 -- 172.21.15.64:0/3303427983 learned_addr learned my addr 172.21.15.64:0/3303427983 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:10.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.889+0000 7f0345f25640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0340153ea0 0x7f03400ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:10.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.889+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345076435 0 0) 0x7f03400ce650 con 0x7f0340154280 2023-11-24T08:07:10.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.889+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f032c003520 con 0x7f0340154280 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314794631 0 0) 0x7f032c003520 con 0x7f0340154280 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0340153ea0 msgr2=0x7f03400ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0340153ea0 0x7f03400ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 >> v1:172.21.15.91:6789/0 conn(0x7f0340070850 legacy=0x7f0340071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034015e660 con 0x7f0340154280 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0330001670 con 0x7f0340154280 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 >> v1:172.21.15.64:6789/0 conn(0x7f0340154280 legacy=0x7f034015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f032c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f032c0047c0 0x7f032c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0340153ea0 0x7f032c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:10.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034015e380 con 0x7f032c003a10 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c00ba80 con 0x7f032c0047c0 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c00bc40 con 0x7f0340153ea0 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0345f25640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f032c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0345f25640 1 -- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0340153ea0 msgr2=0x7f032c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0345f25640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0340153ea0 0x7f032c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0345f25640 1 -- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f032c0047c0 msgr2=0x7f032c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0345f25640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f032c0047c0 0x7f032c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.890+0000 7f0345f25640 1 -- 172.21.15.64:0/3303427983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c003520 con 0x7f032c003a10 2023-11-24T08:07:10.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0345f25640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f032c003e30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f033c009ac0 tx=0x7f032c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2febe3e58fcf054b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c02c710 con 0x7f032c003a10 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c02c870 con 0x7f032c003a10 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0344f23640 1 -- 172.21.15.64:0/3303427983 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c02cc00 con 0x7f032c003a10 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0346f27640 1 -- 172.21.15.64:0/3303427983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 msgr2=0x7f032c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0346f27640 1 --2- 172.21.15.64:0/3303427983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f032c003e30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f033c009ac0 tx=0x7f032c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0346726640 1 -- 172.21.15.64:0/3303427983 reap_dead start 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0346f27640 1 -- 172.21.15.64:0/3303427983 shutdown_connections 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.892+0000 7f0346f27640 1 -- 172.21.15.64:0/3303427983 >> 172.21.15.64:0/3303427983 conn(0x7f03400cec10 msgr2=0x7f03400cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.893+0000 7f0346f27640 1 -- 172.21.15.64:0/3303427983 shutdown_connections 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.893+0000 7f0346f27640 1 -- 172.21.15.64:0/3303427983 wait complete. 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.894+0000 7f0346f27640 1 Processor -- start 2023-11-24T08:07:10.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.895+0000 7f0346f27640 1 -- start start 2023-11-24T08:07:10.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.895+0000 7f0346f27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f0340157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.895+0000 7f0346f27640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03401558b0 0x7f034013c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.895+0000 7f0346f27640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f034013c950 0x7f034013cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0346f27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0340155d30 con 0x7f032c003a10 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0346f27640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0340155f70 con 0x7f034013c950 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0346f27640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0340156250 con 0x7f03401558b0 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345f25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f0340157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345f25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f0340157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55758/0 (socket says 172.21.15.64:55758) 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345f25640 1 -- 172.21.15.64:0/377930892 learned_addr learned my addr 172.21.15.64:0/377930892 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345724640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03401558b0 0x7f034013c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0346726640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f034013c950 0x7f034013cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345f25640 1 -- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03401558b0 msgr2=0x7f034013c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:10.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345f25640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03401558b0 0x7f034013c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345f25640 1 -- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f034013c950 msgr2=0x7f034013cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345f25640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f034013c950 0x7f034013cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0345f25640 1 -- 172.21.15.64:0/377930892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0340156530 con 0x7f032c003a10 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.896+0000 7f0346726640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f034013c950 0x7f034013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.897+0000 7f0345f25640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f0340157010 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f033c02c2b0 tx=0x7f033c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.897+0000 7f03367fc640 1 -- 172.21.15.64:0/377930892 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c003810 con 0x7f032c003a10 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.897+0000 7f03367fc640 1 -- 172.21.15.64:0/377930892 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c02de90 con 0x7f032c003a10 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.897+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0340156810 con 0x7f032c003a10 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.897+0000 7f03367fc640 1 -- 172.21.15.64:0/377930892 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c003b70 con 0x7f032c003a10 2023-11-24T08:07:10.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.897+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034014d8f0 con 0x7f032c003a10 2023-11-24T08:07:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.898+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034004e840 con 0x7f032c003a10 2023-11-24T08:07:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.899+0000 7f03367fc640 1 -- 172.21.15.64:0/377930892 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f033c003cd0 con 0x7f032c003a10 2023-11-24T08:07:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.901+0000 7f03367fc640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f032406b080 0x7f032406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:10.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.901+0000 7f03367fc640 1 -- 172.21.15.64:0/377930892 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f033c0ab870 con 0x7f032c003a10 2023-11-24T08:07:10.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.901+0000 7f0345724640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f032406b080 0x7f032406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:10.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.902+0000 7f0345724640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f032406b080 0x7f032406d540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f03400b4640 tx=0x7f03300016e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:10.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:10.905+0000 7f03367fc640 1 -- 172.21.15.64:0/377930892 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f033c0766d0 con 0x7f032c003a10 2023-11-24T08:07:11.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:11.142+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f03400ad420 con 0x7f032c003a10 2023-11-24T08:07:12.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.105+0000 7f03367fc640 1 -- 172.21.15.64:0/377930892 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 136+0+0 (secure 0 0 0) 0x7f033c079cf0 con 0x7f032c003a10 2023-11-24T08:07:12.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f032406b080 msgr2=0x7f032406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f032406b080 0x7f032406d540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f03400b4640 tx=0x7f03300016e0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 msgr2=0x7f0340157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f0340157010 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f033c02c2b0 tx=0x7f033c004730 comp rx=0 tx=0).stop 2023-11-24T08:07:12.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 shutdown_connections 2023-11-24T08:07:12.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f032406b080 0x7f032406d540 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f034013c950 0x7f034013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f03401558b0 0x7f034013c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 --2- 172.21.15.64:0/377930892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c003a10 0x7f0340157010 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 >> 172.21.15.64:0/377930892 conn(0x7f03400cec10 msgr2=0x7f0340070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:12.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 shutdown_connections 2023-11-24T08:07:12.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.109+0000 7f0346f27640 1 -- 172.21.15.64:0/377930892 wait complete. 2023-11-24T08:07:12.131 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:07:12.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.357+0000 7fa1ff59e640 1 Processor -- start 2023-11-24T08:07:12.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.357+0000 7fa1ff59e640 1 -- start start 2023-11-24T08:07:12.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.359+0000 7fa1ff59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa200071620 0x7fa20015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.359+0000 7fa1ff59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20015b600 con 0x7fa200070e90 2023-11-24T08:07:12.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.359+0000 7fa1ff59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000ce8d0 con 0x7fa200071a00 2023-11-24T08:07:12.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.359+0000 7fa1ff59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000700d0 con 0x7fa200071620 2023-11-24T08:07:12.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.359+0000 7fa1fed9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa200070e90 0x7fa20015aef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32830/0 (socket says 172.21.15.64:32830) 2023-11-24T08:07:12.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.359+0000 7fa1fed9d640 1 -- 172.21.15.64:0/3548450261 learned_addr learned my addr 172.21.15.64:0/3548450261 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:12.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.359+0000 7fa1fe59c640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa200071620 0x7fa20015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621241038 0 0) 0x7fa20015b600 con 0x7fa200070e90 2023-11-24T08:07:12.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1ec005730 con 0x7fa200070e90 2023-11-24T08:07:12.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2087136426 0 0) 0x7fa1ec005730 con 0x7fa200070e90 2023-11-24T08:07:12.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa200071620 msgr2=0x7fa20015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:12.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa200071620 0x7fa20015e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 >> v1:172.21.15.91:6789/0 conn(0x7fa200071a00 legacy=0x7fa2000cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000703b0 con 0x7fa200070e90 2023-11-24T08:07:12.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fa1f8001670 con 0x7fa200070e90 2023-11-24T08:07:12.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 >> v1:172.21.15.64:6789/0 conn(0x7fa200070e90 legacy=0x7fa20015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa200071620 0x7fa1ec005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa1ec0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ec006f10 0x7fa1ec007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000700d0 con 0x7fa1ec0061a0 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec00dd80 con 0x7fa1ec006f10 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.360+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec00df40 con 0x7fa200071620 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.361+0000 7fa1fdd9b640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa1ec0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.361+0000 7fa1fe59c640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa200071620 0x7fa1ec005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.361+0000 7fa1fed9d640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ec006f10 0x7fa1ec007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.361+0000 7fa1fdd9b640 1 -- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa200071620 msgr2=0x7fa1ec005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:12.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.361+0000 7fa1fdd9b640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa200071620 0x7fa1ec005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.361+0000 7fa1fdd9b640 1 -- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ec006f10 msgr2=0x7fa1ec007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:12.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.361+0000 7fa1fdd9b640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1ec006f10 0x7fa1ec007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.361+0000 7fa1fdd9b640 1 -- 172.21.15.64:0/3548450261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec005730 con 0x7fa1ec0061a0 2023-11-24T08:07:12.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.362+0000 7fa1fdd9b640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa1ec0069d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001750 tx=0x7fa1ec007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79db87668fc05c5e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:12.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80052d0 con 0x7fa1ec0061a0 2023-11-24T08:07:12.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e802e050 con 0x7fa1ec0061a0 2023-11-24T08:07:12.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1fd59a640 1 -- 172.21.15.64:0/3548450261 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8003830 con 0x7fa1ec0061a0 2023-11-24T08:07:12.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1ff59e640 1 -- 172.21.15.64:0/3548450261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 msgr2=0x7fa1ec0069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1ff59e640 1 --2- 172.21.15.64:0/3548450261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa1ec0069d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001750 tx=0x7fa1ec007a00 comp rx=0 tx=0).stop 2023-11-24T08:07:12.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1fed9d640 1 -- 172.21.15.64:0/3548450261 reap_dead start 2023-11-24T08:07:12.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1ff59e640 1 -- 172.21.15.64:0/3548450261 shutdown_connections 2023-11-24T08:07:12.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1ff59e640 1 -- 172.21.15.64:0/3548450261 >> 172.21.15.64:0/3548450261 conn(0x7fa2000b0840 msgr2=0x7fa2000b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:12.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1ff59e640 1 -- 172.21.15.64:0/3548450261 shutdown_connections 2023-11-24T08:07:12.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.363+0000 7fa1ff59e640 1 -- 172.21.15.64:0/3548450261 wait complete. 2023-11-24T08:07:12.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.365+0000 7fa1ff59e640 1 Processor -- start 2023-11-24T08:07:12.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.365+0000 7fa1ff59e640 1 -- start start 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.366+0000 7fa1ff59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa20013e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.366+0000 7fa1ff59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa20013ee50 0x7fa2000b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.366+0000 7fa1ff59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa20013f2b0 0x7fa2000b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1ff59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001509b0 con 0x7fa1ec0061a0 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1ff59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000c9f00 con 0x7fa20013f2b0 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1ff59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa200141ca0 con 0x7fa20013ee50 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fe59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa20013e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fe59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa20013e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55782/0 (socket says 172.21.15.64:55782) 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fe59c640 1 -- 172.21.15.64:0/381135913 learned_addr learned my addr 172.21.15.64:0/381135913 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:12.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fe59c640 1 -- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa20013ee50 msgr2=0x7fa2000b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fdd9b640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa20013ee50 0x7fa2000b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fed9d640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa20013f2b0 0x7fa2000b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fe59c640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa20013ee50 0x7fa2000b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fe59c640 1 -- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa20013f2b0 msgr2=0x7fa2000b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fe59c640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa20013f2b0 0x7fa2000b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fe59c640 1 -- 172.21.15.64:0/381135913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20013eb70 con 0x7fa1ec0061a0 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.367+0000 7fa1fed9d640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa20013f2b0 0x7fa2000b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.368+0000 7fa1fe59c640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa20013e630 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0007f90 tx=0x7fa1f0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.368+0000 7fa1f6ffd640 1 -- 172.21.15.64:0/381135913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000e030 con 0x7fa1ec0061a0 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.368+0000 7fa1f6ffd640 1 -- 172.21.15.64:0/381135913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0016470 con 0x7fa1ec0061a0 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.368+0000 7fa1f6ffd640 1 -- 172.21.15.64:0/381135913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0016820 con 0x7fa1ec0061a0 2023-11-24T08:07:12.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.368+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2000c95d0 con 0x7fa1ec0061a0 2023-11-24T08:07:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.368+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2000c9b90 con 0x7fa1ec0061a0 2023-11-24T08:07:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.369+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2000c9d50 con 0x7fa1ec0061a0 2023-11-24T08:07:12.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.370+0000 7fa1f6ffd640 1 -- 172.21.15.64:0/381135913 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa1f0016980 con 0x7fa1ec0061a0 2023-11-24T08:07:12.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.372+0000 7fa1f6ffd640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa1e006b080 0x7fa1e006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.372+0000 7fa1f6ffd640 1 -- 172.21.15.64:0/381135913 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1f000f070 con 0x7fa1ec0061a0 2023-11-24T08:07:12.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.372+0000 7fa1fdd9b640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa1e006b080 0x7fa1e006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.373+0000 7fa1fdd9b640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa1e006b080 0x7fa1e006d540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8033000 tx=0x7fa1e8009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:12.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.377+0000 7fa1f6ffd640 1 -- 172.21.15.64:0/381135913 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1f0055770 con 0x7fa1ec0061a0 2023-11-24T08:07:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.595+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa200151ed0 con 0x7fa1e006b080 2023-11-24T08:07:12.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.600+0000 7fa1f6ffd640 1 -- 172.21.15.64:0/381135913 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa200151ed0 con 0x7fa1e006b080 2023-11-24T08:07:12.601 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:07:12.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:12.601 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-11-24T08:07:12.602 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:12.602 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:07:12.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa1e006b080 msgr2=0x7fa1e006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa1e006b080 0x7fa1e006d540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8033000 tx=0x7fa1e8009030 comp rx=0 tx=0).stop 2023-11-24T08:07:12.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 msgr2=0x7fa20013e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa20013e630 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0007f90 tx=0x7fa1f0011660 comp rx=0 tx=0).stop 2023-11-24T08:07:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 shutdown_connections 2023-11-24T08:07:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa1e006b080 0x7fa1e006d540 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa20013f2b0 0x7fa2000b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa20013ee50 0x7fa2000b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 --2- 172.21.15.64:0/381135913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec0061a0 0x7fa20013e630 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 >> 172.21.15.64:0/381135913 conn(0x7fa2000b0840 msgr2=0x7fa200159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.602+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 shutdown_connections 2023-11-24T08:07:12.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.603+0000 7fa1ff59e640 1 -- 172.21.15.64:0/381135913 wait complete. 2023-11-24T08:07:12.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:07:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.853+0000 7f397ff2c640 1 Processor -- start 2023-11-24T08:07:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.853+0000 7f397ff2c640 1 -- start start 2023-11-24T08:07:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.855+0000 7f397ff2c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f397815aae0 0x7f397806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.855+0000 7f397ff2c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f397806d2a0 0x7f39780b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.855+0000 7f397ff2c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3978157b30 con 0x7f39780b2d70 2023-11-24T08:07:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.855+0000 7f397ff2c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397815aec0 con 0x7f397815aae0 2023-11-24T08:07:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.855+0000 7f397ff2c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397806d730 con 0x7f397806d2a0 2023-11-24T08:07:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397dcc8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f397815aae0 0x7f397806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397dcc8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f397815aae0 0x7f397806cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55786/0 (socket says 172.21.15.64:55786) 2023-11-24T08:07:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397dcc8640 1 -- 172.21.15.64:0/2838436536 learned_addr learned my addr 172.21.15.64:0/2838436536 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397d4c7640 1 --2- 172.21.15.64:0/2838436536 >> v2:172.21.15.91:3300/0 conn(0x7f397806d2a0 0x7f39780b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397dcc8640 1 -- 172.21.15.64:0/2838436536 >> v2:172.21.15.91:3300/0 conn(0x7f397806d2a0 msgr2=0x7f39780b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397dcc8640 1 --2- 172.21.15.64:0/2838436536 >> v2:172.21.15.91:3300/0 conn(0x7f397806d2a0 0x7f39780b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397dcc8640 1 -- 172.21.15.64:0/2838436536 >> v1:172.21.15.91:6789/0 conn(0x7f39780b2d70 legacy=0x7f397815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397dcc8640 1 -- 172.21.15.64:0/2838436536 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780b34f0 con 0x7f397815aae0 2023-11-24T08:07:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.856+0000 7f397d4c7640 1 --2- 172.21.15.64:0/2838436536 >> v2:172.21.15.91:3300/0 conn(0x7f397806d2a0 0x7f39780b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.857+0000 7f397dcc8640 1 --2- 172.21.15.64:0/2838436536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f397806cd60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f39780d0060 tx=0x7f397402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482679250b1e444d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.858+0000 7f397ccc6640 1 -- 172.21.15.64:0/2838436536 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974031870 con 0x7f397815aae0 2023-11-24T08:07:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.858+0000 7f397ccc6640 1 -- 172.21.15.64:0/2838436536 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39740319d0 con 0x7f397815aae0 2023-11-24T08:07:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.858+0000 7f397ccc6640 1 -- 172.21.15.64:0/2838436536 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974031d60 con 0x7f397815aae0 2023-11-24T08:07:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.858+0000 7f397ff2c640 1 -- 172.21.15.64:0/2838436536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 msgr2=0x7f397806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.858+0000 7f397ff2c640 1 --2- 172.21.15.64:0/2838436536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f397806cd60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f39780d0060 tx=0x7f397402b910 comp rx=0 tx=0).stop 2023-11-24T08:07:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.859+0000 7f397ff2c640 1 -- 172.21.15.64:0/2838436536 shutdown_connections 2023-11-24T08:07:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.859+0000 7f397ff2c640 1 --2- 172.21.15.64:0/2838436536 >> v2:172.21.15.91:3300/0 conn(0x7f397806d2a0 0x7f39780b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.859+0000 7f397ff2c640 1 --2- 172.21.15.64:0/2838436536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f397806cd60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.859+0000 7f397ff2c640 1 -- 172.21.15.64:0/2838436536 >> 172.21.15.64:0/2838436536 conn(0x7f39780b0ba0 msgr2=0x7f39780b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.859+0000 7f397ff2c640 1 -- 172.21.15.64:0/2838436536 shutdown_connections 2023-11-24T08:07:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.859+0000 7f397ff2c640 1 -- 172.21.15.64:0/2838436536 wait complete. 2023-11-24T08:07:12.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.861+0000 7f397ff2c640 1 Processor -- start 2023-11-24T08:07:12.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.861+0000 7f397ff2c640 1 -- start start 2023-11-24T08:07:12.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.862+0000 7f397ff2c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f397806d2a0 0x7f397814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.862+0000 7f397ff2c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39780b2d70 0x7f397814e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397ff2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f3978146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397ff2c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397814c700 con 0x7f397815aae0 2023-11-24T08:07:12.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397ff2c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3978099aa0 con 0x7f39780b2d70 2023-11-24T08:07:12.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397ff2c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397814dda0 con 0x7f397806d2a0 2023-11-24T08:07:12.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397e4c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f3978146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397e4c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f3978146a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55798/0 (socket says 172.21.15.64:55798) 2023-11-24T08:07:12.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397e4c9640 1 -- 172.21.15.64:0/5960311 learned_addr learned my addr 172.21.15.64:0/5960311 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397dcc8640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f397806d2a0 0x7f397814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397d4c7640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39780b2d70 0x7f397814e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397e4c9640 1 -- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f397806d2a0 msgr2=0x7f397814d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397e4c9640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f397806d2a0 0x7f397814d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397e4c9640 1 -- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39780b2d70 msgr2=0x7f397814e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397e4c9640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39780b2d70 0x7f397814e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397e4c9640 1 -- 172.21.15.64:0/5960311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740094d0 con 0x7f397815aae0 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.863+0000 7f397d4c7640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39780b2d70 0x7f397814e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.864+0000 7f397e4c9640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f3978146a70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f397000c780 tx=0x7f397000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.864+0000 7f396e7fc640 1 -- 172.21.15.64:0/5960311 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970004b60 con 0x7f397815aae0 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.864+0000 7f396e7fc640 1 -- 172.21.15.64:0/5960311 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397000ce90 con 0x7f397815aae0 2023-11-24T08:07:12.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.864+0000 7f396e7fc640 1 -- 172.21.15.64:0/5960311 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397000a850 con 0x7f397815aae0 2023-11-24T08:07:12.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.864+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397814c9a0 con 0x7f397815aae0 2023-11-24T08:07:12.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.864+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397814cc80 con 0x7f397815aae0 2023-11-24T08:07:12.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.865+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3978099f80 con 0x7f397815aae0 2023-11-24T08:07:12.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.866+0000 7f396e7fc640 1 -- 172.21.15.64:0/5960311 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f397000a9b0 con 0x7f397815aae0 2023-11-24T08:07:12.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.867+0000 7f396e7fc640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f395c06b080 0x7f395c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:12.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.867+0000 7f396e7fc640 1 -- 172.21.15.64:0/5960311 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3970089c10 con 0x7f397815aae0 2023-11-24T08:07:12.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.868+0000 7f397dcc8640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f395c06b080 0x7f395c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:12.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.869+0000 7f397dcc8640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f395c06b080 0x7f395c06d540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3974036000 tx=0x7f397403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:12.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:12.872+0000 7f396e7fc640 1 -- 172.21.15.64:0/5960311 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3978099f80 con 0x7f397815aae0 2023-11-24T08:07:13.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.067+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3978099f80 con 0x7f397815aae0 2023-11-24T08:07:13.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.068+0000 7f396e7fc640 1 -- 172.21.15.64:0/5960311 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3978099f80 con 0x7f397815aae0 2023-11-24T08:07:13.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f395c06b080 msgr2=0x7f395c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f395c06b080 0x7f395c06d540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3974036000 tx=0x7f397403c040 comp rx=0 tx=0).stop 2023-11-24T08:07:13.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 msgr2=0x7f3978146a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f3978146a70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f397000c780 tx=0x7f397000cc40 comp rx=0 tx=0).stop 2023-11-24T08:07:13.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 shutdown_connections 2023-11-24T08:07:13.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f395c06b080 0x7f395c06d540 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f397815aae0 0x7f3978146a70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f39780b2d70 0x7f397814e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 --2- 172.21.15.64:0/5960311 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f397806d2a0 0x7f397814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 >> 172.21.15.64:0/5960311 conn(0x7f39780b0ba0 msgr2=0x7f397815d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:13.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 shutdown_connections 2023-11-24T08:07:13.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.071+0000 7f397ff2c640 1 -- 172.21.15.64:0/5960311 wait complete. 2023-11-24T08:07:13.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol6780 2023-11-24T08:07:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.323+0000 7f79622d4640 1 Processor -- start 2023-11-24T08:07:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.323+0000 7f79622d4640 1 -- start start 2023-11-24T08:07:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.325+0000 7f79622d4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f795c0acd70 0x7f795c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.325+0000 7f79622d4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0ae490 0x7f795c0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.325+0000 7f79622d4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795c06d7a0 con 0x7f795c0ad690 2023-11-24T08:07:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.325+0000 7f79622d4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795c15a5b0 con 0x7f795c0acd70 2023-11-24T08:07:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.325+0000 7f79622d4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795c06ce70 con 0x7f795c0ae490 2023-11-24T08:07:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.325+0000 7f795bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f795c0acd70 0x7f795c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.326+0000 7f795bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f795c0acd70 0x7f795c06dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55814/0 (socket says 172.21.15.64:55814) 2023-11-24T08:07:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.326+0000 7f795bfff640 1 -- 172.21.15.64:0/1257425995 learned_addr learned my addr 172.21.15.64:0/1257425995 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.326+0000 7f79537fe640 1 --2- 172.21.15.64:0/1257425995 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0ae490 0x7f795c0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.326+0000 7f795bfff640 1 -- 172.21.15.64:0/1257425995 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0ae490 msgr2=0x7f795c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.326+0000 7f795bfff640 1 --2- 172.21.15.64:0/1257425995 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0ae490 0x7f795c0ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.326+0000 7f795bfff640 1 -- 172.21.15.64:0/1257425995 >> v1:172.21.15.64:6789/0 conn(0x7f795c0ad690 legacy=0x7f795c070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.326+0000 7f795bfff640 1 -- 172.21.15.64:0/1257425995 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c06d150 con 0x7f795c0acd70 2023-11-24T08:07:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.326+0000 7f795b7fe640 1 -- 172.21.15.64:0/1257425995 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089351588 0 0) 0x7f795c06d7a0 con 0x7f795c0ad690 2023-11-24T08:07:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.327+0000 7f795bfff640 1 --2- 172.21.15.64:0/1257425995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0acd70 0x7f795c06dc30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f794c009e50 tx=0x7f794c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2438ffcba35fbbc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.328+0000 7f795b7fe640 1 -- 172.21.15.64:0/1257425995 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c02cb00 con 0x7f795c0acd70 2023-11-24T08:07:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.328+0000 7f795b7fe640 1 -- 172.21.15.64:0/1257425995 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c02cc60 con 0x7f795c0acd70 2023-11-24T08:07:13.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.328+0000 7f795b7fe640 1 -- 172.21.15.64:0/1257425995 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c0343f0 con 0x7f795c0acd70 2023-11-24T08:07:13.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.329+0000 7f79622d4640 1 -- 172.21.15.64:0/1257425995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0acd70 msgr2=0x7f795c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.329+0000 7f79622d4640 1 --2- 172.21.15.64:0/1257425995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0acd70 0x7f795c06dc30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f794c009e50 tx=0x7f794c02bf60 comp rx=0 tx=0).stop 2023-11-24T08:07:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.329+0000 7f79622d4640 1 -- 172.21.15.64:0/1257425995 shutdown_connections 2023-11-24T08:07:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.329+0000 7f79622d4640 1 --2- 172.21.15.64:0/1257425995 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0ae490 0x7f795c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.329+0000 7f79622d4640 1 --2- 172.21.15.64:0/1257425995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0acd70 0x7f795c06dc30 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.329+0000 7f79622d4640 1 -- 172.21.15.64:0/1257425995 >> 172.21.15.64:0/1257425995 conn(0x7f795c0b0ba0 msgr2=0x7f795c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.329+0000 7f79622d4640 1 -- 172.21.15.64:0/1257425995 shutdown_connections 2023-11-24T08:07:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.329+0000 7f79622d4640 1 -- 172.21.15.64:0/1257425995 wait complete. 2023-11-24T08:07:13.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.331+0000 7f79622d4640 1 Processor -- start 2023-11-24T08:07:13.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.332+0000 7f79622d4640 1 -- start start 2023-11-24T08:07:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f79622d4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0acd70 0x7f795c0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f79622d4640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795c0ad690 0x7f795c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f79622d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0ae490 0x7f795c0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f79622d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c1536c0 con 0x7f795c0ae490 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f79622d4640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c15b600 con 0x7f795c0ad690 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f79622d4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795c15b8e0 con 0x7f795c0acd70 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f7960871640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0ae490 0x7f795c0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f7960871640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0ae490 0x7f795c0c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55820/0 (socket says 172.21.15.64:55820) 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.333+0000 7f7960871640 1 -- 172.21.15.64:0/2717675036 learned_addr learned my addr 172.21.15.64:0/2717675036 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f795bfff640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0acd70 0x7f795c0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f79537fe640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795c0ad690 0x7f795c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f7960871640 1 -- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0acd70 msgr2=0x7f795c0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f7960871640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0acd70 0x7f795c0ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f7960871640 1 -- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795c0ad690 msgr2=0x7f795c1553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f7960871640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795c0ad690 0x7f795c1553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f7960871640 1 -- 172.21.15.64:0/2717675036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c009b30 con 0x7f795c0ae490 2023-11-24T08:07:13.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f7960871640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0ae490 0x7f795c0c8260 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f795400c410 tx=0x7f795400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.334+0000 7f79597fa640 1 -- 172.21.15.64:0/2717675036 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540035f0 con 0x7f795c0ae490 2023-11-24T08:07:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.335+0000 7f79597fa640 1 -- 172.21.15.64:0/2717675036 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7954003750 con 0x7f795c0ae490 2023-11-24T08:07:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.335+0000 7f79597fa640 1 -- 172.21.15.64:0/2717675036 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954003b00 con 0x7f795c0ae490 2023-11-24T08:07:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.335+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c15bbc0 con 0x7f795c0ae490 2023-11-24T08:07:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.335+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c155930 con 0x7f795c0ae490 2023-11-24T08:07:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.336+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c153830 con 0x7f795c0ae490 2023-11-24T08:07:13.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.337+0000 7f79597fa640 1 -- 172.21.15.64:0/2717675036 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7954003c60 con 0x7f795c0ae490 2023-11-24T08:07:13.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.338+0000 7f79597fa640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f793c06b080 0x7f793c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.338+0000 7f79597fa640 1 -- 172.21.15.64:0/2717675036 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7954088fe0 con 0x7f795c0ae490 2023-11-24T08:07:13.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.339+0000 7f795bfff640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f793c06b080 0x7f793c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.340+0000 7f795bfff640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f793c06b080 0x7f793c06d540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f794c035000 tx=0x7f794c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:13.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.345+0000 7f79597fa640 1 -- 172.21.15.64:0/2717675036 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f795c153830 con 0x7f795c0ae490 2023-11-24T08:07:13.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.569+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol6780", "target": ["mon-mgr", ""]}) v1 -- 0x7f795c0b3d90 con 0x7f793c06b080 2023-11-24T08:07:13.583 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:13.581+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:13.583 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:13.582+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:13.584 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:13.582+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:13.584 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:13.582+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:13.584 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:13.582+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:13.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.661+0000 7f79597fa640 1 -- 172.21.15.64:0/2717675036 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f794c02ca80 con 0x7f793c06b080 2023-11-24T08:07:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.664+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f793c06b080 msgr2=0x7f793c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.664+0000 7f79622d4640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f793c06b080 0x7f793c06d540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f794c035000 tx=0x7f794c030000 comp rx=0 tx=0).stop 2023-11-24T08:07:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.664+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0ae490 msgr2=0x7f795c0c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.664+0000 7f79622d4640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0ae490 0x7f795c0c8260 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f795400c410 tx=0x7f795400c8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.664+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 shutdown_connections 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.665+0000 7f79622d4640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f793c06b080 0x7f793c06d540 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.665+0000 7f79622d4640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c0ae490 0x7f795c0c8260 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.665+0000 7f79622d4640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f795c0ad690 0x7f795c1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.665+0000 7f79622d4640 1 --2- 172.21.15.64:0/2717675036 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f795c0acd70 0x7f795c0ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.665+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 >> 172.21.15.64:0/2717675036 conn(0x7f795c0b0ba0 msgr2=0x7f795c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.665+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 shutdown_connections 2023-11-24T08:07:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.665+0000 7f79622d4640 1 -- 172.21.15.64:0/2717675036 wait complete. 2023-11-24T08:07:13.692 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol6780 subvol_snap5258 2023-11-24T08:07:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.915+0000 7f487208c640 1 Processor -- start 2023-11-24T08:07:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.915+0000 7f487208c640 1 -- start start 2023-11-24T08:07:13.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.917+0000 7f487208c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 0x7f486c0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.917+0000 7f487208c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f486c06d9e0 0x7f486c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.917+0000 7f487208c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486c15f930 con 0x7f486c144770 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.917+0000 7f487208c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486c0b43b0 con 0x7f486c06d9e0 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.917+0000 7f487208c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486c0b3b10 con 0x7f486c15f270 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.917+0000 7f487108a640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f486c144770 0x7f486c15eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32838/0 (socket says 172.21.15.64:32838) 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.917+0000 7f487108a640 1 -- 172.21.15.64:0/139945544 learned_addr learned my addr 172.21.15.64:0/139945544 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f487188b640 1 --2- 172.21.15.64:0/139945544 >> v2:172.21.15.91:3300/0 conn(0x7f486c06d9e0 0x7f486c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4870889640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 0x7f486c0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111521707 0 0) 0x7f486c15f930 con 0x7f486c144770 2023-11-24T08:07:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4858003520 con 0x7f486c144770 2023-11-24T08:07:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974184333 0 0) 0x7f4858003520 con 0x7f486c144770 2023-11-24T08:07:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 msgr2=0x7f486c0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 0x7f486c0cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 >> v2:172.21.15.91:3300/0 conn(0x7f486c06d9e0 msgr2=0x7f486c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 --2- 172.21.15.64:0/139945544 >> v2:172.21.15.91:3300/0 conn(0x7f486c06d9e0 0x7f486c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c0b3df0 con 0x7f486c144770 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f4868001670 con 0x7f486c144770 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 >> v1:172.21.15.64:6789/0 conn(0x7f486c144770 legacy=0x7f486c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.918+0000 7f4863fff640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580039e0 0x7f4858003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4863fff640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580047c0 0x7f4858004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4863fff640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 0x7f4858005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c0b43b0 con 0x7f48580047c0 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c0b3b10 con 0x7f48580039e0 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485800b9b0 con 0x7f486c15f270 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4870889640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580047c0 0x7f4858004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f487108a640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580039e0 0x7f4858003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4870889640 1 -- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 msgr2=0x7f4858005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f487188b640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 0x7f4858005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4870889640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 0x7f4858005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4870889640 1 -- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580039e0 msgr2=0x7f4858003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4870889640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580039e0 0x7f4858003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f4870889640 1 -- 172.21.15.64:0/139945544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858003520 con 0x7f48580047c0 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f487188b640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15f270 0x7f4858005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.919+0000 7f487108a640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580039e0 0x7f4858003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.920+0000 7f4870889640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580047c0 0x7f4858004c40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f485c009ac0 tx=0x7f48580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808176e76551eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c02c6b0 con 0x7f48580047c0 2023-11-24T08:07:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c02c810 con 0x7f48580047c0 2023-11-24T08:07:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f4863fff640 1 -- 172.21.15.64:0/139945544 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c02cba0 con 0x7f48580047c0 2023-11-24T08:07:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f487208c640 1 -- 172.21.15.64:0/139945544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580047c0 msgr2=0x7f4858004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f487208c640 1 --2- 172.21.15.64:0/139945544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580047c0 0x7f4858004c40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f485c009ac0 tx=0x7f48580057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f487188b640 1 -- 172.21.15.64:0/139945544 reap_dead start 2023-11-24T08:07:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f487208c640 1 -- 172.21.15.64:0/139945544 shutdown_connections 2023-11-24T08:07:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f487208c640 1 -- 172.21.15.64:0/139945544 >> 172.21.15.64:0/139945544 conn(0x7f486c0cec10 msgr2=0x7f486c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f487208c640 1 -- 172.21.15.64:0/139945544 shutdown_connections 2023-11-24T08:07:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.921+0000 7f487208c640 1 -- 172.21.15.64:0/139945544 wait complete. 2023-11-24T08:07:13.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.923+0000 7f487208c640 1 Processor -- start 2023-11-24T08:07:13.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.924+0000 7f487208c640 1 -- start start 2023-11-24T08:07:13.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.924+0000 7f487208c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580047c0 0x7f486c13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487208c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15d0c0 0x7f486c13a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487208c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486c1f2ba0 0x7f486c1f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487208c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c1f3f70 con 0x7f486c1f2ba0 2023-11-24T08:07:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487208c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c1f3560 con 0x7f48580047c0 2023-11-24T08:07:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487208c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486c1f3c70 con 0x7f486c15d0c0 2023-11-24T08:07:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487188b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486c1f2ba0 0x7f486c1f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487188b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486c1f2ba0 0x7f486c1f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55832/0 (socket says 172.21.15.64:55832) 2023-11-24T08:07:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487188b640 1 -- 172.21.15.64:0/2331850679 learned_addr learned my addr 172.21.15.64:0/2331850679 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487108a640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580047c0 0x7f486c13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f4870889640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15d0c0 0x7f486c13a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487188b640 1 -- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15d0c0 msgr2=0x7f486c13a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487188b640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15d0c0 0x7f486c13a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487188b640 1 -- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580047c0 msgr2=0x7f486c13eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487188b640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580047c0 0x7f486c13eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.925+0000 7f487188b640 1 -- 172.21.15.64:0/2331850679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c15d8d0 con 0x7f486c1f2ba0 2023-11-24T08:07:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.926+0000 7f487188b640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486c1f2ba0 0x7f486c1f3020 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f486c144e80 tx=0x7f486400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.926+0000 7f4861ffb640 1 -- 172.21.15.64:0/2331850679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864017400 con 0x7f486c1f2ba0 2023-11-24T08:07:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.926+0000 7f4861ffb640 1 -- 172.21.15.64:0/2331850679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864011040 con 0x7f486c1f2ba0 2023-11-24T08:07:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.926+0000 7f4861ffb640 1 -- 172.21.15.64:0/2331850679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864017780 con 0x7f486c1f2ba0 2023-11-24T08:07:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.926+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486c15dbb0 con 0x7f486c1f2ba0 2023-11-24T08:07:13.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.926+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486c15d540 con 0x7f486c1f2ba0 2023-11-24T08:07:13.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.927+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486c04e840 con 0x7f486c1f2ba0 2023-11-24T08:07:13.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.928+0000 7f4861ffb640 1 -- 172.21.15.64:0/2331850679 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f48640203b0 con 0x7f486c1f2ba0 2023-11-24T08:07:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.930+0000 7f4861ffb640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f485006b080 0x7f485006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.930+0000 7f4861ffb640 1 -- 172.21.15.64:0/2331850679 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4864089e20 con 0x7f486c1f2ba0 2023-11-24T08:07:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.930+0000 7f487108a640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f485006b080 0x7f485006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.931+0000 7f487108a640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f485006b080 0x7f485006d540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4868001860 tx=0x7f4868009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:13.934+0000 7f4861ffb640 1 -- 172.21.15.64:0/2331850679 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48640549c0 con 0x7f486c1f2ba0 2023-11-24T08:07:14.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.163+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol6780", "snap_name": "subvol_snap5258", "target": ["mon-mgr", ""]}) v1 -- 0x7f486c144b50 con 0x7f485006b080 2023-11-24T08:07:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.202+0000 7f4861ffb640 1 -- 172.21.15.64:0/2331850679 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f486800d070 con 0x7f485006b080 2023-11-24T08:07:14.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f485006b080 msgr2=0x7f485006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f485006b080 0x7f485006d540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4868001860 tx=0x7f4868009040 comp rx=0 tx=0).stop 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486c1f2ba0 msgr2=0x7f486c1f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486c1f2ba0 0x7f486c1f3020 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f486c144e80 tx=0x7f486400f4a0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 shutdown_connections 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f485006b080 0x7f485006d540 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486c1f2ba0 0x7f486c1f3020 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f486c15d0c0 0x7f486c13a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 --2- 172.21.15.64:0/2331850679 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f48580047c0 0x7f486c13eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 >> 172.21.15.64:0/2331850679 conn(0x7f486c0cec10 msgr2=0x7f486c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:14.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 shutdown_connections 2023-11-24T08:07:14.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.205+0000 7f487208c640 1 -- 172.21.15.64:0/2331850679 wait complete. 2023-11-24T08:07:14.227 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol6780 subvol_snap5258 2023-11-24T08:07:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.454+0000 7fbdee978640 1 Processor -- start 2023-11-24T08:07:14.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.454+0000 7fbdee978640 1 -- start start 2023-11-24T08:07:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.456+0000 7fbdee978640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 0x7fbde80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.456+0000 7fbdee978640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbde80adb60 0x7fbde80adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.456+0000 7fbdee978640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde815b730 con 0x7fbde815b350 2023-11-24T08:07:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.456+0000 7fbdee978640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde80ac740 con 0x7fbde80adb60 2023-11-24T08:07:14.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.456+0000 7fbdee978640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80ac460 con 0x7fbde809cb30 2023-11-24T08:07:14.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.457+0000 7fbde7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbde815b350 0x7fbde80b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32840/0 (socket says 172.21.15.64:32840) 2023-11-24T08:07:14.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.457+0000 7fbde7fff640 1 -- 172.21.15.64:0/3268441460 learned_addr learned my addr 172.21.15.64:0/3268441460 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:14.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.457+0000 7fbddffff640 1 --2- 172.21.15.64:0/3268441460 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 0x7fbde80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.457+0000 7fbdecf15640 1 --2- 172.21.15.64:0/3268441460 >> v2:172.21.15.64:3300/0 conn(0x7fbde80adb60 0x7fbde80adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.457+0000 7fbde77fe640 1 -- 172.21.15.64:0/3268441460 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327677356 0 0) 0x7fbdd8001110 con 0x7fbde815b350 2023-11-24T08:07:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.457+0000 7fbde77fe640 1 -- 172.21.15.64:0/3268441460 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd4003520 con 0x7fbde815b350 2023-11-24T08:07:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.458+0000 7fbdecf15640 1 -- 172.21.15.64:0/3268441460 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 msgr2=0x7fbde80abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.458+0000 7fbdecf15640 1 --2- 172.21.15.64:0/3268441460 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 0x7fbde80abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.458+0000 7fbde77fe640 1 -- 172.21.15.64:0/3268441460 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4124501952 0 0) 0x7fbdd4003520 con 0x7fbde815b350 2023-11-24T08:07:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.458+0000 7fbdecf15640 1 -- 172.21.15.64:0/3268441460 >> v1:172.21.15.64:6789/0 conn(0x7fbde815b350 legacy=0x7fbde80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.458+0000 7fbdecf15640 1 -- 172.21.15.64:0/3268441460 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80b2a50 con 0x7fbde80adb60 2023-11-24T08:07:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.459+0000 7fbdecf15640 1 --2- 172.21.15.64:0/3268441460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde80adfb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d90 tx=0x7fbde002bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a475db43d94e0b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.459+0000 7fbde77fe640 1 -- 172.21.15.64:0/3268441460 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8001800 con 0x7fbde80adb60 2023-11-24T08:07:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.459+0000 7fbde77fe640 1 -- 172.21.15.64:0/3268441460 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde002c8a0 con 0x7fbde80adb60 2023-11-24T08:07:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.459+0000 7fbde77fe640 1 -- 172.21.15.64:0/3268441460 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde002cc30 con 0x7fbde80adb60 2023-11-24T08:07:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.459+0000 7fbdee978640 1 -- 172.21.15.64:0/3268441460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 msgr2=0x7fbde80adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.459+0000 7fbdee978640 1 --2- 172.21.15.64:0/3268441460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde80adfb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d90 tx=0x7fbde002bd70 comp rx=0 tx=0).stop 2023-11-24T08:07:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.460+0000 7fbdee978640 1 -- 172.21.15.64:0/3268441460 shutdown_connections 2023-11-24T08:07:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.460+0000 7fbdee978640 1 --2- 172.21.15.64:0/3268441460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde80adfb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.460+0000 7fbdee978640 1 --2- 172.21.15.64:0/3268441460 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 0x7fbde80abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.460+0000 7fbdee978640 1 -- 172.21.15.64:0/3268441460 >> 172.21.15.64:0/3268441460 conn(0x7fbde80b0ba0 msgr2=0x7fbde80b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.460+0000 7fbdee978640 1 -- 172.21.15.64:0/3268441460 shutdown_connections 2023-11-24T08:07:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.460+0000 7fbdee978640 1 -- 172.21.15.64:0/3268441460 wait complete. 2023-11-24T08:07:14.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.461+0000 7fbdee978640 1 Processor -- start 2023-11-24T08:07:14.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.462+0000 7fbdee978640 1 -- start start 2023-11-24T08:07:14.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbdee978640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 0x7fbde813c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:14.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbdee978640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde8149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:14.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbdee978640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbde815b350 0x7fbde81f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:14.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbdee978640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde814c660 con 0x7fbde80adb60 2023-11-24T08:07:14.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbdee978640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde814bae0 con 0x7fbde815b350 2023-11-24T08:07:14.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbdee978640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde814bdc0 con 0x7fbde809cb30 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbddffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde8149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbddffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde8149290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55848/0 (socket says 172.21.15.64:55848) 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbddffff640 1 -- 172.21.15.64:0/4182641376 learned_addr learned my addr 172.21.15.64:0/4182641376 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbde7fff640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 0x7fbde813c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbdecf15640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbde815b350 0x7fbde81f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbddffff640 1 -- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 msgr2=0x7fbde813c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbddffff640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 0x7fbde813c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbddffff640 1 -- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbde815b350 msgr2=0x7fbde81f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbddffff640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbde815b350 0x7fbde81f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbddffff640 1 -- 172.21.15.64:0/4182641376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0009a70 con 0x7fbde80adb60 2023-11-24T08:07:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.463+0000 7fbdecf15640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbde815b350 0x7fbde81f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.464+0000 7fbddffff640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde8149290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e870 tx=0x7fbdd000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.464+0000 7fbde57fa640 1 -- 172.21.15.64:0/4182641376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00094f0 con 0x7fbde80adb60 2023-11-24T08:07:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.464+0000 7fbde57fa640 1 -- 172.21.15.64:0/4182641376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd000c940 con 0x7fbde80adb60 2023-11-24T08:07:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.464+0000 7fbde57fa640 1 -- 172.21.15.64:0/4182641376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd000ccf0 con 0x7fbde80adb60 2023-11-24T08:07:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.464+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde814c0a0 con 0x7fbde80adb60 2023-11-24T08:07:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.464+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde8148c40 con 0x7fbde80adb60 2023-11-24T08:07:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.465+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde8148e30 con 0x7fbde80adb60 2023-11-24T08:07:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.466+0000 7fbde57fa640 1 -- 172.21.15.64:0/4182641376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fbdd0013470 con 0x7fbde80adb60 2023-11-24T08:07:14.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.467+0000 7fbde57fa640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdc806b080 0x7fbdc806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:14.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.467+0000 7fbde57fa640 1 -- 172.21.15.64:0/4182641376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdd0089b60 con 0x7fbde80adb60 2023-11-24T08:07:14.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.468+0000 7fbde7fff640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdc806b080 0x7fbdc806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:14.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.469+0000 7fbde7fff640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdc806b080 0x7fbdc806d540 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8001930 tx=0x7fbdd8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:14.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.472+0000 7fbde57fa640 1 -- 172.21.15.64:0/4182641376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbdd00548b0 con 0x7fbde80adb60 2023-11-24T08:07:14.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.706+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol6780", "snap_name": "subvol_snap5258", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde80ae4f0 con 0x7fbdc806b080 2023-11-24T08:07:14.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.759+0000 7fbde57fa640 1 -- 172.21.15.64:0/4182641376 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbdd8001be0 con 0x7fbdc806b080 2023-11-24T08:07:14.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.762+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdc806b080 msgr2=0x7fbdc806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:14.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.762+0000 7fbdee978640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdc806b080 0x7fbdc806d540 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8001930 tx=0x7fbdd8009040 comp rx=0 tx=0).stop 2023-11-24T08:07:14.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.762+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 msgr2=0x7fbde8149290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:14.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.762+0000 7fbdee978640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde8149290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e870 tx=0x7fbdd000ed30 comp rx=0 tx=0).stop 2023-11-24T08:07:14.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.763+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 shutdown_connections 2023-11-24T08:07:14.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.763+0000 7fbdee978640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbdc806b080 0x7fbdc806d540 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.763+0000 7fbdee978640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbde815b350 0x7fbde81f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.763+0000 7fbdee978640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80adb60 0x7fbde8149290 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.763+0000 7fbdee978640 1 --2- 172.21.15.64:0/4182641376 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbde809cb30 0x7fbde813c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:14.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.763+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 >> 172.21.15.64:0/4182641376 conn(0x7fbde80b0ba0 msgr2=0x7fbde809ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:14.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.763+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 shutdown_connections 2023-11-24T08:07:14.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:14.763+0000 7fbdee978640 1 -- 172.21.15.64:0/4182641376 wait complete. 2023-11-24T08:07:14.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol6780 subvol_snap5258 2023-11-24T08:07:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.016+0000 7fc3e7fff640 1 Processor -- start 2023-11-24T08:07:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.016+0000 7fc3e7fff640 1 -- start start 2023-11-24T08:07:15.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.017+0000 7fc3e7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc3e815d3b0 0x7fc3e80ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.017+0000 7fc3e7fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e80d0fa0 0x7fc3e80d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.017+0000 7fc3e7fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e80d2010 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.017+0000 7fc3e7fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e80c6bf0 con 0x7fc3e815d3b0 2023-11-24T08:07:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.017+0000 7fc3e7fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e80d1400 con 0x7fc3e80d0fa0 2023-11-24T08:07:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc3e815d3b0 0x7fc3e80ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc3e815d3b0 0x7fc3e80ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55864/0 (socket says 172.21.15.64:55864) 2023-11-24T08:07:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e6ffd640 1 -- 172.21.15.64:0/3349540180 learned_addr learned my addr 172.21.15.64:0/3349540180 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e67fc640 1 --2- 172.21.15.64:0/3349540180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e80d0fa0 0x7fc3e80d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e6ffd640 1 -- 172.21.15.64:0/3349540180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e80d0fa0 msgr2=0x7fc3e80d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e6ffd640 1 --2- 172.21.15.64:0/3349540180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e80d0fa0 0x7fc3e80d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e6ffd640 1 -- 172.21.15.64:0/3349540180 >> v1:172.21.15.64:6789/0 conn(0x7fc3e80d1bc0 legacy=0x7fc3e80b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e6ffd640 1 -- 172.21.15.64:0/3349540180 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e80cedc0 con 0x7fc3e815d3b0 2023-11-24T08:07:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e67fc640 1 --2- 172.21.15.64:0/3349540180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e80d0fa0 0x7fc3e80d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.018+0000 7fc3e5ffb640 1 -- 172.21.15.64:0/3349540180 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026434438 0 0) 0x7fc3e0001110 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.020+0000 7fc3e6ffd640 1 --2- 172.21.15.64:0/3349540180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3e815d3b0 0x7fc3e80ce6c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009e50 tx=0x7fc3d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff7fef9e52a672e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.020+0000 7fc3e5ffb640 1 -- 172.21.15.64:0/3349540180 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d802ca80 con 0x7fc3e815d3b0 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.020+0000 7fc3e5ffb640 1 -- 172.21.15.64:0/3349540180 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d802cbe0 con 0x7fc3e815d3b0 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.020+0000 7fc3e5ffb640 1 -- 172.21.15.64:0/3349540180 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80343f0 con 0x7fc3e815d3b0 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.020+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3349540180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3e815d3b0 msgr2=0x7fc3e80ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.020+0000 7fc3e7fff640 1 --2- 172.21.15.64:0/3349540180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3e815d3b0 0x7fc3e80ce6c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009e50 tx=0x7fc3d802bee0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.021+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3349540180 shutdown_connections 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.021+0000 7fc3e7fff640 1 --2- 172.21.15.64:0/3349540180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e80d0fa0 0x7fc3e80d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.021+0000 7fc3e7fff640 1 --2- 172.21.15.64:0/3349540180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3e815d3b0 0x7fc3e80ce6c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.021+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3349540180 >> 172.21.15.64:0/3349540180 conn(0x7fc3e80d30a0 msgr2=0x7fc3e80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.021+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3349540180 shutdown_connections 2023-11-24T08:07:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.021+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3349540180 wait complete. 2023-11-24T08:07:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.022+0000 7fc3e7fff640 1 Processor -- start 2023-11-24T08:07:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.023+0000 7fc3e7fff640 1 -- start start 2023-11-24T08:07:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.023+0000 7fc3e7fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3e80d0fa0 0x7fc3e815f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3e80d1bc0 0x7fc3e815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e7fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e815d3b0 0x7fc3e809fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e7fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8150130 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e7fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e814f4b0 con 0x7fc3e80d0fa0 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e7fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e814f790 con 0x7fc3e815d3b0 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3e80d1bc0 0x7fc3e815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3e80d1bc0 0x7fc3e815fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55868/0 (socket says 172.21.15.64:55868) 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e67fc640 1 -- 172.21.15.64:0/3966070995 learned_addr learned my addr 172.21.15.64:0/3966070995 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e6ffd640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3e80d0fa0 0x7fc3e815f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e77fe640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e815d3b0 0x7fc3e809fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e67fc640 1 -- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e815d3b0 msgr2=0x7fc3e809fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e67fc640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e815d3b0 0x7fc3e809fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e67fc640 1 -- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3e80d0fa0 msgr2=0x7fc3e815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e67fc640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc3e80d0fa0 0x7fc3e815f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e67fc640 1 -- 172.21.15.64:0/3966070995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8009b30 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.024+0000 7fc3e77fe640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc3e815d3b0 0x7fc3e809fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.025+0000 7fc3e67fc640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3e80d1bc0 0x7fc3e815fef0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c7f0 tx=0x7fc3d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.025+0000 7fc3df7fe640 1 -- 172.21.15.64:0/3966070995 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0007820 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.025+0000 7fc3df7fe640 1 -- 172.21.15.64:0/3966070995 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0007980 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.025+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3966070995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e814fa70 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.025+0000 7fc3df7fe640 1 -- 172.21.15.64:0/3966070995 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0007cf0 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.025+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3966070995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e814ef40 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.026+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3966070995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e804e840 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.027+0000 7fc3df7fe640 1 -- 172.21.15.64:0/3966070995 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc3d00113f0 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.028+0000 7fc3df7fe640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc3c806b100 0x7fc3c806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.028+0000 7fc3df7fe640 1 -- 172.21.15.64:0/3966070995 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3d0089d50 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.029+0000 7fc3e6ffd640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc3c806b100 0x7fc3c806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.030+0000 7fc3e6ffd640 1 --2- 172.21.15.64:0/3966070995 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc3c806b100 0x7fc3c806d5c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8035000 tx=0x7fc3d8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:15.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.033+0000 7fc3df7fe640 1 -- 172.21.15.64:0/3966070995 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3d0054a70 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.126+0000 7fc3df7fe640 1 -- 172.21.15.64:0/3966070995 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fc3d0054e40 con 0x7fc3e80d1bc0 2023-11-24T08:07:15.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.267+0000 7fc3e7fff640 1 -- 172.21.15.64:0/3966070995 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol6780", "snap_name": "subvol_snap5258", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3e80cebb0 con 0x7fc3c806b100 2023-11-24T08:07:15.302 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.300+0000 7fe78a34d640 -1 mgr.server reply reply (2) No such file or directory snapshot 'b'subvol_snap5258'' does not exist 2023-11-24T08:07:15.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.301+0000 7fc3df7fe640 1 -- 172.21.15.64:0/3966070995 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: -2 snapshot 'b'subvol_snap5258'' does not exist) v1 ==== 52+0+0 (secure 0 0 0) 0x7fc3e80cebb0 con 0x7fc3c806b100 2023-11-24T08:07:15.303 INFO:teuthology.orchestra.run.smithi064.stderr:Error ENOENT: snapshot 'b'subvol_snap5258'' does not exist 2023-11-24T08:07:15.316 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-24T08:07:15.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol6780 2023-11-24T08:07:15.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.544+0000 7f93cc9e2640 1 Processor -- start 2023-11-24T08:07:15.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.545+0000 7f93cc9e2640 1 -- start start 2023-11-24T08:07:15.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.546+0000 7f93cc9e2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93c815d3b0 0x7f93c80ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.546+0000 7f93cc9e2640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f93c80d0fa0 0x7f93c80d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.546+0000 7f93cc9e2640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c80c6bf0 con 0x7f93c80d1bc0 2023-11-24T08:07:15.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.546+0000 7f93cc9e2640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c80d1430 con 0x7f93c815d3b0 2023-11-24T08:07:15.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.546+0000 7f93cc9e2640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c80d1fa0 con 0x7f93c80d0fa0 2023-11-24T08:07:15.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.546+0000 7f93c6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93c815d3b0 0x7f93c80ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.546+0000 7f93c6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93c815d3b0 0x7f93c80ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55882/0 (socket says 172.21.15.64:55882) 2023-11-24T08:07:15.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.546+0000 7f93c6d9d640 1 -- 172.21.15.64:0/2514924160 learned_addr learned my addr 172.21.15.64:0/2514924160 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:15.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.547+0000 7f93c659c640 1 --2- 172.21.15.64:0/2514924160 >> v2:172.21.15.91:3300/0 conn(0x7f93c80d0fa0 0x7f93c80d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.547+0000 7f93c6d9d640 1 -- 172.21.15.64:0/2514924160 >> v2:172.21.15.91:3300/0 conn(0x7f93c80d0fa0 msgr2=0x7f93c80d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:15.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.547+0000 7f93c6d9d640 1 --2- 172.21.15.64:0/2514924160 >> v2:172.21.15.91:3300/0 conn(0x7f93c80d0fa0 0x7f93c80d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.547+0000 7f93c6d9d640 1 -- 172.21.15.64:0/2514924160 >> v1:172.21.15.91:6789/0 conn(0x7f93c80d1bc0 legacy=0x7f93c80b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:15.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.547+0000 7f93c6d9d640 1 -- 172.21.15.64:0/2514924160 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c80cedc0 con 0x7f93c815d3b0 2023-11-24T08:07:15.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.547+0000 7f93c659c640 1 --2- 172.21.15.64:0/2514924160 >> v2:172.21.15.91:3300/0 conn(0x7f93c80d0fa0 0x7f93c80d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:15.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.548+0000 7f93c6d9d640 1 --2- 172.21.15.64:0/2514924160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c80ce6c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f93c808c4a0 tx=0x7f93b802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a066bb7b6fa4d446 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:15.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93c5d9b640 1 -- 172.21.15.64:0/2514924160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b802c9f0 con 0x7f93c815d3b0 2023-11-24T08:07:15.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93c5d9b640 1 -- 172.21.15.64:0/2514924160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b802cb50 con 0x7f93c815d3b0 2023-11-24T08:07:15.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93c5d9b640 1 -- 172.21.15.64:0/2514924160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b80343f0 con 0x7f93c815d3b0 2023-11-24T08:07:15.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93cc9e2640 1 -- 172.21.15.64:0/2514924160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 msgr2=0x7f93c80ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/2514924160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c80ce6c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f93c808c4a0 tx=0x7f93b802be50 comp rx=0 tx=0).stop 2023-11-24T08:07:15.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93cc9e2640 1 -- 172.21.15.64:0/2514924160 shutdown_connections 2023-11-24T08:07:15.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/2514924160 >> v2:172.21.15.91:3300/0 conn(0x7f93c80d0fa0 0x7f93c80d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/2514924160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c80ce6c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.549+0000 7f93cc9e2640 1 -- 172.21.15.64:0/2514924160 >> 172.21.15.64:0/2514924160 conn(0x7f93c80d30a0 msgr2=0x7f93c80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.550+0000 7f93cc9e2640 1 -- 172.21.15.64:0/2514924160 shutdown_connections 2023-11-24T08:07:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.550+0000 7f93cc9e2640 1 -- 172.21.15.64:0/2514924160 wait complete. 2023-11-24T08:07:15.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.551+0000 7f93cc9e2640 1 Processor -- start 2023-11-24T08:07:15.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.551+0000 7f93cc9e2640 1 -- start start 2023-11-24T08:07:15.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.552+0000 7f93cc9e2640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c80d0fa0 0x7f93c809cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.552+0000 7f93cc9e2640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93c80d1bc0 0x7f93c814f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.552+0000 7f93cc9e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c809d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93cc9e2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80ccc50 con 0x7f93c815d3b0 2023-11-24T08:07:15.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93cc9e2640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80ccf30 con 0x7f93c80d0fa0 2023-11-24T08:07:15.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93cc9e2640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80cd210 con 0x7f93c80d1bc0 2023-11-24T08:07:15.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c809d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c809d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55894/0 (socket says 172.21.15.64:55894) 2023-11-24T08:07:15.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c759e640 1 -- 172.21.15.64:0/1983493297 learned_addr learned my addr 172.21.15.64:0/1983493297 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:15.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c6d9d640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c80d0fa0 0x7f93c809cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c659c640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93c80d1bc0 0x7f93c814f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c759e640 1 -- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93c80d1bc0 msgr2=0x7f93c814f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c759e640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93c80d1bc0 0x7f93c814f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c759e640 1 -- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c80d0fa0 msgr2=0x7f93c809cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c759e640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c80d0fa0 0x7f93c809cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c759e640 1 -- 172.21.15.64:0/1983493297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b8009b30 con 0x7f93c815d3b0 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.553+0000 7f93c6d9d640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c80d0fa0 0x7f93c809cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.554+0000 7f93c759e640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c809d660 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f93c000b410 tx=0x7f93c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.554+0000 7f93bf7fe640 1 -- 172.21.15.64:0/1983493297 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0004020 con 0x7f93c815d3b0 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.554+0000 7f93bf7fe640 1 -- 172.21.15.64:0/1983493297 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c0004180 con 0x7f93c815d3b0 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.554+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c8150a00 con 0x7f93c815d3b0 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.554+0000 7f93bf7fe640 1 -- 172.21.15.64:0/1983493297 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0011630 con 0x7f93c815d3b0 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.554+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c814f9b0 con 0x7f93c815d3b0 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.555+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c8150f20 con 0x7f93c815d3b0 2023-11-24T08:07:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.556+0000 7f93bf7fe640 1 -- 172.21.15.64:0/1983493297 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f93c0011790 con 0x7f93c815d3b0 2023-11-24T08:07:15.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.557+0000 7f93bf7fe640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93a806b110 0x7f93a806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:15.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.557+0000 7f93bf7fe640 1 -- 172.21.15.64:0/1983493297 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93c0089880 con 0x7f93c815d3b0 2023-11-24T08:07:15.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.558+0000 7f93c6d9d640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93a806b110 0x7f93a806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:15.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.559+0000 7f93c6d9d640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93a806b110 0x7f93a806d5d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f93b8035000 tx=0x7f93b8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.562+0000 7f93bf7fe640 1 -- 172.21.15.64:0/1983493297 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93c8150f20 con 0x7f93c815d3b0 2023-11-24T08:07:15.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:07:15.601 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:07:15.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.776+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol6780", "target": ["mon-mgr", ""]}) v1 -- 0x7f93c80cd0a0 con 0x7f93a806b110 2023-11-24T08:07:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.815+0000 7f93bf7fe640 1 -- 172.21.15.64:0/1983493297 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f93c80cd0a0 con 0x7f93a806b110 2023-11-24T08:07:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.818+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93a806b110 msgr2=0x7f93a806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.818+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93a806b110 0x7f93a806d5d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f93b8035000 tx=0x7f93b8030000 comp rx=0 tx=0).stop 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.818+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 msgr2=0x7f93c809d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.818+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c809d660 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f93c000b410 tx=0x7f93c000b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.819+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 shutdown_connections 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.819+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f93a806b110 0x7f93a806d5d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.819+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c815d3b0 0x7f93c809d660 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.819+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f93c80d1bc0 0x7f93c814f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.819+0000 7f93cc9e2640 1 --2- 172.21.15.64:0/1983493297 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f93c80d0fa0 0x7f93c809cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.819+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 >> 172.21.15.64:0/1983493297 conn(0x7f93c80d30a0 msgr2=0x7f93c80d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.819+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 shutdown_connections 2023-11-24T08:07:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:15.819+0000 7f93cc9e2640 1 -- 172.21.15.64:0/1983493297 wait complete. 2023-11-24T08:07:15.833 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.832+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.834 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.832+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.834 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.832+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.834 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.832+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.834 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.832+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:15.838 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T08:07:15.860 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.859+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.861 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.859+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.861 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.859+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.861 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.859+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.861 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:15.859+0000 7fe78d353640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:15.898 INFO:teuthology.orchestra.run.smithi064.stdout:2 2023-11-24T08:07:15.899 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:07:15.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:07:15.996 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:07:16.119 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:07:16.120 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:07:16.120 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:16.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:16.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.397+0000 7f384759e640 1 Processor -- start 2023-11-24T08:07:16.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.397+0000 7f384759e640 1 -- start start 2023-11-24T08:07:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.399+0000 7f384759e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f384813dfd0 0x7f384815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.399+0000 7f384759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38480cf500 0x7f384806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.399+0000 7f384759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38480ce7c0 con 0x7f38480cf050 2023-11-24T08:07:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.399+0000 7f384759e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3848140900 con 0x7f384813dfd0 2023-11-24T08:07:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.399+0000 7f384759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384815fa70 con 0x7f38480cf500 2023-11-24T08:07:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.399+0000 7f383ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f38480cf050 0x7f384806c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32852/0 (socket says 172.21.15.64:32852) 2023-11-24T08:07:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.399+0000 7f383ffff640 1 -- 172.21.15.64:0/1934785880 learned_addr learned my addr 172.21.15.64:0/1934785880 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:16.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f384659c640 1 --2- 172.21.15.64:0/1934785880 >> v2:172.21.15.91:3300/0 conn(0x7f384813dfd0 0x7f384815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3846d9d640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38480cf500 0x7f384806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750201314 0 0) 0x7f38480ce7c0 con 0x7f38480cf050 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3834003520 con 0x7f38480cf050 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 959382827 0 0) 0x7f3834003520 con 0x7f38480cf050 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38480cf500 msgr2=0x7f384806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38480cf500 0x7f384806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 >> v2:172.21.15.91:3300/0 conn(0x7f384813dfd0 msgr2=0x7f384815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 --2- 172.21.15.64:0/1934785880 >> v2:172.21.15.91:3300/0 conn(0x7f384813dfd0 0x7f384815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480cf950 con 0x7f38480cf050 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f3830001690 con 0x7f38480cf050 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 >> v1:172.21.15.64:6789/0 conn(0x7f38480cf050 legacy=0x7f384806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38480cf500 0x7f38340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3834003f60 0x7f38340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3834005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848140900 con 0x7f3834004d00 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815fa70 con 0x7f3834003f60 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.400+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383400b9b0 con 0x7f38480cf500 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.401+0000 7f3846d9d640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3834005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.401+0000 7f384659c640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38480cf500 0x7f38340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.401+0000 7f383ffff640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3834003f60 0x7f38340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.401+0000 7f3846d9d640 1 -- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38480cf500 msgr2=0x7f38340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.401+0000 7f3846d9d640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38480cf500 0x7f38340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.401+0000 7f3846d9d640 1 -- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3834003f60 msgr2=0x7f38340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.401+0000 7f3846d9d640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3834003f60 0x7f38340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.401+0000 7f3846d9d640 1 -- 172.21.15.64:0/1934785880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3834003520 con 0x7f3834004d00 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.402+0000 7f3846d9d640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3834005180 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3840009fc0 tx=0x7f38340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d800e9b8e807a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384002cc20 con 0x7f3834004d00 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384000f040 con 0x7f3834004d00 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f3845d9b640 1 -- 172.21.15.64:0/1934785880 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840034440 con 0x7f3834004d00 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f384759e640 1 -- 172.21.15.64:0/1934785880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 msgr2=0x7f3834005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f384759e640 1 --2- 172.21.15.64:0/1934785880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3834005180 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3840009fc0 tx=0x7f38340057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f3846d9d640 1 -- 172.21.15.64:0/1934785880 reap_dead start 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f384759e640 1 -- 172.21.15.64:0/1934785880 shutdown_connections 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f384759e640 1 -- 172.21.15.64:0/1934785880 >> 172.21.15.64:0/1934785880 conn(0x7f384806a860 msgr2=0x7f384806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f384759e640 1 -- 172.21.15.64:0/1934785880 shutdown_connections 2023-11-24T08:07:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.403+0000 7f384759e640 1 -- 172.21.15.64:0/1934785880 wait complete. 2023-11-24T08:07:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.405+0000 7f384759e640 1 Processor -- start 2023-11-24T08:07:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.405+0000 7f384759e640 1 -- start start 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.406+0000 7f384759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3848071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.406+0000 7f384759e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f384813b6a0 0x7f384813a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.406+0000 7f384759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f384813b0c0 0x7f38480b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.406+0000 7f384759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815adf0 con 0x7f3834004d00 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.406+0000 7f384759e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848159ce0 con 0x7f384813b6a0 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.406+0000 7f384759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384815b040 con 0x7f384813b0c0 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3848071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3848071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55912/0 (socket says 172.21.15.64:55912) 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 -- 172.21.15.64:0/2749131672 learned_addr learned my addr 172.21.15.64:0/2749131672 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f383ffff640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f384813b6a0 0x7f384813a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 -- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f384813b0c0 msgr2=0x7f38480b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f3846d9d640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f384813b0c0 0x7f38480b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f384813b0c0 0x7f38480b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 -- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f384813b6a0 msgr2=0x7f384813a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f384813b6a0 0x7f384813a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 -- 172.21.15.64:0/2749131672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384815b320 con 0x7f3834004d00 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f383ffff640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f384813b6a0 0x7f384813a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.407+0000 7f384659c640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3848071cd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f38380116d0 tx=0x7f3838011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.408+0000 7f383effd640 1 -- 172.21.15.64:0/2749131672 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383800f9c0 con 0x7f3834004d00 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.408+0000 7f383effd640 1 -- 172.21.15.64:0/2749131672 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383800fb20 con 0x7f3834004d00 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.408+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384813ae00 con 0x7f3834004d00 2023-11-24T08:07:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.408+0000 7f383effd640 1 -- 172.21.15.64:0/2749131672 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383800fe90 con 0x7f3834004d00 2023-11-24T08:07:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.408+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3848159a50 con 0x7f3834004d00 2023-11-24T08:07:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.409+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384804e840 con 0x7f3834004d00 2023-11-24T08:07:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.410+0000 7f383effd640 1 -- 172.21.15.64:0/2749131672 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f3838015460 con 0x7f3834004d00 2023-11-24T08:07:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.411+0000 7f383effd640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f382806b110 0x7f382806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.411+0000 7f383effd640 1 -- 172.21.15.64:0/2749131672 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f383808a480 con 0x7f3834004d00 2023-11-24T08:07:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.412+0000 7f383ffff640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f382806b110 0x7f382806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.413+0000 7f383ffff640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f382806b110 0x7f382806d5d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38300018c0 tx=0x7f3830001830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.416+0000 7f383effd640 1 -- 172.21.15.64:0/2749131672 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3838055150 con 0x7f3834004d00 2023-11-24T08:07:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.616+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f384815b4b0 con 0x7f3834004d00 2023-11-24T08:07:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.618+0000 7f383effd640 1 -- 172.21.15.64:0/2749131672 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f3838058770 con 0x7f3834004d00 2023-11-24T08:07:16.621 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:16.621 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:06:58.644237+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:55.038686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:56.142053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f382806b110 msgr2=0x7f382806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f382806b110 0x7f382806d5d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38300018c0 tx=0x7f3830001830 comp rx=0 tx=0).stop 2023-11-24T08:07:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 msgr2=0x7f3848071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3848071cd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f38380116d0 tx=0x7f3838011b90 comp rx=0 tx=0).stop 2023-11-24T08:07:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 shutdown_connections 2023-11-24T08:07:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f382806b110 0x7f382806d5d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f384813b0c0 0x7f38480b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f384813b6a0 0x7f384813a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 --2- 172.21.15.64:0/2749131672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3834004d00 0x7f3848071cd0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 >> 172.21.15.64:0/2749131672 conn(0x7f384806a860 msgr2=0x7f38480af940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 shutdown_connections 2023-11-24T08:07:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.621+0000 7f384759e640 1 -- 172.21.15.64:0/2749131672 wait complete. 2023-11-24T08:07:16.644 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:06:55.038686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:06:56.142053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:07:16.644 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:07:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.871+0000 7f947e373640 1 Processor -- start 2023-11-24T08:07:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.871+0000 7f947e373640 1 -- start start 2023-11-24T08:07:16.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.872+0000 7f947e373640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9478144770 0x7f947815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.872+0000 7f947e373640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94780b35a0 0x7f947815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.872+0000 7f947e373640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947815f930 con 0x7f947806c9d0 2023-11-24T08:07:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.872+0000 7f947e373640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780b43b0 con 0x7f9478144770 2023-11-24T08:07:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.872+0000 7f947e373640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94780b3b10 con 0x7f94780b35a0 2023-11-24T08:07:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f947d371640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9478144770 0x7f947815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f947db72640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f947806c9d0 0x7f947806ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32856/0 (socket says 172.21.15.64:32856) 2023-11-24T08:07:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f947db72640 1 -- 172.21.15.64:0/2633144909 learned_addr learned my addr 172.21.15.64:0/2633144909 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f947cb70640 1 --2- 172.21.15.64:0/2633144909 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94780b35a0 0x7f947815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f947d371640 1 -- 172.21.15.64:0/2633144909 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94780b35a0 msgr2=0x7f947815f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f947d371640 1 --2- 172.21.15.64:0/2633144909 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94780b35a0 0x7f947815f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f947d371640 1 -- 172.21.15.64:0/2633144909 >> v1:172.21.15.64:6789/0 conn(0x7f947806c9d0 legacy=0x7f947806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f947d371640 1 -- 172.21.15.64:0/2633144909 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780b3df0 con 0x7f9478144770 2023-11-24T08:07:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.873+0000 7f946ffff640 1 -- 172.21.15.64:0/2633144909 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343544190 0 0) 0x7f947815f930 con 0x7f947806c9d0 2023-11-24T08:07:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.875+0000 7f947d371640 1 --2- 172.21.15.64:0/2633144909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9478144770 0x7f947815eb60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f94740097d0 tx=0x7f947402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b8f1578af633c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.875+0000 7f946ffff640 1 -- 172.21.15.64:0/2633144909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474031800 con 0x7f9478144770 2023-11-24T08:07:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.875+0000 7f946ffff640 1 -- 172.21.15.64:0/2633144909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9474031960 con 0x7f9478144770 2023-11-24T08:07:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.875+0000 7f946ffff640 1 -- 172.21.15.64:0/2633144909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474031cf0 con 0x7f9478144770 2023-11-24T08:07:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.876+0000 7f947e373640 1 -- 172.21.15.64:0/2633144909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9478144770 msgr2=0x7f947815eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.876+0000 7f947e373640 1 --2- 172.21.15.64:0/2633144909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9478144770 0x7f947815eb60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f94740097d0 tx=0x7f947402b8a0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.876+0000 7f947e373640 1 -- 172.21.15.64:0/2633144909 shutdown_connections 2023-11-24T08:07:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.876+0000 7f947e373640 1 --2- 172.21.15.64:0/2633144909 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94780b35a0 0x7f947815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.876+0000 7f947e373640 1 --2- 172.21.15.64:0/2633144909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9478144770 0x7f947815eb60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.876+0000 7f947e373640 1 -- 172.21.15.64:0/2633144909 >> 172.21.15.64:0/2633144909 conn(0x7f94780cec10 msgr2=0x7f94780cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:16.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.876+0000 7f947e373640 1 -- 172.21.15.64:0/2633144909 shutdown_connections 2023-11-24T08:07:16.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.876+0000 7f947e373640 1 -- 172.21.15.64:0/2633144909 wait complete. 2023-11-24T08:07:16.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.878+0000 7f947e373640 1 Processor -- start 2023-11-24T08:07:16.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.878+0000 7f947e373640 1 -- start start 2023-11-24T08:07:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947e373640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947806c9d0 0x7f94781f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947e373640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94780b35a0 0x7f947813a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947e373640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9478144770 0x7f9478071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947e373640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9478140ba0 con 0x7f94780b35a0 2023-11-24T08:07:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947e373640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781f3100 con 0x7f947806c9d0 2023-11-24T08:07:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947e373640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947813aba0 con 0x7f9478144770 2023-11-24T08:07:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947cb70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94780b35a0 0x7f947813a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947cb70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94780b35a0 0x7f947813a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55926/0 (socket says 172.21.15.64:55926) 2023-11-24T08:07:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.879+0000 7f947cb70640 1 -- 172.21.15.64:0/132970454 learned_addr learned my addr 172.21.15.64:0/132970454 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947db72640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9478144770 0x7f9478071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947d371640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947806c9d0 0x7f94781f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947cb70640 1 -- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9478144770 msgr2=0x7f9478071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947cb70640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9478144770 0x7f9478071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947cb70640 1 -- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947806c9d0 msgr2=0x7f94781f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947cb70640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947806c9d0 0x7f94781f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947cb70640 1 -- 172.21.15.64:0/132970454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009400 con 0x7f94780b35a0 2023-11-24T08:07:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947db72640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9478144770 0x7f9478071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947d371640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947806c9d0 0x7f94781f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.880+0000 7f947cb70640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94780b35a0 0x7f947813a660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f946800d830 tx=0x7f946800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.881+0000 7f946dffb640 1 -- 172.21.15.64:0/132970454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946800b6c0 con 0x7f94780b35a0 2023-11-24T08:07:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.881+0000 7f946dffb640 1 -- 172.21.15.64:0/132970454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946800b820 con 0x7f94780b35a0 2023-11-24T08:07:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.881+0000 7f946dffb640 1 -- 172.21.15.64:0/132970454 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946800bbb0 con 0x7f94780b35a0 2023-11-24T08:07:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.881+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9478070680 con 0x7f94780b35a0 2023-11-24T08:07:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.881+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9478070c40 con 0x7f94780b35a0 2023-11-24T08:07:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.882+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947804e840 con 0x7f94780b35a0 2023-11-24T08:07:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.883+0000 7f946dffb640 1 -- 172.21.15.64:0/132970454 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f946801e460 con 0x7f94780b35a0 2023-11-24T08:07:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.884+0000 7f946dffb640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b110 0x7f945c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.884+0000 7f946dffb640 1 -- 172.21.15.64:0/132970454 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f946808a1c0 con 0x7f94780b35a0 2023-11-24T08:07:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.884+0000 7f947d371640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b110 0x7f945c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.886+0000 7f947d371640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b110 0x7f945c06d5d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9474036000 tx=0x7f947403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:16.888+0000 7f946dffb640 1 -- 172.21.15.64:0/132970454 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9468054f70 con 0x7f94780b35a0 2023-11-24T08:07:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.020+0000 7f946dffb640 1 -- 172.21.15.64:0/132970454 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9468055340 con 0x7f94780b35a0 2023-11-24T08:07:17.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.097+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94780b2460 con 0x7f94780b35a0 2023-11-24T08:07:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.098+0000 7f946dffb640 1 -- 172.21.15.64:0/132970454 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9468073f60 con 0x7f94780b35a0 2023-11-24T08:07:17.101 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b110 msgr2=0x7f945c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b110 0x7f945c06d5d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9474036000 tx=0x7f947403c040 comp rx=0 tx=0).stop 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94780b35a0 msgr2=0x7f947813a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94780b35a0 0x7f947813a660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f946800d830 tx=0x7f946800dcf0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 shutdown_connections 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b110 0x7f945c06d5d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9478144770 0x7f9478071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94780b35a0 0x7f947813a660 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 --2- 172.21.15.64:0/132970454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947806c9d0 0x7f94781f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 >> 172.21.15.64:0/132970454 conn(0x7f94780cec10 msgr2=0x7f947815cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:17.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 shutdown_connections 2023-11-24T08:07:17.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.101+0000 7f947e373640 1 -- 172.21.15.64:0/132970454 wait complete. 2023-11-24T08:07:17.123 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:07:17.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.346+0000 7f2e32e59640 1 Processor -- start 2023-11-24T08:07:17.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.347+0000 7f2e32e59640 1 -- start start 2023-11-24T08:07:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.348+0000 7f2e32e59640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 0x7f2e2c165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.348+0000 7f2e32e59640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c15f930 con 0x7f2e2c15f270 2023-11-24T08:07:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.348+0000 7f2e32e59640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c0b3b10 con 0x7f2e2c144770 2023-11-24T08:07:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.348+0000 7f2e32e59640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b43b0 con 0x7f2e2c06d9e0 2023-11-24T08:07:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.348+0000 7f2e31656640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2e2c15f270 0x7f2e2c0cd4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56512/0 (socket says 172.21.15.64:56512) 2023-11-24T08:07:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.348+0000 7f2e31656640 1 -- 172.21.15.64:0/3261137944 learned_addr learned my addr 172.21.15.64:0/3261137944 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e32658640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 0x7f2e2c165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192596978 0 0) 0x7f2e2c15f930 con 0x7f2e2c15f270 2023-11-24T08:07:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e18003520 con 0x7f2e2c15f270 2023-11-24T08:07:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3321611554 0 0) 0x7f2e18003520 con 0x7f2e2c15f270 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 msgr2=0x7f2e2c165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 0x7f2e2c165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 >> v1:172.21.15.91:6789/0 conn(0x7f2e2c144770 legacy=0x7f2e2c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0b3df0 con 0x7f2e2c15f270 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2e1c001670 con 0x7f2e2c15f270 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.349+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 >> v1:172.21.15.64:6789/0 conn(0x7f2e2c15f270 legacy=0x7f2e2c0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e30e55640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e18003a10 0x7f2e18003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e30e55640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 0x7f2e180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e30e55640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e18004d00 0x7f2e18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b43b0 con 0x7f2e18004d00 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1800ba30 con 0x7f2e18003a10 2023-11-24T08:07:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1800bba0 con 0x7f2e2c06d9e0 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e32658640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e18004d00 0x7f2e18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e31e57640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e18003a10 0x7f2e18003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e31656640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 0x7f2e180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e32658640 1 -- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 msgr2=0x7f2e180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e32658640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 0x7f2e180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e32658640 1 -- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e18003a10 msgr2=0x7f2e18003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e32658640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e18003a10 0x7f2e18003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e32658640 1 -- 172.21.15.64:0/3261137944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18003520 con 0x7f2e18004d00 2023-11-24T08:07:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.350+0000 7f2e31656640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e2c06d9e0 0x7f2e180047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:17.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.352+0000 7f2e32658640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e18004d00 0x7f2e18005180 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009fc0 tx=0x7f2e180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7b8e27ab790130 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:17.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.352+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2402cbf0 con 0x7f2e18004d00 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.352+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2400f040 con 0x7f2e18004d00 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.352+0000 7f2e30e55640 1 -- 172.21.15.64:0/3261137944 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24034440 con 0x7f2e18004d00 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.352+0000 7f2e32e59640 1 -- 172.21.15.64:0/3261137944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e18004d00 msgr2=0x7f2e18005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.352+0000 7f2e32e59640 1 --2- 172.21.15.64:0/3261137944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e18004d00 0x7f2e18005180 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009fc0 tx=0x7f2e180057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.352+0000 7f2e32658640 1 -- 172.21.15.64:0/3261137944 reap_dead start 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.353+0000 7f2e32e59640 1 -- 172.21.15.64:0/3261137944 shutdown_connections 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.353+0000 7f2e32e59640 1 -- 172.21.15.64:0/3261137944 >> 172.21.15.64:0/3261137944 conn(0x7f2e2c0cec10 msgr2=0x7f2e2c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.353+0000 7f2e32e59640 1 -- 172.21.15.64:0/3261137944 shutdown_connections 2023-11-24T08:07:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.353+0000 7f2e32e59640 1 -- 172.21.15.64:0/3261137944 wait complete. 2023-11-24T08:07:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.354+0000 7f2e32e59640 1 Processor -- start 2023-11-24T08:07:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.355+0000 7f2e32e59640 1 -- start start 2023-11-24T08:07:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e32e59640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e18004d00 0x7f2e2c140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e32e59640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e2c0a9000 0x7f2e2c14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e32e59640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e2c1f2f00 0x7f2e2c1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e32e59640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c140670 con 0x7f2e2c0a9000 2023-11-24T08:07:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e32e59640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c1408a0 con 0x7f2e2c1f2f00 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e32e59640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c140b80 con 0x7f2e18004d00 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31656640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e2c0a9000 0x7f2e2c14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31656640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e2c0a9000 0x7f2e2c14f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51914/0 (socket says 172.21.15.64:51914) 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31656640 1 -- 172.21.15.64:0/2883029027 learned_addr learned my addr 172.21.15.64:0/2883029027 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31e57640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e18004d00 0x7f2e2c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31656640 1 -- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e18004d00 msgr2=0x7f2e2c140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31656640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e18004d00 0x7f2e2c140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e32658640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e2c1f2f00 0x7f2e2c1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31656640 1 -- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e2c1f2f00 msgr2=0x7f2e2c1f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31656640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e2c1f2f00 0x7f2e2c1f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.356+0000 7f2e31656640 1 -- 172.21.15.64:0/2883029027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c140e60 con 0x7f2e2c0a9000 2023-11-24T08:07:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.357+0000 7f2e31656640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e2c0a9000 0x7f2e2c14f6e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c000f60 tx=0x7f2e1c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.357+0000 7f2e227fc640 1 -- 172.21.15.64:0/2883029027 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0074e0 con 0x7f2e2c0a9000 2023-11-24T08:07:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.357+0000 7f2e227fc640 1 -- 172.21.15.64:0/2883029027 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c007640 con 0x7f2e2c0a9000 2023-11-24T08:07:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.357+0000 7f2e227fc640 1 -- 172.21.15.64:0/2883029027 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c004a00 con 0x7f2e2c0a9000 2023-11-24T08:07:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.357+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c141140 con 0x7f2e2c0a9000 2023-11-24T08:07:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.357+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c0a9460 con 0x7f2e2c0a9000 2023-11-24T08:07:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.358+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c0cd7d0 con 0x7f2e2c0a9000 2023-11-24T08:07:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.359+0000 7f2e227fc640 1 -- 172.21.15.64:0/2883029027 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f2e1c004b60 con 0x7f2e2c0a9000 2023-11-24T08:07:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.360+0000 7f2e227fc640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2e1006b230 0x7f2e1006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.360+0000 7f2e227fc640 1 -- 172.21.15.64:0/2883029027 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e1c089cf0 con 0x7f2e2c0a9000 2023-11-24T08:07:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.361+0000 7f2e31e57640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2e1006b230 0x7f2e1006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.362+0000 7f2e31e57640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2e1006b230 0x7f2e1006d6f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005500 tx=0x7f2e28007f10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:17.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.364+0000 7f2e227fc640 1 -- 172.21.15.64:0/2883029027 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e1c054690 con 0x7f2e2c0a9000 2023-11-24T08:07:17.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.558+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e2c04e840 con 0x7f2e2c0a9000 2023-11-24T08:07:17.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.559+0000 7f2e227fc640 1 -- 172.21.15.64:0/2883029027 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e1c0190a0 con 0x7f2e2c0a9000 2023-11-24T08:07:17.561 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:07:17.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2e1006b230 msgr2=0x7f2e1006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2e1006b230 0x7f2e1006d6f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005500 tx=0x7f2e28007f10 comp rx=0 tx=0).stop 2023-11-24T08:07:17.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e2c0a9000 msgr2=0x7f2e2c14f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e2c0a9000 0x7f2e2c14f6e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c000f60 tx=0x7f2e1c00df70 comp rx=0 tx=0).stop 2023-11-24T08:07:17.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 shutdown_connections 2023-11-24T08:07:17.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2e1006b230 0x7f2e1006d6f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2e2c1f2f00 0x7f2e2c1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e2c0a9000 0x7f2e2c14f6e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 --2- 172.21.15.64:0/2883029027 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2e18004d00 0x7f2e2c140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 >> 172.21.15.64:0/2883029027 conn(0x7f2e2c0cec10 msgr2=0x7f2e2c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 shutdown_connections 2023-11-24T08:07:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.562+0000 7f2e32e59640 1 -- 172.21.15.64:0/2883029027 wait complete. 2023-11-24T08:07:17.583 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:07:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.809+0000 7ff590c7e640 1 Processor -- start 2023-11-24T08:07:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.809+0000 7ff590c7e640 1 -- start start 2023-11-24T08:07:17.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.810+0000 7ff590c7e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff58c0d1d90 0x7ff58c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.810+0000 7ff590c7e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c0d0fa0 0x7ff58c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.810+0000 7ff590c7e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c0c6bf0 con 0x7ff58c15d3b0 2023-11-24T08:07:17.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.810+0000 7ff590c7e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c0d13f0 con 0x7ff58c0d1d90 2023-11-24T08:07:17.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.810+0000 7ff590c7e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0cedc0 con 0x7ff58c0d0fa0 2023-11-24T08:07:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.811+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c0d0fa0 0x7ff58c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.811+0000 7ff58affd640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff58c0d1d90 0x7ff58c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.811+0000 7ff58b7fe640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7ff58c15d3b0 0x7ff58c0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:42970/0 (socket says 172.21.15.64:42970) 2023-11-24T08:07:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.811+0000 7ff58b7fe640 1 -- 172.21.15.64:0/1038088628 learned_addr learned my addr 172.21.15.64:0/1038088628 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.811+0000 7ff58affd640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff58c0d1d90 0x7ff58c0ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:44210/0 (socket says 172.21.15.64:44210) 2023-11-24T08:07:17.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.812+0000 7ff58bfff640 1 -- 172.21.15.64:0/1038088628 >> v2:172.21.15.91:3300/0 conn(0x7ff58c0d1d90 msgr2=0x7ff58c0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:17.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.812+0000 7ff58bfff640 1 --2- 172.21.15.64:0/1038088628 >> v2:172.21.15.91:3300/0 conn(0x7ff58c0d1d90 0x7ff58c0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.812+0000 7ff58bfff640 1 -- 172.21.15.64:0/1038088628 >> v1:172.21.15.91:6789/0 conn(0x7ff58c15d3b0 legacy=0x7ff58c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.812+0000 7ff58bfff640 1 -- 172.21.15.64:0/1038088628 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0c6e40 con 0x7ff58c0d0fa0 2023-11-24T08:07:17.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.812+0000 7ff58a7fc640 1 -- 172.21.15.64:0/1038088628 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120430373 0 0) 0x7ff58c0c6bf0 con 0x7ff58c15d3b0 2023-11-24T08:07:17.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.812+0000 7ff58affd640 1 --2- 172.21.15.64:0/1038088628 >> v2:172.21.15.91:3300/0 conn(0x7ff58c0d1d90 0x7ff58c0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:07:17.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.813+0000 7ff58bfff640 1 --2- 172.21.15.64:0/1038088628 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c0d0fa0 0x7ff58c0b3ca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57ba5c483919270b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:17.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.814+0000 7ff58a7fc640 1 -- 172.21.15.64:0/1038088628 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584031780 con 0x7ff58c0d0fa0 2023-11-24T08:07:17.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.814+0000 7ff58a7fc640 1 -- 172.21.15.64:0/1038088628 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5840318e0 con 0x7ff58c0d0fa0 2023-11-24T08:07:17.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.814+0000 7ff58a7fc640 1 -- 172.21.15.64:0/1038088628 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584031c70 con 0x7ff58c0d0fa0 2023-11-24T08:07:17.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.814+0000 7ff590c7e640 1 -- 172.21.15.64:0/1038088628 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c0d0fa0 msgr2=0x7ff58c0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.814+0000 7ff590c7e640 1 --2- 172.21.15.64:0/1038088628 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c0d0fa0 0x7ff58c0b3ca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402b820 comp rx=0 tx=0).stop 2023-11-24T08:07:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.815+0000 7ff590c7e640 1 -- 172.21.15.64:0/1038088628 shutdown_connections 2023-11-24T08:07:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.815+0000 7ff590c7e640 1 --2- 172.21.15.64:0/1038088628 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c0d0fa0 0x7ff58c0b3ca0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.815+0000 7ff590c7e640 1 --2- 172.21.15.64:0/1038088628 >> v2:172.21.15.91:3300/0 conn(0x7ff58c0d1d90 0x7ff58c0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.815+0000 7ff590c7e640 1 -- 172.21.15.64:0/1038088628 >> 172.21.15.64:0/1038088628 conn(0x7ff58c0d30a0 msgr2=0x7ff58c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.815+0000 7ff590c7e640 1 -- 172.21.15.64:0/1038088628 shutdown_connections 2023-11-24T08:07:17.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.815+0000 7ff590c7e640 1 -- 172.21.15.64:0/1038088628 wait complete. 2023-11-24T08:07:17.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.816+0000 7ff590c7e640 1 Processor -- start 2023-11-24T08:07:17.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.817+0000 7ff590c7e640 1 -- start start 2023-11-24T08:07:17.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff590c7e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c0d0fa0 0x7ff58c1f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff590c7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58c0d1d90 0x7ff58c0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff590c7e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c15d3b0 0x7ff58c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff590c7e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c1f2700 con 0x7ff58c0d1d90 2023-11-24T08:07:17.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff590c7e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0cac30 con 0x7ff58c0d0fa0 2023-11-24T08:07:17.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff590c7e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0caf10 con 0x7ff58c15d3b0 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff58affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58c0d1d90 0x7ff58c0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff58affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58c0d1d90 0x7ff58c0ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51918/0 (socket says 172.21.15.64:51918) 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff58affd640 1 -- 172.21.15.64:0/964034158 learned_addr learned my addr 172.21.15.64:0/964034158 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff58b7fe640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c0d0fa0 0x7ff58c1f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.818+0000 7ff58bfff640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c15d3b0 0x7ff58c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff58affd640 1 -- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c15d3b0 msgr2=0x7ff58c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff58affd640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c15d3b0 0x7ff58c0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff58affd640 1 -- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c0d0fa0 msgr2=0x7ff58c1f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff58affd640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c0d0fa0 0x7ff58c1f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:17.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff58affd640 1 -- 172.21.15.64:0/964034158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff574009400 con 0x7ff58c0d1d90 2023-11-24T08:07:17.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff58affd640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58c0d1d90 0x7ff58c0ca170 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff57400ebe0 tx=0x7ff57400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:17.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff583fff640 1 -- 172.21.15.64:0/964034158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574009760 con 0x7ff58c0d1d90 2023-11-24T08:07:17.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff583fff640 1 -- 172.21.15.64:0/964034158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57400cc30 con 0x7ff58c0d1d90 2023-11-24T08:07:17.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff583fff640 1 -- 172.21.15.64:0/964034158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57400eee0 con 0x7ff58c0d1d90 2023-11-24T08:07:17.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff584009400 con 0x7ff58c0d1d90 2023-11-24T08:07:17.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.819+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c0cb4d0 con 0x7ff58c0d1d90 2023-11-24T08:07:17.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.821+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c04e840 con 0x7ff58c0d1d90 2023-11-24T08:07:17.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.822+0000 7ff583fff640 1 -- 172.21.15.64:0/964034158 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff574013470 con 0x7ff58c0d1d90 2023-11-24T08:07:17.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.823+0000 7ff583fff640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff56c06b230 0x7ff56c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:17.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.823+0000 7ff583fff640 1 -- 172.21.15.64:0/964034158 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff57408a050 con 0x7ff58c0d1d90 2023-11-24T08:07:17.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.823+0000 7ff58b7fe640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff56c06b230 0x7ff56c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:17.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.824+0000 7ff58b7fe640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff56c06b230 0x7ff56c06d6f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff57c007ed0 tx=0x7ff57c007e60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:17.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:17.827+0000 7ff583fff640 1 -- 172.21.15.64:0/964034158 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5740bce90 con 0x7ff58c0d1d90 2023-11-24T08:07:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.029+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff58c15c7b0 con 0x7ff58c0d1d90 2023-11-24T08:07:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.030+0000 7ff583fff640 1 -- 172.21.15.64:0/964034158 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5740585b0 con 0x7ff58c0d1d90 2023-11-24T08:07:18.032 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:07:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.032+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff56c06b230 msgr2=0x7ff56c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.032+0000 7ff590c7e640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff56c06b230 0x7ff56c06d6f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff57c007ed0 tx=0x7ff57c007e60 comp rx=0 tx=0).stop 2023-11-24T08:07:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.032+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58c0d1d90 msgr2=0x7ff58c0ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.032+0000 7ff590c7e640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58c0d1d90 0x7ff58c0ca170 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff57400ebe0 tx=0x7ff57400c520 comp rx=0 tx=0).stop 2023-11-24T08:07:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.033+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 shutdown_connections 2023-11-24T08:07:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.033+0000 7ff590c7e640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff56c06b230 0x7ff56c06d6f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.033+0000 7ff590c7e640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff58c15d3b0 0x7ff58c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.033+0000 7ff590c7e640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58c0d1d90 0x7ff58c0ca170 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.033+0000 7ff590c7e640 1 --2- 172.21.15.64:0/964034158 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff58c0d0fa0 0x7ff58c1f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.033+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 >> 172.21.15.64:0/964034158 conn(0x7ff58c0d30a0 msgr2=0x7ff58c0d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.033+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 shutdown_connections 2023-11-24T08:07:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.033+0000 7ff590c7e640 1 -- 172.21.15.64:0/964034158 wait complete. 2023-11-24T08:07:18.056 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:07:18.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.281+0000 7f621969b640 1 Processor -- start 2023-11-24T08:07:18.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.281+0000 7f621969b640 1 -- start start 2023-11-24T08:07:18.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f621969b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62140cbcb0 0x7f621406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f621969b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62140ce880 0x7f621406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f621969b640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62140b2080 con 0x7f621415b340 2023-11-24T08:07:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f621969b640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62140b4130 con 0x7f62140ce880 2023-11-24T08:07:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f621969b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62140b2360 con 0x7f62140cbcb0 2023-11-24T08:07:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f62137fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62140ce880 0x7f621406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f62127fc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62140cbcb0 0x7f621406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f62137fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f62140ce880 0x7f621406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:44232/0 (socket says 172.21.15.64:44232) 2023-11-24T08:07:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f62137fe640 1 -- 172.21.15.64:0/220469967 learned_addr learned my addr 172.21.15.64:0/220469967 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f62137fe640 1 -- 172.21.15.64:0/220469967 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62140cbcb0 msgr2=0x7f621406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f62137fe640 1 --2- 172.21.15.64:0/220469967 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62140cbcb0 0x7f621406d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f62137fe640 1 -- 172.21.15.64:0/220469967 >> v1:172.21.15.91:6789/0 conn(0x7f621415b340 legacy=0x7f62140abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.283+0000 7f62137fe640 1 -- 172.21.15.64:0/220469967 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62140b2640 con 0x7f62140ce880 2023-11-24T08:07:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.285+0000 7f62137fe640 1 --2- 172.21.15.64:0/220469967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140ce880 0x7f621406da60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f620c009e50 tx=0x7f620c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aaddc6f8e1019ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.286+0000 7f6211ffb640 1 -- 172.21.15.64:0/220469967 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c02cb00 con 0x7f62140ce880 2023-11-24T08:07:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.286+0000 7f6211ffb640 1 -- 172.21.15.64:0/220469967 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c02cc60 con 0x7f62140ce880 2023-11-24T08:07:18.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.286+0000 7f6211ffb640 1 -- 172.21.15.64:0/220469967 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0343f0 con 0x7f62140ce880 2023-11-24T08:07:18.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.287+0000 7f621969b640 1 -- 172.21.15.64:0/220469967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140ce880 msgr2=0x7f621406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.287+0000 7f621969b640 1 --2- 172.21.15.64:0/220469967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140ce880 0x7f621406da60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f620c009e50 tx=0x7f620c02bf60 comp rx=0 tx=0).stop 2023-11-24T08:07:18.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.287+0000 7f621969b640 1 -- 172.21.15.64:0/220469967 shutdown_connections 2023-11-24T08:07:18.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.287+0000 7f621969b640 1 --2- 172.21.15.64:0/220469967 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140ce880 0x7f621406da60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.287+0000 7f621969b640 1 --2- 172.21.15.64:0/220469967 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f62140cbcb0 0x7f621406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.287+0000 7f621969b640 1 -- 172.21.15.64:0/220469967 >> 172.21.15.64:0/220469967 conn(0x7f62140b0d80 msgr2=0x7f62140b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.287+0000 7f621969b640 1 -- 172.21.15.64:0/220469967 shutdown_connections 2023-11-24T08:07:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.287+0000 7f621969b640 1 -- 172.21.15.64:0/220469967 wait complete. 2023-11-24T08:07:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.289+0000 7f621969b640 1 Processor -- start 2023-11-24T08:07:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.290+0000 7f621969b640 1 -- start start 2023-11-24T08:07:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f621969b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140cbcb0 0x7f621415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f621969b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62140ce880 0x7f6214158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f621969b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f621415b340 0x7f6214159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f621969b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62141f0800 con 0x7f62140ce880 2023-11-24T08:07:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f621969b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6214158660 con 0x7f62140cbcb0 2023-11-24T08:07:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f621969b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6214158940 con 0x7f621415b340 2023-11-24T08:07:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f62127fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62140ce880 0x7f6214158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f62127fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62140ce880 0x7f6214158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51930/0 (socket says 172.21.15.64:51930) 2023-11-24T08:07:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.291+0000 7f62127fc640 1 -- 172.21.15.64:0/1918368887 learned_addr learned my addr 172.21.15.64:0/1918368887 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.292+0000 7f6212ffd640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140cbcb0 0x7f621415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.292+0000 7f62127fc640 1 -- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f621415b340 msgr2=0x7f6214159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.292+0000 7f62137fe640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f621415b340 0x7f6214159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.292+0000 7f62127fc640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f621415b340 0x7f6214159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.292+0000 7f62127fc640 1 -- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140cbcb0 msgr2=0x7f621415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.292+0000 7f62127fc640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140cbcb0 0x7f621415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.292+0000 7f62127fc640 1 -- 172.21.15.64:0/1918368887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c009b30 con 0x7f62140ce880 2023-11-24T08:07:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.292+0000 7f62127fc640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62140ce880 0x7f6214158c20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f620800e980 tx=0x7f620800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.293+0000 7f61fb7fe640 1 -- 172.21.15.64:0/1918368887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208009670 con 0x7f62140ce880 2023-11-24T08:07:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.293+0000 7f61fb7fe640 1 -- 172.21.15.64:0/1918368887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620800ca00 con 0x7f62140ce880 2023-11-24T08:07:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.293+0000 7f61fb7fe640 1 -- 172.21.15.64:0/1918368887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800cd90 con 0x7f62140ce880 2023-11-24T08:07:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.293+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621415b8c0 con 0x7f62140ce880 2023-11-24T08:07:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.293+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62141596a0 con 0x7f62140ce880 2023-11-24T08:07:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.294+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621415b720 con 0x7f62140ce880 2023-11-24T08:07:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.295+0000 7f61fb7fe640 1 -- 172.21.15.64:0/1918368887 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6208013470 con 0x7f62140ce880 2023-11-24T08:07:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.296+0000 7f61fb7fe640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f620006b230 0x7f620006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.296+0000 7f61fb7fe640 1 -- 172.21.15.64:0/1918368887 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f620808a070 con 0x7f62140ce880 2023-11-24T08:07:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.297+0000 7f6212ffd640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f620006b230 0x7f620006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.298+0000 7f6212ffd640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f620006b230 0x7f620006d6f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f621409d1c0 tx=0x7f6204009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:18.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.301+0000 7f61fb7fe640 1 -- 172.21.15.64:0/1918368887 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62080bce90 con 0x7f62140ce880 2023-11-24T08:07:18.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.513+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6214074770 con 0x7f62140ce880 2023-11-24T08:07:18.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.514+0000 7f61fb7fe640 1 -- 172.21.15.64:0/1918368887 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6208017080 con 0x7f62140ce880 2023-11-24T08:07:18.516 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:07:18.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f620006b230 msgr2=0x7f620006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f620006b230 0x7f620006d6f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f621409d1c0 tx=0x7f6204009040 comp rx=0 tx=0).stop 2023-11-24T08:07:18.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62140ce880 msgr2=0x7f6214158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62140ce880 0x7f6214158c20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f620800e980 tx=0x7f620800ee40 comp rx=0 tx=0).stop 2023-11-24T08:07:18.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 shutdown_connections 2023-11-24T08:07:18.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f620006b230 0x7f620006d6f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f621415b340 0x7f6214159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62140ce880 0x7f6214158c20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 --2- 172.21.15.64:0/1918368887 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f62140cbcb0 0x7f621415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 >> 172.21.15.64:0/1918368887 conn(0x7f62140b0d80 msgr2=0x7f62140b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:18.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.516+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 shutdown_connections 2023-11-24T08:07:18.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.517+0000 7f621969b640 1 -- 172.21.15.64:0/1918368887 wait complete. 2023-11-24T08:07:18.537 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T08:07:18.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:18.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:18.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[109190]: fuse finished with error 0 and tester_r 0 2023-11-24T08:07:18.636 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:07:18.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:18.636 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:07:18.691 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:07:18.691 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:18.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:18.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.981+0000 7f0e25c6d640 1 Processor -- start 2023-11-24T08:07:18.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.981+0000 7f0e25c6d640 1 -- start start 2023-11-24T08:07:18.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.982+0000 7f0e25c6d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2013dfd0 0x7f0e2015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.982+0000 7f0e25c6d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0e200cf050 0x7f0e200cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.983+0000 7f0e25c6d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e200ce7c0 con 0x7f0e2006c130 2023-11-24T08:07:18.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.983+0000 7f0e25c6d640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e200cfa10 con 0x7f0e200cf050 2023-11-24T08:07:18.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.983+0000 7f0e25c6d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20140900 con 0x7f0e2013dfd0 2023-11-24T08:07:18.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.983+0000 7f0e24c6b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2013dfd0 0x7f0e2015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.983+0000 7f0e1ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0e200cf050 0x7f0e200cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.983+0000 7f0e1ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0e200cf050 0x7f0e200cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:44256/0 (socket says 172.21.15.64:44256) 2023-11-24T08:07:18.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.983+0000 7f0e24c6b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2013dfd0 0x7f0e2015be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:39904/0 (socket says 172.21.15.64:39904) 2023-11-24T08:07:18.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.983+0000 7f0e1ffff640 1 -- 172.21.15.64:0/982216433 learned_addr learned my addr 172.21.15.64:0/982216433 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:18.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.984+0000 7f0e1ffff640 1 -- 172.21.15.64:0/982216433 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2013dfd0 msgr2=0x7f0e2015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:18.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.984+0000 7f0e1ffff640 1 --2- 172.21.15.64:0/982216433 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2013dfd0 0x7f0e2015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.984+0000 7f0e1ffff640 1 -- 172.21.15.64:0/982216433 >> v1:172.21.15.91:6789/0 conn(0x7f0e2006c130 legacy=0x7f0e2006c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.984+0000 7f0e1ffff640 1 -- 172.21.15.64:0/982216433 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2015fa70 con 0x7f0e200cf050 2023-11-24T08:07:18.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.984+0000 7f0e24c6b640 1 --2- 172.21.15.64:0/982216433 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2013dfd0 0x7f0e2015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:07:18.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.986+0000 7f0e1ffff640 1 --2- 172.21.15.64:0/982216433 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e200cf050 0x7f0e200cf4d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009870 tx=0x7f0e0c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9043ce47662a96b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.986+0000 7f0e1f7fe640 1 -- 172.21.15.64:0/982216433 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c030840 con 0x7f0e200cf050 2023-11-24T08:07:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.986+0000 7f0e1f7fe640 1 -- 172.21.15.64:0/982216433 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c0309a0 con 0x7f0e200cf050 2023-11-24T08:07:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.986+0000 7f0e1f7fe640 1 -- 172.21.15.64:0/982216433 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c030d30 con 0x7f0e200cf050 2023-11-24T08:07:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.987+0000 7f0e25c6d640 1 -- 172.21.15.64:0/982216433 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e200cf050 msgr2=0x7f0e200cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.987+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/982216433 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e200cf050 0x7f0e200cf4d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009870 tx=0x7f0e0c02b980 comp rx=0 tx=0).stop 2023-11-24T08:07:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.987+0000 7f0e25c6d640 1 -- 172.21.15.64:0/982216433 shutdown_connections 2023-11-24T08:07:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.987+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/982216433 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e200cf050 0x7f0e200cf4d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.987+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/982216433 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2013dfd0 0x7f0e2015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.987+0000 7f0e25c6d640 1 -- 172.21.15.64:0/982216433 >> 172.21.15.64:0/982216433 conn(0x7f0e2006a860 msgr2=0x7f0e2006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.987+0000 7f0e25c6d640 1 -- 172.21.15.64:0/982216433 shutdown_connections 2023-11-24T08:07:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.987+0000 7f0e25c6d640 1 -- 172.21.15.64:0/982216433 wait complete. 2023-11-24T08:07:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.989+0000 7f0e25c6d640 1 Processor -- start 2023-11-24T08:07:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.989+0000 7f0e25c6d640 1 -- start start 2023-11-24T08:07:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.990+0000 7f0e25c6d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2006c130 0x7f0e20154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.990+0000 7f0e25c6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e200cf050 0x7f0e2013e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.990+0000 7f0e25c6d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e2013dfd0 0x7f0e2013e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.990+0000 7f0e25c6d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20157850 con 0x7f0e200cf050 2023-11-24T08:07:18.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.990+0000 7f0e25c6d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e201e5910 con 0x7f0e2013dfd0 2023-11-24T08:07:18.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.990+0000 7f0e25c6d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e201e5630 con 0x7f0e2006c130 2023-11-24T08:07:18.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e1ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e200cf050 0x7f0e2013e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e1ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e200cf050 0x7f0e2013e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51934/0 (socket says 172.21.15.64:51934) 2023-11-24T08:07:18.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e1ffff640 1 -- 172.21.15.64:0/1645333358 learned_addr learned my addr 172.21.15.64:0/1645333358 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e24c6b640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2006c130 0x7f0e20154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e2546c640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e2013dfd0 0x7f0e2013e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e1ffff640 1 -- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2006c130 msgr2=0x7f0e20154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e1ffff640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2006c130 0x7f0e20154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e1ffff640 1 -- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e2013dfd0 msgr2=0x7f0e2013e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e1ffff640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e2013dfd0 0x7f0e2013e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e1ffff640 1 -- 172.21.15.64:0/1645333358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14009400 con 0x7f0e200cf050 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.991+0000 7f0e2546c640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e2013dfd0 0x7f0e2013e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.992+0000 7f0e1ffff640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e200cf050 0x7f0e2013e3b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000c00 tx=0x7f0e0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.992+0000 7f0e1d7fa640 1 -- 172.21.15.64:0/1645333358 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c039c90 con 0x7f0e200cf050 2023-11-24T08:07:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.992+0000 7f0e1d7fa640 1 -- 172.21.15.64:0/1645333358 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c039df0 con 0x7f0e200cf050 2023-11-24T08:07:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.992+0000 7f0e1d7fa640 1 -- 172.21.15.64:0/1645333358 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c043640 con 0x7f0e200cf050 2023-11-24T08:07:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.992+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c0094d0 con 0x7f0e200cf050 2023-11-24T08:07:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.992+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2015c0c0 con 0x7f0e200cf050 2023-11-24T08:07:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.994+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2015bc40 con 0x7f0e200cf050 2023-11-24T08:07:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.995+0000 7f0e1d7fa640 1 -- 172.21.15.64:0/1645333358 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f0e0c0437a0 con 0x7f0e200cf050 2023-11-24T08:07:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.995+0000 7f0e1d7fa640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e0406b230 0x7f0e0406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.995+0000 7f0e1d7fa640 1 -- 172.21.15.64:0/1645333358 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0e0c03e070 con 0x7f0e200cf050 2023-11-24T08:07:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.997+0000 7f0e24c6b640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e0406b230 0x7f0e0406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:18.998+0000 7f0e24c6b640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e0406b230 0x7f0e0406d6f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0e2013d140 tx=0x7f0e14008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.000+0000 7f0e1d7fa640 1 -- 172.21.15.64:0/1645333358 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e2015bc40 con 0x7f0e200cf050 2023-11-24T08:07:19.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.198+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e2015bc40 con 0x7f0e200cf050 2023-11-24T08:07:19.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.200+0000 7f0e1d7fa640 1 -- 172.21.15.64:0/1645333358 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16625 (secure 0 0 0) 0x7f0e2015bc40 con 0x7f0e200cf050 2023-11-24T08:07:19.202 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:19.202 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":61,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:18.141438+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:55.038686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:56.142053+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:19.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e0406b230 msgr2=0x7f0e0406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e0406b230 0x7f0e0406d6f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0e2013d140 tx=0x7f0e14008040 comp rx=0 tx=0).stop 2023-11-24T08:07:19.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e200cf050 msgr2=0x7f0e2013e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e200cf050 0x7f0e2013e3b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000c00 tx=0x7f0e0c004730 comp rx=0 tx=0).stop 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 shutdown_connections 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e0406b230 0x7f0e0406d6f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e2013dfd0 0x7f0e2013e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e200cf050 0x7f0e2013e3b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 --2- 172.21.15.64:0/1645333358 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e2006c130 0x7f0e20154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.203+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 >> 172.21.15.64:0/1645333358 conn(0x7f0e2006a860 msgr2=0x7f0e200cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.204+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 shutdown_connections 2023-11-24T08:07:19.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.204+0000 7f0e25c6d640 1 -- 172.21.15.64:0/1645333358 wait complete. 2023-11-24T08:07:19.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:06:55.038686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:06:56.142053+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:07:19.225 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:07:19.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.452+0000 7f1efd4b2640 1 Processor -- start 2023-11-24T08:07:19.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.452+0000 7f1efd4b2640 1 -- start start 2023-11-24T08:07:19.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1efd4b2640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1ef8160b10 0x7f1ef806e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1efd4b2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ef806cad0 0x7f1ef80b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1efd4b2640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef8071780 con 0x7f1ef8072280 2023-11-24T08:07:19.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1efd4b2640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef80b2320 con 0x7f1ef806cad0 2023-11-24T08:07:19.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1efd4b2640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef806df30 con 0x7f1ef8160b10 2023-11-24T08:07:19.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1ef77fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ef806cad0 0x7f1ef80b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1ef77fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ef806cad0 0x7f1ef80b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51946/0 (socket says 172.21.15.64:51946) 2023-11-24T08:07:19.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1ef77fe640 1 -- 172.21.15.64:0/2758639832 learned_addr learned my addr 172.21.15.64:0/2758639832 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:19.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.454+0000 7f1ef6ffd640 1 --2- 172.21.15.64:0/2758639832 >> v2:172.21.15.91:3300/0 conn(0x7f1ef8160b10 0x7f1ef806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.455+0000 7f1ef77fe640 1 -- 172.21.15.64:0/2758639832 >> v2:172.21.15.91:3300/0 conn(0x7f1ef8160b10 msgr2=0x7f1ef806e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:19.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.455+0000 7f1ef77fe640 1 --2- 172.21.15.64:0/2758639832 >> v2:172.21.15.91:3300/0 conn(0x7f1ef8160b10 0x7f1ef806e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.455+0000 7f1ef77fe640 1 -- 172.21.15.64:0/2758639832 >> v1:172.21.15.91:6789/0 conn(0x7f1ef8072280 legacy=0x7f1ef806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:19.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.455+0000 7f1ef77fe640 1 -- 172.21.15.64:0/2758639832 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef80711c0 con 0x7f1ef806cad0 2023-11-24T08:07:19.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.456+0000 7f1ef77fe640 1 --2- 172.21.15.64:0/2758639832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef80b2740 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80b4070 tx=0x7f1ef002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86159ee7529278da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:19.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.456+0000 7f1ef5ffb640 1 -- 172.21.15.64:0/2758639832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00317e0 con 0x7f1ef806cad0 2023-11-24T08:07:19.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.456+0000 7f1ef5ffb640 1 -- 172.21.15.64:0/2758639832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0031940 con 0x7f1ef806cad0 2023-11-24T08:07:19.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.456+0000 7f1ef5ffb640 1 -- 172.21.15.64:0/2758639832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0031cd0 con 0x7f1ef806cad0 2023-11-24T08:07:19.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.457+0000 7f1efd4b2640 1 -- 172.21.15.64:0/2758639832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 msgr2=0x7f1ef80b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.457+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/2758639832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef80b2740 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80b4070 tx=0x7f1ef002b880 comp rx=0 tx=0).stop 2023-11-24T08:07:19.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.457+0000 7f1efd4b2640 1 -- 172.21.15.64:0/2758639832 shutdown_connections 2023-11-24T08:07:19.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.457+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/2758639832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef80b2740 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.457+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/2758639832 >> v2:172.21.15.91:3300/0 conn(0x7f1ef8160b10 0x7f1ef806e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.457+0000 7f1efd4b2640 1 -- 172.21.15.64:0/2758639832 >> 172.21.15.64:0/2758639832 conn(0x7f1ef80b0ba0 msgr2=0x7f1ef80b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:19.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.457+0000 7f1efd4b2640 1 -- 172.21.15.64:0/2758639832 shutdown_connections 2023-11-24T08:07:19.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.457+0000 7f1efd4b2640 1 -- 172.21.15.64:0/2758639832 wait complete. 2023-11-24T08:07:19.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.459+0000 7f1efd4b2640 1 Processor -- start 2023-11-24T08:07:19.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.459+0000 7f1efd4b2640 1 -- start start 2023-11-24T08:07:19.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.460+0000 7f1efd4b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef8157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.460+0000 7f1efd4b2640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ef8072280 0x7f1ef80cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.460+0000 7f1efd4b2640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ef8160b10 0x7f1ef81f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.460+0000 7f1efd4b2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef80ce9a0 con 0x7f1ef806cad0 2023-11-24T08:07:19.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.460+0000 7f1efd4b2640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef80ce040 con 0x7f1ef8160b10 2023-11-24T08:07:19.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.460+0000 7f1efd4b2640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81f2f70 con 0x7f1ef8072280 2023-11-24T08:07:19.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.460+0000 7f1ef6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef8157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef8157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51958/0 (socket says 172.21.15.64:51958) 2023-11-24T08:07:19.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef6ffd640 1 -- 172.21.15.64:0/3842224236 learned_addr learned my addr 172.21.15.64:0/3842224236 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:19.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef77fe640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ef8160b10 0x7f1ef81f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef67fc640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ef8072280 0x7f1ef80cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef6ffd640 1 -- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ef8072280 msgr2=0x7f1ef80cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef6ffd640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ef8072280 0x7f1ef80cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef6ffd640 1 -- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ef8160b10 msgr2=0x7f1ef81f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef6ffd640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ef8160b10 0x7f1ef81f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef6ffd640 1 -- 172.21.15.64:0/3842224236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00094d0 con 0x7f1ef806cad0 2023-11-24T08:07:19.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef67fc640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ef8072280 0x7f1ef80cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:19.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef77fe640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ef8160b10 0x7f1ef81f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:19.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.461+0000 7f1ef6ffd640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef8157e80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8007e40 tx=0x7f1ee800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:19.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.462+0000 7f1eef7fe640 1 -- 172.21.15.64:0/3842224236 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8014620 con 0x7f1ef806cad0 2023-11-24T08:07:19.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.462+0000 7f1eef7fe640 1 -- 172.21.15.64:0/3842224236 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee800ee90 con 0x7f1ef806cad0 2023-11-24T08:07:19.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.462+0000 7f1eef7fe640 1 -- 172.21.15.64:0/3842224236 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80149c0 con 0x7f1ef806cad0 2023-11-24T08:07:19.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.462+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef8161390 con 0x7f1ef806cad0 2023-11-24T08:07:19.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.462+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef8161950 con 0x7f1ef806cad0 2023-11-24T08:07:19.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.463+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef809d200 con 0x7f1ef806cad0 2023-11-24T08:07:19.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.464+0000 7f1eef7fe640 1 -- 172.21.15.64:0/3842224236 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1ee8014b20 con 0x7f1ef806cad0 2023-11-24T08:07:19.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.465+0000 7f1eef7fe640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b230 0x7f1ed806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.465+0000 7f1eef7fe640 1 -- 172.21.15.64:0/3842224236 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1ee808a6c0 con 0x7f1ef806cad0 2023-11-24T08:07:19.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.466+0000 7f1ef67fc640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b230 0x7f1ed806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.467+0000 7f1ef67fc640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b230 0x7f1ed806d6f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ef81f6410 tx=0x7f1ee0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:19.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.470+0000 7f1eef7fe640 1 -- 172.21.15.64:0/3842224236 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ee80551a0 con 0x7f1ef806cad0 2023-11-24T08:07:19.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.663+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ef804e920 con 0x7f1ef806cad0 2023-11-24T08:07:19.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.664+0000 7f1eef7fe640 1 -- 172.21.15.64:0/3842224236 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ee80587c0 con 0x7f1ef806cad0 2023-11-24T08:07:19.665 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:07:19.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b230 msgr2=0x7f1ed806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b230 0x7f1ed806d6f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ef81f6410 tx=0x7f1ee0009040 comp rx=0 tx=0).stop 2023-11-24T08:07:19.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 msgr2=0x7f1ef8157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef8157e80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8007e40 tx=0x7f1ee800e4b0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 shutdown_connections 2023-11-24T08:07:19.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1ed806b230 0x7f1ed806d6f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ef8160b10 0x7f1ef81f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ef8072280 0x7f1ef80cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 --2- 172.21.15.64:0/3842224236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ef806cad0 0x7f1ef8157e80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 >> 172.21.15.64:0/3842224236 conn(0x7f1ef80b0ba0 msgr2=0x7f1ef80ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:19.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 shutdown_connections 2023-11-24T08:07:19.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.667+0000 7f1efd4b2640 1 -- 172.21.15.64:0/3842224236 wait complete. 2023-11-24T08:07:19.688 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:07:19.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.921+0000 7f0e3aa5c640 1 Processor -- start 2023-11-24T08:07:19.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.922+0000 7f0e3aa5c640 1 -- start start 2023-11-24T08:07:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.923+0000 7f0e3aa5c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0e3406cad0 0x7f0e3406e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.923+0000 7f0e3aa5c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34072280 0x7f0e340b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.923+0000 7f0e3aa5c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e34071780 con 0x7f0e34160b10 2023-11-24T08:07:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.923+0000 7f0e3aa5c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e340b2320 con 0x7f0e3406cad0 2023-11-24T08:07:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.923+0000 7f0e3aa5c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3406df30 con 0x7f0e34072280 2023-11-24T08:07:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.924+0000 7f0e33fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0e34160b10 0x7f0e3406c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56522/0 (socket says 172.21.15.64:56522) 2023-11-24T08:07:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.924+0000 7f0e33fff640 1 -- 172.21.15.64:0/4294224154 learned_addr learned my addr 172.21.15.64:0/4294224154 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.924+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34072280 0x7f0e340b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.924+0000 7f0e2b7fe640 1 --2- 172.21.15.64:0/4294224154 >> v2:172.21.15.91:3300/0 conn(0x7f0e3406cad0 0x7f0e3406e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.924+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458775245 0 0) 0x7f0e34071780 con 0x7f0e34160b10 2023-11-24T08:07:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.924+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e20003520 con 0x7f0e34160b10 2023-11-24T08:07:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1284522479 0 0) 0x7f0e20003520 con 0x7f0e34160b10 2023-11-24T08:07:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34072280 msgr2=0x7f0e340b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34072280 0x7f0e340b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 >> v2:172.21.15.91:3300/0 conn(0x7f0e3406cad0 msgr2=0x7f0e3406e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 --2- 172.21.15.64:0/4294224154 >> v2:172.21.15.91:3300/0 conn(0x7f0e3406cad0 0x7f0e3406e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340711c0 con 0x7f0e34160b10 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0e240019f0 con 0x7f0e34160b10 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 >> v1:172.21.15.64:6789/0 conn(0x7f0e34160b10 legacy=0x7f0e3406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e20003970 0x7f0e20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20004360 0x7f0e20004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e20005000 0x7f0e2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340b2320 con 0x7f0e20005000 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3406df30 con 0x7f0e20004360 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000bfa0 con 0x7f0e20003970 2023-11-24T08:07:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34072280 0x7f0e340b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e20005000 0x7f0e2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.925+0000 7f0e33fff640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e20003970 0x7f0e20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.926+0000 7f0e2b7fe640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20004360 0x7f0e20004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.926+0000 7f0e38ff9640 1 -- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e20003970 msgr2=0x7f0e20003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.926+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e20003970 0x7f0e20003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.926+0000 7f0e38ff9640 1 -- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20004360 msgr2=0x7f0e20004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.926+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20004360 0x7f0e20004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.926+0000 7f0e38ff9640 1 -- 172.21.15.64:0/4294224154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20003520 con 0x7f0e20005000 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.926+0000 7f0e2b7fe640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20004360 0x7f0e20004840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.927+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e20005000 0x7f0e2000b980 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c006e80 tx=0x7f0e200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b140e099704c5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.927+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c032600 con 0x7f0e20005000 2023-11-24T08:07:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.927+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c02e050 con 0x7f0e20005000 2023-11-24T08:07:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.927+0000 7f0e337fe640 1 -- 172.21.15.64:0/4294224154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c032980 con 0x7f0e20005000 2023-11-24T08:07:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.928+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/4294224154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e20005000 msgr2=0x7f0e2000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.928+0000 7f0e3aa5c640 1 --2- 172.21.15.64:0/4294224154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e20005000 0x7f0e2000b980 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c006e80 tx=0x7f0e200057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.928+0000 7f0e38ff9640 1 -- 172.21.15.64:0/4294224154 reap_dead start 2023-11-24T08:07:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.928+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/4294224154 shutdown_connections 2023-11-24T08:07:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.928+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/4294224154 >> 172.21.15.64:0/4294224154 conn(0x7f0e340b0ba0 msgr2=0x7f0e340b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.928+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/4294224154 shutdown_connections 2023-11-24T08:07:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.928+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/4294224154 wait complete. 2023-11-24T08:07:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.930+0000 7f0e3aa5c640 1 Processor -- start 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.930+0000 7f0e3aa5c640 1 -- start start 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e3aa5c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20005000 0x7f0e34144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e3aa5c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34147490 0x7f0e340aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e3aa5c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e34099a00 0x7f0e34143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e3aa5c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3409a930 con 0x7f0e34099a00 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e3aa5c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3409a370 con 0x7f0e20005000 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e3aa5c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3409a650 con 0x7f0e34147490 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e38ff9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e34099a00 0x7f0e34143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e38ff9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e34099a00 0x7f0e34143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51986/0 (socket says 172.21.15.64:51986) 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.931+0000 7f0e38ff9640 1 -- 172.21.15.64:0/3040882814 learned_addr learned my addr 172.21.15.64:0/3040882814 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e2b7fe640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34147490 0x7f0e340aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e33fff640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20005000 0x7f0e34144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e38ff9640 1 -- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34147490 msgr2=0x7f0e340aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34147490 0x7f0e340aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e38ff9640 1 -- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20005000 msgr2=0x7f0e34144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20005000 0x7f0e34144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e38ff9640 1 -- 172.21.15.64:0/3040882814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3409b580 con 0x7f0e34099a00 2023-11-24T08:07:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e2b7fe640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34147490 0x7f0e340aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e38ff9640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e34099a00 0x7f0e34143ab0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0e340d00b0 tx=0x7f0e2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e317fa640 1 -- 172.21.15.64:0/3040882814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c046410 con 0x7f0e34099a00 2023-11-24T08:07:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.932+0000 7f0e317fa640 1 -- 172.21.15.64:0/3040882814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c046570 con 0x7f0e34099a00 2023-11-24T08:07:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.933+0000 7f0e317fa640 1 -- 172.21.15.64:0/3040882814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0468e0 con 0x7f0e34099a00 2023-11-24T08:07:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.933+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3409b860 con 0x7f0e34099a00 2023-11-24T08:07:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.933+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e340af490 con 0x7f0e34099a00 2023-11-24T08:07:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.934+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3409ba50 con 0x7f0e34099a00 2023-11-24T08:07:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.935+0000 7f0e317fa640 1 -- 172.21.15.64:0/3040882814 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f0e2c03c410 con 0x7f0e34099a00 2023-11-24T08:07:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.936+0000 7f0e317fa640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e1406b230 0x7f0e1406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.936+0000 7f0e317fa640 1 -- 172.21.15.64:0/3040882814 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0e2c0ab680 con 0x7f0e34099a00 2023-11-24T08:07:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.936+0000 7f0e33fff640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e1406b230 0x7f0e1406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:19.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.937+0000 7f0e33fff640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e1406b230 0x7f0e1406d6f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0e240013d0 tx=0x7f0e240020f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:19.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:19.940+0000 7f0e317fa640 1 -- 172.21.15.64:0/3040882814 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e2c075fc0 con 0x7f0e34099a00 2023-11-24T08:07:20.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.138+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e3409d200 con 0x7f0e34099a00 2023-11-24T08:07:20.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.139+0000 7f0e317fa640 1 -- 172.21.15.64:0/3040882814 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e2c00f040 con 0x7f0e34099a00 2023-11-24T08:07:20.140 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:07:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e1406b230 msgr2=0x7f0e1406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e1406b230 0x7f0e1406d6f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0e240013d0 tx=0x7f0e240020f0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e34099a00 msgr2=0x7f0e34143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e34099a00 0x7f0e34143ab0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0e340d00b0 tx=0x7f0e2c004730 comp rx=0 tx=0).stop 2023-11-24T08:07:20.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 shutdown_connections 2023-11-24T08:07:20.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0e1406b230 0x7f0e1406d6f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e34099a00 0x7f0e34143ab0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0e34147490 0x7f0e340aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 --2- 172.21.15.64:0/3040882814 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0e20005000 0x7f0e34144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 >> 172.21.15.64:0/3040882814 conn(0x7f0e340b0ba0 msgr2=0x7f0e3415ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:20.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 shutdown_connections 2023-11-24T08:07:20.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.142+0000 7f0e3aa5c640 1 -- 172.21.15.64:0/3040882814 wait complete. 2023-11-24T08:07:20.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:07:20.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.390+0000 7f9bbec83640 1 Processor -- start 2023-11-24T08:07:20.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.390+0000 7f9bbec83640 1 -- start start 2023-11-24T08:07:20.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.392+0000 7f9bbec83640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9bb815b080 0x7f9bb806a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.392+0000 7f9bbec83640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9bb80cf2c0 0x7f9bb815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.392+0000 7f9bbec83640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb80cc6e0 con 0x7f9bb806a980 2023-11-24T08:07:20.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.392+0000 7f9bbec83640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb80cc9c0 con 0x7f9bb80cf2c0 2023-11-24T08:07:20.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.392+0000 7f9bbec83640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb80ce3b0 con 0x7f9bb815b080 2023-11-24T08:07:20.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.392+0000 7f9bb7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9bb806a980 0x7f9bb80cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56528/0 (socket says 172.21.15.64:56528) 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.392+0000 7f9bb7fff640 1 -- 172.21.15.64:0/3157102397 learned_addr learned my addr 172.21.15.64:0/3157102397 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.393+0000 7f9bbd220640 1 --2- 172.21.15.64:0/3157102397 >> v2:172.21.15.64:3300/0 conn(0x7f9bb80cf2c0 0x7f9bb815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.393+0000 7f9bbca1f640 1 --2- 172.21.15.64:0/3157102397 >> v2:172.21.15.91:3300/0 conn(0x7f9bb815b080 0x7f9bb806a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.393+0000 7f9bb77fe640 1 -- 172.21.15.64:0/3157102397 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939292027 0 0) 0x7f9bb80cc6e0 con 0x7f9bb806a980 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.393+0000 7f9bb77fe640 1 -- 172.21.15.64:0/3157102397 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba8003520 con 0x7f9bb806a980 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.393+0000 7f9bbd220640 1 -- 172.21.15.64:0/3157102397 >> v2:172.21.15.91:3300/0 conn(0x7f9bb815b080 msgr2=0x7f9bb806a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.393+0000 7f9bbd220640 1 --2- 172.21.15.64:0/3157102397 >> v2:172.21.15.91:3300/0 conn(0x7f9bb815b080 0x7f9bb806a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.393+0000 7f9bbd220640 1 -- 172.21.15.64:0/3157102397 >> v1:172.21.15.64:6789/0 conn(0x7f9bb806a980 legacy=0x7f9bb80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.393+0000 7f9bbd220640 1 -- 172.21.15.64:0/3157102397 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb80ce690 con 0x7f9bb80cf2c0 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.394+0000 7f9bbd220640 1 --2- 172.21.15.64:0/3157102397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80cf2c0 0x7f9bb815ed60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8072060 tx=0x7f9bb002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be2cb31bd8ac7b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bb77fe640 1 -- 172.21.15.64:0/3157102397 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb002cbf0 con 0x7f9bb80cf2c0 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bb77fe640 1 -- 172.21.15.64:0/3157102397 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb002cd50 con 0x7f9bb80cf2c0 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bb77fe640 1 -- 172.21.15.64:0/3157102397 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0034640 con 0x7f9bb80cf2c0 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bbec83640 1 -- 172.21.15.64:0/3157102397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80cf2c0 msgr2=0x7f9bb815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bbec83640 1 --2- 172.21.15.64:0/3157102397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80cf2c0 0x7f9bb815ed60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8072060 tx=0x7f9bb002be40 comp rx=0 tx=0).stop 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bbec83640 1 -- 172.21.15.64:0/3157102397 shutdown_connections 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bbec83640 1 --2- 172.21.15.64:0/3157102397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80cf2c0 0x7f9bb815ed60 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bbec83640 1 --2- 172.21.15.64:0/3157102397 >> v2:172.21.15.91:3300/0 conn(0x7f9bb815b080 0x7f9bb806a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bbec83640 1 -- 172.21.15.64:0/3157102397 >> 172.21.15.64:0/3157102397 conn(0x7f9bb80cf6f0 msgr2=0x7f9bb80cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:20.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bbec83640 1 -- 172.21.15.64:0/3157102397 shutdown_connections 2023-11-24T08:07:20.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.395+0000 7f9bbec83640 1 -- 172.21.15.64:0/3157102397 wait complete. 2023-11-24T08:07:20.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.397+0000 7f9bbec83640 1 Processor -- start 2023-11-24T08:07:20.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.397+0000 7f9bbec83640 1 -- start start 2023-11-24T08:07:20.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.398+0000 7f9bbec83640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80cf2c0 0x7f9bb813dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.398+0000 7f9bbec83640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb815b080 0x7f9bb8150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.398+0000 7f9bbec83640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb81509b0 0x7f9bb80acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.398+0000 7f9bbec83640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8150e60 con 0x7f9bb81509b0 2023-11-24T08:07:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.398+0000 7f9bbec83640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb813eb30 con 0x7f9bb815b080 2023-11-24T08:07:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.398+0000 7f9bbec83640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb80ad8f0 con 0x7f9bb80cf2c0 2023-11-24T08:07:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb81509b0 0x7f9bb80acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb81509b0 0x7f9bb80acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52000/0 (socket says 172.21.15.64:52000) 2023-11-24T08:07:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 -- 172.21.15.64:0/2956410801 learned_addr learned my addr 172.21.15.64:0/2956410801 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bb7fff640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb815b080 0x7f9bb8150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbca1f640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80cf2c0 0x7f9bb813dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 -- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80cf2c0 msgr2=0x7f9bb813dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80cf2c0 0x7f9bb813dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 -- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb815b080 msgr2=0x7f9bb8150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb815b080 0x7f9bb8150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 -- 172.21.15.64:0/2956410801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb0009a70 con 0x7f9bb81509b0 2023-11-24T08:07:20.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bb7fff640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb815b080 0x7f9bb8150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:20.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.399+0000 7f9bbd220640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb81509b0 0x7f9bb80acb80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009e50 tx=0x7f9bb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:20.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.400+0000 7f9bb57fa640 1 -- 172.21.15.64:0/2956410801 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb003e610 con 0x7f9bb81509b0 2023-11-24T08:07:20.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.400+0000 7f9bb57fa640 1 -- 172.21.15.64:0/2956410801 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb003e770 con 0x7f9bb81509b0 2023-11-24T08:07:20.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.400+0000 7f9bb57fa640 1 -- 172.21.15.64:0/2956410801 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb003eb60 con 0x7f9bb81509b0 2023-11-24T08:07:20.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.400+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb806cb00 con 0x7f9bb81509b0 2023-11-24T08:07:20.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.400+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb806d0c0 con 0x7f9bb81509b0 2023-11-24T08:07:20.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.401+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb80676c0 con 0x7f9bb81509b0 2023-11-24T08:07:20.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.402+0000 7f9bb57fa640 1 -- 172.21.15.64:0/2956410801 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9bb003b050 con 0x7f9bb81509b0 2023-11-24T08:07:20.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.404+0000 7f9bb57fa640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9b9c06b230 0x7f9b9c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.404+0000 7f9bb57fa640 1 -- 172.21.15.64:0/2956410801 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9bb00abf70 con 0x7f9bb81509b0 2023-11-24T08:07:20.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.404+0000 7f9bbca1f640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9b9c06b230 0x7f9b9c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.405+0000 7f9bbca1f640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9b9c06b230 0x7f9b9c06d6f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9bac006fd0 tx=0x7f9bac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:20.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.409+0000 7f9bb57fa640 1 -- 172.21.15.64:0/2956410801 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bb0076bb0 con 0x7f9bb81509b0 2023-11-24T08:07:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.604+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bb80725e0 con 0x7f9bb81509b0 2023-11-24T08:07:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.604+0000 7f9bb57fa640 1 -- 172.21.15.64:0/2956410801 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bb004d020 con 0x7f9bb81509b0 2023-11-24T08:07:20.605 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:07:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9b9c06b230 msgr2=0x7f9b9c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9b9c06b230 0x7f9b9c06d6f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9bac006fd0 tx=0x7f9bac008040 comp rx=0 tx=0).stop 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb81509b0 msgr2=0x7f9bb80acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb81509b0 0x7f9bb80acb80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009e50 tx=0x7f9bb0004a40 comp rx=0 tx=0).stop 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 shutdown_connections 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9b9c06b230 0x7f9b9c06d6f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb81509b0 0x7f9bb80acb80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bb815b080 0x7f9bb8150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 --2- 172.21.15.64:0/2956410801 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bb80cf2c0 0x7f9bb813dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 >> 172.21.15.64:0/2956410801 conn(0x7f9bb80cf6f0 msgr2=0x7f9bb80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 shutdown_connections 2023-11-24T08:07:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.607+0000 7f9bbec83640 1 -- 172.21.15.64:0/2956410801 wait complete. 2023-11-24T08:07:20.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:07:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.850+0000 7fcb262a3640 1 Processor -- start 2023-11-24T08:07:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.850+0000 7fcb262a3640 1 -- start start 2023-11-24T08:07:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.852+0000 7fcb262a3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb2015f460 0x7fcb2006cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.852+0000 7fcb262a3640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb20070bf0 con 0x7fcb20141e80 2023-11-24T08:07:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.852+0000 7fcb262a3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2006d850 con 0x7fcb201416f0 2023-11-24T08:07:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.852+0000 7fcb262a3640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2006db30 con 0x7fcb2015f460 2023-11-24T08:07:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.853+0000 7fcb24aa0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb2015f460 0x7fcb2006cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.853+0000 7fcb25aa2640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcb20141e80 0x7fcb2006d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56544/0 (socket says 172.21.15.64:56544) 2023-11-24T08:07:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.853+0000 7fcb25aa2640 1 -- 172.21.15.64:0/4072294562 learned_addr learned my addr 172.21.15.64:0/4072294562 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.853+0000 7fcb24aa0640 1 -- 172.21.15.64:0/4072294562 >> v1:172.21.15.91:6789/0 conn(0x7fcb201416f0 legacy=0x7fcb2015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.853+0000 7fcb24aa0640 1 -- 172.21.15.64:0/4072294562 >> v1:172.21.15.64:6789/0 conn(0x7fcb20141e80 legacy=0x7fcb2006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.853+0000 7fcb24aa0640 1 -- 172.21.15.64:0/4072294562 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200705a0 con 0x7fcb2015f460 2023-11-24T08:07:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.854+0000 7fcb24aa0640 1 --2- 172.21.15.64:0/4072294562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb2015f460 0x7fcb2006cb20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcb100097a0 tx=0x7fcb1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2263a288ed353ba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.854+0000 7fcb17fff640 1 -- 172.21.15.64:0/4072294562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10031830 con 0x7fcb2015f460 2023-11-24T08:07:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb17fff640 1 -- 172.21.15.64:0/4072294562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10031990 con 0x7fcb2015f460 2023-11-24T08:07:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb17fff640 1 -- 172.21.15.64:0/4072294562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10031d20 con 0x7fcb2015f460 2023-11-24T08:07:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb262a3640 1 -- 172.21.15.64:0/4072294562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb2015f460 msgr2=0x7fcb2006cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb262a3640 1 --2- 172.21.15.64:0/4072294562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb2015f460 0x7fcb2006cb20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcb100097a0 tx=0x7fcb1002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb262a3640 1 -- 172.21.15.64:0/4072294562 shutdown_connections 2023-11-24T08:07:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb262a3640 1 --2- 172.21.15.64:0/4072294562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb2015f460 0x7fcb2006cb20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb262a3640 1 -- 172.21.15.64:0/4072294562 >> 172.21.15.64:0/4072294562 conn(0x7fcb200b0840 msgr2=0x7fcb200b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb262a3640 1 -- 172.21.15.64:0/4072294562 shutdown_connections 2023-11-24T08:07:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.855+0000 7fcb262a3640 1 -- 172.21.15.64:0/4072294562 wait complete. 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.857+0000 7fcb262a3640 1 Processor -- start 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.857+0000 7fcb262a3640 1 -- start start 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.858+0000 7fcb262a3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb201416f0 0x7fcb20158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.858+0000 7fcb262a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20141e80 0x7fcb2015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.858+0000 7fcb262a3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb2015f460 0x7fcb20134c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.858+0000 7fcb262a3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb201598e0 con 0x7fcb20141e80 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.858+0000 7fcb262a3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20159bc0 con 0x7fcb201416f0 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.858+0000 7fcb262a3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20159ea0 con 0x7fcb2015f460 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20141e80 0x7fcb2015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20141e80 0x7fcb2015a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52014/0 (socket says 172.21.15.64:52014) 2023-11-24T08:07:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 -- 172.21.15.64:0/2703707442 learned_addr learned my addr 172.21.15.64:0/2703707442 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb252a1640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb201416f0 0x7fcb20158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb25aa2640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb2015f460 0x7fcb20134c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 -- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb2015f460 msgr2=0x7fcb20134c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb2015f460 0x7fcb20134c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 -- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb201416f0 msgr2=0x7fcb20158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb201416f0 0x7fcb20158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 -- 172.21.15.64:0/2703707442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10009400 con 0x7fcb20141e80 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb25aa2640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb2015f460 0x7fcb20134c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.859+0000 7fcb24aa0640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20141e80 0x7fcb2015a980 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcb100098c0 tx=0x7fcb100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.860+0000 7fcb15ffb640 1 -- 172.21.15.64:0/2703707442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1003ba70 con 0x7fcb20141e80 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.860+0000 7fcb15ffb640 1 -- 172.21.15.64:0/2703707442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1003bbf0 con 0x7fcb20141e80 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.860+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2015a180 con 0x7fcb20141e80 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.860+0000 7fcb15ffb640 1 -- 172.21.15.64:0/2703707442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100453f0 con 0x7fcb20141e80 2023-11-24T08:07:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.860+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb201351a0 con 0x7fcb20141e80 2023-11-24T08:07:20.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.861+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2004e840 con 0x7fcb20141e80 2023-11-24T08:07:20.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.862+0000 7fcb15ffb640 1 -- 172.21.15.64:0/2703707442 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcb1004e450 con 0x7fcb20141e80 2023-11-24T08:07:20.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.863+0000 7fcb15ffb640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406b230 0x7fcb0406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:20.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.863+0000 7fcb15ffb640 1 -- 172.21.15.64:0/2703707442 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcb100ac0c0 con 0x7fcb20141e80 2023-11-24T08:07:20.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.864+0000 7fcb252a1640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406b230 0x7fcb0406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:20.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.865+0000 7fcb252a1640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406b230 0x7fcb0406d6f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c006fd0 tx=0x7fcb1c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:20.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:20.867+0000 7fcb15ffb640 1 -- 172.21.15.64:0/2703707442 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb100dee90 con 0x7fcb20141e80 2023-11-24T08:07:21.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.080+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb2015f220 con 0x7fcb20141e80 2023-11-24T08:07:21.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.081+0000 7fcb15ffb640 1 -- 172.21.15.64:0/2703707442 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcb10035080 con 0x7fcb20141e80 2023-11-24T08:07:21.082 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:07:21.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.083+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406b230 msgr2=0x7fcb0406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:21.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.083+0000 7fcb262a3640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406b230 0x7fcb0406d6f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c006fd0 tx=0x7fcb1c008040 comp rx=0 tx=0).stop 2023-11-24T08:07:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.083+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20141e80 msgr2=0x7fcb2015a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.083+0000 7fcb262a3640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20141e80 0x7fcb2015a980 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcb100098c0 tx=0x7fcb100046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.084+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 shutdown_connections 2023-11-24T08:07:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.084+0000 7fcb262a3640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcb0406b230 0x7fcb0406d6f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.084+0000 7fcb262a3640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcb2015f460 0x7fcb20134c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.084+0000 7fcb262a3640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20141e80 0x7fcb2015a980 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.084+0000 7fcb262a3640 1 --2- 172.21.15.64:0/2703707442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcb201416f0 0x7fcb20158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.084+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 >> 172.21.15.64:0/2703707442 conn(0x7fcb200b0840 msgr2=0x7fcb200aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.084+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 shutdown_connections 2023-11-24T08:07:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.084+0000 7fcb262a3640 1 -- 172.21.15.64:0/2703707442 wait complete. 2023-11-24T08:07:21.105 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi091.front.sepia.ceph.com... 2023-11-24T08:07:21.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:21.105 DEBUG:teuthology.orchestra.run.smithi091:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:21.207 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[97180]: fuse finished with error 0 and tester_r 0 2023-11-24T08:07:21.213 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:07:21.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:21.213 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:07:21.280 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:07:21.280 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:21.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:07:21.343 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:07:21.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:21.344 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T08:07:21.399 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T08:07:21.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:07:21.401 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:07:21.401 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:21.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:07:21.462 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:07:21.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:21.462 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T08:07:21.517 INFO:teuthology.orchestra.run.smithi091.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T08:07:21.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:07:21.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.757+0000 7fa3cfa90640 1 Processor -- start 2023-11-24T08:07:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.758+0000 7fa3cfa90640 1 -- start start 2023-11-24T08:07:21.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.759+0000 7fa3cfa90640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3c815d4d0 0x7fa3c80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:21.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.759+0000 7fa3cfa90640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa3c80d1f60 0x7fa3c80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:21.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.759+0000 7fa3cfa90640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c815d8b0 con 0x7fa3c80ce260 2023-11-24T08:07:21.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.759+0000 7fa3cfa90640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c80b91b0 con 0x7fa3c815d4d0 2023-11-24T08:07:21.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.759+0000 7fa3cfa90640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c80cdfb0 con 0x7fa3c80d1f60 2023-11-24T08:07:21.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.759+0000 7fa3cd82c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3c815d4d0 0x7fa3c80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:21.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.759+0000 7fa3cd82c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3c815d4d0 0x7fa3c80cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52028/0 (socket says 172.21.15.64:52028) 2023-11-24T08:07:21.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.759+0000 7fa3cd82c640 1 -- 172.21.15.64:0/3889776107 learned_addr learned my addr 172.21.15.64:0/3889776107 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:21.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.760+0000 7fa3ce02d640 1 --2- 172.21.15.64:0/3889776107 >> v2:172.21.15.91:3300/0 conn(0x7fa3c80d1f60 0x7fa3c80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:21.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.760+0000 7fa3cd82c640 1 -- 172.21.15.64:0/3889776107 >> v2:172.21.15.91:3300/0 conn(0x7fa3c80d1f60 msgr2=0x7fa3c80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:21.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.760+0000 7fa3cd82c640 1 --2- 172.21.15.64:0/3889776107 >> v2:172.21.15.91:3300/0 conn(0x7fa3c80d1f60 0x7fa3c80b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.760+0000 7fa3cd82c640 1 -- 172.21.15.64:0/3889776107 >> v1:172.21.15.91:6789/0 conn(0x7fa3c80ce260 legacy=0x7fa3c80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:21.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.760+0000 7fa3cd82c640 1 -- 172.21.15.64:0/3889776107 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80b2400 con 0x7fa3c815d4d0 2023-11-24T08:07:21.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.761+0000 7fa3cd82c640 1 --2- 172.21.15.64:0/3889776107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c815d4d0 0x7fa3c80cda70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa3c806c470 tx=0x7fa3c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40728bf13ef403f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:21.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.761+0000 7fa3cc82a640 1 -- 172.21.15.64:0/3889776107 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4031640 con 0x7fa3c815d4d0 2023-11-24T08:07:21.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.761+0000 7fa3cc82a640 1 -- 172.21.15.64:0/3889776107 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c40317a0 con 0x7fa3c815d4d0 2023-11-24T08:07:21.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.761+0000 7fa3cc82a640 1 -- 172.21.15.64:0/3889776107 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4031b30 con 0x7fa3c815d4d0 2023-11-24T08:07:21.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.762+0000 7fa3cfa90640 1 -- 172.21.15.64:0/3889776107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c815d4d0 msgr2=0x7fa3c80cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:21.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.762+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/3889776107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c815d4d0 0x7fa3c80cda70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa3c806c470 tx=0x7fa3c402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.762+0000 7fa3cfa90640 1 -- 172.21.15.64:0/3889776107 shutdown_connections 2023-11-24T08:07:21.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.762+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/3889776107 >> v2:172.21.15.91:3300/0 conn(0x7fa3c80d1f60 0x7fa3c80b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.762+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/3889776107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c815d4d0 0x7fa3c80cda70 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.762+0000 7fa3cfa90640 1 -- 172.21.15.64:0/3889776107 >> 172.21.15.64:0/3889776107 conn(0x7fa3c8071f90 msgr2=0x7fa3c80723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:21.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.762+0000 7fa3cfa90640 1 -- 172.21.15.64:0/3889776107 shutdown_connections 2023-11-24T08:07:21.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.762+0000 7fa3cfa90640 1 -- 172.21.15.64:0/3889776107 wait complete. 2023-11-24T08:07:21.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.764+0000 7fa3cfa90640 1 Processor -- start 2023-11-24T08:07:21.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.764+0000 7fa3cfa90640 1 -- start start 2023-11-24T08:07:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cfa90640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3c80ce260 0x7fa3c81516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cfa90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c80d1f60 0x7fa3c80cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cfa90640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3c815d4d0 0x7fa3c81530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cfa90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80cba10 con 0x7fa3c80d1f60 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cfa90640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80cbcf0 con 0x7fa3c80ce260 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cfa90640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80cbfd0 con 0x7fa3c815d4d0 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cd02b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c80d1f60 0x7fa3c80cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cd02b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c80d1f60 0x7fa3c80cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52042/0 (socket says 172.21.15.64:52042) 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.765+0000 7fa3cd02b640 1 -- 172.21.15.64:0/2806846577 learned_addr learned my addr 172.21.15.64:0/2806846577 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3cd82c640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3c80ce260 0x7fa3c81516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3ce02d640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3c815d4d0 0x7fa3c81530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3cd02b640 1 -- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3c815d4d0 msgr2=0x7fa3c81530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3cd02b640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3c815d4d0 0x7fa3c81530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3cd02b640 1 -- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3c80ce260 msgr2=0x7fa3c81516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3cd02b640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3c80ce260 0x7fa3c81516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3cd02b640 1 -- 172.21.15.64:0/2806846577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40094d0 con 0x7fa3c80d1f60 2023-11-24T08:07:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3cd02b640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c80d1f60 0x7fa3c80cafc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800c430 tx=0x7fa3b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3be7fc640 1 -- 172.21.15.64:0/2806846577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8011790 con 0x7fa3c80d1f60 2023-11-24T08:07:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3be7fc640 1 -- 172.21.15.64:0/2806846577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b80118f0 con 0x7fa3c80d1f60 2023-11-24T08:07:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3be7fc640 1 -- 172.21.15.64:0/2806846577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8011ce0 con 0x7fa3c80d1f60 2023-11-24T08:07:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.766+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c80c73c0 con 0x7fa3c80d1f60 2023-11-24T08:07:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.767+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c815f870 con 0x7fa3c80d1f60 2023-11-24T08:07:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.768+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c80c75e0 con 0x7fa3c80d1f60 2023-11-24T08:07:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.769+0000 7fa3be7fc640 1 -- 172.21.15.64:0/2806846577 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa3b801c460 con 0x7fa3c80d1f60 2023-11-24T08:07:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.770+0000 7fa3be7fc640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3ac06b260 0x7fa3ac06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.770+0000 7fa3be7fc640 1 -- 172.21.15.64:0/2806846577 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa3b8089cc0 con 0x7fa3c80d1f60 2023-11-24T08:07:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.770+0000 7fa3cd82c640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3ac06b260 0x7fa3ac06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.771+0000 7fa3cd82c640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3ac06b260 0x7fa3ac06d720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4036000 tx=0x7fa3c403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:21.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:21.774+0000 7fa3be7fc640 1 -- 172.21.15.64:0/2806846577 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3b80547c0 con 0x7fa3c80d1f60 2023-11-24T08:07:22.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.008+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3c8074770 con 0x7fa3c80d1f60 2023-11-24T08:07:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.009+0000 7fa3be7fc640 1 -- 172.21.15.64:0/2806846577 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fa3b8057de0 con 0x7fa3c80d1f60 2023-11-24T08:07:22.010 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:22.010 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2873504597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2873504597},{"type":"v1","addr":"172.21.15.91:6835","nonce":2873504597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3295227147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3295227147},{"type":"v1","addr":"172.21.15.91:6837","nonce":3295227147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2949237565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2949237565},{"type":"v1","addr":"172.21.15.64:6834","nonce":2949237565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:57.112228+0000","modified":"2023-11-24T08:07:00.539822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4200493009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":4200493009},{"type":"v1","addr":"172.21.15.64:6835","nonce":4200493009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:07:22.010 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2023-11-24T08:07:22.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.011+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3ac06b260 msgr2=0x7fa3ac06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.011+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3ac06b260 0x7fa3ac06d720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4036000 tx=0x7fa3c403c040 comp rx=0 tx=0).stop 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.011+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c80d1f60 msgr2=0x7fa3c80cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.011+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c80d1f60 0x7fa3c80cafc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800c430 tx=0x7fa3b800c8f0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.012+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 shutdown_connections 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.012+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3ac06b260 0x7fa3ac06d720 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.012+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3c815d4d0 0x7fa3c81530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.012+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c80d1f60 0x7fa3c80cafc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.012+0000 7fa3cfa90640 1 --2- 172.21.15.64:0/2806846577 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3c80ce260 0x7fa3c81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.012+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 >> 172.21.15.64:0/2806846577 conn(0x7fa3c8071f90 msgr2=0x7fa3c80b7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.012+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 shutdown_connections 2023-11-24T08:07:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.012+0000 7fa3cfa90640 1 -- 172.21.15.64:0/2806846577 wait complete. 2023-11-24T08:07:22.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.257+0000 7f159f59e640 1 Processor -- start 2023-11-24T08:07:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.257+0000 7f159f59e640 1 -- start start 2023-11-24T08:07:22.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159f59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f15a0143570 0x7f15a00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159f59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15a0143950 0x7f15a00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a015e6b0 0x7f15a015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159f59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a0071700 con 0x7f15a0143950 2023-11-24T08:07:22.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159f59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a015b410 con 0x7f15a0143570 2023-11-24T08:07:22.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a015b130 con 0x7f15a015e6b0 2023-11-24T08:07:22.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159dd9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15a0143950 0x7f15a00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159dd9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15a0143950 0x7f15a00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52052/0 (socket says 172.21.15.64:52052) 2023-11-24T08:07:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.259+0000 7f159dd9b640 1 -- 172.21.15.64:0/2732443823 learned_addr learned my addr 172.21.15.64:0/2732443823 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159e59c640 1 --2- 172.21.15.64:0/2732443823 >> v2:172.21.15.91:3300/0 conn(0x7f15a0143570 0x7f15a00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159dd9b640 1 -- 172.21.15.64:0/2732443823 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a015e6b0 msgr2=0x7f15a015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159ed9d640 1 --2- 172.21.15.64:0/2732443823 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a015e6b0 0x7f15a015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159dd9b640 1 --2- 172.21.15.64:0/2732443823 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a015e6b0 0x7f15a015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159dd9b640 1 -- 172.21.15.64:0/2732443823 >> v2:172.21.15.91:3300/0 conn(0x7f15a0143570 msgr2=0x7f15a00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159dd9b640 1 --2- 172.21.15.64:0/2732443823 >> v2:172.21.15.91:3300/0 conn(0x7f15a0143570 0x7f15a00cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159dd9b640 1 -- 172.21.15.64:0/2732443823 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a015dbd0 con 0x7f15a0143950 2023-11-24T08:07:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159ed9d640 1 --2- 172.21.15.64:0/2732443823 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a015e6b0 0x7f15a015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.260+0000 7f159e59c640 1 --2- 172.21.15.64:0/2732443823 >> v2:172.21.15.91:3300/0 conn(0x7f15a0143570 0x7f15a00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:22.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.261+0000 7f159dd9b640 1 --2- 172.21.15.64:0/2732443823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a0143950 0x7f15a00cf590 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f15a0070c70 tx=0x7f159402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba174179b153b87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:22.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.261+0000 7f159d59a640 1 -- 172.21.15.64:0/2732443823 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15940317e0 con 0x7f15a0143950 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.261+0000 7f159d59a640 1 -- 172.21.15.64:0/2732443823 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1594031940 con 0x7f15a0143950 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.261+0000 7f159d59a640 1 -- 172.21.15.64:0/2732443823 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594031cd0 con 0x7f15a0143950 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 -- 172.21.15.64:0/2732443823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a0143950 msgr2=0x7f15a00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 --2- 172.21.15.64:0/2732443823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a0143950 0x7f15a00cf590 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f15a0070c70 tx=0x7f159402b880 comp rx=0 tx=0).stop 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 -- 172.21.15.64:0/2732443823 shutdown_connections 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 --2- 172.21.15.64:0/2732443823 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a015e6b0 0x7f15a015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 --2- 172.21.15.64:0/2732443823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a0143950 0x7f15a00cf590 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 --2- 172.21.15.64:0/2732443823 >> v2:172.21.15.91:3300/0 conn(0x7f15a0143570 0x7f15a00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 -- 172.21.15.64:0/2732443823 >> 172.21.15.64:0/2732443823 conn(0x7f15a006a860 msgr2=0x7f15a006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 -- 172.21.15.64:0/2732443823 shutdown_connections 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.262+0000 7f159f59e640 1 -- 172.21.15.64:0/2732443823 wait complete. 2023-11-24T08:07:22.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.264+0000 7f159f59e640 1 Processor -- start 2023-11-24T08:07:22.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.264+0000 7f159f59e640 1 -- start start 2023-11-24T08:07:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.265+0000 7f159f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a0143570 0x7f15a0159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.265+0000 7f159f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a0143950 0x7f15a0140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.265+0000 7f159f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a015e6b0 0x7f15a01408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.265+0000 7f159f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0141ad0 con 0x7f15a015e6b0 2023-11-24T08:07:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.265+0000 7f159f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0159740 con 0x7f15a0143570 2023-11-24T08:07:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.265+0000 7f159f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0140ec0 con 0x7f15a0143950 2023-11-24T08:07:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.265+0000 7f159ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a015e6b0 0x7f15a01408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a015e6b0 0x7f15a01408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52054/0 (socket says 172.21.15.64:52054) 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159ed9d640 1 -- 172.21.15.64:0/2604684574 learned_addr learned my addr 172.21.15.64:0/2604684574 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159dd9b640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a0143950 0x7f15a0140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159e59c640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a0143570 0x7f15a0159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159ed9d640 1 -- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a0143950 msgr2=0x7f15a0140370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159ed9d640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a0143950 0x7f15a0140370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159ed9d640 1 -- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a0143570 msgr2=0x7f15a0159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159ed9d640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a0143570 0x7f15a0159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159ed9d640 1 -- 172.21.15.64:0/2604684574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940094d0 con 0x7f15a015e6b0 2023-11-24T08:07:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.266+0000 7f159ed9d640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a015e6b0 0x7f15a01408b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f159800cd50 tx=0x7f159800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.267+0000 7f158effd640 1 -- 172.21.15.64:0/2604684574 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800f680 con 0x7f15a015e6b0 2023-11-24T08:07:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.267+0000 7f158effd640 1 -- 172.21.15.64:0/2604684574 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800f800 con 0x7f15a015e6b0 2023-11-24T08:07:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.267+0000 7f158effd640 1 -- 172.21.15.64:0/2604684574 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800fbb0 con 0x7f15a015e6b0 2023-11-24T08:07:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.267+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a01411a0 con 0x7f15a015e6b0 2023-11-24T08:07:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.267+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a0141760 con 0x7f15a015e6b0 2023-11-24T08:07:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.268+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a0141c40 con 0x7f15a015e6b0 2023-11-24T08:07:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.269+0000 7f158effd640 1 -- 172.21.15.64:0/2604684574 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f159800b040 con 0x7f15a015e6b0 2023-11-24T08:07:22.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.270+0000 7f158effd640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f158006b260 0x7f158006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.271+0000 7f158effd640 1 -- 172.21.15.64:0/2604684574 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f159808a6c0 con 0x7f15a015e6b0 2023-11-24T08:07:22.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.271+0000 7f159e59c640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f158006b260 0x7f158006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.272+0000 7f159e59c640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f158006b260 0x7f158006d720 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f159000a990 tx=0x7f1590008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:22.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.275+0000 7f158effd640 1 -- 172.21.15.64:0/2604684574 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1598055190 con 0x7f15a015e6b0 2023-11-24T08:07:22.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.477+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15a004e840 con 0x7f15a015e6b0 2023-11-24T08:07:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.479+0000 7f158effd640 1 -- 172.21.15.64:0/2604684574 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f15980587b0 con 0x7f15a015e6b0 2023-11-24T08:07:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:22.480 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":62,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:21.002292+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:55.038686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:56.142053+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f158006b260 msgr2=0x7f158006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f158006b260 0x7f158006d720 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f159000a990 tx=0x7f1590008040 comp rx=0 tx=0).stop 2023-11-24T08:07:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a015e6b0 msgr2=0x7f15a01408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a015e6b0 0x7f15a01408b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f159800cd50 tx=0x7f159800a650 comp rx=0 tx=0).stop 2023-11-24T08:07:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 shutdown_connections 2023-11-24T08:07:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f158006b260 0x7f158006d720 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a015e6b0 0x7f15a01408b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15a0143950 0x7f15a0140370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 --2- 172.21.15.64:0/2604684574 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f15a0143570 0x7f15a0159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.481+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 >> 172.21.15.64:0/2604684574 conn(0x7f15a006a860 msgr2=0x7f15a0071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.482+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 shutdown_connections 2023-11-24T08:07:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.482+0000 7f159f59e640 1 -- 172.21.15.64:0/2604684574 wait complete. 2023-11-24T08:07:22.503 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:06:55.038686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:06:56.142053+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:07:22.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:07:22.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.733+0000 7f691554e640 1 Processor -- start 2023-11-24T08:07:22.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.734+0000 7f691554e640 1 -- start start 2023-11-24T08:07:22.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.735+0000 7f691554e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69100d1d90 0x7f69100ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.735+0000 7f691554e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 0x7f69100b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.735+0000 7f691554e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69100c6bf0 con 0x7f691015d3b0 2023-11-24T08:07:22.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.735+0000 7f691554e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69100d13f0 con 0x7f69100d1d90 2023-11-24T08:07:22.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.735+0000 7f691554e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69100cedc0 con 0x7f69100d0fa0 2023-11-24T08:07:22.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.736+0000 7f690f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69100d1d90 0x7f69100ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.736+0000 7f690f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69100d1d90 0x7f69100ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52060/0 (socket says 172.21.15.64:52060) 2023-11-24T08:07:22.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.736+0000 7f690f7fe640 1 -- 172.21.15.64:0/2219053683 learned_addr learned my addr 172.21.15.64:0/2219053683 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:22.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.736+0000 7f6914d4d640 1 --2- 172.21.15.64:0/2219053683 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 0x7f69100b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.736+0000 7f690f7fe640 1 -- 172.21.15.64:0/2219053683 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 msgr2=0x7f69100b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:22.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.736+0000 7f690f7fe640 1 --2- 172.21.15.64:0/2219053683 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 0x7f69100b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.736+0000 7f690f7fe640 1 -- 172.21.15.64:0/2219053683 >> v1:172.21.15.91:6789/0 conn(0x7f691015d3b0 legacy=0x7f69100d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:22.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.736+0000 7f690f7fe640 1 -- 172.21.15.64:0/2219053683 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69100c6e40 con 0x7f69100d1d90 2023-11-24T08:07:22.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.737+0000 7f690f7fe640 1 --2- 172.21.15.64:0/2219053683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69100ce6c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6904009870 tx=0x7f690402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708350e3385b4645 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:22.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f690effd640 1 -- 172.21.15.64:0/2219053683 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040317c0 con 0x7f69100d1d90 2023-11-24T08:07:22.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f690effd640 1 -- 172.21.15.64:0/2219053683 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6904031920 con 0x7f69100d1d90 2023-11-24T08:07:22.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f690effd640 1 -- 172.21.15.64:0/2219053683 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904031cb0 con 0x7f69100d1d90 2023-11-24T08:07:22.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f691554e640 1 -- 172.21.15.64:0/2219053683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 msgr2=0x7f69100ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f691554e640 1 --2- 172.21.15.64:0/2219053683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69100ce6c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6904009870 tx=0x7f690402b860 comp rx=0 tx=0).stop 2023-11-24T08:07:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f691554e640 1 -- 172.21.15.64:0/2219053683 shutdown_connections 2023-11-24T08:07:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f691554e640 1 --2- 172.21.15.64:0/2219053683 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 0x7f69100b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f691554e640 1 --2- 172.21.15.64:0/2219053683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69100ce6c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.738+0000 7f691554e640 1 -- 172.21.15.64:0/2219053683 >> 172.21.15.64:0/2219053683 conn(0x7f69100d30a0 msgr2=0x7f69100d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.739+0000 7f691554e640 1 -- 172.21.15.64:0/2219053683 shutdown_connections 2023-11-24T08:07:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.739+0000 7f691554e640 1 -- 172.21.15.64:0/2219053683 wait complete. 2023-11-24T08:07:22.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.740+0000 7f691554e640 1 Processor -- start 2023-11-24T08:07:22.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.741+0000 7f691554e640 1 -- start start 2023-11-24T08:07:22.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f691554e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 0x7f691015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f691554e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69101545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f691554e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691015d3b0 0x7f691006c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f691554e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69101f3b90 con 0x7f69100d1d90 2023-11-24T08:07:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f691554e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69101f2fe0 con 0x7f691015d3b0 2023-11-24T08:07:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f691554e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69101f3260 con 0x7f69100d0fa0 2023-11-24T08:07:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69101545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69101545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52070/0 (socket says 172.21.15.64:52070) 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690f7fe640 1 -- 172.21.15.64:0/3725761959 learned_addr learned my addr 172.21.15.64:0/3725761959 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690ffff640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 0x7f691015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f6914d4d640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691015d3b0 0x7f691006c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690f7fe640 1 -- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 msgr2=0x7f691015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690f7fe640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 0x7f691015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690f7fe640 1 -- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691015d3b0 msgr2=0x7f691006c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690f7fe640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691015d3b0 0x7f691006c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.742+0000 7f690f7fe640 1 -- 172.21.15.64:0/3725761959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040094d0 con 0x7f69100d1d90 2023-11-24T08:07:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.743+0000 7f690f7fe640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69101545e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f690402b8a0 tx=0x7f69040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.743+0000 7f690cff9640 1 -- 172.21.15.64:0/3725761959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904032860 con 0x7f69100d1d90 2023-11-24T08:07:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.743+0000 7f690cff9640 1 -- 172.21.15.64:0/3725761959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69040329c0 con 0x7f69100d1d90 2023-11-24T08:07:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.743+0000 7f690cff9640 1 -- 172.21.15.64:0/3725761959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904032d70 con 0x7f69100d1d90 2023-11-24T08:07:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.743+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69101f3540 con 0x7f69100d1d90 2023-11-24T08:07:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.743+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69101f4020 con 0x7f69100d1d90 2023-11-24T08:07:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.744+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69101f33a0 con 0x7f69100d1d90 2023-11-24T08:07:22.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.745+0000 7f690cff9640 1 -- 172.21.15.64:0/3725761959 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f690404e450 con 0x7f69100d1d90 2023-11-24T08:07:22.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.746+0000 7f690cff9640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f68fc06b230 0x7f68fc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:22.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.746+0000 7f690cff9640 1 -- 172.21.15.64:0/3725761959 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f69040abf90 con 0x7f69100d1d90 2023-11-24T08:07:22.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.747+0000 7f690ffff640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f68fc06b230 0x7f68fc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:22.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.748+0000 7f690ffff640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f68fc06b230 0x7f68fc06d6f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6900009f20 tx=0x7f69000094d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:22.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.751+0000 7f690cff9640 1 -- 172.21.15.64:0/3725761959 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69101f33a0 con 0x7f69100d1d90 2023-11-24T08:07:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.946+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69101f33a0 con 0x7f69100d1d90 2023-11-24T08:07:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.947+0000 7f690cff9640 1 -- 172.21.15.64:0/3725761959 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69101f33a0 con 0x7f69100d1d90 2023-11-24T08:07:22.950 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:07:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.949+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f68fc06b230 msgr2=0x7f68fc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.949+0000 7f691554e640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f68fc06b230 0x7f68fc06d6f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6900009f20 tx=0x7f69000094d0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 msgr2=0x7f69101545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69101545e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f690402b8a0 tx=0x7f69040046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 shutdown_connections 2023-11-24T08:07:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f68fc06b230 0x7f68fc06d6f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f691015d3b0 0x7f691006c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69100d1d90 0x7f69101545e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 --2- 172.21.15.64:0/3725761959 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69100d0fa0 0x7f691015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 >> 172.21.15.64:0/3725761959 conn(0x7f69100d30a0 msgr2=0x7f691006b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 shutdown_connections 2023-11-24T08:07:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:22.950+0000 7f691554e640 1 -- 172.21.15.64:0/3725761959 wait complete. 2023-11-24T08:07:22.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:07:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.201+0000 7fcf5bf5a640 1 Processor -- start 2023-11-24T08:07:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.201+0000 7fcf5bf5a640 1 -- start start 2023-11-24T08:07:23.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.203+0000 7fcf5bf5a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcf54160b10 0x7fcf5406e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.203+0000 7fcf5bf5a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcf5406cad0 0x7fcf540b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.203+0000 7fcf5bf5a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf54071780 con 0x7fcf54072280 2023-11-24T08:07:23.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.203+0000 7fcf5bf5a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf540b2320 con 0x7fcf54160b10 2023-11-24T08:07:23.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.203+0000 7fcf5bf5a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5406df30 con 0x7fcf5406cad0 2023-11-24T08:07:23.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.203+0000 7fcf59cf6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcf54160b10 0x7fcf5406e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.204+0000 7fcf59cf6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcf54160b10 0x7fcf5406e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52084/0 (socket says 172.21.15.64:52084) 2023-11-24T08:07:23.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.204+0000 7fcf59cf6640 1 -- 172.21.15.64:0/3030227417 learned_addr learned my addr 172.21.15.64:0/3030227417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:23.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.204+0000 7fcf5a4f7640 1 --2- 172.21.15.64:0/3030227417 >> v2:172.21.15.91:3300/0 conn(0x7fcf5406cad0 0x7fcf540b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.204+0000 7fcf59cf6640 1 -- 172.21.15.64:0/3030227417 >> v2:172.21.15.91:3300/0 conn(0x7fcf5406cad0 msgr2=0x7fcf540b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:23.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.204+0000 7fcf59cf6640 1 --2- 172.21.15.64:0/3030227417 >> v2:172.21.15.91:3300/0 conn(0x7fcf5406cad0 0x7fcf540b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.204+0000 7fcf59cf6640 1 -- 172.21.15.64:0/3030227417 >> v1:172.21.15.91:6789/0 conn(0x7fcf54072280 legacy=0x7fcf5406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:23.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.204+0000 7fcf59cf6640 1 -- 172.21.15.64:0/3030227417 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf540711c0 con 0x7fcf54160b10 2023-11-24T08:07:23.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.205+0000 7fcf59cf6640 1 --2- 172.21.15.64:0/3030227417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf5406e420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcf540d0090 tx=0x7fcf5002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fe24b5f64011b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:23.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf58cf4640 1 -- 172.21.15.64:0/3030227417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500317e0 con 0x7fcf54160b10 2023-11-24T08:07:23.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf58cf4640 1 -- 172.21.15.64:0/3030227417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf50031940 con 0x7fcf54160b10 2023-11-24T08:07:23.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf58cf4640 1 -- 172.21.15.64:0/3030227417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50031cd0 con 0x7fcf54160b10 2023-11-24T08:07:23.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3030227417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 msgr2=0x7fcf5406e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3030227417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf5406e420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcf540d0090 tx=0x7fcf5002b880 comp rx=0 tx=0).stop 2023-11-24T08:07:23.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3030227417 shutdown_connections 2023-11-24T08:07:23.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3030227417 >> v2:172.21.15.91:3300/0 conn(0x7fcf5406cad0 0x7fcf540b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3030227417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf5406e420 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.206+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3030227417 >> 172.21.15.64:0/3030227417 conn(0x7fcf540b0ba0 msgr2=0x7fcf540b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:23.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.207+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3030227417 shutdown_connections 2023-11-24T08:07:23.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.207+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3030227417 wait complete. 2023-11-24T08:07:23.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.208+0000 7fcf5bf5a640 1 Processor -- start 2023-11-24T08:07:23.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.208+0000 7fcf5bf5a640 1 -- start start 2023-11-24T08:07:23.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.209+0000 7fcf5bf5a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5406cad0 0x7fcf54158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.209+0000 7fcf5bf5a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54072280 0x7fcf540ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.209+0000 7fcf5bf5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf540ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.209+0000 7fcf5bf5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54158d60 con 0x7fcf54160b10 2023-11-24T08:07:23.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.209+0000 7fcf5bf5a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540cba40 con 0x7fcf5406cad0 2023-11-24T08:07:23.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.209+0000 7fcf5bf5a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540cbce0 con 0x7fcf54072280 2023-11-24T08:07:23.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf5a4f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf540ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf5a4f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf540ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52100/0 (socket says 172.21.15.64:52100) 2023-11-24T08:07:23.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf5a4f7640 1 -- 172.21.15.64:0/3702448680 learned_addr learned my addr 172.21.15.64:0/3702448680 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:23.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf59cf6640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5406cad0 0x7fcf54158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf5a4f7640 1 -- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54072280 msgr2=0x7fcf540ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf594f5640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54072280 0x7fcf540ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf5a4f7640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54072280 0x7fcf540ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf5a4f7640 1 -- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5406cad0 msgr2=0x7fcf54158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf5a4f7640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5406cad0 0x7fcf54158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf5a4f7640 1 -- 172.21.15.64:0/3702448680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf500094d0 con 0x7fcf54160b10 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf59cf6640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5406cad0 0x7fcf54158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.210+0000 7fcf594f5640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54072280 0x7fcf540ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.211+0000 7fcf5a4f7640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf540ace30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00cd40 tx=0x7fcf4c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.211+0000 7fcf4a7fc640 1 -- 172.21.15.64:0/3702448680 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c003b90 con 0x7fcf54160b10 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.211+0000 7fcf4a7fc640 1 -- 172.21.15.64:0/3702448680 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c003cf0 con 0x7fcf54160b10 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.211+0000 7fcf4a7fc640 1 -- 172.21.15.64:0/3702448680 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004240 con 0x7fcf54160b10 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.211+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf540cbfc0 con 0x7fcf54160b10 2023-11-24T08:07:23.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.211+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf540cc580 con 0x7fcf54160b10 2023-11-24T08:07:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.212+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5409d200 con 0x7fcf54160b10 2023-11-24T08:07:23.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.213+0000 7fcf4a7fc640 1 -- 172.21.15.64:0/3702448680 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcf4c0043a0 con 0x7fcf54160b10 2023-11-24T08:07:23.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.214+0000 7fcf4a7fc640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3806b230 0x7fcf3806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.214+0000 7fcf4a7fc640 1 -- 172.21.15.64:0/3702448680 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcf4c012070 con 0x7fcf54160b10 2023-11-24T08:07:23.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.215+0000 7fcf59cf6640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3806b230 0x7fcf3806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.216+0000 7fcf59cf6640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3806b230 0x7fcf3806d6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcf50036000 tx=0x7fcf5003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:23.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.218+0000 7fcf4a7fc640 1 -- 172.21.15.64:0/3702448680 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf4c055200 con 0x7fcf54160b10 2023-11-24T08:07:23.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.413+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf5404e920 con 0x7fcf54160b10 2023-11-24T08:07:23.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.413+0000 7fcf4a7fc640 1 -- 172.21.15.64:0/3702448680 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf4c058820 con 0x7fcf54160b10 2023-11-24T08:07:23.414 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:07:23.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3806b230 msgr2=0x7fcf3806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3806b230 0x7fcf3806d6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcf50036000 tx=0x7fcf5003d040 comp rx=0 tx=0).stop 2023-11-24T08:07:23.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 msgr2=0x7fcf540ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf540ace30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00cd40 tx=0x7fcf4c00a660 comp rx=0 tx=0).stop 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 shutdown_connections 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3806b230 0x7fcf3806d6f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54160b10 0x7fcf540ace30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54072280 0x7fcf540ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 --2- 172.21.15.64:0/3702448680 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5406cad0 0x7fcf54158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 >> 172.21.15.64:0/3702448680 conn(0x7fcf540b0ba0 msgr2=0x7fcf540b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 shutdown_connections 2023-11-24T08:07:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.416+0000 7fcf5bf5a640 1 -- 172.21.15.64:0/3702448680 wait complete. 2023-11-24T08:07:23.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:07:23.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.662+0000 7fbac5a46640 1 Processor -- start 2023-11-24T08:07:23.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.662+0000 7fbac5a46640 1 -- start start 2023-11-24T08:07:23.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.664+0000 7fbac5a46640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbac00acd70 0x7fbac006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.664+0000 7fbac5a46640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac006d7a0 con 0x7fbac0070490 2023-11-24T08:07:23.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.664+0000 7fbac5a46640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac006ce70 con 0x7fbac00ae490 2023-11-24T08:07:23.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.664+0000 7fbac5a46640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac015a5b0 con 0x7fbac00acd70 2023-11-24T08:07:23.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.665+0000 7fbabf7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbac0070490 0x7fbac0070900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56550/0 (socket says 172.21.15.64:56550) 2023-11-24T08:07:23.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.665+0000 7fbabf7fe640 1 -- 172.21.15.64:0/3406391250 learned_addr learned my addr 172.21.15.64:0/3406391250 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:23.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.665+0000 7fbabeffd640 1 --2- 172.21.15.64:0/3406391250 >> v2:172.21.15.91:3300/0 conn(0x7fbac00acd70 0x7fbac006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.665+0000 7fbabe7fc640 1 -- 172.21.15.64:0/3406391250 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580611324 0 0) 0x7fbac006d7a0 con 0x7fbac0070490 2023-11-24T08:07:23.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.665+0000 7fbabe7fc640 1 -- 172.21.15.64:0/3406391250 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaac003520 con 0x7fbac0070490 2023-11-24T08:07:23.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.666+0000 7fbabeffd640 1 -- 172.21.15.64:0/3406391250 >> v1:172.21.15.91:6789/0 conn(0x7fbac00ae490 legacy=0x7fbac00ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.666+0000 7fbabeffd640 1 -- 172.21.15.64:0/3406391250 >> v1:172.21.15.64:6789/0 conn(0x7fbac0070490 legacy=0x7fbac0070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.666+0000 7fbabe7fc640 1 -- 172.21.15.64:0/3406391250 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4000268317 0 0) 0x7fbaac003520 con 0x7fbac0070490 2023-11-24T08:07:23.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.666+0000 7fbabeffd640 1 -- 172.21.15.64:0/3406391250 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac006d150 con 0x7fbac00acd70 2023-11-24T08:07:23.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.667+0000 7fbabeffd640 1 --2- 172.21.15.64:0/3406391250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 0x7fbac006dc30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbab00097a0 tx=0x7fbab002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=827a2bb2bbaf4a7a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:23.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.667+0000 7fbabe7fc640 1 -- 172.21.15.64:0/3406391250 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab002f050 con 0x7fbac00acd70 2023-11-24T08:07:23.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.667+0000 7fbabe7fc640 1 -- 172.21.15.64:0/3406391250 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab0031500 con 0x7fbac00acd70 2023-11-24T08:07:23.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.668+0000 7fbabe7fc640 1 -- 172.21.15.64:0/3406391250 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80019f0 con 0x7fbac00acd70 2023-11-24T08:07:23.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.668+0000 7fbac5a46640 1 -- 172.21.15.64:0/3406391250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 msgr2=0x7fbac006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.668+0000 7fbac5a46640 1 --2- 172.21.15.64:0/3406391250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 0x7fbac006dc30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbab00097a0 tx=0x7fbab002b820 comp rx=0 tx=0).stop 2023-11-24T08:07:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.668+0000 7fbac5a46640 1 -- 172.21.15.64:0/3406391250 shutdown_connections 2023-11-24T08:07:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.668+0000 7fbac5a46640 1 --2- 172.21.15.64:0/3406391250 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 0x7fbac006dc30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.668+0000 7fbac5a46640 1 -- 172.21.15.64:0/3406391250 >> 172.21.15.64:0/3406391250 conn(0x7fbac00b0ba0 msgr2=0x7fbac00b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.668+0000 7fbac5a46640 1 -- 172.21.15.64:0/3406391250 shutdown_connections 2023-11-24T08:07:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.668+0000 7fbac5a46640 1 -- 172.21.15.64:0/3406391250 wait complete. 2023-11-24T08:07:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.670+0000 7fbac5a46640 1 Processor -- start 2023-11-24T08:07:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.670+0000 7fbac5a46640 1 -- start start 2023-11-24T08:07:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.671+0000 7fbac5a46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbac0070490 0x7fbac01f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.671+0000 7fbac5a46640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 0x7fbac00c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.671+0000 7fbac5a46640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbac00ae490 0x7fbac01486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.671+0000 7fbac5a46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac01f6030 con 0x7fbac0070490 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.671+0000 7fbac5a46640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac01f51c0 con 0x7fbac00acd70 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.671+0000 7fbac5a46640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac01562e0 con 0x7fbac00ae490 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbac0070490 0x7fbac01f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbac0070490 0x7fbac01f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52116/0 (socket says 172.21.15.64:52116) 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 -- 172.21.15.64:0/1528527356 learned_addr learned my addr 172.21.15.64:0/1528527356 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbab67fc640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 0x7fbac00c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 -- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbac00ae490 msgr2=0x7fbac01486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabf7fe640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbac00ae490 0x7fbac01486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbac00ae490 0x7fbac01486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 -- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 msgr2=0x7fbac00c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 0x7fbac00c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 -- 172.21.15.64:0/1528527356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0009400 con 0x7fbac0070490 2023-11-24T08:07:23.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbab67fc640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 0x7fbac00c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:23.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.672+0000 7fbabeffd640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbac0070490 0x7fbac01f4c80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:23.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.673+0000 7fbabcff9640 1 -- 172.21.15.64:0/1528527356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0033980 con 0x7fbac0070490 2023-11-24T08:07:23.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.673+0000 7fbabcff9640 1 -- 172.21.15.64:0/1528527356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab0033b00 con 0x7fbac0070490 2023-11-24T08:07:23.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.673+0000 7fbabcff9640 1 -- 172.21.15.64:0/1528527356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab003e480 con 0x7fbac0070490 2023-11-24T08:07:23.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.673+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac01f53b0 con 0x7fbac0070490 2023-11-24T08:07:23.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.673+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac01f5970 con 0x7fbac0070490 2023-11-24T08:07:23.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.674+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac004e920 con 0x7fbac0070490 2023-11-24T08:07:23.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.675+0000 7fbabcff9640 1 -- 172.21.15.64:0/1528527356 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbab003a040 con 0x7fbac0070490 2023-11-24T08:07:23.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.676+0000 7fbabcff9640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbaa006b260 0x7fbaa006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:23.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.676+0000 7fbabcff9640 1 -- 172.21.15.64:0/1528527356 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbab00abd20 con 0x7fbac0070490 2023-11-24T08:07:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.677+0000 7fbab67fc640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbaa006b260 0x7fbaa006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.678+0000 7fbab67fc640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbaa006b260 0x7fbaa006d720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbac00c8310 tx=0x7fbaa8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:23.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.680+0000 7fbabcff9640 1 -- 172.21.15.64:0/1528527356 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbab0076710 con 0x7fbac0070490 2023-11-24T08:07:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.873+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbac00ad2d0 con 0x7fbac0070490 2023-11-24T08:07:23.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.874+0000 7fbabcff9640 1 -- 172.21.15.64:0/1528527356 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbab00380a0 con 0x7fbac0070490 2023-11-24T08:07:23.875 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:07:23.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbaa006b260 msgr2=0x7fbaa006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbaa006b260 0x7fbaa006d720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbac00c8310 tx=0x7fbaa8009040 comp rx=0 tx=0).stop 2023-11-24T08:07:23.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbac0070490 msgr2=0x7fbac01f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:23.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbac0070490 0x7fbac01f4c80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab00046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 shutdown_connections 2023-11-24T08:07:23.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbaa006b260 0x7fbaa006d720 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbac00ae490 0x7fbac01486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbac00acd70 0x7fbac00c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 --2- 172.21.15.64:0/1528527356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbac0070490 0x7fbac01f4c80 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:23.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 >> 172.21.15.64:0/1528527356 conn(0x7fbac00b0ba0 msgr2=0x7fbac00b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:23.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.877+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 shutdown_connections 2023-11-24T08:07:23.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:23.878+0000 7fbac5a46640 1 -- 172.21.15.64:0/1528527356 wait complete. 2023-11-24T08:07:23.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:24.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.109+0000 7fce5c6bb640 1 Processor -- start 2023-11-24T08:07:24.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.109+0000 7fce5c6bb640 1 -- start start 2023-11-24T08:07:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.111+0000 7fce5c6bb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fce540abff0 0x7fce54071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.111+0000 7fce5c6bb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5413da30 con 0x7fce5415a100 2023-11-24T08:07:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.111+0000 7fce5c6bb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce540ac560 con 0x7fce540b39a0 2023-11-24T08:07:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.111+0000 7fce5c6bb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce540ac840 con 0x7fce540abff0 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.111+0000 7fce5a457640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fce5415a100 0x7fce540b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56558/0 (socket says 172.21.15.64:56558) 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.111+0000 7fce5a457640 1 -- 172.21.15.64:0/1403110859 learned_addr learned my addr 172.21.15.64:0/1403110859 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce5ac58640 1 --2- 172.21.15.64:0/1403110859 >> v2:172.21.15.91:3300/0 conn(0x7fce540abff0 0x7fce54071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930082739 0 0) 0x7fce5413da30 con 0x7fce5415a100 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce40003520 con 0x7fce5415a100 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522747958 0 0) 0x7fce40003520 con 0x7fce5415a100 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 >> v2:172.21.15.91:3300/0 conn(0x7fce540abff0 msgr2=0x7fce54071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce59455640 1 --2- 172.21.15.64:0/1403110859 >> v2:172.21.15.91:3300/0 conn(0x7fce540abff0 0x7fce54071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 >> v1:172.21.15.91:6789/0 conn(0x7fce540b39a0 legacy=0x7fce54071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54070650 con 0x7fce5415a100 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.112+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fce50001670 con 0x7fce5415a100 2023-11-24T08:07:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 >> v1:172.21.15.64:6789/0 conn(0x7fce5415a100 legacy=0x7fce540b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce59455640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce400039c0 0x7fce40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce59455640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce400047c0 0x7fce40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce59455640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540ac840 con 0x7fce40005180 2023-11-24T08:07:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4000c080 con 0x7fce400047c0 2023-11-24T08:07:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4000c240 con 0x7fce400039c0 2023-11-24T08:07:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce5ac58640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce59c56640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce400047c0 0x7fce40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce5a457640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce400039c0 0x7fce40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce5ac58640 1 -- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce400039c0 msgr2=0x7fce40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce5ac58640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce400039c0 0x7fce40003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.113+0000 7fce5ac58640 1 -- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce400047c0 msgr2=0x7fce40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.114+0000 7fce5ac58640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce400047c0 0x7fce40004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.114+0000 7fce5ac58640 1 -- 172.21.15.64:0/1403110859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce40003520 con 0x7fce40005180 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.114+0000 7fce59c56640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce400047c0 0x7fce40004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.114+0000 7fce5a457640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce400039c0 0x7fce40003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.114+0000 7fce5ac58640 1 -- 172.21.15.64:0/1403110859 reap_dead start 2023-11-24T08:07:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.115+0000 7fce5ac58640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce4000b980 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fce4c009fc0 tx=0x7fce400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94393bd02262d0a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:24.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.115+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c02cc30 con 0x7fce40005180 2023-11-24T08:07:24.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.115+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c00f040 con 0x7fce40005180 2023-11-24T08:07:24.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.115+0000 7fce59455640 1 -- 172.21.15.64:0/1403110859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c003990 con 0x7fce40005180 2023-11-24T08:07:24.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.116+0000 7fce5c6bb640 1 -- 172.21.15.64:0/1403110859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 msgr2=0x7fce4000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.116+0000 7fce5c6bb640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce4000b980 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fce4c009fc0 tx=0x7fce400057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.116+0000 7fce5c6bb640 1 -- 172.21.15.64:0/1403110859 shutdown_connections 2023-11-24T08:07:24.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.116+0000 7fce5c6bb640 1 --2- 172.21.15.64:0/1403110859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce4000b980 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.116+0000 7fce5c6bb640 1 -- 172.21.15.64:0/1403110859 >> 172.21.15.64:0/1403110859 conn(0x7fce540b0d80 msgr2=0x7fce540b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:24.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.116+0000 7fce5c6bb640 1 -- 172.21.15.64:0/1403110859 shutdown_connections 2023-11-24T08:07:24.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.116+0000 7fce5c6bb640 1 -- 172.21.15.64:0/1403110859 wait complete. 2023-11-24T08:07:24.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.118+0000 7fce5c6bb640 1 Processor -- start 2023-11-24T08:07:24.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.118+0000 7fce5c6bb640 1 -- start start 2023-11-24T08:07:24.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5c6bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce54097a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5c6bb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce5415c440 0x7fce5415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5c6bb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce5415ce00 0x7fce540cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5c6bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5415c0f0 con 0x7fce40005180 2023-11-24T08:07:24.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5c6bb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540cc210 con 0x7fce5415c440 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5c6bb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5415d280 con 0x7fce5415ce00 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5a457640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce54097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5a457640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce54097a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52142/0 (socket says 172.21.15.64:52142) 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.119+0000 7fce5a457640 1 -- 172.21.15.64:0/287744804 learned_addr learned my addr 172.21.15.64:0/287744804 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.120+0000 7fce59c56640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce5415c440 0x7fce5415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.120+0000 7fce5ac58640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce5415ce00 0x7fce540cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.120+0000 7fce5a457640 1 -- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce5415ce00 msgr2=0x7fce540cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.120+0000 7fce5a457640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce5415ce00 0x7fce540cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.120+0000 7fce5a457640 1 -- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce5415c440 msgr2=0x7fce5415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.120+0000 7fce5a457640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce5415c440 0x7fce5415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.120+0000 7fce5a457640 1 -- 172.21.15.64:0/287744804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54097f80 con 0x7fce40005180 2023-11-24T08:07:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.120+0000 7fce5a457640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce54097a40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fce5000d7a0 tx=0x7fce5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:24.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.121+0000 7fce4affd640 1 -- 172.21.15.64:0/287744804 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500074e0 con 0x7fce40005180 2023-11-24T08:07:24.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.121+0000 7fce4affd640 1 -- 172.21.15.64:0/287744804 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50007640 con 0x7fce40005180 2023-11-24T08:07:24.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.121+0000 7fce4affd640 1 -- 172.21.15.64:0/287744804 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500049e0 con 0x7fce40005180 2023-11-24T08:07:24.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.121+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5415be10 con 0x7fce40005180 2023-11-24T08:07:24.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.121+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce540cbea0 con 0x7fce40005180 2023-11-24T08:07:24.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.122+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce54067960 con 0x7fce40005180 2023-11-24T08:07:24.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.123+0000 7fce4affd640 1 -- 172.21.15.64:0/287744804 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fce50004b40 con 0x7fce40005180 2023-11-24T08:07:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.124+0000 7fce4affd640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce3806b230 0x7fce3806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.124+0000 7fce4affd640 1 -- 172.21.15.64:0/287744804 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fce50089980 con 0x7fce40005180 2023-11-24T08:07:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.125+0000 7fce59c56640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce3806b230 0x7fce3806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.126+0000 7fce59c56640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce3806b230 0x7fce3806d6f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fce540afb30 tx=0x7fce44007bd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:24.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.128+0000 7fce4affd640 1 -- 172.21.15.64:0/287744804 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce50057e20 con 0x7fce40005180 2023-11-24T08:07:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.362+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce54074770 con 0x7fce40005180 2023-11-24T08:07:24.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.363+0000 7fce4affd640 1 -- 172.21.15.64:0/287744804 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fce50019090 con 0x7fce40005180 2023-11-24T08:07:24.365 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:24.365 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2873504597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2873504597},{"type":"v1","addr":"172.21.15.91:6835","nonce":2873504597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3295227147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3295227147},{"type":"v1","addr":"172.21.15.91:6837","nonce":3295227147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2949237565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2949237565},{"type":"v1","addr":"172.21.15.64:6834","nonce":2949237565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:57.112228+0000","modified":"2023-11-24T08:07:00.539822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4200493009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":4200493009},{"type":"v1","addr":"172.21.15.64:6835","nonce":4200493009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:07:24.365 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2023-11-24T08:07:24.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce3806b230 msgr2=0x7fce3806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce3806b230 0x7fce3806d6f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fce540afb30 tx=0x7fce44007bd0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 msgr2=0x7fce54097a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce54097a40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fce5000d7a0 tx=0x7fce5000dc60 comp rx=0 tx=0).stop 2023-11-24T08:07:24.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 shutdown_connections 2023-11-24T08:07:24.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fce3806b230 0x7fce3806d6f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fce5415ce00 0x7fce540cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fce5415c440 0x7fce5415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 --2- 172.21.15.64:0/287744804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce40005180 0x7fce54097a40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.365+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 >> 172.21.15.64:0/287744804 conn(0x7fce540b0d80 msgr2=0x7fce5406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:24.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.366+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 shutdown_connections 2023-11-24T08:07:24.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.366+0000 7fce5c6bb640 1 -- 172.21.15.64:0/287744804 wait complete. 2023-11-24T08:07:24.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:24.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.607+0000 7f1fe11ec640 1 Processor -- start 2023-11-24T08:07:24.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.608+0000 7f1fe11ec640 1 -- start start 2023-11-24T08:07:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.609+0000 7f1fe11ec640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fdc0b39a0 0x7f1fdc071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.609+0000 7f1fe11ec640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fdc13da30 con 0x7f1fdc0abff0 2023-11-24T08:07:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.609+0000 7f1fe11ec640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fdc0ac560 con 0x7f1fdc15a100 2023-11-24T08:07:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.609+0000 7f1fe11ec640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fdc0ac840 con 0x7f1fdc0b39a0 2023-11-24T08:07:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.610+0000 7f1fdb59e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1fdc0abff0 0x7f1fdc071670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56574/0 (socket says 172.21.15.64:56574) 2023-11-24T08:07:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.610+0000 7f1fdb59e640 1 -- 172.21.15.64:0/1297314811 learned_addr learned my addr 172.21.15.64:0/1297314811 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.610+0000 7f1fda59c640 1 --2- 172.21.15.64:0/1297314811 >> v2:172.21.15.64:3300/0 conn(0x7f1fdc0b39a0 0x7f1fdc071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.610+0000 7f1fd9d9b640 1 -- 172.21.15.64:0/1297314811 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757687670 0 0) 0x7f1fdc13da30 con 0x7f1fdc0abff0 2023-11-24T08:07:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.610+0000 7f1fd9d9b640 1 -- 172.21.15.64:0/1297314811 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc4003520 con 0x7f1fdc0abff0 2023-11-24T08:07:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.610+0000 7f1fda59c640 1 -- 172.21.15.64:0/1297314811 >> v1:172.21.15.91:6789/0 conn(0x7f1fdc15a100 legacy=0x7f1fdc0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.611+0000 7f1fda59c640 1 -- 172.21.15.64:0/1297314811 >> v1:172.21.15.64:6789/0 conn(0x7f1fdc0abff0 legacy=0x7f1fdc071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.611+0000 7f1fda59c640 1 -- 172.21.15.64:0/1297314811 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc070650 con 0x7f1fdc0b39a0 2023-11-24T08:07:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.612+0000 7f1fda59c640 1 --2- 172.21.15.64:0/1297314811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0b39a0 0x7f1fdc071130 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00097a0 tx=0x7f1fd002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89381da061640a31 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.612+0000 7f1fd9d9b640 1 -- 172.21.15.64:0/1297314811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0030610 con 0x7f1fdc0b39a0 2023-11-24T08:07:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.612+0000 7f1fd9d9b640 1 -- 172.21.15.64:0/1297314811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0030770 con 0x7f1fdc0b39a0 2023-11-24T08:07:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.613+0000 7f1fd9d9b640 1 -- 172.21.15.64:0/1297314811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0030ae0 con 0x7f1fdc0b39a0 2023-11-24T08:07:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.613+0000 7f1fe11ec640 1 -- 172.21.15.64:0/1297314811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0b39a0 msgr2=0x7f1fdc071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.613+0000 7f1fe11ec640 1 --2- 172.21.15.64:0/1297314811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0b39a0 0x7f1fdc071130 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00097a0 tx=0x7f1fd002b960 comp rx=0 tx=0).stop 2023-11-24T08:07:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.613+0000 7f1fe11ec640 1 -- 172.21.15.64:0/1297314811 shutdown_connections 2023-11-24T08:07:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.613+0000 7f1fe11ec640 1 --2- 172.21.15.64:0/1297314811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0b39a0 0x7f1fdc071130 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.613+0000 7f1fe11ec640 1 -- 172.21.15.64:0/1297314811 >> 172.21.15.64:0/1297314811 conn(0x7f1fdc0b0d80 msgr2=0x7f1fdc0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.613+0000 7f1fe11ec640 1 -- 172.21.15.64:0/1297314811 shutdown_connections 2023-11-24T08:07:24.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.613+0000 7f1fe11ec640 1 -- 172.21.15.64:0/1297314811 wait complete. 2023-11-24T08:07:24.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.615+0000 7f1fe11ec640 1 Processor -- start 2023-11-24T08:07:24.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.615+0000 7f1fe11ec640 1 -- start start 2023-11-24T08:07:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.616+0000 7f1fe11ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0abff0 0x7f1fdc06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.616+0000 7f1fe11ec640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1fdc15a100 0x7f1fdc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.616+0000 7f1fe11ec640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fdc15d0d0 0x7f1fdc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.616+0000 7f1fe11ec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0cc390 con 0x7f1fdc0abff0 2023-11-24T08:07:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.616+0000 7f1fe11ec640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0c84a0 con 0x7f1fdc15d0d0 2023-11-24T08:07:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.616+0000 7f1fe11ec640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0c8780 con 0x7f1fdc15a100 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0abff0 0x7f1fdc06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0abff0 0x7f1fdc06c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52158/0 (socket says 172.21.15.64:52158) 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdad9d640 1 -- 172.21.15.64:0/3097498401 learned_addr learned my addr 172.21.15.64:0/3097498401 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdb59e640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fdc15d0d0 0x7f1fdc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fda59c640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1fdc15a100 0x7f1fdc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdad9d640 1 -- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1fdc15a100 msgr2=0x7f1fdc15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdad9d640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1fdc15a100 0x7f1fdc15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdad9d640 1 -- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fdc15d0d0 msgr2=0x7f1fdc15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdad9d640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fdc15d0d0 0x7f1fdc15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.617+0000 7f1fdad9d640 1 -- 172.21.15.64:0/3097498401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd0009400 con 0x7f1fdc0abff0 2023-11-24T08:07:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.618+0000 7f1fdad9d640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0abff0 0x7f1fdc06c7b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00c870 tx=0x7f1fcc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.618+0000 7f1fc37fe640 1 -- 172.21.15.64:0/3097498401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc003b70 con 0x7f1fdc0abff0 2023-11-24T08:07:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.618+0000 7f1fc37fe640 1 -- 172.21.15.64:0/3097498401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc003cf0 con 0x7f1fdc0abff0 2023-11-24T08:07:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.618+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fdc0c8a60 con 0x7f1fdc0abff0 2023-11-24T08:07:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.618+0000 7f1fc37fe640 1 -- 172.21.15.64:0/3097498401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00aa90 con 0x7f1fdc0abff0 2023-11-24T08:07:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.618+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc15d560 con 0x7f1fdc0abff0 2023-11-24T08:07:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.619+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc067960 con 0x7f1fdc0abff0 2023-11-24T08:07:24.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.620+0000 7f1fc37fe640 1 -- 172.21.15.64:0/3097498401 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1fcc00abf0 con 0x7f1fdc0abff0 2023-11-24T08:07:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.621+0000 7f1fc37fe640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1fc806b230 0x7f1fc806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.621+0000 7f1fc37fe640 1 -- 172.21.15.64:0/3097498401 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1fcc089c50 con 0x7f1fdc0abff0 2023-11-24T08:07:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.622+0000 7f1fda59c640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1fc806b230 0x7f1fc806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:24.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.623+0000 7f1fda59c640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1fc806b230 0x7f1fc806d6f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0036000 tx=0x7f1fd0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.625+0000 7f1fc37fe640 1 -- 172.21.15.64:0/3097498401 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1fcc054700 con 0x7f1fdc0abff0 2023-11-24T08:07:24.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.816+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fdc0cb3c0 con 0x7f1fdc0abff0 2023-11-24T08:07:24.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.818+0000 7f1fc37fe640 1 -- 172.21.15.64:0/3097498401 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f1fdc0cb3c0 con 0x7f1fdc0abff0 2023-11-24T08:07:24.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:24.820 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":62,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:21.002292+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:55.038686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:56.142053+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:24.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.820+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1fc806b230 msgr2=0x7f1fc806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.820+0000 7f1fe11ec640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1fc806b230 0x7f1fc806d6f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0036000 tx=0x7f1fd0008030 comp rx=0 tx=0).stop 2023-11-24T08:07:24.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.820+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0abff0 msgr2=0x7f1fdc06c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:24.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.820+0000 7f1fe11ec640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0abff0 0x7f1fdc06c7b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00c870 tx=0x7f1fcc00cd30 comp rx=0 tx=0).stop 2023-11-24T08:07:24.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.821+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 shutdown_connections 2023-11-24T08:07:24.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.821+0000 7f1fe11ec640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1fc806b230 0x7f1fc806d6f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.821+0000 7f1fe11ec640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1fdc15d0d0 0x7f1fdc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.821+0000 7f1fe11ec640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1fdc15a100 0x7f1fdc15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.821+0000 7f1fe11ec640 1 --2- 172.21.15.64:0/3097498401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc0abff0 0x7f1fdc06c7b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:24.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.821+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 >> 172.21.15.64:0/3097498401 conn(0x7f1fdc0b0d80 msgr2=0x7f1fdc0712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:24.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.821+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 shutdown_connections 2023-11-24T08:07:24.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:24.821+0000 7f1fe11ec640 1 -- 172.21.15.64:0/3097498401 wait complete. 2023-11-24T08:07:24.840 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:07:24.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:25.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.051+0000 7fe41adcb640 1 Processor -- start 2023-11-24T08:07:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.051+0000 7fe41adcb640 1 -- start start 2023-11-24T08:07:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.052+0000 7fe41adcb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe41413dfd0 0x7fe41415be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.052+0000 7fe41adcb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4140cf500 0x7fe41406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.052+0000 7fe41adcb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140ce7c0 con 0x7fe4140cf050 2023-11-24T08:07:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.052+0000 7fe41adcb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe414140900 con 0x7fe41413dfd0 2023-11-24T08:07:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.052+0000 7fe41adcb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41415fa70 con 0x7fe4140cf500 2023-11-24T08:07:25.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.052+0000 7fe4195c8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe4140cf050 0x7fe41406c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56580/0 (socket says 172.21.15.64:56580) 2023-11-24T08:07:25.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.052+0000 7fe4195c8640 1 -- 172.21.15.64:0/3730892136 learned_addr learned my addr 172.21.15.64:0/3730892136 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:25.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe41a5ca640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4140cf500 0x7fe41406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79722700 0 0) 0x7fe4140ce7c0 con 0x7fe4140cf050 2023-11-24T08:07:25.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe419dc9640 1 --2- 172.21.15.64:0/3730892136 >> v2:172.21.15.91:3300/0 conn(0x7fe41413dfd0 0x7fe41415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe400003520 con 0x7fe4140cf050 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 821396493 0 0) 0x7fe400003520 con 0x7fe4140cf050 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4140cf500 msgr2=0x7fe41406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4140cf500 0x7fe41406c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 >> v2:172.21.15.91:3300/0 conn(0x7fe41413dfd0 msgr2=0x7fe41415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 --2- 172.21.15.64:0/3730892136 >> v2:172.21.15.91:3300/0 conn(0x7fe41413dfd0 0x7fe41415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140cf950 con 0x7fe4140cf050 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fe404001690 con 0x7fe4140cf050 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe419dc9640 1 --2- 172.21.15.64:0/3730892136 >> v2:172.21.15.91:3300/0 conn(0x7fe41413dfd0 0x7fe41415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.053+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 >> v1:172.21.15.64:6789/0 conn(0x7fe4140cf050 legacy=0x7fe41406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe418dc7640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4140cf500 0x7fe4000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe418dc7640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe4000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe418dc7640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe400004d00 0x7fe400005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414140900 con 0x7fe400003fb0 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41415fa70 con 0x7fe400004d00 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40000bab0 con 0x7fe4140cf500 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe4195c8640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe4000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe419dc9640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4140cf500 0x7fe4000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe41a5ca640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe400004d00 0x7fe400005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe4195c8640 1 -- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4140cf500 msgr2=0x7fe4000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.054+0000 7fe4195c8640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe4140cf500 0x7fe4000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.055+0000 7fe4195c8640 1 -- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe400004d00 msgr2=0x7fe400005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.055+0000 7fe4195c8640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe400004d00 0x7fe400005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.055+0000 7fe4195c8640 1 -- 172.21.15.64:0/3730892136 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400003520 con 0x7fe400003fb0 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.056+0000 7fe4195c8640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe4000047c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe40400c160 tx=0x7fe4000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c033c18b0b0368d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.056+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404005ad0 con 0x7fe400003fb0 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.056+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe404005c30 con 0x7fe400003fb0 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.056+0000 7fe418dc7640 1 -- 172.21.15.64:0/3730892136 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40402c7f0 con 0x7fe400003fb0 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.057+0000 7fe41adcb640 1 -- 172.21.15.64:0/3730892136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 msgr2=0x7fe4000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.057+0000 7fe41adcb640 1 --2- 172.21.15.64:0/3730892136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe4000047c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe40400c160 tx=0x7fe4000057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.057+0000 7fe41a5ca640 1 -- 172.21.15.64:0/3730892136 reap_dead start 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.057+0000 7fe41adcb640 1 -- 172.21.15.64:0/3730892136 shutdown_connections 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.057+0000 7fe41adcb640 1 -- 172.21.15.64:0/3730892136 >> 172.21.15.64:0/3730892136 conn(0x7fe41406a860 msgr2=0x7fe41406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.057+0000 7fe41adcb640 1 -- 172.21.15.64:0/3730892136 shutdown_connections 2023-11-24T08:07:25.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.057+0000 7fe41adcb640 1 -- 172.21.15.64:0/3730892136 wait complete. 2023-11-24T08:07:25.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.059+0000 7fe41adcb640 1 Processor -- start 2023-11-24T08:07:25.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.059+0000 7fe41adcb640 1 -- start start 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.060+0000 7fe41adcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe414155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.060+0000 7fe41adcb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe41413b930 0x7fe41413a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.060+0000 7fe41adcb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe41413ae90 0x7fe4140ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.060+0000 7fe41adcb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414071fc0 con 0x7fe400003fb0 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.060+0000 7fe41adcb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140719e0 con 0x7fe41413b930 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.060+0000 7fe41adcb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41415d5b0 con 0x7fe41413ae90 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe414155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe414155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52174/0 (socket says 172.21.15.64:52174) 2023-11-24T08:07:25.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 -- 172.21.15.64:0/398438064 learned_addr learned my addr 172.21.15.64:0/398438064 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:25.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe41a5ca640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe41413ae90 0x7fe4140ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe4195c8640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe41413b930 0x7fe41413a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 -- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe41413ae90 msgr2=0x7fe4140ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe41413ae90 0x7fe4140ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 -- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe41413b930 msgr2=0x7fe41413a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe41413b930 0x7fe41413a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 -- 172.21.15.64:0/398438064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414071c50 con 0x7fe400003fb0 2023-11-24T08:07:25.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.061+0000 7fe419dc9640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe414155220 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe410010890 tx=0x7fe410010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:25.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.062+0000 7fe40a7fc640 1 -- 172.21.15.64:0/398438064 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000cb40 con 0x7fe400003fb0 2023-11-24T08:07:25.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.062+0000 7fe40a7fc640 1 -- 172.21.15.64:0/398438064 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41000cca0 con 0x7fe400003fb0 2023-11-24T08:07:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.062+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41415d890 con 0x7fe400003fb0 2023-11-24T08:07:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.062+0000 7fe40a7fc640 1 -- 172.21.15.64:0/398438064 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410015690 con 0x7fe400003fb0 2023-11-24T08:07:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.062+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe414154d20 con 0x7fe400003fb0 2023-11-24T08:07:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.063+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe414071840 con 0x7fe400003fb0 2023-11-24T08:07:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.064+0000 7fe40a7fc640 1 -- 172.21.15.64:0/398438064 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe4100157f0 con 0x7fe400003fb0 2023-11-24T08:07:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.065+0000 7fe40a7fc640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe3f806b230 0x7fe3f806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.065+0000 7fe40a7fc640 1 -- 172.21.15.64:0/398438064 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe41008a460 con 0x7fe400003fb0 2023-11-24T08:07:25.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.066+0000 7fe4195c8640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe3f806b230 0x7fe3f806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.067+0000 7fe4195c8640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe3f806b230 0x7fe3f806d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe414141720 tx=0x7fe404007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:25.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.069+0000 7fe40a7fc640 1 -- 172.21.15.64:0/398438064 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe414071840 con 0x7fe400003fb0 2023-11-24T08:07:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.302+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe414071840 con 0x7fe400003fb0 2023-11-24T08:07:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.303+0000 7fe40a7fc640 1 -- 172.21.15.64:0/398438064 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fe414071840 con 0x7fe400003fb0 2023-11-24T08:07:25.305 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:25.306 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2873504597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2873504597},{"type":"v1","addr":"172.21.15.91:6835","nonce":2873504597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3295227147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3295227147},{"type":"v1","addr":"172.21.15.91:6837","nonce":3295227147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2949237565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2949237565},{"type":"v1","addr":"172.21.15.64:6834","nonce":2949237565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:57.112228+0000","modified":"2023-11-24T08:07:00.539822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4200493009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":4200493009},{"type":"v1","addr":"172.21.15.64:6835","nonce":4200493009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:07:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2023-11-24T08:07:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe3f806b230 msgr2=0x7fe3f806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe3f806b230 0x7fe3f806d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe414141720 tx=0x7fe404007000 comp rx=0 tx=0).stop 2023-11-24T08:07:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 msgr2=0x7fe414155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe414155220 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe410010890 tx=0x7fe410010d50 comp rx=0 tx=0).stop 2023-11-24T08:07:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 shutdown_connections 2023-11-24T08:07:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe3f806b230 0x7fe3f806d6f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe41413ae90 0x7fe4140ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe41413b930 0x7fe41413a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 --2- 172.21.15.64:0/398438064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe400003fb0 0x7fe414155220 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 >> 172.21.15.64:0/398438064 conn(0x7fe41406a860 msgr2=0x7fe4140cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:25.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 shutdown_connections 2023-11-24T08:07:25.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.306+0000 7fe41adcb640 1 -- 172.21.15.64:0/398438064 wait complete. 2023-11-24T08:07:25.326 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.549+0000 7f47fb521640 1 Processor -- start 2023-11-24T08:07:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.550+0000 7f47fb521640 1 -- start start 2023-11-24T08:07:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.551+0000 7f47fb521640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f47f415d5b0 0x7f47f40d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.551+0000 7f47fb521640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b1650 0x7f47f40d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.551+0000 7f47fb521640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40cf0c0 con 0x7f47f40d14a0 2023-11-24T08:07:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.551+0000 7f47fb521640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40b01b0 con 0x7f47f415d5b0 2023-11-24T08:07:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.551+0000 7f47fb521640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f415d990 con 0x7f47f40b1650 2023-11-24T08:07:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f9d1e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f47f40d14a0 0x7f47f40cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56590/0 (socket says 172.21.15.64:56590) 2023-11-24T08:07:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f9d1e640 1 -- 172.21.15.64:0/2413417975 learned_addr learned my addr 172.21.15.64:0/2413417975 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47fad20640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b1650 0x7f47f40d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47fa51f640 1 --2- 172.21.15.64:0/2413417975 >> v2:172.21.15.91:3300/0 conn(0x7f47f415d5b0 0x7f47f40d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000350553 0 0) 0x7f47f40cf0c0 con 0x7f47f40d14a0 2023-11-24T08:07:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e0003520 con 0x7f47f40d14a0 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2835579345 0 0) 0x7f47e0003520 con 0x7f47f40d14a0 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b1650 msgr2=0x7f47f40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b1650 0x7f47f40d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 >> v2:172.21.15.91:3300/0 conn(0x7f47f415d5b0 msgr2=0x7f47f40d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 --2- 172.21.15.64:0/2413417975 >> v2:172.21.15.91:3300/0 conn(0x7f47f415d5b0 0x7f47f40d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40d1e40 con 0x7f47f40d14a0 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f47e4001670 con 0x7f47f40d14a0 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 >> v1:172.21.15.64:6789/0 conn(0x7f47f40d14a0 legacy=0x7f47f40cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.552+0000 7f47f951d640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47f951d640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47e00047c0 0x7f47e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47f951d640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b1650 0x7f47e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40b01b0 con 0x7f47e00039e0 2023-11-24T08:07:25.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f415d990 con 0x7f47e00047c0 2023-11-24T08:07:25.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e000b9d0 con 0x7f47f40b1650 2023-11-24T08:07:25.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47fa51f640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47fa51f640 1 -- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b1650 msgr2=0x7f47e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:25.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47fa51f640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b1650 0x7f47e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47fa51f640 1 -- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47e00047c0 msgr2=0x7f47e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:25.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47fa51f640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47e00047c0 0x7f47e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.553+0000 7f47fa51f640 1 -- 172.21.15.64:0/2413417975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0003520 con 0x7f47e00039e0 2023-11-24T08:07:25.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.554+0000 7f47fa51f640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47e0003e00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009af0 tx=0x7f47e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aeecfe75da3406 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:25.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f002c710 con 0x7f47e00039e0 2023-11-24T08:07:25.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f002c870 con 0x7f47e00039e0 2023-11-24T08:07:25.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47f951d640 1 -- 172.21.15.64:0/2413417975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f002cc00 con 0x7f47e00039e0 2023-11-24T08:07:25.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47fb521640 1 -- 172.21.15.64:0/2413417975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 msgr2=0x7f47e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47fb521640 1 --2- 172.21.15.64:0/2413417975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47e0003e00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009af0 tx=0x7f47e00057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47fad20640 1 -- 172.21.15.64:0/2413417975 reap_dead start 2023-11-24T08:07:25.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47fb521640 1 -- 172.21.15.64:0/2413417975 shutdown_connections 2023-11-24T08:07:25.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47fb521640 1 -- 172.21.15.64:0/2413417975 >> 172.21.15.64:0/2413417975 conn(0x7f47f409c3b0 msgr2=0x7f47f40d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47fb521640 1 -- 172.21.15.64:0/2413417975 shutdown_connections 2023-11-24T08:07:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.555+0000 7f47fb521640 1 -- 172.21.15.64:0/2413417975 wait complete. 2023-11-24T08:07:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.557+0000 7f47fb521640 1 Processor -- start 2023-11-24T08:07:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.557+0000 7f47fb521640 1 -- start start 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.558+0000 7f47fb521640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47f40b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.558+0000 7f47fb521640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b5c90 0x7f47f40c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.558+0000 7f47fb521640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47f40b6110 0x7f47f40c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.558+0000 7f47fb521640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40b31c0 con 0x7f47e00039e0 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.558+0000 7f47fb521640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40b2330 con 0x7f47f40b6110 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.558+0000 7f47fb521640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40b2540 con 0x7f47f40b5c90 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47f40b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47f40b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52196/0 (socket says 172.21.15.64:52196) 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 -- 172.21.15.64:0/600224047 learned_addr learned my addr 172.21.15.64:0/600224047 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47f9d1e640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b5c90 0x7f47f40c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fad20640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47f40b6110 0x7f47f40c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 -- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b5c90 msgr2=0x7f47f40c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b5c90 0x7f47f40c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 -- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47f40b6110 msgr2=0x7f47f40c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47f40b6110 0x7f47f40c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 -- 172.21.15.64:0/600224047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40b2820 con 0x7f47e00039e0 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fad20640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47f40b6110 0x7f47f40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.559+0000 7f47fa51f640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47f40b8710 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f47f0037040 tx=0x7f47f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.560+0000 7f47eaffd640 1 -- 172.21.15.64:0/600224047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f0003810 con 0x7f47e00039e0 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.560+0000 7f47eaffd640 1 -- 172.21.15.64:0/600224047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f002de90 con 0x7f47e00039e0 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.560+0000 7f47eaffd640 1 -- 172.21.15.64:0/600224047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f0003b90 con 0x7f47e00039e0 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.560+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f40b2b00 con 0x7f47e00039e0 2023-11-24T08:07:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.560+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f40b8c50 con 0x7f47e00039e0 2023-11-24T08:07:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.561+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f40b8de0 con 0x7f47e00039e0 2023-11-24T08:07:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.562+0000 7f47eaffd640 1 -- 172.21.15.64:0/600224047 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f47f0003cf0 con 0x7f47e00039e0 2023-11-24T08:07:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.563+0000 7f47eaffd640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f47d806b260 0x7f47d806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.564+0000 7f47eaffd640 1 -- 172.21.15.64:0/600224047 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f47f00abac0 con 0x7f47e00039e0 2023-11-24T08:07:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.564+0000 7f47f9d1e640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f47d806b260 0x7f47d806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:25.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.565+0000 7f47f9d1e640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f47d806b260 0x7f47d806d720 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f47f40ce7f0 tx=0x7f47e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:25.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.568+0000 7f47eaffd640 1 -- 172.21.15.64:0/600224047 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47f00766e0 con 0x7f47e00039e0 2023-11-24T08:07:25.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.802+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47f404e840 con 0x7f47e00039e0 2023-11-24T08:07:25.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.804+0000 7f47eaffd640 1 -- 172.21.15.64:0/600224047 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f47f0079d00 con 0x7f47e00039e0 2023-11-24T08:07:25.806 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:25.806 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2873504597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2873504597},{"type":"v1","addr":"172.21.15.91:6835","nonce":2873504597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3295227147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3295227147},{"type":"v1","addr":"172.21.15.91:6837","nonce":3295227147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2949237565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2949237565},{"type":"v1","addr":"172.21.15.64:6834","nonce":2949237565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:06:57.112228+0000","modified":"2023-11-24T08:07:00.539822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4200493009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":4200493009},{"type":"v1","addr":"172.21.15.64:6835","nonce":4200493009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:07:25.806 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2023-11-24T08:07:25.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f47d806b260 msgr2=0x7f47d806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f47d806b260 0x7f47d806d720 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f47f40ce7f0 tx=0x7f47e4009040 comp rx=0 tx=0).stop 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 msgr2=0x7f47f40b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47f40b8710 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f47f0037040 tx=0x7f47f0004730 comp rx=0 tx=0).stop 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 shutdown_connections 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f47d806b260 0x7f47d806d720 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f47f40b6110 0x7f47f40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f47f40b5c90 0x7f47f40c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 --2- 172.21.15.64:0/600224047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e00039e0 0x7f47f40b8710 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.806+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 >> 172.21.15.64:0/600224047 conn(0x7f47f409c3b0 msgr2=0x7f47f415dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.807+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 shutdown_connections 2023-11-24T08:07:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:25.807+0000 7f47fb521640 1 -- 172.21.15.64:0/600224047 wait complete. 2023-11-24T08:07:25.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:26.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.048+0000 7f29bd59e640 1 Processor -- start 2023-11-24T08:07:26.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.048+0000 7f29bd59e640 1 -- start start 2023-11-24T08:07:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.050+0000 7f29bd59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.050+0000 7f29bd59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29b80cf050 0x7f29b80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.050+0000 7f29bd59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b80ce7c0 con 0x7f29b806c130 2023-11-24T08:07:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.050+0000 7f29bd59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b80cfa10 con 0x7f29b80cf050 2023-11-24T08:07:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.050+0000 7f29bd59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b8140900 con 0x7f29b813dfd0 2023-11-24T08:07:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.051+0000 7f29affff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29b80cf050 0x7f29b80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.051+0000 7f29affff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29b80cf050 0x7f29b80cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52200/0 (socket says 172.21.15.64:52200) 2023-11-24T08:07:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.051+0000 7f29affff640 1 -- 172.21.15.64:0/2117581201 learned_addr learned my addr 172.21.15.64:0/2117581201 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.051+0000 7f29b7fff640 1 --2- 172.21.15.64:0/2117581201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.051+0000 7f29affff640 1 -- 172.21.15.64:0/2117581201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 msgr2=0x7f29b815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.051+0000 7f29affff640 1 --2- 172.21.15.64:0/2117581201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.051+0000 7f29affff640 1 -- 172.21.15.64:0/2117581201 >> v1:172.21.15.91:6789/0 conn(0x7f29b806c130 legacy=0x7f29b806c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.051+0000 7f29affff640 1 -- 172.21.15.64:0/2117581201 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b815fa70 con 0x7f29b80cf050 2023-11-24T08:07:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.053+0000 7f29affff640 1 --2- 172.21.15.64:0/2117581201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b80cf4d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009870 tx=0x7f29a002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9ffafb03af5958 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.053+0000 7f29b77fe640 1 -- 172.21.15.64:0/2117581201 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0030840 con 0x7f29b80cf050 2023-11-24T08:07:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.053+0000 7f29b77fe640 1 -- 172.21.15.64:0/2117581201 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a00309a0 con 0x7f29b80cf050 2023-11-24T08:07:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.053+0000 7f29b77fe640 1 -- 172.21.15.64:0/2117581201 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0030d30 con 0x7f29b80cf050 2023-11-24T08:07:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.053+0000 7f29bd59e640 1 -- 172.21.15.64:0/2117581201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 msgr2=0x7f29b80cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.053+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2117581201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b80cf4d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009870 tx=0x7f29a002b980 comp rx=0 tx=0).stop 2023-11-24T08:07:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.054+0000 7f29bd59e640 1 -- 172.21.15.64:0/2117581201 shutdown_connections 2023-11-24T08:07:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.054+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2117581201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b80cf4d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.054+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2117581201 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.054+0000 7f29bd59e640 1 -- 172.21.15.64:0/2117581201 >> 172.21.15.64:0/2117581201 conn(0x7f29b806a860 msgr2=0x7f29b806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.054+0000 7f29bd59e640 1 -- 172.21.15.64:0/2117581201 shutdown_connections 2023-11-24T08:07:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.054+0000 7f29bd59e640 1 -- 172.21.15.64:0/2117581201 wait complete. 2023-11-24T08:07:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.055+0000 7f29bd59e640 1 Processor -- start 2023-11-24T08:07:26.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.056+0000 7f29bd59e640 1 -- start start 2023-11-24T08:07:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29bd59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b806c130 0x7f29b80c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29bd59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29bd59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b8155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29bd59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b813d480 con 0x7f29b80cf050 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29bd59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c8f00 con 0x7f29b806c130 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29bd59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c91a0 con 0x7f29b813dfd0 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b815a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52202/0 (socket says 172.21.15.64:52202) 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29affff640 1 -- 172.21.15.64:0/2134538851 learned_addr learned my addr 172.21.15.64:0/2134538851 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29b7fff640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b806c130 0x7f29b80c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29bcd9d640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b8155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29affff640 1 -- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 msgr2=0x7f29b8155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29affff640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b8155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29affff640 1 -- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b806c130 msgr2=0x7f29b80c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29affff640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b806c130 0x7f29b80c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29affff640 1 -- 172.21.15.64:0/2134538851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00094d0 con 0x7f29b80cf050 2023-11-24T08:07:26.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.057+0000 7f29bcd9d640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b8155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:26.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.058+0000 7f29affff640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b815a920 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f29a0000c00 tx=0x7f29a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:26.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.058+0000 7f29b57fa640 1 -- 172.21.15.64:0/2134538851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0039c90 con 0x7f29b80cf050 2023-11-24T08:07:26.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.058+0000 7f29b57fa640 1 -- 172.21.15.64:0/2134538851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a0039df0 con 0x7f29b80cf050 2023-11-24T08:07:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.058+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b80c9480 con 0x7f29b80cf050 2023-11-24T08:07:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.058+0000 7f29b57fa640 1 -- 172.21.15.64:0/2134538851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0043640 con 0x7f29b80cf050 2023-11-24T08:07:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.058+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b813d0e0 con 0x7f29b80cf050 2023-11-24T08:07:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.059+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b80c9690 con 0x7f29b80cf050 2023-11-24T08:07:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.060+0000 7f29b57fa640 1 -- 172.21.15.64:0/2134538851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f29a00437a0 con 0x7f29b80cf050 2023-11-24T08:07:26.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.062+0000 7f29b57fa640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f299806b230 0x7f299806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.062+0000 7f29b57fa640 1 -- 172.21.15.64:0/2134538851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f29a003e070 con 0x7f29b80cf050 2023-11-24T08:07:26.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.062+0000 7f29b7fff640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f299806b230 0x7f299806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.063+0000 7f29b7fff640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f299806b230 0x7f299806d6f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f29a8006fd0 tx=0x7f29a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:26.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.066+0000 7f29b57fa640 1 -- 172.21.15.64:0/2134538851 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29a00b1070 con 0x7f29b80cf050 2023-11-24T08:07:26.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.256+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29b80c9870 con 0x7f29b80cf050 2023-11-24T08:07:26.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.258+0000 7f29b57fa640 1 -- 172.21.15.64:0/2134538851 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f29a0077920 con 0x7f29b80cf050 2023-11-24T08:07:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:26.260 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":62,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:21.002292+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:06:55.038686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:06:56.142053+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:26.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.260+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f299806b230 msgr2=0x7f299806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.260+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f299806b230 0x7f299806d6f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f29a8006fd0 tx=0x7f29a8008040 comp rx=0 tx=0).stop 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.260+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 msgr2=0x7f29b815a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.260+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b815a920 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f29a0000c00 tx=0x7f29a0004730 comp rx=0 tx=0).stop 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.260+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 shutdown_connections 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.260+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f299806b230 0x7f299806d6f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.260+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f29b813dfd0 0x7f29b8155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.261+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29b80cf050 0x7f29b815a920 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.261+0000 7f29bd59e640 1 --2- 172.21.15.64:0/2134538851 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f29b806c130 0x7f29b80c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.261+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 >> 172.21.15.64:0/2134538851 conn(0x7f29b806a860 msgr2=0x7f29b813fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.261+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 shutdown_connections 2023-11-24T08:07:26.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.261+0000 7f29bd59e640 1 -- 172.21.15.64:0/2134538851 wait complete. 2023-11-24T08:07:26.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:07:26.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.501+0000 7f855a489640 1 Processor -- start 2023-11-24T08:07:26.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.501+0000 7f855a489640 1 -- start start 2023-11-24T08:07:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.502+0000 7f855a489640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540b39a0 0x7f8554072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.502+0000 7f855a489640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85540abff0 0x7f855406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.502+0000 7f855a489640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85540ac760 con 0x7f855415a090 2023-11-24T08:07:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.502+0000 7f855a489640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855415a7f0 con 0x7f85540abff0 2023-11-24T08:07:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f855a489640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855415aad0 con 0x7f85540b39a0 2023-11-24T08:07:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f8558a26640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85540abff0 0x7f855406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f8558a26640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85540abff0 0x7f855406d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52210/0 (socket says 172.21.15.64:52210) 2023-11-24T08:07:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f8558a26640 1 -- 172.21.15.64:0/2986462245 learned_addr learned my addr 172.21.15.64:0/2986462245 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:26.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f854bfff640 1 --2- 172.21.15.64:0/2986462245 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540b39a0 0x7f8554072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f8558a26640 1 -- 172.21.15.64:0/2986462245 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540b39a0 msgr2=0x7f8554072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:26.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f8558a26640 1 --2- 172.21.15.64:0/2986462245 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540b39a0 0x7f8554072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f8558a26640 1 -- 172.21.15.64:0/2986462245 >> v1:172.21.15.91:6789/0 conn(0x7f855415a090 legacy=0x7f85540b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:26.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.503+0000 7f8558a26640 1 -- 172.21.15.64:0/2986462245 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855415adb0 con 0x7f85540abff0 2023-11-24T08:07:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.504+0000 7f8558a26640 1 --2- 172.21.15.64:0/2986462245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85540abff0 0x7f855406d520 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f854c009e50 tx=0x7f854c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331ef9f420dcd444 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f85537fe640 1 -- 172.21.15.64:0/2986462245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c02ca70 con 0x7f85540abff0 2023-11-24T08:07:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f85537fe640 1 -- 172.21.15.64:0/2986462245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c02cbd0 con 0x7f85540abff0 2023-11-24T08:07:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f85537fe640 1 -- 172.21.15.64:0/2986462245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c0343f0 con 0x7f85540abff0 2023-11-24T08:07:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f855a489640 1 -- 172.21.15.64:0/2986462245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85540abff0 msgr2=0x7f855406d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f855a489640 1 --2- 172.21.15.64:0/2986462245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85540abff0 0x7f855406d520 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f854c009e50 tx=0x7f854c02bed0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f855a489640 1 -- 172.21.15.64:0/2986462245 shutdown_connections 2023-11-24T08:07:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f855a489640 1 --2- 172.21.15.64:0/2986462245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85540abff0 0x7f855406d520 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f855a489640 1 --2- 172.21.15.64:0/2986462245 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540b39a0 0x7f8554072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f855a489640 1 -- 172.21.15.64:0/2986462245 >> 172.21.15.64:0/2986462245 conn(0x7f85540b0d80 msgr2=0x7f85540b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f855a489640 1 -- 172.21.15.64:0/2986462245 shutdown_connections 2023-11-24T08:07:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.505+0000 7f855a489640 1 -- 172.21.15.64:0/2986462245 wait complete. 2023-11-24T08:07:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.507+0000 7f855a489640 1 Processor -- start 2023-11-24T08:07:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.507+0000 7f855a489640 1 -- start start 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.508+0000 7f855a489640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540abff0 0x7f855414a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.508+0000 7f855a489640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85540b39a0 0x7f85540cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.508+0000 7f855a489640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855415a090 0x7f855415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.508+0000 7f855a489640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85540cd070 con 0x7f855415a090 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.508+0000 7f855a489640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85540cd350 con 0x7f85540b39a0 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.508+0000 7f855a489640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85540cd630 con 0x7f85540abff0 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855415a090 0x7f855415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855415a090 0x7f855415cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52218/0 (socket says 172.21.15.64:52218) 2023-11-24T08:07:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 -- 172.21.15.64:0/2889045442 learned_addr learned my addr 172.21.15.64:0/2889045442 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f854bfff640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85540b39a0 0x7f85540cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8553fff640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540abff0 0x7f855414a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 -- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540abff0 msgr2=0x7f855414a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540abff0 0x7f855414a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 -- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85540b39a0 msgr2=0x7f85540cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85540b39a0 0x7f85540cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 -- 172.21.15.64:0/2889045442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c009b30 con 0x7f855415a090 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8553fff640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540abff0 0x7f855414a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.509+0000 7f8558a26640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855415a090 0x7f855415cee0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f854c036040 tx=0x7f854c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.510+0000 7f85517fa640 1 -- 172.21.15.64:0/2889045442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c03e450 con 0x7f855415a090 2023-11-24T08:07:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.510+0000 7f85517fa640 1 -- 172.21.15.64:0/2889045442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c03e5d0 con 0x7f855415a090 2023-11-24T08:07:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.510+0000 7f85517fa640 1 -- 172.21.15.64:0/2889045442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c03e980 con 0x7f855415a090 2023-11-24T08:07:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.510+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85540cd910 con 0x7f855415a090 2023-11-24T08:07:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.510+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85540ccd60 con 0x7f855415a090 2023-11-24T08:07:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.511+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8554074770 con 0x7f855415a090 2023-11-24T08:07:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.512+0000 7f85517fa640 1 -- 172.21.15.64:0/2889045442 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f854c03eae0 con 0x7f855415a090 2023-11-24T08:07:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.513+0000 7f85517fa640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f853406b230 0x7f853406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.513+0000 7f85517fa640 1 -- 172.21.15.64:0/2889045442 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f854c0acbf0 con 0x7f855415a090 2023-11-24T08:07:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.513+0000 7f8553fff640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f853406b230 0x7f853406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.514+0000 7f8553fff640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f853406b230 0x7f853406d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8544003eb0 tx=0x7f8544009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:26.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.517+0000 7f85517fa640 1 -- 172.21.15.64:0/2889045442 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f854c0777a0 con 0x7f855415a090 2023-11-24T08:07:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.748+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f85541595d0 con 0x7f855415a090 2023-11-24T08:07:26.751 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-11-24T08:07:26.749+0000 7fb13dee4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:07:26.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.753+0000 7f85517fa640 1 -- 172.21.15.64:0/2889045442 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f854c07adc0 con 0x7f855415a090 2023-11-24T08:07:26.755 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:07:26.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.757+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f853406b230 msgr2=0x7f853406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.757+0000 7f855a489640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f853406b230 0x7f853406d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8544003eb0 tx=0x7f8544009040 comp rx=0 tx=0).stop 2023-11-24T08:07:26.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.757+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855415a090 msgr2=0x7f855415cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.757+0000 7f855a489640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855415a090 0x7f855415cee0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f854c036040 tx=0x7f854c004a40 comp rx=0 tx=0).stop 2023-11-24T08:07:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.758+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 shutdown_connections 2023-11-24T08:07:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.758+0000 7f855a489640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f853406b230 0x7f853406d6f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.758+0000 7f855a489640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855415a090 0x7f855415cee0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.758+0000 7f855a489640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f85540b39a0 0x7f85540cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.758+0000 7f855a489640 1 --2- 172.21.15.64:0/2889045442 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f85540abff0 0x7f855414a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.758+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 >> 172.21.15.64:0/2889045442 conn(0x7f85540b0d80 msgr2=0x7f855406c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.758+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 shutdown_connections 2023-11-24T08:07:26.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.758+0000 7f855a489640 1 -- 172.21.15.64:0/2889045442 wait complete. 2023-11-24T08:07:26.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:07:26.836 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:07:26.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.985+0000 7f025f252640 1 Processor -- start 2023-11-24T08:07:26.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.985+0000 7f025f252640 1 -- start start 2023-11-24T08:07:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.987+0000 7f025f252640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f0258071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.987+0000 7f025f252640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f02581439b0 0x7f0258070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.987+0000 7f025f252640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0258070d40 con 0x7f02581435d0 2023-11-24T08:07:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.987+0000 7f025f252640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0258071020 con 0x7f02581439b0 2023-11-24T08:07:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.987+0000 7f025f252640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0258071300 con 0x7f02580b4140 2023-11-24T08:07:26.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.988+0000 7f025ea51640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f02581439b0 0x7f0258070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.988+0000 7f025da4f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f0258071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.988+0000 7f025da4f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f0258071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:40178/0 (socket says 172.21.15.64:40178) 2023-11-24T08:07:26.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.988+0000 7f025da4f640 1 -- 172.21.15.64:0/2689006145 learned_addr learned my addr 172.21.15.64:0/2689006145 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:26.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.989+0000 7f025da4f640 1 -- 172.21.15.64:0/2689006145 >> v2:172.21.15.91:3300/0 conn(0x7f02581439b0 msgr2=0x7f0258070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:26.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.989+0000 7f025da4f640 1 --2- 172.21.15.64:0/2689006145 >> v2:172.21.15.91:3300/0 conn(0x7f02581439b0 0x7f0258070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.989+0000 7f025da4f640 1 -- 172.21.15.64:0/2689006145 >> v1:172.21.15.91:6789/0 conn(0x7f02581435d0 legacy=0x7f02580b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.989+0000 7f025da4f640 1 -- 172.21.15.64:0/2689006145 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02580715e0 con 0x7f02580b4140 2023-11-24T08:07:26.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.990+0000 7f025da4f640 1 --2- 172.21.15.64:0/2689006145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f0258071de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0248009870 tx=0x7f024802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce798b23e189c1f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:26.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025d24e640 1 -- 172.21.15.64:0/2689006145 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248030840 con 0x7f02580b4140 2023-11-24T08:07:26.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025d24e640 1 -- 172.21.15.64:0/2689006145 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02480309a0 con 0x7f02580b4140 2023-11-24T08:07:26.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025d24e640 1 -- 172.21.15.64:0/2689006145 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248030d30 con 0x7f02580b4140 2023-11-24T08:07:26.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025f252640 1 -- 172.21.15.64:0/2689006145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 msgr2=0x7f0258071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025f252640 1 --2- 172.21.15.64:0/2689006145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f0258071de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0248009870 tx=0x7f024802b980 comp rx=0 tx=0).stop 2023-11-24T08:07:26.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025f252640 1 -- 172.21.15.64:0/2689006145 shutdown_connections 2023-11-24T08:07:26.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025f252640 1 --2- 172.21.15.64:0/2689006145 >> v2:172.21.15.91:3300/0 conn(0x7f02581439b0 0x7f0258070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025f252640 1 --2- 172.21.15.64:0/2689006145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f0258071de0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.991+0000 7f025f252640 1 -- 172.21.15.64:0/2689006145 >> 172.21.15.64:0/2689006145 conn(0x7f02580cec10 msgr2=0x7f02580cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:26.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.992+0000 7f025f252640 1 -- 172.21.15.64:0/2689006145 shutdown_connections 2023-11-24T08:07:26.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.992+0000 7f025f252640 1 -- 172.21.15.64:0/2689006145 wait complete. 2023-11-24T08:07:26.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.994+0000 7f025f252640 1 Processor -- start 2023-11-24T08:07:26.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.994+0000 7f025f252640 1 -- start start 2023-11-24T08:07:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.995+0000 7f025f252640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f025815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.995+0000 7f025f252640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02581435d0 0x7f025813b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.995+0000 7f025f252640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02581439b0 0x7f0258139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.995+0000 7f025f252640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025813b930 con 0x7f02581435d0 2023-11-24T08:07:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.995+0000 7f025f252640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025813bc10 con 0x7f02581439b0 2023-11-24T08:07:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.995+0000 7f025f252640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025813bef0 con 0x7f02580b4140 2023-11-24T08:07:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.995+0000 7f025da4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02581435d0 0x7f025813b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025da4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02581435d0 0x7f025813b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52252/0 (socket says 172.21.15.64:52252) 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025da4f640 1 -- 172.21.15.64:0/1046821153 learned_addr learned my addr 172.21.15.64:0/1046821153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025ea51640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02581439b0 0x7f0258139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025e250640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f025815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025da4f640 1 -- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 msgr2=0x7f025815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025da4f640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f025815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025da4f640 1 -- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02581439b0 msgr2=0x7f0258139900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025da4f640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02581439b0 0x7f0258139900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025da4f640 1 -- 172.21.15.64:0/1046821153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480094d0 con 0x7f02581435d0 2023-11-24T08:07:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025ea51640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02581439b0 0x7f0258139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:07:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.996+0000 7f025da4f640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02581435d0 0x7f025813b2a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0248000c00 tx=0x7f0248004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:27.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.997+0000 7f024effd640 1 -- 172.21.15.64:0/1046821153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248039c90 con 0x7f02581435d0 2023-11-24T08:07:27.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.997+0000 7f024effd640 1 -- 172.21.15.64:0/1046821153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0248039df0 con 0x7f02581435d0 2023-11-24T08:07:27.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.997+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025813c1d0 con 0x7f02581435d0 2023-11-24T08:07:27.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.997+0000 7f024effd640 1 -- 172.21.15.64:0/1046821153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248043640 con 0x7f02581435d0 2023-11-24T08:07:27.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.997+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02580afe70 con 0x7f02581435d0 2023-11-24T08:07:27.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.998+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025813c6f0 con 0x7f02581435d0 2023-11-24T08:07:27.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:26.999+0000 7f024effd640 1 -- 172.21.15.64:0/1046821153 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f02480437a0 con 0x7f02581435d0 2023-11-24T08:07:27.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.000+0000 7f024effd640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f023c06b230 0x7f023c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:27.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.000+0000 7f024effd640 1 -- 172.21.15.64:0/1046821153 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f024803e070 con 0x7f02581435d0 2023-11-24T08:07:27.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.001+0000 7f025e250640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f023c06b230 0x7f023c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:27.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.002+0000 7f025e250640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f023c06b230 0x7f023c06d6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0254007fd0 tx=0x7f0254009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:27.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.006+0000 7f024effd640 1 -- 172.21.15.64:0/1046821153 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f025813c6f0 con 0x7f02581435d0 2023-11-24T08:07:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.228+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f025813c6f0 con 0x7f02581435d0 2023-11-24T08:07:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.757+0000 7f024effd640 1 -- 172.21.15.64:0/1046821153 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f025813c6f0 con 0x7f02581435d0 2023-11-24T08:07:27.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.760+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f023c06b230 msgr2=0x7f023c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.760+0000 7f025f252640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f023c06b230 0x7f023c06d6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0254007fd0 tx=0x7f0254009040 comp rx=0 tx=0).stop 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.760+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02581435d0 msgr2=0x7f025813b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.760+0000 7f025f252640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02581435d0 0x7f025813b2a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0248000c00 tx=0x7f0248004730 comp rx=0 tx=0).stop 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.760+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 shutdown_connections 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.760+0000 7f025f252640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f023c06b230 0x7f023c06d6f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.760+0000 7f025f252640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f02581439b0 0x7f0258139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.760+0000 7f025f252640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02581435d0 0x7f025813b2a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.761+0000 7f025f252640 1 --2- 172.21.15.64:0/1046821153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f02580b4140 0x7f025815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.761+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 >> 172.21.15.64:0/1046821153 conn(0x7f02580cec10 msgr2=0x7f0258140bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:27.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.761+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 shutdown_connections 2023-11-24T08:07:27.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:27.761+0000 7f025f252640 1 -- 172.21.15.64:0/1046821153 wait complete. 2023-11-24T08:07:27.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:07:28.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.002+0000 7f013fed3640 1 Processor -- start 2023-11-24T08:07:28.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.002+0000 7f013fed3640 1 -- start start 2023-11-24T08:07:28.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.004+0000 7f013fed3640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f01380ae490 0x7f0138070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:28.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.004+0000 7f013fed3640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013806d7a0 con 0x7f013806dc30 2023-11-24T08:07:28.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.004+0000 7f013fed3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01380ae900 con 0x7f01380acd70 2023-11-24T08:07:28.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.004+0000 7f013fed3640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f013815a5b0 con 0x7f01380ae490 2023-11-24T08:07:28.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013d46e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f013806dc30 0x7f0138070490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39502/0 (socket says 172.21.15.64:39502) 2023-11-24T08:07:28.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013d46e640 1 -- 172.21.15.64:0/1107280791 learned_addr learned my addr 172.21.15.64:0/1107280791 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:28.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013e470640 1 --2- 172.21.15.64:0/1107280791 >> v2:172.21.15.91:3300/0 conn(0x7f01380ae490 0x7f0138070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:28.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339765468 0 0) 0x7f013806d7a0 con 0x7f013806dc30 2023-11-24T08:07:28.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0124003520 con 0x7f013806dc30 2023-11-24T08:07:28.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 182413331 0 0) 0x7f0124003520 con 0x7f013806dc30 2023-11-24T08:07:28.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 >> v2:172.21.15.91:3300/0 conn(0x7f01380ae490 msgr2=0x7f0138070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:28.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013cc6d640 1 --2- 172.21.15.64:0/1107280791 >> v2:172.21.15.91:3300/0 conn(0x7f01380ae490 0x7f0138070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:28.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 >> v1:172.21.15.91:6789/0 conn(0x7f01380acd70 legacy=0x7f01380ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:28.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013806ce70 con 0x7f013806dc30 2023-11-24T08:07:28.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.005+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0128001670 con 0x7f013806dc30 2023-11-24T08:07:28.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 >> v1:172.21.15.64:6789/0 conn(0x7f013806dc30 legacy=0x7f0138070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:28.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013cc6d640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0124003970 0x7f0124003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:28.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013cc6d640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01240047c0 0x7f0124004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:28.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013cc6d640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124005180 0x7f012400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:28.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013815a5b0 con 0x7f0124003970 2023-11-24T08:07:28.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012400c020 con 0x7f01240047c0 2023-11-24T08:07:28.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012400c1e0 con 0x7f0124005180 2023-11-24T08:07:28.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013dc6f640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0124003970 0x7f0124003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:28.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013e470640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124005180 0x7f012400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:28.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013dc6f640 1 -- 172.21.15.64:0/1107280791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124005180 msgr2=0x7f012400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:28.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013dc6f640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124005180 0x7f012400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:28.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013dc6f640 1 -- 172.21.15.64:0/1107280791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01240047c0 msgr2=0x7f0124004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013dc6f640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01240047c0 0x7f0124004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013dc6f640 1 -- 172.21.15.64:0/1107280791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0124003520 con 0x7f0124003970 2023-11-24T08:07:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.006+0000 7f013e470640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124005180 0x7f012400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.007+0000 7f013e470640 1 -- 172.21.15.64:0/1107280791 reap_dead start 2023-11-24T08:07:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.007+0000 7f013dc6f640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0124003970 0x7f0124003d90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0134001750 tx=0x7f01240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbaccc304cabe54 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01340052d0 con 0x7f0124003970 2023-11-24T08:07:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013402e050 con 0x7f0124003970 2023-11-24T08:07:28.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013cc6d640 1 -- 172.21.15.64:0/1107280791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134003830 con 0x7f0124003970 2023-11-24T08:07:28.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013fed3640 1 -- 172.21.15.64:0/1107280791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0124003970 msgr2=0x7f0124003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:28.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013fed3640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0124003970 0x7f0124003d90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0134001750 tx=0x7f01240057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:28.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013fed3640 1 -- 172.21.15.64:0/1107280791 shutdown_connections 2023-11-24T08:07:28.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013fed3640 1 --2- 172.21.15.64:0/1107280791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0124003970 0x7f0124003d90 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:28.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013fed3640 1 -- 172.21.15.64:0/1107280791 >> 172.21.15.64:0/1107280791 conn(0x7f01380b0ba0 msgr2=0x7f01380b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:28.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.008+0000 7f013fed3640 1 -- 172.21.15.64:0/1107280791 shutdown_connections 2023-11-24T08:07:28.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.009+0000 7f013fed3640 1 -- 172.21.15.64:0/1107280791 wait complete. 2023-11-24T08:07:28.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.010+0000 7f013fed3640 1 Processor -- start 2023-11-24T08:07:28.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.011+0000 7f013fed3640 1 -- start start 2023-11-24T08:07:28.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.011+0000 7f013fed3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124003970 0x7f013815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:28.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.011+0000 7f013fed3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01380cb500 0x7f01380ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.011+0000 7f013fed3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013815b740 0x7f013815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013fed3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01380cc6b0 con 0x7f013815b740 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013fed3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013815d510 con 0x7f01380cb500 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013fed3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01380cbaa0 con 0x7f0124003970 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013e470640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013815b740 0x7f013815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013e470640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013815b740 0x7f013815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57904/0 (socket says 172.21.15.64:57904) 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013e470640 1 -- 172.21.15.64:0/3242370501 learned_addr learned my addr 172.21.15.64:0/3242370501 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013dc6f640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124003970 0x7f013815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:28.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013d46e640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01380cb500 0x7f01380ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:28.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013e470640 1 -- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124003970 msgr2=0x7f013815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:28.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013e470640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124003970 0x7f013815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:28.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013e470640 1 -- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01380cb500 msgr2=0x7f01380ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:28.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013e470640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01380cb500 0x7f01380ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:28.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.012+0000 7f013e470640 1 -- 172.21.15.64:0/3242370501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380cbd80 con 0x7f013815b740 2023-11-24T08:07:28.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.013+0000 7f013e470640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013815b740 0x7f013815bbc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f013806d710 tx=0x7f013000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:28.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.013+0000 7f012e7fc640 1 -- 172.21.15.64:0/3242370501 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130016620 con 0x7f013815b740 2023-11-24T08:07:28.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.013+0000 7f012e7fc640 1 -- 172.21.15.64:0/3242370501 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0130016780 con 0x7f013815b740 2023-11-24T08:07:28.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.013+0000 7f012e7fc640 1 -- 172.21.15.64:0/3242370501 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130016af0 con 0x7f013815b740 2023-11-24T08:07:28.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.013+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01380cc060 con 0x7f013815b740 2023-11-24T08:07:28.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.013+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01380caa20 con 0x7f013815b740 2023-11-24T08:07:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.014+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01380cc820 con 0x7f013815b740 2023-11-24T08:07:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.015+0000 7f012e7fc640 1 -- 172.21.15.64:0/3242370501 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f0130016c50 con 0x7f013815b740 2023-11-24T08:07:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.016+0000 7f012e7fc640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f011c06b260 0x7f011c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.016+0000 7f012e7fc640 1 -- 172.21.15.64:0/3242370501 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f013008abf0 con 0x7f013815b740 2023-11-24T08:07:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.017+0000 7f013dc6f640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f011c06b260 0x7f011c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:28.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.018+0000 7f013dc6f640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f011c06b260 0x7f011c06d720 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0134033000 tx=0x7f0134009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:28.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.021+0000 7f012e7fc640 1 -- 172.21.15.64:0/3242370501 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0130055510 con 0x7f013815b740 2023-11-24T08:07:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:28.207+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f013809d200 con 0x7f013815b740 2023-11-24T08:07:29.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.214+0000 7f012e7fc640 1 -- 172.21.15.64:0/3242370501 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0130058b30 con 0x7f013815b740 2023-11-24T08:07:29.217 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:07:29.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f011c06b260 msgr2=0x7f011c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:29.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f011c06b260 0x7f011c06d720 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0134033000 tx=0x7f0134009030 comp rx=0 tx=0).stop 2023-11-24T08:07:29.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013815b740 msgr2=0x7f013815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:29.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013815b740 0x7f013815bbc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f013806d710 tx=0x7f013000f520 comp rx=0 tx=0).stop 2023-11-24T08:07:29.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 shutdown_connections 2023-11-24T08:07:29.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f011c06b260 0x7f011c06d720 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013815b740 0x7f013815bbc0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01380cb500 0x7f01380ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 --2- 172.21.15.64:0/3242370501 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0124003970 0x7f013815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 >> 172.21.15.64:0/3242370501 conn(0x7f01380b0ba0 msgr2=0x7f01380aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:29.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 shutdown_connections 2023-11-24T08:07:29.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.219+0000 7f013fed3640 1 -- 172.21.15.64:0/3242370501 wait complete. 2023-11-24T08:07:29.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:07:29.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.467+0000 7f6860abb640 1 Processor -- start 2023-11-24T08:07:29.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.467+0000 7f6860abb640 1 -- start start 2023-11-24T08:07:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.469+0000 7f6860abb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f685c15f270 0x7f685c0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.469+0000 7f6860abb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c06d9e0 0x7f685c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.469+0000 7f6860abb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c15f930 con 0x7f685c144770 2023-11-24T08:07:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.469+0000 7f6860abb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0b43b0 con 0x7f685c15f270 2023-11-24T08:07:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.469+0000 7f6860abb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0b3b10 con 0x7f685c06d9e0 2023-11-24T08:07:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.470+0000 7f6853fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f685c15f270 0x7f685c0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.470+0000 7f6853fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f685c15f270 0x7f685c0cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:58872/0 (socket says 172.21.15.64:58872) 2023-11-24T08:07:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.470+0000 7f685bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c06d9e0 0x7f685c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:29.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.470+0000 7f6853fff640 1 -- 172.21.15.64:0/3097540375 learned_addr learned my addr 172.21.15.64:0/3097540375 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:29.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.470+0000 7f6853fff640 1 -- 172.21.15.64:0/3097540375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c06d9e0 msgr2=0x7f685c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:29.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.470+0000 7f6853fff640 1 --2- 172.21.15.64:0/3097540375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c06d9e0 0x7f685c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.470+0000 7f6853fff640 1 -- 172.21.15.64:0/3097540375 >> v1:172.21.15.91:6789/0 conn(0x7f685c144770 legacy=0x7f685c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:29.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.470+0000 7f6853fff640 1 -- 172.21.15.64:0/3097540375 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0b3df0 con 0x7f685c15f270 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.472+0000 7f6853fff640 1 --2- 172.21.15.64:0/3097540375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c15f270 0x7f685c0cd4a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6844009870 tx=0x7f684402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf367ed628b53f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.472+0000 7f685affd640 1 -- 172.21.15.64:0/3097540375 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844030840 con 0x7f685c15f270 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.472+0000 7f685affd640 1 -- 172.21.15.64:0/3097540375 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68440309a0 con 0x7f685c15f270 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.472+0000 7f685affd640 1 -- 172.21.15.64:0/3097540375 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844030d30 con 0x7f685c15f270 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.473+0000 7f6860abb640 1 -- 172.21.15.64:0/3097540375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c15f270 msgr2=0x7f685c0cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.473+0000 7f6860abb640 1 --2- 172.21.15.64:0/3097540375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c15f270 0x7f685c0cd4a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6844009870 tx=0x7f684402b980 comp rx=0 tx=0).stop 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.473+0000 7f6860abb640 1 -- 172.21.15.64:0/3097540375 shutdown_connections 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.473+0000 7f6860abb640 1 --2- 172.21.15.64:0/3097540375 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c06d9e0 0x7f685c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.473+0000 7f6860abb640 1 --2- 172.21.15.64:0/3097540375 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c15f270 0x7f685c0cd4a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.473+0000 7f6860abb640 1 -- 172.21.15.64:0/3097540375 >> 172.21.15.64:0/3097540375 conn(0x7f685c0cec10 msgr2=0x7f685c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:29.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.473+0000 7f6860abb640 1 -- 172.21.15.64:0/3097540375 shutdown_connections 2023-11-24T08:07:29.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.473+0000 7f6860abb640 1 -- 172.21.15.64:0/3097540375 wait complete. 2023-11-24T08:07:29.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.475+0000 7f6860abb640 1 Processor -- start 2023-11-24T08:07:29.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.475+0000 7f6860abb640 1 -- start start 2023-11-24T08:07:29.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.476+0000 7f6860abb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c06d9e0 0x7f685c13a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:29.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.476+0000 7f6860abb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685c144770 0x7f685c1f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:29.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.476+0000 7f6860abb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c15f270 0x7f685c13d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:29.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.476+0000 7f6860abb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c15c330 con 0x7f685c144770 2023-11-24T08:07:29.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.476+0000 7f6860abb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c15c750 con 0x7f685c06d9e0 2023-11-24T08:07:29.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.476+0000 7f6860abb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c13acc0 con 0x7f685c15f270 2023-11-24T08:07:29.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.476+0000 7f6853fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685c144770 0x7f685c1f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:29.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f6853fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685c144770 0x7f685c1f47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57918/0 (socket says 172.21.15.64:57918) 2023-11-24T08:07:29.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f6853fff640 1 -- 172.21.15.64:0/89287559 learned_addr learned my addr 172.21.15.64:0/89287559 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:29.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f685bfff640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c15f270 0x7f685c13d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:29.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f685b7fe640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c06d9e0 0x7f685c13a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:29.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f6853fff640 1 -- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c15f270 msgr2=0x7f685c13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:29.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f6853fff640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c15f270 0x7f685c13d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f6853fff640 1 -- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c06d9e0 msgr2=0x7f685c13a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f6853fff640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c06d9e0 0x7f685c13a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f6853fff640 1 -- 172.21.15.64:0/89287559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440094d0 con 0x7f685c144770 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.477+0000 7f685bfff640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c15f270 0x7f685c13d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.478+0000 7f6853fff640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685c144770 0x7f685c1f47b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f68440090b0 tx=0x7f68440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.478+0000 7f6858ff9640 1 -- 172.21.15.64:0/89287559 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684403e070 con 0x7f685c144770 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.478+0000 7f6858ff9640 1 -- 172.21.15.64:0/89287559 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844039a10 con 0x7f685c144770 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.478+0000 7f6858ff9640 1 -- 172.21.15.64:0/89287559 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844039de0 con 0x7f685c144770 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.478+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c15b6b0 con 0x7f685c144770 2023-11-24T08:07:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.478+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c15bc70 con 0x7f685c144770 2023-11-24T08:07:29.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.479+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c15c470 con 0x7f685c144770 2023-11-24T08:07:29.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.480+0000 7f6858ff9640 1 -- 172.21.15.64:0/89287559 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f684404b400 con 0x7f685c144770 2023-11-24T08:07:29.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.481+0000 7f6858ff9640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683c06b280 0x7f683c06d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:29.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.481+0000 7f6858ff9640 1 -- 172.21.15.64:0/89287559 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f68440ac170 con 0x7f685c144770 2023-11-24T08:07:29.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.482+0000 7f685b7fe640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683c06b280 0x7f683c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:29.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.483+0000 7f685b7fe640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683c06b280 0x7f683c06d740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f684c000f60 tx=0x7f684c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:29.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.485+0000 7f6858ff9640 1 -- 172.21.15.64:0/89287559 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f685c15c470 con 0x7f685c144770 2023-11-24T08:07:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:29.686+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f685c15c470 con 0x7f685c144770 2023-11-24T08:07:30.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.228+0000 7f6858ff9640 1 -- 172.21.15.64:0/89287559 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) v1 ==== 169+0+0 (secure 0 0 0) 0x7f685c15c470 con 0x7f685c144770 2023-11-24T08:07:30.230 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-11-24T08:07:30.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683c06b280 msgr2=0x7f683c06d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683c06b280 0x7f683c06d740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f684c000f60 tx=0x7f684c009040 comp rx=0 tx=0).stop 2023-11-24T08:07:30.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685c144770 msgr2=0x7f685c1f47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685c144770 0x7f685c1f47b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f68440090b0 tx=0x7f68440046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 shutdown_connections 2023-11-24T08:07:30.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f683c06b280 0x7f683c06d740 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f685c15f270 0x7f685c13d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f685c144770 0x7f685c1f47b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 --2- 172.21.15.64:0/89287559 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f685c06d9e0 0x7f685c13a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.232+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 >> 172.21.15.64:0/89287559 conn(0x7f685c0cec10 msgr2=0x7f685c1434b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:30.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.233+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 shutdown_connections 2023-11-24T08:07:30.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.233+0000 7f6860abb640 1 -- 172.21.15.64:0/89287559 wait complete. 2023-11-24T08:07:30.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:07:30.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.472+0000 7fdcd23ce640 1 Processor -- start 2023-11-24T08:07:30.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.472+0000 7fdcd23ce640 1 -- start start 2023-11-24T08:07:30.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.474+0000 7fdcd23ce640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdccc06cad0 0x7fdccc162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.474+0000 7fdcd23ce640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc162fa0 con 0x7fdccc162220 2023-11-24T08:07:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.474+0000 7fdcd23ce640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0706a0 con 0x7fdccc162600 2023-11-24T08:07:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.474+0000 7fdcd23ce640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdccc070c60 con 0x7fdccc06cad0 2023-11-24T08:07:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.474+0000 7fdccbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdccc162220 0x7fdccc0b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39510/0 (socket says 172.21.15.64:39510) 2023-11-24T08:07:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.474+0000 7fdccbfff640 1 -- 172.21.15.64:0/263422733 learned_addr learned my addr 172.21.15.64:0/263422733 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.474+0000 7fdcd096b640 1 --2- 172.21.15.64:0/263422733 >> v2:172.21.15.91:3300/0 conn(0x7fdccc06cad0 0x7fdccc162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.474+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730826902 0 0) 0x7fdccc162fa0 con 0x7fdccc162220 2023-11-24T08:07:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb8003520 con 0x7fdccc162220 2023-11-24T08:07:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2581295187 0 0) 0x7fdcb8003520 con 0x7fdccc162220 2023-11-24T08:07:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 >> v2:172.21.15.91:3300/0 conn(0x7fdccc06cad0 msgr2=0x7fdccc162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 --2- 172.21.15.64:0/263422733 >> v2:172.21.15.91:3300/0 conn(0x7fdccc06cad0 0x7fdccc162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 >> v1:172.21.15.91:6789/0 conn(0x7fdccc162600 legacy=0x7fdccc06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc0722b0 con 0x7fdccc162220 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fdcbc0019f0 con 0x7fdccc162220 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 >> v1:172.21.15.64:6789/0 conn(0x7fdccc162220 legacy=0x7fdccc0b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb80039c0 0x7fdcb8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcb80047c0 0x7fdcb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdcb8005180 0x7fdcb800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc070c60 con 0x7fdcb80039c0 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb800c080 con 0x7fdcb80047c0 2023-11-24T08:07:30.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb800c240 con 0x7fdcb8005180 2023-11-24T08:07:30.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.475+0000 7fdccbfff640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb80039c0 0x7fdcb8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.476+0000 7fdccb7fe640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcb80047c0 0x7fdcb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.476+0000 7fdccbfff640 1 -- 172.21.15.64:0/263422733 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdcb8005180 msgr2=0x7fdcb800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:30.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.476+0000 7fdccbfff640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdcb8005180 0x7fdcb800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.476+0000 7fdccbfff640 1 -- 172.21.15.64:0/263422733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcb80047c0 msgr2=0x7fdcb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:30.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.476+0000 7fdccbfff640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcb80047c0 0x7fdcb8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.476+0000 7fdccbfff640 1 -- 172.21.15.64:0/263422733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8003520 con 0x7fdcb80039c0 2023-11-24T08:07:30.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.476+0000 7fdccb7fe640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdcb80047c0 0x7fdcb8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:30.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.476+0000 7fdcd096b640 1 -- 172.21.15.64:0/263422733 reap_dead start 2023-11-24T08:07:30.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.477+0000 7fdccbfff640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb80039c0 0x7fdcb8003de0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00a760 tx=0x7fdcb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c829f87a564b9f12 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:30.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.477+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc034630 con 0x7fdcb80039c0 2023-11-24T08:07:30.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.477+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc02f070 con 0x7fdcb80039c0 2023-11-24T08:07:30.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.477+0000 7fdccaffd640 1 -- 172.21.15.64:0/263422733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc034a50 con 0x7fdcb80039c0 2023-11-24T08:07:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.478+0000 7fdcd23ce640 1 -- 172.21.15.64:0/263422733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb80039c0 msgr2=0x7fdcb8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.478+0000 7fdcd23ce640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb80039c0 0x7fdcb8003de0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00a760 tx=0x7fdcb80057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.478+0000 7fdcd23ce640 1 -- 172.21.15.64:0/263422733 shutdown_connections 2023-11-24T08:07:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.478+0000 7fdcd23ce640 1 --2- 172.21.15.64:0/263422733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb80039c0 0x7fdcb8003de0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.478+0000 7fdcd23ce640 1 -- 172.21.15.64:0/263422733 >> 172.21.15.64:0/263422733 conn(0x7fdccc0b0ba0 msgr2=0x7fdccc0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.478+0000 7fdcd23ce640 1 -- 172.21.15.64:0/263422733 shutdown_connections 2023-11-24T08:07:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.478+0000 7fdcd23ce640 1 -- 172.21.15.64:0/263422733 wait complete. 2023-11-24T08:07:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.480+0000 7fdcd23ce640 1 Processor -- start 2023-11-24T08:07:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.480+0000 7fdcd23ce640 1 -- start start 2023-11-24T08:07:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdcd23ce640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdcb80039c0 0x7fdccc099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdcd23ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdccc152b40 0x7fdccc160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdcd23ce640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdccc152300 0x7fdccc09b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdcd23ce640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc1610d0 con 0x7fdccc152b40 2023-11-24T08:07:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdcd23ce640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc164140 con 0x7fdccc152300 2023-11-24T08:07:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdcd23ce640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc1612e0 con 0x7fdcb80039c0 2023-11-24T08:07:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdccc152b40 0x7fdccc160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdccc152b40 0x7fdccc160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57930/0 (socket says 172.21.15.64:57930) 2023-11-24T08:07:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.481+0000 7fdccb7fe640 1 -- 172.21.15.64:0/504706335 learned_addr learned my addr 172.21.15.64:0/504706335 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:30.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdccbfff640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdcb80039c0 0x7fdccc099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdcd096b640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdccc152300 0x7fdccc09b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdccb7fe640 1 -- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdcb80039c0 msgr2=0x7fdccc099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdccb7fe640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdcb80039c0 0x7fdccc099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdccb7fe640 1 -- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdccc152300 msgr2=0x7fdccc09b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdccb7fe640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdccc152300 0x7fdccc09b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdccb7fe640 1 -- 172.21.15.64:0/504706335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc1615c0 con 0x7fdccc152b40 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdccb7fe640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdccc152b40 0x7fdccc160b90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400d480 tx=0x7fdcb400d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdcc8ff9640 1 -- 172.21.15.64:0/504706335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4005480 con 0x7fdccc152b40 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdcc8ff9640 1 -- 172.21.15.64:0/504706335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4003610 con 0x7fdccc152b40 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdcc8ff9640 1 -- 172.21.15.64:0/504706335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4003a00 con 0x7fdccc152b40 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.482+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc1643e0 con 0x7fdccc152b40 2023-11-24T08:07:30.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.483+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc1527b0 con 0x7fdccc152b40 2023-11-24T08:07:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.484+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc152970 con 0x7fdccc152b40 2023-11-24T08:07:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.485+0000 7fdcc8ff9640 1 -- 172.21.15.64:0/504706335 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fdcb4003b60 con 0x7fdccc152b40 2023-11-24T08:07:30.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.486+0000 7fdcc8ff9640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcac06b230 0x7fdcac06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.486+0000 7fdcc8ff9640 1 -- 172.21.15.64:0/504706335 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdcb4089a10 con 0x7fdccc152b40 2023-11-24T08:07:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.486+0000 7fdccbfff640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcac06b230 0x7fdcac06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.487+0000 7fdccbfff640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcac06b230 0x7fdcac06d6f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc034000 tx=0x7fdcbc02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:30.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.490+0000 7fdcc8ff9640 1 -- 172.21.15.64:0/504706335 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdcb40547e0 con 0x7fdccc152b40 2023-11-24T08:07:30.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.695+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdccc1617b0 con 0x7fdccc152b40 2023-11-24T08:07:30.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.701+0000 7fdcc8ff9640 1 -- 172.21.15.64:0/504706335 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdcb4057e00 con 0x7fdccc152b40 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.704+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcac06b230 msgr2=0x7fdcac06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.704+0000 7fdcd23ce640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcac06b230 0x7fdcac06d6f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc034000 tx=0x7fdcbc02f000 comp rx=0 tx=0).stop 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.704+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdccc152b40 msgr2=0x7fdccc160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.704+0000 7fdcd23ce640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdccc152b40 0x7fdccc160b90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400d480 tx=0x7fdcb400d940 comp rx=0 tx=0).stop 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.705+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 shutdown_connections 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.705+0000 7fdcd23ce640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcac06b230 0x7fdcac06d6f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.705+0000 7fdcd23ce640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdccc152300 0x7fdccc09b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.705+0000 7fdcd23ce640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdccc152b40 0x7fdccc160b90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.705+0000 7fdcd23ce640 1 --2- 172.21.15.64:0/504706335 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdcb80039c0 0x7fdccc099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.705+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 >> 172.21.15.64:0/504706335 conn(0x7fdccc0b0ba0 msgr2=0x7fdccc0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:30.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.705+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 shutdown_connections 2023-11-24T08:07:30.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.705+0000 7fdcd23ce640 1 -- 172.21.15.64:0/504706335 wait complete. 2023-11-24T08:07:30.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T08:07:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.946+0000 7f759c161640 1 Processor -- start 2023-11-24T08:07:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.946+0000 7f759c161640 1 -- start start 2023-11-24T08:07:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.947+0000 7f759c161640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f759406d520 0x7f759406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.947+0000 7f759c161640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f75940ad720 0x7f759415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.947+0000 7f759c161640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759415c360 con 0x7f75940abff0 2023-11-24T08:07:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.947+0000 7f759c161640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75940b41a0 con 0x7f75940ad720 2023-11-24T08:07:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.948+0000 7f759c161640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759406de40 con 0x7f759406d520 2023-11-24T08:07:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.948+0000 7f75996fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f75940ad720 0x7f759415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.948+0000 7f759a6fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f75940abff0 0x7f759415f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39524/0 (socket says 172.21.15.64:39524) 2023-11-24T08:07:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.948+0000 7f759a6fe640 1 -- 172.21.15.64:0/1514593717 learned_addr learned my addr 172.21.15.64:0/1514593717 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.948+0000 7f7599efd640 1 --2- 172.21.15.64:0/1514593717 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f759406d520 0x7f759406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.949+0000 7f75996fc640 1 -- 172.21.15.64:0/1514593717 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f759406d520 msgr2=0x7f759406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.949+0000 7f75996fc640 1 --2- 172.21.15.64:0/1514593717 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f759406d520 0x7f759406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.949+0000 7f75996fc640 1 -- 172.21.15.64:0/1514593717 >> v1:172.21.15.64:6789/0 conn(0x7f75940abff0 legacy=0x7f759415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.949+0000 7f75996fc640 1 -- 172.21.15.64:0/1514593717 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759415ba30 con 0x7f75940ad720 2023-11-24T08:07:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.950+0000 7f75996fc640 1 --2- 172.21.15.64:0/1514593717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75940ad720 0x7f759415ecd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f75840097a0 tx=0x7f758402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138262995c256289 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.950+0000 7f7598efb640 1 -- 172.21.15.64:0/1514593717 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758402d040 con 0x7f75940ad720 2023-11-24T08:07:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.950+0000 7f7598efb640 1 -- 172.21.15.64:0/1514593717 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7584031810 con 0x7f75940ad720 2023-11-24T08:07:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.950+0000 7f7598efb640 1 -- 172.21.15.64:0/1514593717 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584031ba0 con 0x7f75940ad720 2023-11-24T08:07:30.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.951+0000 7f759c161640 1 -- 172.21.15.64:0/1514593717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75940ad720 msgr2=0x7f759415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.951+0000 7f759c161640 1 --2- 172.21.15.64:0/1514593717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75940ad720 0x7f759415ecd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f75840097a0 tx=0x7f758402b8b0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.951+0000 7f759c161640 1 -- 172.21.15.64:0/1514593717 shutdown_connections 2023-11-24T08:07:30.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.951+0000 7f759c161640 1 --2- 172.21.15.64:0/1514593717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75940ad720 0x7f759415ecd0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.951+0000 7f759c161640 1 --2- 172.21.15.64:0/1514593717 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f759406d520 0x7f759406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.951+0000 7f759c161640 1 -- 172.21.15.64:0/1514593717 >> 172.21.15.64:0/1514593717 conn(0x7f75940b0d80 msgr2=0x7f75940b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:30.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.951+0000 7f759c161640 1 -- 172.21.15.64:0/1514593717 shutdown_connections 2023-11-24T08:07:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.951+0000 7f759c161640 1 -- 172.21.15.64:0/1514593717 wait complete. 2023-11-24T08:07:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.953+0000 7f759c161640 1 Processor -- start 2023-11-24T08:07:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.953+0000 7f759c161640 1 -- start start 2023-11-24T08:07:30.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f759c161640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f759406d520 0x7f7594071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f759c161640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75940abff0 0x7f75940b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f759c161640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75940ad720 0x7f759414bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f759c161640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7594071ac0 con 0x7f759406d520 2023-11-24T08:07:30.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f759c161640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7594070bf0 con 0x7f75940abff0 2023-11-24T08:07:30.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f759c161640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7594070e40 con 0x7f75940ad720 2023-11-24T08:07:30.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f7599efd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f759406d520 0x7f7594071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f7599efd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f759406d520 0x7f7594071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57936/0 (socket says 172.21.15.64:57936) 2023-11-24T08:07:30.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f7599efd640 1 -- 172.21.15.64:0/3710658637 learned_addr learned my addr 172.21.15.64:0/3710658637 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:30.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f759a6fe640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75940ad720 0x7f759414bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f7599efd640 1 -- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75940ad720 msgr2=0x7f759414bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f7599efd640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75940ad720 0x7f759414bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f75996fc640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75940abff0 0x7f75940b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.954+0000 7f7599efd640 1 -- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75940abff0 msgr2=0x7f75940b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:30.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.955+0000 7f7599efd640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75940abff0 0x7f75940b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:30.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.955+0000 7f7599efd640 1 -- 172.21.15.64:0/3710658637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009400 con 0x7f759406d520 2023-11-24T08:07:30.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.955+0000 7f7599efd640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f759406d520 0x7f7594071ee0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f759000d800 tx=0x7f759000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:30.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.955+0000 7f758a7fc640 1 -- 172.21.15.64:0/3710658637 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759000b6c0 con 0x7f759406d520 2023-11-24T08:07:30.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.955+0000 7f758a7fc640 1 -- 172.21.15.64:0/3710658637 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759000b820 con 0x7f759406d520 2023-11-24T08:07:30.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.955+0000 7f758a7fc640 1 -- 172.21.15.64:0/3710658637 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759000bbb0 con 0x7f759406d520 2023-11-24T08:07:30.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.955+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7594071120 con 0x7f759406d520 2023-11-24T08:07:30.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.955+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75940716e0 con 0x7f759406d520 2023-11-24T08:07:30.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.957+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7594071d30 con 0x7f759406d520 2023-11-24T08:07:30.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.957+0000 7f758a7fc640 1 -- 172.21.15.64:0/3710658637 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f759000bd10 con 0x7f759406d520 2023-11-24T08:07:30.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.958+0000 7f758a7fc640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757806b230 0x7f757806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:30.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.958+0000 7f758a7fc640 1 -- 172.21.15.64:0/3710658637 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7590089f70 con 0x7f759406d520 2023-11-24T08:07:30.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.959+0000 7f75996fc640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757806b230 0x7f757806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:30.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.960+0000 7f75996fc640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757806b230 0x7f757806d6f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7584036000 tx=0x7f758403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:30.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:30.963+0000 7f758a7fc640 1 -- 172.21.15.64:0/3710658637 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7590054dd0 con 0x7f759406d520 2023-11-24T08:07:31.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:31.238+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7f7594074770 con 0x7f759406d520 2023-11-24T08:07:32.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.243+0000 7f758a7fc640 1 -- 172.21.15.64:0/3710658637 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v216) v1 ==== 166+0+0 (secure 0 0 0) 0x7f75900583f0 con 0x7f759406d520 2023-11-24T08:07:32.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.246+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757806b230 msgr2=0x7f757806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.246+0000 7f759c161640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757806b230 0x7f757806d6f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7584036000 tx=0x7f758403c040 comp rx=0 tx=0).stop 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.246+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f759406d520 msgr2=0x7f7594071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.246+0000 7f759c161640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f759406d520 0x7f7594071ee0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f759000d800 tx=0x7f759000dcc0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.247+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 shutdown_connections 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.247+0000 7f759c161640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f757806b230 0x7f757806d6f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.247+0000 7f759c161640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f75940ad720 0x7f759414bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.247+0000 7f759c161640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f75940abff0 0x7f75940b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.247+0000 7f759c161640 1 --2- 172.21.15.64:0/3710658637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f759406d520 0x7f7594071ee0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.247+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 >> 172.21.15.64:0/3710658637 conn(0x7f75940b0d80 msgr2=0x7f759415d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.247+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 shutdown_connections 2023-11-24T08:07:32.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.247+0000 7f759c161640 1 -- 172.21.15.64:0/3710658637 wait complete. 2023-11-24T08:07:32.270 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ok 2023-11-24T08:07:32.271 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) tests the scenario where 'fs subvolume snapshot info' command 2023-11-24T08:07:32.271 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:07:32.271 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:32.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.501+0000 7fc27dae7640 1 Processor -- start 2023-11-24T08:07:32.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.501+0000 7fc27dae7640 1 -- start start 2023-11-24T08:07:32.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.503+0000 7fc27dae7640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc27806d520 0x7fc27806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.503+0000 7fc27dae7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2780ad720 0x7fc27815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.503+0000 7fc27dae7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27815c360 con 0x7fc2780abff0 2023-11-24T08:07:32.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.503+0000 7fc27dae7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2780b41a0 con 0x7fc2780ad720 2023-11-24T08:07:32.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.503+0000 7fc27dae7640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27806de40 con 0x7fc27806d520 2023-11-24T08:07:32.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.503+0000 7fc277fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc2780abff0 0x7fc27815f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39534/0 (socket says 172.21.15.64:39534) 2023-11-24T08:07:32.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.503+0000 7fc277fff640 1 -- 172.21.15.64:0/2105489430 learned_addr learned my addr 172.21.15.64:0/2105489430 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:32.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.503+0000 7fc26ffff640 1 --2- 172.21.15.64:0/2105489430 >> v2:172.21.15.64:3300/0 conn(0x7fc2780ad720 0x7fc27815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.504+0000 7fc2777fe640 1 --2- 172.21.15.64:0/2105489430 >> v2:172.21.15.91:3300/0 conn(0x7fc27806d520 0x7fc27806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.504+0000 7fc26ffff640 1 -- 172.21.15.64:0/2105489430 >> v2:172.21.15.91:3300/0 conn(0x7fc27806d520 msgr2=0x7fc27806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:32.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.504+0000 7fc26ffff640 1 --2- 172.21.15.64:0/2105489430 >> v2:172.21.15.91:3300/0 conn(0x7fc27806d520 0x7fc27806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.504+0000 7fc26ffff640 1 -- 172.21.15.64:0/2105489430 >> v1:172.21.15.64:6789/0 conn(0x7fc2780abff0 legacy=0x7fc27815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.504+0000 7fc26ffff640 1 -- 172.21.15.64:0/2105489430 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27815ba30 con 0x7fc2780ad720 2023-11-24T08:07:32.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.505+0000 7fc26ffff640 1 --2- 172.21.15.64:0/2105489430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc27815ecd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc27806c070 tx=0x7fc26002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414abb469d707f73 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:32.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc276ffd640 1 -- 172.21.15.64:0/2105489430 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260031830 con 0x7fc2780ad720 2023-11-24T08:07:32.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc276ffd640 1 -- 172.21.15.64:0/2105489430 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260031990 con 0x7fc2780ad720 2023-11-24T08:07:32.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc276ffd640 1 -- 172.21.15.64:0/2105489430 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260031d20 con 0x7fc2780ad720 2023-11-24T08:07:32.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc27dae7640 1 -- 172.21.15.64:0/2105489430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 msgr2=0x7fc27815ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc27dae7640 1 --2- 172.21.15.64:0/2105489430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc27815ecd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc27806c070 tx=0x7fc26002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc27dae7640 1 -- 172.21.15.64:0/2105489430 shutdown_connections 2023-11-24T08:07:32.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc27dae7640 1 --2- 172.21.15.64:0/2105489430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc27815ecd0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc27dae7640 1 --2- 172.21.15.64:0/2105489430 >> v2:172.21.15.91:3300/0 conn(0x7fc27806d520 0x7fc27806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc27dae7640 1 -- 172.21.15.64:0/2105489430 >> 172.21.15.64:0/2105489430 conn(0x7fc2780b0d80 msgr2=0x7fc2780b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:32.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc27dae7640 1 -- 172.21.15.64:0/2105489430 shutdown_connections 2023-11-24T08:07:32.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.506+0000 7fc27dae7640 1 -- 172.21.15.64:0/2105489430 wait complete. 2023-11-24T08:07:32.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.508+0000 7fc27dae7640 1 Processor -- start 2023-11-24T08:07:32.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.508+0000 7fc27dae7640 1 -- start start 2023-11-24T08:07:32.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.509+0000 7fc27dae7640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc27806d520 0x7fc27813f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.509+0000 7fc27dae7640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2780abff0 0x7fc278152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.509+0000 7fc27dae7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc278141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.509+0000 7fc27dae7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc278154cf0 con 0x7fc2780ad720 2023-11-24T08:07:32.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.509+0000 7fc27dae7640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781421a0 con 0x7fc2780abff0 2023-11-24T08:07:32.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.509+0000 7fc27dae7640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc278141be0 con 0x7fc27806d520 2023-11-24T08:07:32.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc277fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc278141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc277fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc278141210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57950/0 (socket says 172.21.15.64:57950) 2023-11-24T08:07:32.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc277fff640 1 -- 172.21.15.64:0/1239814550 learned_addr learned my addr 172.21.15.64:0/1239814550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:32.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc26ffff640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2780abff0 0x7fc278152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc2777fe640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc27806d520 0x7fc27813f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc277fff640 1 -- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc27806d520 msgr2=0x7fc27813f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc277fff640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc27806d520 0x7fc27813f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc277fff640 1 -- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2780abff0 msgr2=0x7fc278152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc277fff640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2780abff0 0x7fc278152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc277fff640 1 -- 172.21.15.64:0/1239814550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009400 con 0x7fc2780ad720 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.510+0000 7fc26ffff640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2780abff0 0x7fc278152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.511+0000 7fc277fff640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc278141210 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc27000caf0 tx=0x7fc27000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.511+0000 7fc274ff9640 1 -- 172.21.15.64:0/1239814550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27000aca0 con 0x7fc2780ad720 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.511+0000 7fc274ff9640 1 -- 172.21.15.64:0/1239814550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27000ae00 con 0x7fc2780ad720 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.511+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc278141ec0 con 0x7fc2780ad720 2023-11-24T08:07:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.511+0000 7fc274ff9640 1 -- 172.21.15.64:0/1239814550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270012600 con 0x7fc2780ad720 2023-11-24T08:07:32.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.511+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc278155200 con 0x7fc2780ad720 2023-11-24T08:07:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.512+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27813f780 con 0x7fc2780ad720 2023-11-24T08:07:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.513+0000 7fc274ff9640 1 -- 172.21.15.64:0/1239814550 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fc270007070 con 0x7fc2780ad720 2023-11-24T08:07:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.514+0000 7fc274ff9640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc25806b230 0x7fc25806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.514+0000 7fc274ff9640 1 -- 172.21.15.64:0/1239814550 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc270089960 con 0x7fc2780ad720 2023-11-24T08:07:32.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.515+0000 7fc2777fe640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc25806b230 0x7fc25806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.516+0000 7fc2777fe640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc25806b230 0x7fc25806d6f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc268006fd0 tx=0x7fc268008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.519+0000 7fc274ff9640 1 -- 172.21.15.64:0/1239814550 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc270054710 con 0x7fc2780ad720 2023-11-24T08:07:32.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.716+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2781553f0 con 0x7fc2780ad720 2023-11-24T08:07:32.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.718+0000 7fc274ff9640 1 -- 172.21.15.64:0/1239814550 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+12938 (secure 0 0 0) 0x7fc270017070 con 0x7fc2780ad720 2023-11-24T08:07:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:32.721 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":65,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:30.216353+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6835/4200493009":"2023-11-25T08:07:26.750145+0000","172.21.15.64:6833/4200493009":"2023-11-25T08:07:26.750145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.721+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc25806b230 msgr2=0x7fc25806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.721+0000 7fc27dae7640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc25806b230 0x7fc25806d6f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc268006fd0 tx=0x7fc268008040 comp rx=0 tx=0).stop 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.721+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 msgr2=0x7fc278141210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.721+0000 7fc27dae7640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc278141210 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc27000caf0 tx=0x7fc27000a060 comp rx=0 tx=0).stop 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.722+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 shutdown_connections 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.722+0000 7fc27dae7640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc25806b230 0x7fc25806d6f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.722+0000 7fc27dae7640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2780ad720 0x7fc278141210 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.722+0000 7fc27dae7640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc2780abff0 0x7fc278152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.722+0000 7fc27dae7640 1 --2- 172.21.15.64:0/1239814550 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc27806d520 0x7fc27813f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.722+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 >> 172.21.15.64:0/1239814550 conn(0x7fc2780b0d80 msgr2=0x7fc2780b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.722+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 shutdown_connections 2023-11-24T08:07:32.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.722+0000 7fc27dae7640 1 -- 172.21.15.64:0/1239814550 wait complete. 2023-11-24T08:07:32.742 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:07:32.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:07:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.961+0000 7ff41ffff640 1 Processor -- start 2023-11-24T08:07:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.962+0000 7ff41ffff640 1 -- start start 2023-11-24T08:07:32.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.963+0000 7ff41ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff420153ea0 0x7ff4200ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.963+0000 7ff41ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff420070850 0x7ff420071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.963+0000 7ff41ffff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4200ce650 con 0x7ff420154280 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.963+0000 7ff41ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff420154710 con 0x7ff420070850 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.963+0000 7ff41ffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42015e0a0 con 0x7ff420153ea0 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.963+0000 7ff41f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff420070850 0x7ff420071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.963+0000 7ff41f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff420070850 0x7ff420071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57954/0 (socket says 172.21.15.64:57954) 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.963+0000 7ff41f7fe640 1 -- 172.21.15.64:0/1091121204 learned_addr learned my addr 172.21.15.64:0/1091121204 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.964+0000 7ff41effd640 1 --2- 172.21.15.64:0/1091121204 >> v2:172.21.15.91:3300/0 conn(0x7ff420153ea0 0x7ff4200ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.964+0000 7ff41f7fe640 1 -- 172.21.15.64:0/1091121204 >> v2:172.21.15.91:3300/0 conn(0x7ff420153ea0 msgr2=0x7ff4200ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:32.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.964+0000 7ff41f7fe640 1 --2- 172.21.15.64:0/1091121204 >> v2:172.21.15.91:3300/0 conn(0x7ff420153ea0 0x7ff4200ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.964+0000 7ff41f7fe640 1 -- 172.21.15.64:0/1091121204 >> v1:172.21.15.91:6789/0 conn(0x7ff420154280 legacy=0x7ff42015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:32.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.964+0000 7ff41f7fe640 1 -- 172.21.15.64:0/1091121204 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42015e380 con 0x7ff420070850 2023-11-24T08:07:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.965+0000 7ff41f7fe640 1 --2- 172.21.15.64:0/1091121204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420070850 0x7ff420071350 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff4200cfc10 tx=0x7ff41802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f556e2fa614799b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.965+0000 7ff41dffb640 1 -- 172.21.15.64:0/1091121204 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418031870 con 0x7ff420070850 2023-11-24T08:07:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.965+0000 7ff41dffb640 1 -- 172.21.15.64:0/1091121204 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4180319d0 con 0x7ff420070850 2023-11-24T08:07:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.965+0000 7ff41dffb640 1 -- 172.21.15.64:0/1091121204 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418031d60 con 0x7ff420070850 2023-11-24T08:07:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.966+0000 7ff41ffff640 1 -- 172.21.15.64:0/1091121204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420070850 msgr2=0x7ff420071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.966+0000 7ff41ffff640 1 --2- 172.21.15.64:0/1091121204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420070850 0x7ff420071350 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff4200cfc10 tx=0x7ff41802b910 comp rx=0 tx=0).stop 2023-11-24T08:07:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.966+0000 7ff41ffff640 1 -- 172.21.15.64:0/1091121204 shutdown_connections 2023-11-24T08:07:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.966+0000 7ff41ffff640 1 --2- 172.21.15.64:0/1091121204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420070850 0x7ff420071350 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.966+0000 7ff41ffff640 1 --2- 172.21.15.64:0/1091121204 >> v2:172.21.15.91:3300/0 conn(0x7ff420153ea0 0x7ff4200ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.966+0000 7ff41ffff640 1 -- 172.21.15.64:0/1091121204 >> 172.21.15.64:0/1091121204 conn(0x7ff4200cec10 msgr2=0x7ff4200cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.966+0000 7ff41ffff640 1 -- 172.21.15.64:0/1091121204 shutdown_connections 2023-11-24T08:07:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.966+0000 7ff41ffff640 1 -- 172.21.15.64:0/1091121204 wait complete. 2023-11-24T08:07:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.968+0000 7ff41ffff640 1 Processor -- start 2023-11-24T08:07:32.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.968+0000 7ff41ffff640 1 -- start start 2023-11-24T08:07:32.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff420070850 0x7ff42014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff420153ea0 0x7ff4201f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420154280 0x7ff4201f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201f3010 con 0x7ff420154280 2023-11-24T08:07:32.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff420157200 con 0x7ff420070850 2023-11-24T08:07:32.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff420156680 con 0x7ff420153ea0 2023-11-24T08:07:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420154280 0x7ff4201f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420154280 0x7ff4201f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57966/0 (socket says 172.21.15.64:57966) 2023-11-24T08:07:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.969+0000 7ff41f7fe640 1 -- 172.21.15.64:0/839756160 learned_addr learned my addr 172.21.15.64:0/839756160 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.970+0000 7ff41effd640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff420070850 0x7ff42014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.970+0000 7ff41e7fc640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff420153ea0 0x7ff4201f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.970+0000 7ff41f7fe640 1 -- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff420153ea0 msgr2=0x7ff4201f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.970+0000 7ff41f7fe640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff420153ea0 0x7ff4201f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.970+0000 7ff41f7fe640 1 -- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff420070850 msgr2=0x7ff42014c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.970+0000 7ff41f7fe640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff420070850 0x7ff42014c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.970+0000 7ff41f7fe640 1 -- 172.21.15.64:0/839756160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4180094d0 con 0x7ff420154280 2023-11-24T08:07:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.970+0000 7ff41f7fe640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420154280 0x7ff4201f2ad0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff4180090b0 tx=0x7ff4180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.971+0000 7ff4177fe640 1 -- 172.21.15.64:0/839756160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41803ac60 con 0x7ff420154280 2023-11-24T08:07:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.971+0000 7ff4177fe640 1 -- 172.21.15.64:0/839756160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41803adc0 con 0x7ff420154280 2023-11-24T08:07:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.971+0000 7ff4177fe640 1 -- 172.21.15.64:0/839756160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418044630 con 0x7ff420154280 2023-11-24T08:07:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.971+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff420156960 con 0x7ff420154280 2023-11-24T08:07:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.971+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff420156f20 con 0x7ff420154280 2023-11-24T08:07:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.972+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42004e840 con 0x7ff420154280 2023-11-24T08:07:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.973+0000 7ff4177fe640 1 -- 172.21.15.64:0/839756160 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff41804d450 con 0x7ff420154280 2023-11-24T08:07:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.974+0000 7ff4177fe640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff40006b230 0x7ff40006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.975+0000 7ff4177fe640 1 -- 172.21.15.64:0/839756160 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff41802d040 con 0x7ff420154280 2023-11-24T08:07:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.975+0000 7ff41effd640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff40006b230 0x7ff40006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.976+0000 7ff41effd640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff40006b230 0x7ff40006d6f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff410009a60 tx=0x7ff410003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:32.979+0000 7ff4177fe640 1 -- 172.21.15.64:0/839756160 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff418076e50 con 0x7ff420154280 2023-11-24T08:07:33.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.170+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff42015fa00 con 0x7ff420154280 2023-11-24T08:07:33.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.171+0000 7ff4177fe640 1 -- 172.21.15.64:0/839756160 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff418035020 con 0x7ff420154280 2023-11-24T08:07:33.173 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:07:33.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff40006b230 msgr2=0x7ff40006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:33.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff40006b230 0x7ff40006d6f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff410009a60 tx=0x7ff410003990 comp rx=0 tx=0).stop 2023-11-24T08:07:33.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420154280 msgr2=0x7ff4201f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:33.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420154280 0x7ff4201f2ad0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff4180090b0 tx=0x7ff4180046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 shutdown_connections 2023-11-24T08:07:33.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff40006b230 0x7ff40006d6f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff420154280 0x7ff4201f2ad0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff420153ea0 0x7ff4201f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 --2- 172.21.15.64:0/839756160 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff420070850 0x7ff42014c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.173+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 >> 172.21.15.64:0/839756160 conn(0x7ff4200cec10 msgr2=0x7ff42015d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:33.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.174+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 shutdown_connections 2023-11-24T08:07:33.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.174+0000 7ff41ffff640 1 -- 172.21.15.64:0/839756160 wait complete. 2023-11-24T08:07:33.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T08:07:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.411+0000 7f5129ddb640 1 Processor -- start 2023-11-24T08:07:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.412+0000 7f5129ddb640 1 -- start start 2023-11-24T08:07:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.413+0000 7f5129ddb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f51240d18f0 0x7f51240b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.413+0000 7f5129ddb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51240cf0b0 con 0x7f512415d5a0 2023-11-24T08:07:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.414+0000 7f5129ddb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51240b01a0 con 0x7f51240d1490 2023-11-24T08:07:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.414+0000 7f5129ddb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512415d980 con 0x7f51240d18f0 2023-11-24T08:07:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.414+0000 7f5128dd9640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f512415d5a0 0x7f51240d3980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39544/0 (socket says 172.21.15.64:39544) 2023-11-24T08:07:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.414+0000 7f5128dd9640 1 -- 172.21.15.64:0/2090680364 learned_addr learned my addr 172.21.15.64:0/2090680364 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.414+0000 7f51295da640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f51240d18f0 0x7f51240b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.414+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746985867 0 0) 0x7f51240cf0b0 con 0x7f512415d5a0 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.414+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f510c003520 con 0x7f512415d5a0 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2215747878 0 0) 0x7f510c003520 con 0x7f512415d5a0 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f51240d18f0 msgr2=0x7f51240b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f51240d18f0 0x7f51240b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 >> v1:172.21.15.91:6789/0 conn(0x7f51240d1490 legacy=0x7f51240b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240ceaf0 con 0x7f512415d5a0 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f51140019f0 con 0x7f512415d5a0 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 >> v1:172.21.15.64:6789/0 conn(0x7f512415d5a0 legacy=0x7f51240d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f51240d18f0 0x7f510c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c003fe0 0x7f510c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c004d00 0x7f510c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512415d980 con 0x7f510c004d00 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c00bb40 con 0x7f510c003fe0 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c00bd00 con 0x7f51240d18f0 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f51295da640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c004d00 0x7f510c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f5123fff640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c003fe0 0x7f510c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.415+0000 7f5128dd9640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f51240d18f0 0x7f510c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.416+0000 7f51295da640 1 -- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f51240d18f0 msgr2=0x7f510c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.416+0000 7f51295da640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f51240d18f0 0x7f510c003a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.416+0000 7f51295da640 1 -- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c003fe0 msgr2=0x7f510c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.416+0000 7f51295da640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c003fe0 0x7f510c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.416+0000 7f51295da640 1 -- 172.21.15.64:0/2090680364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c003520 con 0x7f510c004d00 2023-11-24T08:07:33.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.416+0000 7f5123fff640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f510c003fe0 0x7f510c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.417+0000 7f51295da640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c004d00 0x7f510c005180 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f511c009ac0 tx=0x7f510c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557c5618d6010c83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.417+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c02c6b0 con 0x7f510c004d00 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.417+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c02c810 con 0x7f510c004d00 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.417+0000 7f51237fe640 1 -- 172.21.15.64:0/2090680364 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c02cba0 con 0x7f510c004d00 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.418+0000 7f5129ddb640 1 -- 172.21.15.64:0/2090680364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c004d00 msgr2=0x7f510c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.418+0000 7f5129ddb640 1 --2- 172.21.15.64:0/2090680364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f510c004d00 0x7f510c005180 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f511c009ac0 tx=0x7f510c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.418+0000 7f51295da640 1 -- 172.21.15.64:0/2090680364 reap_dead start 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.418+0000 7f5129ddb640 1 -- 172.21.15.64:0/2090680364 shutdown_connections 2023-11-24T08:07:33.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.418+0000 7f5129ddb640 1 -- 172.21.15.64:0/2090680364 >> 172.21.15.64:0/2090680364 conn(0x7f512409c3a0 msgr2=0x7f51240d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:33.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.418+0000 7f5129ddb640 1 -- 172.21.15.64:0/2090680364 shutdown_connections 2023-11-24T08:07:33.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.418+0000 7f5129ddb640 1 -- 172.21.15.64:0/2090680364 wait complete. 2023-11-24T08:07:33.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.420+0000 7f5129ddb640 1 Processor -- start 2023-11-24T08:07:33.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.420+0000 7f5129ddb640 1 -- start start 2023-11-24T08:07:33.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.421+0000 7f5129ddb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c004d00 0x7f51240b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:33.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.421+0000 7f5129ddb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51241500c0 0x7f51240b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:33.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.421+0000 7f5129ddb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f512415fea0 0x7f5124160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.421+0000 7f5129ddb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51241509d0 con 0x7f51241500c0 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.421+0000 7f5129ddb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51240b8b60 con 0x7f512415fea0 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.421+0000 7f5129ddb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512414ff30 con 0x7f510c004d00 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.421+0000 7f5123fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51241500c0 0x7f51240b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.421+0000 7f5123fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51241500c0 0x7f51240b8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57986/0 (socket says 172.21.15.64:57986) 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f5123fff640 1 -- 172.21.15.64:0/3298188511 learned_addr learned my addr 172.21.15.64:0/3298188511 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f5128dd9640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c004d00 0x7f51240b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f51295da640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f512415fea0 0x7f5124160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f5123fff640 1 -- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c004d00 msgr2=0x7f51240b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f5123fff640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c004d00 0x7f51240b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f5123fff640 1 -- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f512415fea0 msgr2=0x7f5124160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f5123fff640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f512415fea0 0x7f5124160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:33.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f5123fff640 1 -- 172.21.15.64:0/3298188511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240b7120 con 0x7f51241500c0 2023-11-24T08:07:33.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f5123fff640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51241500c0 0x7f51240b8620 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f511800d7a0 tx=0x7f511800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:33.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.422+0000 7f51217fa640 1 -- 172.21.15.64:0/3298188511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118017070 con 0x7f51241500c0 2023-11-24T08:07:33.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.423+0000 7f51217fa640 1 -- 172.21.15.64:0/3298188511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118005070 con 0x7f51241500c0 2023-11-24T08:07:33.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.423+0000 7f51217fa640 1 -- 172.21.15.64:0/3298188511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118005420 con 0x7f51241500c0 2023-11-24T08:07:33.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.423+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51240b7400 con 0x7f51241500c0 2023-11-24T08:07:33.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.423+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51240b79c0 con 0x7f51241500c0 2023-11-24T08:07:33.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.424+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5124067440 con 0x7f51241500c0 2023-11-24T08:07:33.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.425+0000 7f51217fa640 1 -- 172.21.15.64:0/3298188511 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5118007030 con 0x7f51241500c0 2023-11-24T08:07:33.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.426+0000 7f51217fa640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f511006b230 0x7f511006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:33.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.426+0000 7f51217fa640 1 -- 172.21.15.64:0/3298188511 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f51180892d0 con 0x7f51241500c0 2023-11-24T08:07:33.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.426+0000 7f5128dd9640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f511006b230 0x7f511006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:33.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.428+0000 7f5128dd9640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f511006b230 0x7f511006d6f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5114001370 tx=0x7f51140018d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:33.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.430+0000 7f51217fa640 1 -- 172.21.15.64:0/3298188511 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5118054120 con 0x7f51241500c0 2023-11-24T08:07:33.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:33.699+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f512404e840 con 0x7f51241500c0 2023-11-24T08:07:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.257+0000 7f51217fa640 1 -- 172.21.15.64:0/3298188511 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v218) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5118057740 con 0x7f51241500c0 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.261+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f511006b230 msgr2=0x7f511006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.261+0000 7f5129ddb640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f511006b230 0x7f511006d6f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5114001370 tx=0x7f51140018d0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.261+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51241500c0 msgr2=0x7f51240b8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.261+0000 7f5129ddb640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51241500c0 0x7f51240b8620 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f511800d7a0 tx=0x7f511800dc60 comp rx=0 tx=0).stop 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.262+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 shutdown_connections 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.262+0000 7f5129ddb640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f511006b230 0x7f511006d6f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.262+0000 7f5129ddb640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f512415fea0 0x7f5124160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.262+0000 7f5129ddb640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51241500c0 0x7f51240b8620 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.262+0000 7f5129ddb640 1 --2- 172.21.15.64:0/3298188511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f510c004d00 0x7f51240b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.262+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 >> 172.21.15.64:0/3298188511 conn(0x7f512409c3a0 msgr2=0x7f512406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.262+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 shutdown_connections 2023-11-24T08:07:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.262+0000 7f5129ddb640 1 -- 172.21.15.64:0/3298188511 wait complete. 2023-11-24T08:07:34.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:07:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.509+0000 7efc90bfb640 1 Processor -- start 2023-11-24T08:07:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.509+0000 7efc90bfb640 1 -- start start 2023-11-24T08:07:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.511+0000 7efc90bfb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efc8c06cad0 0x7efc8c06e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.511+0000 7efc90bfb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8c071780 con 0x7efc8c072280 2023-11-24T08:07:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.511+0000 7efc90bfb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8c0711c0 con 0x7efc8c160b10 2023-11-24T08:07:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.511+0000 7efc90bfb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8c0b2320 con 0x7efc8c06cad0 2023-11-24T08:07:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.511+0000 7efc8ad9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efc8c072280 0x7efc8c0b2740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39556/0 (socket says 172.21.15.64:39556) 2023-11-24T08:07:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.511+0000 7efc8ad9d640 1 -- 172.21.15.64:0/80925618 learned_addr learned my addr 172.21.15.64:0/80925618 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc83fff640 1 --2- 172.21.15.64:0/80925618 >> v2:172.21.15.91:3300/0 conn(0x7efc8c06cad0 0x7efc8c06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805304548 0 0) 0x7efc8c071780 con 0x7efc8c072280 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc78003520 con 0x7efc8c072280 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 77881461 0 0) 0x7efc78003520 con 0x7efc8c072280 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 >> v2:172.21.15.91:3300/0 conn(0x7efc8c06cad0 msgr2=0x7efc8c06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 --2- 172.21.15.64:0/80925618 >> v2:172.21.15.91:3300/0 conn(0x7efc8c06cad0 0x7efc8c06e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 >> v1:172.21.15.91:6789/0 conn(0x7efc8c160b10 legacy=0x7efc8c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c0714a0 con 0x7efc8c072280 2023-11-24T08:07:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7efc840019f0 con 0x7efc8c072280 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 >> v1:172.21.15.64:6789/0 conn(0x7efc8c072280 legacy=0x7efc8c0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780039c0 0x7efc78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc780047c0 0x7efc78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc780052a0 0x7efc7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0b2320 con 0x7efc780052a0 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7800c1c0 con 0x7efc780047c0 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.512+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7800c380 con 0x7efc780039c0 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc8ad9d640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc780052a0 0x7efc7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc8a59c640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780039c0 0x7efc78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc83fff640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc780047c0 0x7efc78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc8ad9d640 1 -- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780039c0 msgr2=0x7efc78003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc8ad9d640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780039c0 0x7efc78003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc8ad9d640 1 -- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc780047c0 msgr2=0x7efc78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc8ad9d640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc780047c0 0x7efc78004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc8ad9d640 1 -- 172.21.15.64:0/80925618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78003520 con 0x7efc780052a0 2023-11-24T08:07:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.513+0000 7efc83fff640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc780047c0 0x7efc78004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.514+0000 7efc8ad9d640 1 -- 172.21.15.64:0/80925618 reap_dead start 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.514+0000 7efc8ad9d640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc780052a0 0x7efc7800b980 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7efc8400a760 tx=0x7efc780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cace6d3ac193e2ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84034630 con 0x7efc780052a0 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8402f070 con 0x7efc780052a0 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc89d9b640 1 -- 172.21.15.64:0/80925618 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84034a10 con 0x7efc780052a0 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc90bfb640 1 -- 172.21.15.64:0/80925618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc780052a0 msgr2=0x7efc7800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc90bfb640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc780052a0 0x7efc7800b980 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7efc8400a760 tx=0x7efc780057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc90bfb640 1 -- 172.21.15.64:0/80925618 shutdown_connections 2023-11-24T08:07:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc90bfb640 1 --2- 172.21.15.64:0/80925618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc780052a0 0x7efc7800b980 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc90bfb640 1 -- 172.21.15.64:0/80925618 >> 172.21.15.64:0/80925618 conn(0x7efc8c0b0ba0 msgr2=0x7efc8c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc90bfb640 1 -- 172.21.15.64:0/80925618 shutdown_connections 2023-11-24T08:07:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.515+0000 7efc90bfb640 1 -- 172.21.15.64:0/80925618 wait complete. 2023-11-24T08:07:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.517+0000 7efc90bfb640 1 Processor -- start 2023-11-24T08:07:34.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.518+0000 7efc90bfb640 1 -- start start 2023-11-24T08:07:34.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc90bfb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780052a0 0x7efc8c0cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:34.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc90bfb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c09b6a0 0x7efc8c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:34.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc90bfb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc8c0ad140 0x7efc8c14d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:34.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc90bfb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c09c130 con 0x7efc8c09b6a0 2023-11-24T08:07:34.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc90bfb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c099f40 con 0x7efc8c0ad140 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc90bfb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c09a1e0 con 0x7efc780052a0 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc83fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c09b6a0 0x7efc8c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc83fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c09b6a0 0x7efc8c099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58000/0 (socket says 172.21.15.64:58000) 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc83fff640 1 -- 172.21.15.64:0/1899869275 learned_addr learned my addr 172.21.15.64:0/1899869275 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc8a59c640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780052a0 0x7efc8c0cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc8ad9d640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc8c0ad140 0x7efc8c14d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc83fff640 1 -- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780052a0 msgr2=0x7efc8c0cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc83fff640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780052a0 0x7efc8c0cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc83fff640 1 -- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc8c0ad140 msgr2=0x7efc8c14d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc83fff640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc8c0ad140 0x7efc8c14d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc83fff640 1 -- 172.21.15.64:0/1899869275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c09a4c0 con 0x7efc8c09b6a0 2023-11-24T08:07:34.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.519+0000 7efc8a59c640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780052a0 0x7efc8c0cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:34.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.520+0000 7efc83fff640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c09b6a0 0x7efc8c099a00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7efc7400e7e0 tx=0x7efc7400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:34.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.520+0000 7efc82ffd640 1 -- 172.21.15.64:0/1899869275 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7400c6d0 con 0x7efc8c09b6a0 2023-11-24T08:07:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.520+0000 7efc82ffd640 1 -- 172.21.15.64:0/1899869275 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7400c830 con 0x7efc8c09b6a0 2023-11-24T08:07:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.520+0000 7efc82ffd640 1 -- 172.21.15.64:0/1899869275 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7400cba0 con 0x7efc8c09b6a0 2023-11-24T08:07:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.520+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c09a7a0 con 0x7efc8c09b6a0 2023-11-24T08:07:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.520+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8c0c8300 con 0x7efc8c09b6a0 2023-11-24T08:07:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.522+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8c09d200 con 0x7efc8c09b6a0 2023-11-24T08:07:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.522+0000 7efc82ffd640 1 -- 172.21.15.64:0/1899869275 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7efc7400cd00 con 0x7efc8c09b6a0 2023-11-24T08:07:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.523+0000 7efc82ffd640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc6c06b230 0x7efc6c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.524+0000 7efc82ffd640 1 -- 172.21.15.64:0/1899869275 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efc74089960 con 0x7efc8c09b6a0 2023-11-24T08:07:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.524+0000 7efc8a59c640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc6c06b230 0x7efc6c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:34.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.525+0000 7efc8a59c640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc6c06b230 0x7efc6c06d6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc7c000c00 tx=0x7efc7c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:34.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.528+0000 7efc82ffd640 1 -- 172.21.15.64:0/1899869275 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc740545e0 con 0x7efc8c09b6a0 2023-11-24T08:07:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.719+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efc8c06df00 con 0x7efc8c09b6a0 2023-11-24T08:07:34.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.726+0000 7efc82ffd640 1 -- 172.21.15.64:0/1899869275 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7efc74057c00 con 0x7efc8c09b6a0 2023-11-24T08:07:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc6c06b230 msgr2=0x7efc6c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc6c06b230 0x7efc6c06d6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc7c000c00 tx=0x7efc7c009040 comp rx=0 tx=0).stop 2023-11-24T08:07:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c09b6a0 msgr2=0x7efc8c099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c09b6a0 0x7efc8c099a00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7efc7400e7e0 tx=0x7efc7400eca0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 shutdown_connections 2023-11-24T08:07:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efc6c06b230 0x7efc6c06d6f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efc8c0ad140 0x7efc8c14d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc8c09b6a0 0x7efc8c099a00 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 --2- 172.21.15.64:0/1899869275 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efc780052a0 0x7efc8c0cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 >> 172.21.15.64:0/1899869275 conn(0x7efc8c0b0ba0 msgr2=0x7efc8c160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 shutdown_connections 2023-11-24T08:07:34.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:34.729+0000 7efc90bfb640 1 -- 172.21.15.64:0/1899869275 wait complete. 2023-11-24T08:07:34.750 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:07:34.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:07:34.872 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:07:34.998 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:07:34.998 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:07:34.998 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:35.062 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:07:35.062 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:07:35.062 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:35.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:07:35.124 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:07:35.124 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:35.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:07:35.187 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:07:35.187 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:35.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:07:35.248 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:07:35.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:35.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.476+0000 7f87c1e2d640 1 Processor -- start 2023-11-24T08:07:35.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.476+0000 7f87c1e2d640 1 -- start start 2023-11-24T08:07:35.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.478+0000 7f87c1e2d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87bc15a100 0x7f87bc0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:35.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.478+0000 7f87c1e2d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc13da30 con 0x7f87bc0ae7e0 2023-11-24T08:07:35.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.478+0000 7f87c1e2d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc06e340 con 0x7f87bc0abff0 2023-11-24T08:07:35.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.478+0000 7f87c1e2d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0ac840 con 0x7f87bc15a100 2023-11-24T08:07:35.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.478+0000 7f87bb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87bc15a100 0x7f87bc0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:35.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.479+0000 7f87bbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f87bc0ae7e0 0x7f87bc06dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39572/0 (socket says 172.21.15.64:39572) 2023-11-24T08:07:35.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.479+0000 7f87bbfff640 1 -- 172.21.15.64:0/1248879179 learned_addr learned my addr 172.21.15.64:0/1248879179 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:35.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.479+0000 7f87bb7fe640 1 -- 172.21.15.64:0/1248879179 >> v1:172.21.15.91:6789/0 conn(0x7f87bc0abff0 legacy=0x7f87bc06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:35.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.479+0000 7f87bb7fe640 1 -- 172.21.15.64:0/1248879179 >> v1:172.21.15.64:6789/0 conn(0x7f87bc0ae7e0 legacy=0x7f87bc06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:35.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.479+0000 7f87bb7fe640 1 -- 172.21.15.64:0/1248879179 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0ac560 con 0x7f87bc15a100 2023-11-24T08:07:35.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.480+0000 7f87bb7fe640 1 --2- 172.21.15.64:0/1248879179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc0b3290 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009d90 tx=0x7f87ac02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4845dd8924907555 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:35.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.480+0000 7f87baffd640 1 -- 172.21.15.64:0/1248879179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac02cc60 con 0x7f87bc15a100 2023-11-24T08:07:35.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87baffd640 1 -- 172.21.15.64:0/1248879179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac02cdc0 con 0x7f87bc15a100 2023-11-24T08:07:35.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87baffd640 1 -- 172.21.15.64:0/1248879179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac034660 con 0x7f87bc15a100 2023-11-24T08:07:35.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87c1e2d640 1 -- 172.21.15.64:0/1248879179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 msgr2=0x7f87bc0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:35.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87c1e2d640 1 --2- 172.21.15.64:0/1248879179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc0b3290 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009d90 tx=0x7f87ac02beb0 comp rx=0 tx=0).stop 2023-11-24T08:07:35.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87c1e2d640 1 -- 172.21.15.64:0/1248879179 shutdown_connections 2023-11-24T08:07:35.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87c1e2d640 1 --2- 172.21.15.64:0/1248879179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc0b3290 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:35.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87c1e2d640 1 -- 172.21.15.64:0/1248879179 >> 172.21.15.64:0/1248879179 conn(0x7f87bc0b0d80 msgr2=0x7f87bc0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:35.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87c1e2d640 1 -- 172.21.15.64:0/1248879179 shutdown_connections 2023-11-24T08:07:35.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.481+0000 7f87c1e2d640 1 -- 172.21.15.64:0/1248879179 wait complete. 2023-11-24T08:07:35.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.483+0000 7f87c1e2d640 1 Processor -- start 2023-11-24T08:07:35.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.484+0000 7f87c1e2d640 1 -- start start 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87c1e2d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87bc0abff0 0x7f87bc1f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87c1e2d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87bc0ae7e0 0x7f87bc0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87c1e2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc13f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87c1e2d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0cd850 con 0x7f87bc15a100 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87c1e2d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0cdb30 con 0x7f87bc0abff0 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87c1e2d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0cde10 con 0x7f87bc0ae7e0 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc13f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc13f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58010/0 (socket says 172.21.15.64:58010) 2023-11-24T08:07:35.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87bbfff640 1 -- 172.21.15.64:0/4026339326 learned_addr learned my addr 172.21.15.64:0/4026339326 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87b2ffd640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87bc0ae7e0 0x7f87bc0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.485+0000 7f87bb7fe640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87bc0abff0 0x7f87bc1f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87bbfff640 1 -- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87bc0ae7e0 msgr2=0x7f87bc0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87bbfff640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87bc0ae7e0 0x7f87bc0cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87bbfff640 1 -- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87bc0abff0 msgr2=0x7f87bc1f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87bbfff640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87bc0abff0 0x7f87bc1f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87bbfff640 1 -- 172.21.15.64:0/4026339326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac009a70 con 0x7f87bc15a100 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87b2ffd640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87bc0ae7e0 0x7f87bc0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:35.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87bb7fe640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87bc0abff0 0x7f87bc1f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:35.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87bbfff640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc13f090 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c370 tx=0x7f87b400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:35.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87b8ff9640 1 -- 172.21.15.64:0/4026339326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4005ce0 con 0x7f87bc15a100 2023-11-24T08:07:35.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87b8ff9640 1 -- 172.21.15.64:0/4026339326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4005e40 con 0x7f87bc15a100 2023-11-24T08:07:35.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87b8ff9640 1 -- 172.21.15.64:0/4026339326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40072d0 con 0x7f87bc15a100 2023-11-24T08:07:35.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.486+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc0ce0f0 con 0x7f87bc15a100 2023-11-24T08:07:35.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.487+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc0ce8f0 con 0x7f87bc15a100 2023-11-24T08:07:35.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.488+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc067960 con 0x7f87bc15a100 2023-11-24T08:07:35.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.489+0000 7f87b8ff9640 1 -- 172.21.15.64:0/4026339326 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f87b4007430 con 0x7f87bc15a100 2023-11-24T08:07:35.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.489+0000 7f87b8ff9640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f879c06b260 0x7f879c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:35.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.490+0000 7f87b8ff9640 1 -- 172.21.15.64:0/4026339326 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f87b4088ec0 con 0x7f87bc15a100 2023-11-24T08:07:35.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.490+0000 7f87bb7fe640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f879c06b260 0x7f879c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:35.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.491+0000 7f87bb7fe640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f879c06b260 0x7f879c06d720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f87ac036000 tx=0x7f87ac030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:35.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.494+0000 7f87b8ff9640 1 -- 172.21.15.64:0/4026339326 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87b4053d00 con 0x7f87bc15a100 2023-11-24T08:07:35.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.730+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87bc0ce610 con 0x7f87bc15a100 2023-11-24T08:07:35.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.731+0000 7f87b8ff9640 1 -- 172.21.15.64:0/4026339326 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f87bc0ce610 con 0x7f87bc15a100 2023-11-24T08:07:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:35.733 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6835/2873504597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2873504597},{"type":"v1","addr":"172.21.15.91:6835","nonce":2873504597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/3295227147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6836","nonce":3295227147},{"type":"v1","addr":"172.21.15.91:6837","nonce":3295227147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2949237565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2949237565},{"type":"v1","addr":"172.21.15.64:6834","nonce":2949237565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6606,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/718056019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":718056019},{"type":"v1","addr":"172.21.15.64:6835","nonce":718056019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[]} 2023-11-24T08:07:35.734 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 37 2023-11-24T08:07:35.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f879c06b260 msgr2=0x7f879c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f879c06b260 0x7f879c06d720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f87ac036000 tx=0x7f87ac030000 comp rx=0 tx=0).stop 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 msgr2=0x7f87bc13f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc13f090 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c370 tx=0x7f87b400c830 comp rx=0 tx=0).stop 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 shutdown_connections 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f879c06b260 0x7f879c06d720 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87bc15a100 0x7f87bc13f090 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f87bc0ae7e0 0x7f87bc0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 --2- 172.21.15.64:0/4026339326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f87bc0abff0 0x7f87bc1f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.734+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 >> 172.21.15.64:0/4026339326 conn(0x7f87bc0b0d80 msgr2=0x7f87bc0b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.735+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 shutdown_connections 2023-11-24T08:07:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:35.735+0000 7f87c1e2d640 1 -- 172.21.15.64:0/4026339326 wait complete. 2023-11-24T08:07:35.757 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:07:35.757 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:07:35.757 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:07:35.757 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:07:35.758 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:07:35.758 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:07:35.758 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:07:35.758 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:07:35.758 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:07:35.758 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:07:35.759 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:07:35.759 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:07:35.759 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:07:35.759 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:07:35.759 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:07:35.759 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:07:35.775 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:07:35.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:07:35.777 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:07:35.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:07:35.778 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:07:35.778 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:07:35.780 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:07:35.780 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:07:35.826 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:07:35.826 DEBUG:teuthology.parallel:result is None 2023-11-24T08:07:35.829 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:07:35.829 DEBUG:teuthology.parallel:result is None 2023-11-24T08:07:35.834 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:07:35.834 DEBUG:teuthology.parallel:result is None 2023-11-24T08:07:35.836 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:07:35.836 DEBUG:teuthology.parallel:result is None 2023-11-24T08:07:35.837 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:07:35.986 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:07:35.990 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2023-11-24T08:07:35.990 INFO:tasks.ceph.mds.d.smithi091.stdout:starting mds.d at 2023-11-24T08:07:35.995 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T08:07:36.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.032+0000 7f4d484f3640 1 Processor -- start 2023-11-24T08:07:36.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.032+0000 7f4d484f3640 1 -- start start 2023-11-24T08:07:36.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.033+0000 7f4d484f3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 0x7f4d400d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d484f3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d400d3980 0x7f4d400b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d484f3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d400d0b40 con 0x7f4d400d2740 2023-11-24T08:07:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d484f3640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4015db50 con 0x7f4d400d3980 2023-11-24T08:07:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d484f3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400d0860 con 0x7f4d4015d770 2023-11-24T08:07:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d46a90640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d400d3980 0x7f4d400b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d45a8e640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f4d400d2740 0x7f4d400d3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:50206/0 (socket says 172.21.15.64:50206) 2023-11-24T08:07:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d45a8e640 1 -- 172.21.15.64:0/347855710 learned_addr learned my addr 172.21.15.64:0/347855710 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d4628f640 1 --2- 172.21.15.64:0/347855710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 0x7f4d400d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d46a90640 1 -- 172.21.15.64:0/347855710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 msgr2=0x7f4d400d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d46a90640 1 --2- 172.21.15.64:0/347855710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 0x7f4d400d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d46a90640 1 -- 172.21.15.64:0/347855710 >> v1:172.21.15.91:6789/0 conn(0x7f4d400d2740 legacy=0x7f4d400d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.034+0000 7f4d46a90640 1 -- 172.21.15.64:0/347855710 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d400d1720 con 0x7f4d400d3980 2023-11-24T08:07:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.036+0000 7f4d46a90640 1 --2- 172.21.15.64:0/347855710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d400b6760 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009d90 tx=0x7f4d3802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9802e7f8ed8de072 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.038+0000 7f4d4528d640 1 -- 172.21.15.64:0/347855710 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3802cc50 con 0x7f4d400d3980 2023-11-24T08:07:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.038+0000 7f4d4528d640 1 -- 172.21.15.64:0/347855710 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3802cdb0 con 0x7f4d400d3980 2023-11-24T08:07:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.038+0000 7f4d4528d640 1 -- 172.21.15.64:0/347855710 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38034660 con 0x7f4d400d3980 2023-11-24T08:07:36.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.039+0000 7f4d484f3640 1 -- 172.21.15.64:0/347855710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 msgr2=0x7f4d400b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:36.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.039+0000 7f4d484f3640 1 --2- 172.21.15.64:0/347855710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d400b6760 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009d90 tx=0x7f4d3802bea0 comp rx=0 tx=0).stop 2023-11-24T08:07:36.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.039+0000 7f4d484f3640 1 -- 172.21.15.64:0/347855710 shutdown_connections 2023-11-24T08:07:36.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.039+0000 7f4d484f3640 1 --2- 172.21.15.64:0/347855710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d400b6760 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:36.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.039+0000 7f4d484f3640 1 --2- 172.21.15.64:0/347855710 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 0x7f4d400d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:36.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.039+0000 7f4d484f3640 1 -- 172.21.15.64:0/347855710 >> 172.21.15.64:0/347855710 conn(0x7f4d400721a0 msgr2=0x7f4d4006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:36.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.039+0000 7f4d484f3640 1 -- 172.21.15.64:0/347855710 shutdown_connections 2023-11-24T08:07:36.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.039+0000 7f4d484f3640 1 -- 172.21.15.64:0/347855710 wait complete. 2023-11-24T08:07:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.041+0000 7f4d484f3640 1 Processor -- start 2023-11-24T08:07:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.041+0000 7f4d484f3640 1 -- start start 2023-11-24T08:07:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.042+0000 7f4d484f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d401f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.042+0000 7f4d484f3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 0x7f4d4015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.042+0000 7f4d484f3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d400c9b60 0x7f4d400cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.042+0000 7f4d484f3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400b3f70 con 0x7f4d400d3980 2023-11-24T08:07:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.042+0000 7f4d484f3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400b41c0 con 0x7f4d400c9b60 2023-11-24T08:07:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.042+0000 7f4d484f3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400c6e10 con 0x7f4d4015d770 2023-11-24T08:07:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d401f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d401f2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58050/0 (socket says 172.21.15.64:58050) 2023-11-24T08:07:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 -- 172.21.15.64:0/4232502379 learned_addr learned my addr 172.21.15.64:0/4232502379 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d46a90640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d400c9b60 0x7f4d400cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d45a8e640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 0x7f4d4015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 -- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 msgr2=0x7f4d4015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 0x7f4d4015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 -- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d400c9b60 msgr2=0x7f4d400cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d400c9b60 0x7f4d400cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 -- 172.21.15.64:0/4232502379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d38009a70 con 0x7f4d400d3980 2023-11-24T08:07:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.043+0000 7f4d4628f640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d401f2e40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00e870 tx=0x7f4d3c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.044+0000 7f4d36ffd640 1 -- 172.21.15.64:0/4232502379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0094f0 con 0x7f4d400d3980 2023-11-24T08:07:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.044+0000 7f4d36ffd640 1 -- 172.21.15.64:0/4232502379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3c00c940 con 0x7f4d400d3980 2023-11-24T08:07:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.044+0000 7f4d36ffd640 1 -- 172.21.15.64:0/4232502379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00ccf0 con 0x7f4d400d3980 2023-11-24T08:07:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.044+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d400c70f0 con 0x7f4d400d3980 2023-11-24T08:07:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.044+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4015e230 con 0x7f4d400d3980 2023-11-24T08:07:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.045+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d400cfa30 con 0x7f4d400d3980 2023-11-24T08:07:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.046+0000 7f4d36ffd640 1 -- 172.21.15.64:0/4232502379 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f4d3c013470 con 0x7f4d400d3980 2023-11-24T08:07:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.047+0000 7f4d36ffd640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4d2406b230 0x7f4d2406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:36.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.048+0000 7f4d36ffd640 1 -- 172.21.15.64:0/4232502379 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4d3c0899d0 con 0x7f4d400d3980 2023-11-24T08:07:36.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.048+0000 7f4d45a8e640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4d2406b230 0x7f4d2406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:36.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.049+0000 7f4d45a8e640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4d2406b230 0x7f4d2406d6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4d400c6d30 tx=0x7f4d30009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:36.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.052+0000 7f4d36ffd640 1 -- 172.21.15.64:0/4232502379 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d3c0548e0 con 0x7f4d400d3980 2023-11-24T08:07:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:36.259+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4d40074770 con 0x7f4d400d3980 2023-11-24T08:07:37.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.046+0000 7f4d36ffd640 1 -- 172.21.15.64:0/4232502379 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4d3c017080 con 0x7f4d400d3980 2023-11-24T08:07:37.048 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-11-24T08:07:37.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.050+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4d2406b230 msgr2=0x7f4d2406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.050+0000 7f4d484f3640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4d2406b230 0x7f4d2406d6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4d400c6d30 tx=0x7f4d30009040 comp rx=0 tx=0).stop 2023-11-24T08:07:37.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.050+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 msgr2=0x7f4d401f2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.050+0000 7f4d484f3640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d401f2e40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00e870 tx=0x7f4d3c00ed30 comp rx=0 tx=0).stop 2023-11-24T08:07:37.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.051+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 shutdown_connections 2023-11-24T08:07:37.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.051+0000 7f4d484f3640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f4d2406b230 0x7f4d2406d6f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.051+0000 7f4d484f3640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4d400c9b60 0x7f4d400cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.051+0000 7f4d484f3640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f4d4015d770 0x7f4d4015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.051+0000 7f4d484f3640 1 --2- 172.21.15.64:0/4232502379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d400d3980 0x7f4d401f2e40 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.051+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 >> 172.21.15.64:0/4232502379 conn(0x7f4d400721a0 msgr2=0x7f4d400d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:37.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.051+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 shutdown_connections 2023-11-24T08:07:37.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.051+0000 7f4d484f3640 1 -- 172.21.15.64:0/4232502379 wait complete. 2023-11-24T08:07:37.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:07:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.298+0000 7f5e9982d640 1 Processor -- start 2023-11-24T08:07:37.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.299+0000 7f5e9982d640 1 -- start start 2023-11-24T08:07:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.300+0000 7f5e9982d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e9406d520 0x7f5e9406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.300+0000 7f5e9982d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5e940abff0 0x7f5e9415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.300+0000 7f5e9982d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9415c360 con 0x7f5e940ad720 2023-11-24T08:07:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.300+0000 7f5e9982d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e940b41a0 con 0x7f5e9406d520 2023-11-24T08:07:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.300+0000 7f5e9982d640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9406de40 con 0x7f5e940abff0 2023-11-24T08:07:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e92ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e9406d520 0x7f5e9406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e92ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e9406d520 0x7f5e9406d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54644/0 (socket says 172.21.15.64:54644) 2023-11-24T08:07:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e92ffd640 1 -- 172.21.15.64:0/1377141827 learned_addr learned my addr 172.21.15.64:0/1377141827 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e937fe640 1 --2- 172.21.15.64:0/1377141827 >> v2:172.21.15.91:3300/0 conn(0x7f5e940abff0 0x7f5e9415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e92ffd640 1 -- 172.21.15.64:0/1377141827 >> v2:172.21.15.91:3300/0 conn(0x7f5e940abff0 msgr2=0x7f5e9415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e92ffd640 1 --2- 172.21.15.64:0/1377141827 >> v2:172.21.15.91:3300/0 conn(0x7f5e940abff0 0x7f5e9415f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e92ffd640 1 -- 172.21.15.64:0/1377141827 >> v1:172.21.15.91:6789/0 conn(0x7f5e940ad720 legacy=0x7f5e9415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e92ffd640 1 -- 172.21.15.64:0/1377141827 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9415ba30 con 0x7f5e9406d520 2023-11-24T08:07:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.301+0000 7f5e937fe640 1 --2- 172.21.15.64:0/1377141827 >> v2:172.21.15.91:3300/0 conn(0x7f5e940abff0 0x7f5e9415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.302+0000 7f5e92ffd640 1 --2- 172.21.15.64:0/1377141827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e9406d520 0x7f5e9406d900 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5e94072960 tx=0x7f5e8402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cae6686890cf10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e91ffb640 1 -- 172.21.15.64:0/1377141827 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8402ca80 con 0x7f5e9406d520 2023-11-24T08:07:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e91ffb640 1 -- 172.21.15.64:0/1377141827 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8402cbe0 con 0x7f5e9406d520 2023-11-24T08:07:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e91ffb640 1 -- 172.21.15.64:0/1377141827 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e840343f0 con 0x7f5e9406d520 2023-11-24T08:07:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e9982d640 1 -- 172.21.15.64:0/1377141827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e9406d520 msgr2=0x7f5e9406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e9982d640 1 --2- 172.21.15.64:0/1377141827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e9406d520 0x7f5e9406d900 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5e94072960 tx=0x7f5e8402bee0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e9982d640 1 -- 172.21.15.64:0/1377141827 shutdown_connections 2023-11-24T08:07:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e9982d640 1 --2- 172.21.15.64:0/1377141827 >> v2:172.21.15.91:3300/0 conn(0x7f5e940abff0 0x7f5e9415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e9982d640 1 --2- 172.21.15.64:0/1377141827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e9406d520 0x7f5e9406d900 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.303+0000 7f5e9982d640 1 -- 172.21.15.64:0/1377141827 >> 172.21.15.64:0/1377141827 conn(0x7f5e940b0d80 msgr2=0x7f5e940b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.304+0000 7f5e9982d640 1 -- 172.21.15.64:0/1377141827 shutdown_connections 2023-11-24T08:07:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.304+0000 7f5e9982d640 1 -- 172.21.15.64:0/1377141827 wait complete. 2023-11-24T08:07:37.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.305+0000 7f5e9982d640 1 Processor -- start 2023-11-24T08:07:37.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.305+0000 7f5e9982d640 1 -- start start 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.306+0000 7f5e9982d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e9406d520 0x7f5e940ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.306+0000 7f5e9982d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940abff0 0x7f5e9414c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.306+0000 7f5e9982d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5e940ad720 0x7f5e94071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.306+0000 7f5e9982d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9414c8d0 con 0x7f5e940abff0 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.306+0000 7f5e9982d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e941f6cf0 con 0x7f5e9406d520 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.306+0000 7f5e9982d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e940ce030 con 0x7f5e940ad720 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e927fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940abff0 0x7f5e9414c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e927fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940abff0 0x7f5e9414c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54660/0 (socket says 172.21.15.64:54660) 2023-11-24T08:07:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e927fc640 1 -- 172.21.15.64:0/4081576864 learned_addr learned my addr 172.21.15.64:0/4081576864 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e937fe640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5e940ad720 0x7f5e94071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e92ffd640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e9406d520 0x7f5e940ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e927fc640 1 -- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5e940ad720 msgr2=0x7f5e94071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e927fc640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5e940ad720 0x7f5e94071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e927fc640 1 -- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e9406d520 msgr2=0x7f5e940ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e927fc640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e9406d520 0x7f5e940ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.307+0000 7f5e927fc640 1 -- 172.21.15.64:0/4081576864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e84009b30 con 0x7f5e940abff0 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.308+0000 7f5e927fc640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940abff0 0x7f5e9414c390 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009fd0 tx=0x7f5e7c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.308+0000 7f5e8bfff640 1 -- 172.21.15.64:0/4081576864 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c011830 con 0x7f5e940abff0 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.308+0000 7f5e8bfff640 1 -- 172.21.15.64:0/4081576864 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c011990 con 0x7f5e940abff0 2023-11-24T08:07:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.308+0000 7f5e8bfff640 1 -- 172.21.15.64:0/4081576864 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c011d20 con 0x7f5e940abff0 2023-11-24T08:07:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.308+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e941f63c0 con 0x7f5e940abff0 2023-11-24T08:07:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.308+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e941f6980 con 0x7f5e940abff0 2023-11-24T08:07:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.309+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e941f6e80 con 0x7f5e940abff0 2023-11-24T08:07:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.310+0000 7f5e8bfff640 1 -- 172.21.15.64:0/4081576864 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5e7c01a400 con 0x7f5e940abff0 2023-11-24T08:07:37.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.311+0000 7f5e8bfff640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5e7406b260 0x7f5e7406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.311+0000 7f5e8bfff640 1 -- 172.21.15.64:0/4081576864 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e7c089b20 con 0x7f5e940abff0 2023-11-24T08:07:37.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.312+0000 7f5e92ffd640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5e7406b260 0x7f5e7406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.313+0000 7f5e92ffd640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5e7406b260 0x7f5e7406d720 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5e84035000 tx=0x7f5e84030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.316+0000 7f5e8bfff640 1 -- 172.21.15.64:0/4081576864 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e7c054940 con 0x7f5e940abff0 2023-11-24T08:07:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.567+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5e94074770 con 0x7f5e940abff0 2023-11-24T08:07:37.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.568+0000 7f5e8bfff640 1 -- 172.21.15.64:0/4081576864 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5e7c057f60 con 0x7f5e940abff0 2023-11-24T08:07:37.570 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:37.570 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:07:37.570 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-11-24T08:07:37.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllV/IJxAAMEWwBM8H9uSLw5Bd93bScA==", 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-11-24T08:07:37.571 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBl35hoEBAADvW80b1cRhhgBRkYgNaftg==", 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBlZ1rGORAAjjOrPcL663guE4Q9MKA+6g==", 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:07:37.572 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlqV+1IRAAaLczF6+eDIuMM6X5Jgni/A==", 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:07:37.573 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCuWGBlsrGpIBAAVTJlhEtf7rPd1J4wUZRWTg==", 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCxWGBl+u/pLBAAwh85K8evI69aY9Nij5dBRQ==", 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC1WGBl1kGECxAAtoQ9UI319UpAU7OwZehzew==", 2023-11-24T08:07:37.575 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC4WGBlSlE2JRAAjPT1020nD4Dnrw1hrB2bKQ==", 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.576 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC/WGBlrRSGKRAAaiWZZvvd7O8PuipXF1oAUQ==", 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.577 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDDWGBls+mkBxAAJZXdXFq1+8QC5dBE14+YJA==", 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDGWGBl71+WIBAA/NNCMwpi0ycNJvXZFHKuIA==", 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.578 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJWGBl11BPKxAAnXhK4EqlIyvbHCpHzPCDUg==", 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA==", 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlsyr3DBAASDVvIWUkg06i7YARod5spw==", 2023-11-24T08:07:37.580 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlhsnfABAA5sZEDm1Uyin3dIOEoMX+gg==", 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:07:37.581 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlT9iOFxAAiupeG8QnYzYA8pJYftfxNA==", 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCnWGBlKY7iFBAAmkZ+pYWFmZ7aPH06BiE0+w==", 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-11-24T08:07:37.583 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllqymGhAA+wKOSJeuBn60R4gZJ3edoA==", 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBloKAEDhAAFSBzeCcgrTtmsfXrxKqCRQ==", 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:07:37.584 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:07:37.585 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.585 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:37.585 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T08:07:37.585 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:07:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5e7406b260 msgr2=0x7f5e7406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5e7406b260 0x7f5e7406d720 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5e84035000 tx=0x7f5e84030000 comp rx=0 tx=0).stop 2023-11-24T08:07:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940abff0 msgr2=0x7f5e9414c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940abff0 0x7f5e9414c390 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009fd0 tx=0x7f5e7c00b870 comp rx=0 tx=0).stop 2023-11-24T08:07:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 shutdown_connections 2023-11-24T08:07:37.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5e7406b260 0x7f5e7406d720 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5e940ad720 0x7f5e94071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e940abff0 0x7f5e9414c390 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 --2- 172.21.15.64:0/4081576864 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5e9406d520 0x7f5e940ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 >> 172.21.15.64:0/4081576864 conn(0x7f5e940b0d80 msgr2=0x7f5e940affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:37.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 shutdown_connections 2023-11-24T08:07:37.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.571+0000 7f5e9982d640 1 -- 172.21.15.64:0/4081576864 wait complete. 2023-11-24T08:07:37.590 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:37.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.809+0000 7fe65d981640 1 Processor -- start 2023-11-24T08:07:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.809+0000 7fe65d981640 1 -- start start 2023-11-24T08:07:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.810+0000 7fe65d981640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580d1900 0x7fe6580b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.810+0000 7fe65d981640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6580cf0c0 con 0x7fe6580d14a0 2023-11-24T08:07:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.810+0000 7fe65d981640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6580b01b0 con 0x7fe65815d5b0 2023-11-24T08:07:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.810+0000 7fe65d981640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65815d990 con 0x7fe6580d1900 2023-11-24T08:07:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.810+0000 7fe657fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe6580d14a0 0x7fe6580b0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47502/0 (socket says 172.21.15.64:47502) 2023-11-24T08:07:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.810+0000 7fe657fff640 1 -- 172.21.15.64:0/3259450665 learned_addr learned my addr 172.21.15.64:0/3259450665 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:37.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe65d180640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580d1900 0x7fe6580b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411093709 0 0) 0x7fe6580cf0c0 con 0x7fe6580d14a0 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe640003520 con 0x7fe6580d14a0 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1743454910 0 0) 0x7fe640003520 con 0x7fe6580d14a0 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580d1900 msgr2=0x7fe6580b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580d1900 0x7fe6580b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 >> v1:172.21.15.91:6789/0 conn(0x7fe65815d5b0 legacy=0x7fe6580d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580ceb00 con 0x7fe6580d14a0 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fe64c001670 con 0x7fe6580d14a0 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 >> v1:172.21.15.64:6789/0 conn(0x7fe6580d14a0 legacy=0x7fe6580b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640003a10 0x7fe640003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.811+0000 7fe6577fe640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580d1900 0x7fe6400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe6577fe640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640004d00 0x7fe640005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65815d990 con 0x7fe640003a10 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64000bac0 con 0x7fe640004d00 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64000bc80 con 0x7fe6580d1900 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe65c97f640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640003a10 0x7fe640003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe657fff640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580d1900 0x7fe6400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe65c97f640 1 -- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580d1900 msgr2=0x7fe6400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe65c97f640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580d1900 0x7fe6400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe65c97f640 1 -- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640004d00 msgr2=0x7fe640005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe65d180640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640004d00 0x7fe640005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe65c97f640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640004d00 0x7fe640005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.812+0000 7fe65c97f640 1 -- 172.21.15.64:0/3259450665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe640003520 con 0x7fe640003a10 2023-11-24T08:07:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.813+0000 7fe65c97f640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640003a10 0x7fe640003e30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe648001e20 tx=0x7fe6400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e034d60dd91ff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648003b70 con 0x7fe640003a10 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe648003cd0 con 0x7fe640003a10 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe6577fe640 1 -- 172.21.15.64:0/3259450665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648005310 con 0x7fe640003a10 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe65d981640 1 -- 172.21.15.64:0/3259450665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640003a10 msgr2=0x7fe640003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe65d981640 1 --2- 172.21.15.64:0/3259450665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640003a10 0x7fe640003e30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe648001e20 tx=0x7fe6400057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe65d180640 1 -- 172.21.15.64:0/3259450665 reap_dead start 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe65d981640 1 -- 172.21.15.64:0/3259450665 shutdown_connections 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe65d981640 1 -- 172.21.15.64:0/3259450665 >> 172.21.15.64:0/3259450665 conn(0x7fe65809c3b0 msgr2=0x7fe6580d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe65d981640 1 -- 172.21.15.64:0/3259450665 shutdown_connections 2023-11-24T08:07:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.814+0000 7fe65d981640 1 -- 172.21.15.64:0/3259450665 wait complete. 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.816+0000 7fe65d981640 1 Processor -- start 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.816+0000 7fe65d981640 1 -- start start 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.817+0000 7fe65d981640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640003a10 0x7fe658156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.817+0000 7fe65d981640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6580b3d70 0x7fe6580b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.817+0000 7fe65d981640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580cc670 0x7fe6580ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.817+0000 7fe65d981640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580cd030 con 0x7fe6580b3d70 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.817+0000 7fe65d981640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580c8b20 con 0x7fe640003a10 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.817+0000 7fe65d981640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580c8e00 con 0x7fe6580cc670 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6580b3d70 0x7fe6580b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6580b3d70 0x7fe6580b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54670/0 (socket says 172.21.15.64:54670) 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 -- 172.21.15.64:0/1449564634 learned_addr learned my addr 172.21.15.64:0/1449564634 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe65c97f640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640003a10 0x7fe658156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe65d180640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580cc670 0x7fe6580ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 -- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580cc670 msgr2=0x7fe6580ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580cc670 0x7fe6580ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 -- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640003a10 msgr2=0x7fe658156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640003a10 0x7fe658156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 -- 172.21.15.64:0/1449564634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580c90e0 con 0x7fe6580b3d70 2023-11-24T08:07:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe65d180640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580cc670 0x7fe6580ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:37.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.818+0000 7fe657fff640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6580b3d70 0x7fe6580b41d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00e730 tx=0x7fe64c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:37.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.819+0000 7fe6557fa640 1 -- 172.21.15.64:0/1449564634 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00c6c0 con 0x7fe6580b3d70 2023-11-24T08:07:37.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.819+0000 7fe6557fa640 1 -- 172.21.15.64:0/1449564634 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c00c840 con 0x7fe6580b3d70 2023-11-24T08:07:37.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.819+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6580c6c40 con 0x7fe6580b3d70 2023-11-24T08:07:37.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.819+0000 7fe6557fa640 1 -- 172.21.15.64:0/1449564634 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00cbf0 con 0x7fe6580b3d70 2023-11-24T08:07:37.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.819+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6580c93c0 con 0x7fe6580b3d70 2023-11-24T08:07:37.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.820+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6580c6e60 con 0x7fe6580b3d70 2023-11-24T08:07:37.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.821+0000 7fe6557fa640 1 -- 172.21.15.64:0/1449564634 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe64c01d460 con 0x7fe6580b3d70 2023-11-24T08:07:37.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.822+0000 7fe6557fa640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe64406b230 0x7fe64406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:37.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.823+0000 7fe6557fa640 1 -- 172.21.15.64:0/1449564634 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe64c089b00 con 0x7fe6580b3d70 2023-11-24T08:07:37.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.823+0000 7fe65c97f640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe64406b230 0x7fe64406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:37.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.824+0000 7fe65c97f640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe64406b230 0x7fe64406d6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe648034000 tx=0x7fe648009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:37.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:37.827+0000 7fe6557fa640 1 -- 172.21.15.64:0/1449564634 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe64c054970 con 0x7fe6580b3d70 2023-11-24T08:07:38.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.023+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6580ce7c0 con 0x7fe6580b3d70 2023-11-24T08:07:38.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.025+0000 7fe6557fa640 1 -- 172.21.15.64:0/1449564634 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+12809 (secure 0 0 0) 0x7fe64c057f90 con 0x7fe6580b3d70 2023-11-24T08:07:38.027 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:38.027 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":70,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:37.037928+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:38.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe64406b230 msgr2=0x7fe64406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe64406b230 0x7fe64406d6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe648034000 tx=0x7fe648009030 comp rx=0 tx=0).stop 2023-11-24T08:07:38.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6580b3d70 msgr2=0x7fe6580b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6580b3d70 0x7fe6580b41d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00e730 tx=0x7fe64c00ebf0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 shutdown_connections 2023-11-24T08:07:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe64406b230 0x7fe64406d6f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6580cc670 0x7fe6580ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6580b3d70 0x7fe6580b41d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 --2- 172.21.15.64:0/1449564634 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe640003a10 0x7fe658156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 >> 172.21.15.64:0/1449564634 conn(0x7fe65809c3b0 msgr2=0x7fe6580cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:38.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 shutdown_connections 2023-11-24T08:07:38.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.028+0000 7fe65d981640 1 -- 172.21.15.64:0/1449564634 wait complete. 2023-11-24T08:07:38.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:07:38.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:07:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.272+0000 7f675e178640 1 Processor -- start 2023-11-24T08:07:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.273+0000 7f675e178640 1 -- start start 2023-11-24T08:07:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.274+0000 7f675e178640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f675815b350 0x7f67580abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.274+0000 7f675e178640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675815b730 con 0x7f675809cb30 2023-11-24T08:07:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.274+0000 7f675e178640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67580ac740 con 0x7f67580adb60 2023-11-24T08:07:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.274+0000 7f675e178640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580ac460 con 0x7f675815b350 2023-11-24T08:07:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.275+0000 7f674ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f675809cb30 0x7f67580b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47508/0 (socket says 172.21.15.64:47508) 2023-11-24T08:07:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.275+0000 7f674ffff640 1 -- 172.21.15.64:0/1688439248 learned_addr learned my addr 172.21.15.64:0/1688439248 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.275+0000 7f67577fe640 1 --2- 172.21.15.64:0/1688439248 >> v2:172.21.15.64:3300/0 conn(0x7f675815b350 0x7f67580abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.275+0000 7f6756ffd640 1 -- 172.21.15.64:0/1688439248 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540785379 0 0) 0x7f6740001110 con 0x7f675809cb30 2023-11-24T08:07:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.275+0000 7f6756ffd640 1 -- 172.21.15.64:0/1688439248 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6744003520 con 0x7f675809cb30 2023-11-24T08:07:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.276+0000 7f67577fe640 1 -- 172.21.15.64:0/1688439248 >> v1:172.21.15.91:6789/0 conn(0x7f67580adb60 legacy=0x7f67580adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.276+0000 7f67577fe640 1 -- 172.21.15.64:0/1688439248 >> v1:172.21.15.64:6789/0 conn(0x7f675809cb30 legacy=0x7f67580b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.276+0000 7f67577fe640 1 -- 172.21.15.64:0/1688439248 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580b2a50 con 0x7f675815b350 2023-11-24T08:07:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.277+0000 7f67577fe640 1 --2- 172.21.15.64:0/1688439248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f675815b350 0x7f67580abe80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f67480097a0 tx=0x7f674802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb1d3c55ec6af52 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:38.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.280+0000 7f6756ffd640 1 -- 172.21.15.64:0/1688439248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748031830 con 0x7f675815b350 2023-11-24T08:07:38.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.280+0000 7f6756ffd640 1 -- 172.21.15.64:0/1688439248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6748031990 con 0x7f675815b350 2023-11-24T08:07:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.280+0000 7f6756ffd640 1 -- 172.21.15.64:0/1688439248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748031d20 con 0x7f675815b350 2023-11-24T08:07:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.280+0000 7f675e178640 1 -- 172.21.15.64:0/1688439248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f675815b350 msgr2=0x7f67580abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.280+0000 7f675e178640 1 --2- 172.21.15.64:0/1688439248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f675815b350 0x7f67580abe80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f67480097a0 tx=0x7f674802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.281+0000 7f675e178640 1 -- 172.21.15.64:0/1688439248 shutdown_connections 2023-11-24T08:07:38.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.281+0000 7f675e178640 1 --2- 172.21.15.64:0/1688439248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f675815b350 0x7f67580abe80 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.281+0000 7f675e178640 1 -- 172.21.15.64:0/1688439248 >> 172.21.15.64:0/1688439248 conn(0x7f67580b0ba0 msgr2=0x7f67580b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:38.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.281+0000 7f675e178640 1 -- 172.21.15.64:0/1688439248 shutdown_connections 2023-11-24T08:07:38.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.281+0000 7f675e178640 1 -- 172.21.15.64:0/1688439248 wait complete. 2023-11-24T08:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.283+0000 7f675e178640 1 Processor -- start 2023-11-24T08:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.283+0000 7f675e178640 1 -- start start 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f675e178640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f675809cb30 0x7f6758146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f675e178640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67580adb60 0x7f675814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f675e178640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f675815b350 0x7f675814cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f675e178640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758147190 con 0x7f67580adb60 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f675e178640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675814eab0 con 0x7f675815b350 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f675e178640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675814b060 con 0x7f675809cb30 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f674ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67580adb60 0x7f675814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f674ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67580adb60 0x7f675814ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54698/0 (socket says 172.21.15.64:54698) 2023-11-24T08:07:38.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f674ffff640 1 -- 172.21.15.64:0/3559883176 learned_addr learned my addr 172.21.15.64:0/3559883176 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:38.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f674ffff640 1 -- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f675809cb30 msgr2=0x7f6758146c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f674ffff640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f675809cb30 0x7f6758146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f674ffff640 1 -- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f675815b350 msgr2=0x7f675814cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f674ffff640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f675815b350 0x7f675814cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.285+0000 7f674ffff640 1 -- 172.21.15.64:0/3559883176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6748009400 con 0x7f67580adb60 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.286+0000 7f674ffff640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67580adb60 0x7f675814ab20 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f674000c700 tx=0x7f674000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.286+0000 7f6754ff9640 1 -- 172.21.15.64:0/3559883176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740005ce0 con 0x7f67580adb60 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.286+0000 7f6754ff9640 1 -- 172.21.15.64:0/3559883176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6740005e40 con 0x7f67580adb60 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.286+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675814ed50 con 0x7f67580adb60 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.286+0000 7f6754ff9640 1 -- 172.21.15.64:0/3559883176 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740007290 con 0x7f67580adb60 2023-11-24T08:07:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.286+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675814f310 con 0x7f67580adb60 2023-11-24T08:07:38.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.288+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67580b0f80 con 0x7f67580adb60 2023-11-24T08:07:38.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.288+0000 7f6754ff9640 1 -- 172.21.15.64:0/3559883176 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f67400074b0 con 0x7f67580adb60 2023-11-24T08:07:38.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.289+0000 7f6754ff9640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f673806b230 0x7f673806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.289+0000 7f6754ff9640 1 -- 172.21.15.64:0/3559883176 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6740088da0 con 0x7f67580adb60 2023-11-24T08:07:38.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.290+0000 7f67577fe640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f673806b230 0x7f673806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.291+0000 7f67577fe640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f673806b230 0x7f673806d6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6748036000 tx=0x7f6748030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:38.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.296+0000 7f6754ff9640 1 -- 172.21.15.64:0/3559883176 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67400576e0 con 0x7f67580adb60 2023-11-24T08:07:38.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.487+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f675815b730 con 0x7f67580adb60 2023-11-24T08:07:38.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.488+0000 7f6754ff9640 1 -- 172.21.15.64:0/3559883176 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f675815b730 con 0x7f67580adb60 2023-11-24T08:07:38.490 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:07:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.490+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f673806b230 msgr2=0x7f673806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.490+0000 7f675e178640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f673806b230 0x7f673806d6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6748036000 tx=0x7f6748030000 comp rx=0 tx=0).stop 2023-11-24T08:07:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.490+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67580adb60 msgr2=0x7f675814ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.490+0000 7f675e178640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67580adb60 0x7f675814ab20 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f674000c700 tx=0x7f674000cb00 comp rx=0 tx=0).stop 2023-11-24T08:07:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.491+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 shutdown_connections 2023-11-24T08:07:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.491+0000 7f675e178640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f673806b230 0x7f673806d6f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.491+0000 7f675e178640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f675815b350 0x7f675814cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.491+0000 7f675e178640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67580adb60 0x7f675814ab20 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.491+0000 7f675e178640 1 --2- 172.21.15.64:0/3559883176 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f675809cb30 0x7f6758146c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.491+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 >> 172.21.15.64:0/3559883176 conn(0x7f67580b0ba0 msgr2=0x7f67580b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.491+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 shutdown_connections 2023-11-24T08:07:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.491+0000 7f675e178640 1 -- 172.21.15.64:0/3559883176 wait complete. 2023-11-24T08:07:38.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:07:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.730+0000 7fd7b51bc640 1 Processor -- start 2023-11-24T08:07:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.730+0000 7fd7b51bc640 1 -- start start 2023-11-24T08:07:38.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.732+0000 7fd7b51bc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd7b015d300 0x7fd7b00ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.732+0000 7fd7b51bc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b006f620 0x7fd7b00ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.732+0000 7fd7b51bc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b006fb00 con 0x7fd7b00cf2f0 2023-11-24T08:07:38.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.732+0000 7fd7b51bc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b00cdc30 con 0x7fd7b015d300 2023-11-24T08:07:38.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.732+0000 7fd7b51bc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00cdf10 con 0x7fd7b006f620 2023-11-24T08:07:38.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.732+0000 7fd7b49bb640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd7b00cf2f0 0x7fd7b006d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47520/0 (socket says 172.21.15.64:47520) 2023-11-24T08:07:38.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.732+0000 7fd7b49bb640 1 -- 172.21.15.64:0/1005931819 learned_addr learned my addr 172.21.15.64:0/1005931819 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:38.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.732+0000 7fd7affff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd7b015d300 0x7fd7b00ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.733+0000 7fd7af7fe640 1 --2- 172.21.15.64:0/1005931819 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b006f620 0x7fd7b00ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.733+0000 7fd7affff640 1 -- 172.21.15.64:0/1005931819 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b006f620 msgr2=0x7fd7b00ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:38.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.733+0000 7fd7affff640 1 --2- 172.21.15.64:0/1005931819 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b006f620 0x7fd7b00ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.733+0000 7fd7aeffd640 1 -- 172.21.15.64:0/1005931819 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978923656 0 0) 0x7fd7b006fb00 con 0x7fd7b00cf2f0 2023-11-24T08:07:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.733+0000 7fd7affff640 1 -- 172.21.15.64:0/1005931819 >> v1:172.21.15.64:6789/0 conn(0x7fd7b00cf2f0 legacy=0x7fd7b006d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.733+0000 7fd7affff640 1 -- 172.21.15.64:0/1005931819 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b00ce1f0 con 0x7fd7b015d300 2023-11-24T08:07:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.734+0000 7fd7affff640 1 --2- 172.21.15.64:0/1005931819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b015d300 0x7fd7b00ce9f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00097a0 tx=0x7fd7a002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a3b6b435d4a497 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.734+0000 7fd7aeffd640 1 -- 172.21.15.64:0/1005931819 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a002f050 con 0x7fd7b015d300 2023-11-24T08:07:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.734+0000 7fd7aeffd640 1 -- 172.21.15.64:0/1005931819 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a0031700 con 0x7fd7b015d300 2023-11-24T08:07:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.734+0000 7fd7aeffd640 1 -- 172.21.15.64:0/1005931819 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0031a90 con 0x7fd7b015d300 2023-11-24T08:07:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.735+0000 7fd7b51bc640 1 -- 172.21.15.64:0/1005931819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b015d300 msgr2=0x7fd7b00ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.735+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/1005931819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b015d300 0x7fd7b00ce9f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00097a0 tx=0x7fd7a002b7a0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.735+0000 7fd7b51bc640 1 -- 172.21.15.64:0/1005931819 shutdown_connections 2023-11-24T08:07:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.735+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/1005931819 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b006f620 0x7fd7b00ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.735+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/1005931819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b015d300 0x7fd7b00ce9f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.735+0000 7fd7b51bc640 1 -- 172.21.15.64:0/1005931819 >> 172.21.15.64:0/1005931819 conn(0x7fd7b00cfd20 msgr2=0x7fd7b00d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.735+0000 7fd7b51bc640 1 -- 172.21.15.64:0/1005931819 shutdown_connections 2023-11-24T08:07:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.735+0000 7fd7b51bc640 1 -- 172.21.15.64:0/1005931819 wait complete. 2023-11-24T08:07:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.737+0000 7fd7b51bc640 1 Processor -- start 2023-11-24T08:07:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.737+0000 7fd7b51bc640 1 -- start start 2023-11-24T08:07:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7b51bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b006f620 0x7fd7b00b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7b51bc640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7b00cf2f0 0x7fd7b00b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7b51bc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b015d300 0x7fd7b00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7b51bc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b0160130 con 0x7fd7b006f620 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7b51bc640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b015f520 con 0x7fd7b00cf2f0 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7b51bc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b015f800 con 0x7fd7b015d300 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b006f620 0x7fd7b00b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b006f620 0x7fd7b00b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54716/0 (socket says 172.21.15.64:54716) 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.738+0000 7fd7affff640 1 -- 172.21.15.64:0/3630364493 learned_addr learned my addr 172.21.15.64:0/3630364493 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7affff640 1 -- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b015d300 msgr2=0x7fd7b00c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7b49bb640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b015d300 0x7fd7b00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7af7fe640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7b00cf2f0 0x7fd7b00b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7affff640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b015d300 0x7fd7b00c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7affff640 1 -- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7b00cf2f0 msgr2=0x7fd7b00b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7affff640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7b00cf2f0 0x7fd7b00b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7affff640 1 -- 172.21.15.64:0/3630364493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a0009400 con 0x7fd7b006f620 2023-11-24T08:07:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7affff640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b006f620 0x7fd7b00b4c20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0036040 tx=0x7fd7a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.739+0000 7fd7acff9640 1 -- 172.21.15.64:0/3630364493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0033a30 con 0x7fd7b006f620 2023-11-24T08:07:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.740+0000 7fd7acff9640 1 -- 172.21.15.64:0/3630364493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a0033bb0 con 0x7fd7b006f620 2023-11-24T08:07:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.740+0000 7fd7acff9640 1 -- 172.21.15.64:0/3630364493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a003c400 con 0x7fd7b006f620 2023-11-24T08:07:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.740+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b015fae0 con 0x7fd7b006f620 2023-11-24T08:07:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.740+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b00c7090 con 0x7fd7b006f620 2023-11-24T08:07:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.741+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b00c7220 con 0x7fd7b006f620 2023-11-24T08:07:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.742+0000 7fd7acff9640 1 -- 172.21.15.64:0/3630364493 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd7a003a040 con 0x7fd7b006f620 2023-11-24T08:07:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.743+0000 7fd7acff9640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd79c06b230 0x7fd79c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.743+0000 7fd7acff9640 1 -- 172.21.15.64:0/3630364493 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd7a00aad80 con 0x7fd7b006f620 2023-11-24T08:07:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.743+0000 7fd7af7fe640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd79c06b230 0x7fd79c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:38.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.744+0000 7fd7af7fe640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd79c06b230 0x7fd79c06d6f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00c6d50 tx=0x7fd7a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:38.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.748+0000 7fd7acff9640 1 -- 172.21.15.64:0/3630364493 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7b00c7220 con 0x7fd7b006f620 2023-11-24T08:07:38.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.945+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd7b00c7220 con 0x7fd7b006f620 2023-11-24T08:07:38.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.946+0000 7fd7acff9640 1 -- 172.21.15.64:0/3630364493 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd7b00c7220 con 0x7fd7b006f620 2023-11-24T08:07:38.947 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:38.948 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:07:38.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:38.948 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-11-24T08:07:38.948 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-11-24T08:07:38.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:38.948 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:07:38.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd79c06b230 msgr2=0x7fd79c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd79c06b230 0x7fd79c06d6f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00c6d50 tx=0x7fd7a4008040 comp rx=0 tx=0).stop 2023-11-24T08:07:38.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b006f620 msgr2=0x7fd7b00b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b006f620 0x7fd7b00b4c20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0036040 tx=0x7fd7a00046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 shutdown_connections 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd79c06b230 0x7fd79c06d6f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd7b015d300 0x7fd7b00c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd7b00cf2f0 0x7fd7b00b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 --2- 172.21.15.64:0/3630364493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7b006f620 0x7fd7b00b4c20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.948+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 >> 172.21.15.64:0/3630364493 conn(0x7fd7b00cfd20 msgr2=0x7fd7b00acd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.949+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 shutdown_connections 2023-11-24T08:07:38.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:38.949+0000 7fd7b51bc640 1 -- 172.21.15.64:0/3630364493 wait complete. 2023-11-24T08:07:38.969 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:07:38.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:07:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.202+0000 7f4001583640 1 Processor -- start 2023-11-24T08:07:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.202+0000 7f4001583640 1 -- start start 2023-11-24T08:07:39.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f4001583640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ffc162220 0x7f3ffc0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:39.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f4001583640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3ffc06cad0 0x7f3ffc162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:39.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f4001583640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ffc162fa0 con 0x7f3ffc162600 2023-11-24T08:07:39.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f4001583640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ffc0706a0 con 0x7f3ffc162220 2023-11-24T08:07:39.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f4001583640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ffc070c60 con 0x7f3ffc06cad0 2023-11-24T08:07:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f3ffaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ffc162220 0x7f3ffc0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f3ffaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ffc162220 0x7f3ffc0b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54720/0 (socket says 172.21.15.64:54720) 2023-11-24T08:07:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f3ffaffd640 1 -- 172.21.15.64:0/2270940210 learned_addr learned my addr 172.21.15.64:0/2270940210 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.204+0000 7f3ffb7fe640 1 --2- 172.21.15.64:0/2270940210 >> v2:172.21.15.91:3300/0 conn(0x7f3ffc06cad0 0x7f3ffc162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.205+0000 7f3ffaffd640 1 -- 172.21.15.64:0/2270940210 >> v2:172.21.15.91:3300/0 conn(0x7f3ffc06cad0 msgr2=0x7f3ffc162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.205+0000 7f3ffaffd640 1 --2- 172.21.15.64:0/2270940210 >> v2:172.21.15.91:3300/0 conn(0x7f3ffc06cad0 0x7f3ffc162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.205+0000 7f3ffaffd640 1 -- 172.21.15.64:0/2270940210 >> v1:172.21.15.91:6789/0 conn(0x7f3ffc162600 legacy=0x7f3ffc06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.205+0000 7f3ffaffd640 1 -- 172.21.15.64:0/2270940210 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0722b0 con 0x7f3ffc162220 2023-11-24T08:07:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.206+0000 7f3ffaffd640 1 --2- 172.21.15.64:0/2270940210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc162220 0x7f3ffc0b1ea0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0cb530 tx=0x7f3fec02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eef66c092e310c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.206+0000 7f3ffa7fc640 1 -- 172.21.15.64:0/2270940210 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec031890 con 0x7f3ffc162220 2023-11-24T08:07:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.206+0000 7f3ffa7fc640 1 -- 172.21.15.64:0/2270940210 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec0319f0 con 0x7f3ffc162220 2023-11-24T08:07:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.206+0000 7f3ffa7fc640 1 -- 172.21.15.64:0/2270940210 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec031d80 con 0x7f3ffc162220 2023-11-24T08:07:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.207+0000 7f4001583640 1 -- 172.21.15.64:0/2270940210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc162220 msgr2=0x7f3ffc0b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.207+0000 7f4001583640 1 --2- 172.21.15.64:0/2270940210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc162220 0x7f3ffc0b1ea0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0cb530 tx=0x7f3fec02b930 comp rx=0 tx=0).stop 2023-11-24T08:07:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.207+0000 7f4001583640 1 -- 172.21.15.64:0/2270940210 shutdown_connections 2023-11-24T08:07:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.207+0000 7f4001583640 1 --2- 172.21.15.64:0/2270940210 >> v2:172.21.15.91:3300/0 conn(0x7f3ffc06cad0 0x7f3ffc162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.207+0000 7f4001583640 1 --2- 172.21.15.64:0/2270940210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc162220 0x7f3ffc0b1ea0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.207+0000 7f4001583640 1 -- 172.21.15.64:0/2270940210 >> 172.21.15.64:0/2270940210 conn(0x7f3ffc0b0ba0 msgr2=0x7f3ffc0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:39.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.207+0000 7f4001583640 1 -- 172.21.15.64:0/2270940210 shutdown_connections 2023-11-24T08:07:39.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.207+0000 7f4001583640 1 -- 172.21.15.64:0/2270940210 wait complete. 2023-11-24T08:07:39.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.209+0000 7f4001583640 1 Processor -- start 2023-11-24T08:07:39.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.210+0000 7f4001583640 1 -- start start 2023-11-24T08:07:39.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f4001583640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc06cad0 0x7f3ffc153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:39.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f4001583640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3ffc162220 0x7f3ffc143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:39.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f4001583640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ffc162600 0x7f3ffc1eb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:39.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f4001583640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc1eb6b0 con 0x7f3ffc06cad0 2023-11-24T08:07:39.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f4001583640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc099ab0 con 0x7f3ffc162600 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f4001583640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc09ae50 con 0x7f3ffc162220 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f3ffaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc06cad0 0x7f3ffc153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f3ffaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc06cad0 0x7f3ffc153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54734/0 (socket says 172.21.15.64:54734) 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f3ffaffd640 1 -- 172.21.15.64:0/1709161743 learned_addr learned my addr 172.21.15.64:0/1709161743 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f3ffb7fe640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ffc162600 0x7f3ffc1eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f3ff3fff640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3ffc162220 0x7f3ffc143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.211+0000 7f3ffaffd640 1 -- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3ffc162220 msgr2=0x7f3ffc143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f3ffaffd640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3ffc162220 0x7f3ffc143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f3ffaffd640 1 -- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ffc162600 msgr2=0x7f3ffc1eb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f3ffaffd640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ffc162600 0x7f3ffc1eb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f3ffaffd640 1 -- 172.21.15.64:0/1709161743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec0094d0 con 0x7f3ffc06cad0 2023-11-24T08:07:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f3ffaffd640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc06cad0 0x7f3ffc153870 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3fec000c00 tx=0x7f3fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f3ff37fe640 1 -- 172.21.15.64:0/1709161743 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec03ac60 con 0x7f3ffc06cad0 2023-11-24T08:07:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f3ff37fe640 1 -- 172.21.15.64:0/1709161743 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec03adc0 con 0x7f3ffc06cad0 2023-11-24T08:07:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f3ff37fe640 1 -- 172.21.15.64:0/1709161743 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec044610 con 0x7f3ffc06cad0 2023-11-24T08:07:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc099d00 con 0x7f3ffc06cad0 2023-11-24T08:07:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.212+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc09b0a0 con 0x7f3ffc06cad0 2023-11-24T08:07:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.213+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ffc06e730 con 0x7f3ffc06cad0 2023-11-24T08:07:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.214+0000 7f3ff37fe640 1 -- 172.21.15.64:0/1709161743 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3fec04d450 con 0x7f3ffc06cad0 2023-11-24T08:07:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.215+0000 7f3ff37fe640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3fdc06b230 0x7f3fdc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.216+0000 7f3ff37fe640 1 -- 172.21.15.64:0/1709161743 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3fec0ac160 con 0x7f3ffc06cad0 2023-11-24T08:07:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.216+0000 7f3ff3fff640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3fdc06b230 0x7f3fdc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:39.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.217+0000 7f3ff3fff640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3fdc06b230 0x7f3fdc06d6f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0b1d20 tx=0x7f3fe4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:39.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.220+0000 7f3ff37fe640 1 -- 172.21.15.64:0/1709161743 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fec076e70 con 0x7f3ffc06cad0 2023-11-24T08:07:39.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:39.416+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3ffc09d200 con 0x7f3ffc06cad0 2023-11-24T08:07:40.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.301+0000 7f3ff37fe640 1 -- 172.21.15.64:0/1709161743 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3fec07a490 con 0x7f3ffc06cad0 2023-11-24T08:07:40.304 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-11-24T08:07:40.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3fdc06b230 msgr2=0x7f3fdc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:40.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3fdc06b230 0x7f3fdc06d6f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0b1d20 tx=0x7f3fe4009040 comp rx=0 tx=0).stop 2023-11-24T08:07:40.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc06cad0 msgr2=0x7f3ffc153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:40.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc06cad0 0x7f3ffc153870 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3fec000c00 tx=0x7f3fec0046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 shutdown_connections 2023-11-24T08:07:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3fdc06b230 0x7f3fdc06d6f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3ffc162600 0x7f3ffc1eb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3ffc162220 0x7f3ffc143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 --2- 172.21.15.64:0/1709161743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ffc06cad0 0x7f3ffc153870 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 >> 172.21.15.64:0/1709161743 conn(0x7f3ffc0b0ba0 msgr2=0x7f3ffc15fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 shutdown_connections 2023-11-24T08:07:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.305+0000 7f4001583640 1 -- 172.21.15.64:0/1709161743 wait complete. 2023-11-24T08:07:40.335 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:07:40.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.554+0000 7f2b8b6d4640 1 Processor -- start 2023-11-24T08:07:40.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.554+0000 7f2b8b6d4640 1 -- start start 2023-11-24T08:07:40.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.556+0000 7f2b8b6d4640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2b840ae490 0x7f2b84070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:40.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.556+0000 7f2b8b6d4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8406d7a0 con 0x7f2b8406dc30 2023-11-24T08:07:40.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.556+0000 7f2b8b6d4640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b840ae900 con 0x7f2b840acd70 2023-11-24T08:07:40.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.556+0000 7f2b8b6d4640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8415a5b0 con 0x7f2b840ae490 2023-11-24T08:07:40.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.557+0000 7f2b88c6f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2b8406dc30 0x7f2b84070490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47522/0 (socket says 172.21.15.64:47522) 2023-11-24T08:07:40.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.557+0000 7f2b88c6f640 1 -- 172.21.15.64:0/998256279 learned_addr learned my addr 172.21.15.64:0/998256279 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.557+0000 7f2b89c71640 1 --2- 172.21.15.64:0/998256279 >> v2:172.21.15.91:3300/0 conn(0x7f2b840ae490 0x7f2b84070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.557+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099188631 0 0) 0x7f2b8406d7a0 con 0x7f2b8406dc30 2023-11-24T08:07:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.557+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b70005730 con 0x7f2b8406dc30 2023-11-24T08:07:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 272134263 0 0) 0x7f2b70005730 con 0x7f2b8406dc30 2023-11-24T08:07:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 >> v2:172.21.15.91:3300/0 conn(0x7f2b840ae490 msgr2=0x7f2b84070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 --2- 172.21.15.64:0/998256279 >> v2:172.21.15.91:3300/0 conn(0x7f2b840ae490 0x7f2b84070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 >> v1:172.21.15.91:6789/0 conn(0x7f2b840acd70 legacy=0x7f2b840ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8406ce70 con 0x7f2b8406dc30 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2b74001670 con 0x7f2b8406dc30 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 >> v1:172.21.15.64:6789/0 conn(0x7f2b8406dc30 legacy=0x7f2b84070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b70005bd0 0x7f2b70005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b70006530 0x7f2b700069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70007140 0x7f2b7000ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8415a5b0 con 0x7f2b70006530 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b700104d0 con 0x7f2b70007140 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70010640 con 0x7f2b70005bd0 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b88c6f640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b70006530 0x7f2b700069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.558+0000 7f2b89470640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b70005bd0 0x7f2b70005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.559+0000 7f2b89c71640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70007140 0x7f2b7000ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.559+0000 7f2b88c6f640 1 -- 172.21.15.64:0/998256279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b70005bd0 msgr2=0x7f2b70005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.559+0000 7f2b88c6f640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b70005bd0 0x7f2b70005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.559+0000 7f2b88c6f640 1 -- 172.21.15.64:0/998256279 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70007140 msgr2=0x7f2b7000ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.559+0000 7f2b88c6f640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70007140 0x7f2b7000ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.559+0000 7f2b88c6f640 1 -- 172.21.15.64:0/998256279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70005730 con 0x7f2b70006530 2023-11-24T08:07:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.559+0000 7f2b89c71640 1 -- 172.21.15.64:0/998256279 reap_dead start 2023-11-24T08:07:40.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.560+0000 7f2b88c6f640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b70006530 0x7f2b700069d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400c0e0 tx=0x7f2b70007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cd099af931df7a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:40.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.560+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74005ad0 con 0x7f2b70006530 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.560+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b74005c30 con 0x7f2b70006530 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.560+0000 7f2b7bfff640 1 -- 172.21.15.64:0/998256279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7402ca00 con 0x7f2b70006530 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.561+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/998256279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b70006530 msgr2=0x7f2b700069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.561+0000 7f2b8b6d4640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b70006530 0x7f2b700069d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400c0e0 tx=0x7f2b70007a00 comp rx=0 tx=0).stop 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.561+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/998256279 shutdown_connections 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.561+0000 7f2b8b6d4640 1 --2- 172.21.15.64:0/998256279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b70006530 0x7f2b700069d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.561+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/998256279 >> 172.21.15.64:0/998256279 conn(0x7f2b840b0ba0 msgr2=0x7f2b840b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.561+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/998256279 shutdown_connections 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.561+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/998256279 wait complete. 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.563+0000 7f2b8b6d4640 1 Processor -- start 2023-11-24T08:07:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.563+0000 7f2b8b6d4640 1 -- start start 2023-11-24T08:07:40.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b8b6d4640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70006530 0x7f2b841593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:40.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b8b6d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b84154a80 0x7f2b840cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:40.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b8b6d4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b840cd6a0 0x7f2b841475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:40.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b8b6d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b84156ad0 con 0x7f2b84154a80 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b8b6d4640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b84159910 con 0x7f2b70006530 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b8b6d4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840cdd00 con 0x7f2b840cd6a0 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b88c6f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b84154a80 0x7f2b840cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b88c6f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b84154a80 0x7f2b840cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54746/0 (socket says 172.21.15.64:54746) 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b88c6f640 1 -- 172.21.15.64:0/1703766399 learned_addr learned my addr 172.21.15.64:0/1703766399 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.564+0000 7f2b89470640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70006530 0x7f2b841593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.565+0000 7f2b89c71640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b840cd6a0 0x7f2b841475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.565+0000 7f2b88c6f640 1 -- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b840cd6a0 msgr2=0x7f2b841475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.565+0000 7f2b88c6f640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b840cd6a0 0x7f2b841475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.565+0000 7f2b88c6f640 1 -- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70006530 msgr2=0x7f2b841593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.565+0000 7f2b88c6f640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70006530 0x7f2b841593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.565+0000 7f2b88c6f640 1 -- 172.21.15.64:0/1703766399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b840cdfe0 con 0x7f2b84154a80 2023-11-24T08:07:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.565+0000 7f2b88c6f640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b84154a80 0x7f2b840cd160 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2b74000c00 tx=0x7f2b7402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.565+0000 7f2b79ffb640 1 -- 172.21.15.64:0/1703766399 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7403c040 con 0x7f2b84154a80 2023-11-24T08:07:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.566+0000 7f2b79ffb640 1 -- 172.21.15.64:0/1703766399 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7402ccb0 con 0x7f2b84154a80 2023-11-24T08:07:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.566+0000 7f2b79ffb640 1 -- 172.21.15.64:0/1703766399 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b740072b0 con 0x7f2b84154a80 2023-11-24T08:07:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.566+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b840ce2c0 con 0x7f2b84154a80 2023-11-24T08:07:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.566+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b84147ae0 con 0x7f2b84154a80 2023-11-24T08:07:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.567+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b840cdab0 con 0x7f2b84154a80 2023-11-24T08:07:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.568+0000 7f2b79ffb640 1 -- 172.21.15.64:0/1703766399 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f2b74032070 con 0x7f2b84154a80 2023-11-24T08:07:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.569+0000 7f2b79ffb640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b6806b230 0x7f2b6806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.569+0000 7f2b79ffb640 1 -- 172.21.15.64:0/1703766399 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2b740ab110 con 0x7f2b84154a80 2023-11-24T08:07:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.570+0000 7f2b89470640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b6806b230 0x7f2b6806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:40.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.571+0000 7f2b89470640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b6806b230 0x7f2b6806d6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2b84068d90 tx=0x7f2b80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:40.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.574+0000 7f2b79ffb640 1 -- 172.21.15.64:0/1703766399 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b74075d80 con 0x7f2b84154a80 2023-11-24T08:07:40.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:40.772+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2b840ce450 con 0x7f2b84154a80 2023-11-24T08:07:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.316+0000 7f2b79ffb640 1 -- 172.21.15.64:0/1703766399 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2b740793a0 con 0x7f2b84154a80 2023-11-24T08:07:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-11-24T08:07:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.320+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b6806b230 msgr2=0x7f2b6806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.320+0000 7f2b8b6d4640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b6806b230 0x7f2b6806d6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2b84068d90 tx=0x7f2b80009040 comp rx=0 tx=0).stop 2023-11-24T08:07:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.320+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b84154a80 msgr2=0x7f2b840cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.320+0000 7f2b8b6d4640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b84154a80 0x7f2b840cd160 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2b74000c00 tx=0x7f2b7402e070 comp rx=0 tx=0).stop 2023-11-24T08:07:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.321+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 shutdown_connections 2023-11-24T08:07:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.321+0000 7f2b8b6d4640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2b6806b230 0x7f2b6806d6f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.321+0000 7f2b8b6d4640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b840cd6a0 0x7f2b841475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.321+0000 7f2b8b6d4640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b84154a80 0x7f2b840cd160 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.321+0000 7f2b8b6d4640 1 --2- 172.21.15.64:0/1703766399 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b70006530 0x7f2b841593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.321+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 >> 172.21.15.64:0/1703766399 conn(0x7f2b840b0ba0 msgr2=0x7f2b840b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.321+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 shutdown_connections 2023-11-24T08:07:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.321+0000 7f2b8b6d4640 1 -- 172.21.15.64:0/1703766399 wait complete. 2023-11-24T08:07:41.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:07:41.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.548+0000 7fe9da8ea640 1 Processor -- start 2023-11-24T08:07:41.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.549+0000 7fe9da8ea640 1 -- start start 2023-11-24T08:07:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.550+0000 7fe9da8ea640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe9d40d1d90 0x7fe9d40ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.550+0000 7fe9da8ea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.550+0000 7fe9da8ea640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d40c6bf0 con 0x7fe9d415d3b0 2023-11-24T08:07:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.550+0000 7fe9da8ea640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d40d13f0 con 0x7fe9d40d1d90 2023-11-24T08:07:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.550+0000 7fe9da8ea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40cedc0 con 0x7fe9d40d0fa0 2023-11-24T08:07:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.551+0000 7fe9da0e9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.551+0000 7fe9d90e7640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe9d40d1d90 0x7fe9d40ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.551+0000 7fe9da0e9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d40b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:60968/0 (socket says 172.21.15.64:60968) 2023-11-24T08:07:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.551+0000 7fe9da0e9640 1 -- 172.21.15.64:0/3510995151 learned_addr learned my addr 172.21.15.64:0/3510995151 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.552+0000 7fe9d90e7640 1 -- 172.21.15.64:0/3510995151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 msgr2=0x7fe9d40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:41.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.552+0000 7fe9d90e7640 1 --2- 172.21.15.64:0/3510995151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d40b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.552+0000 7fe9d90e7640 1 -- 172.21.15.64:0/3510995151 >> v1:172.21.15.91:6789/0 conn(0x7fe9d415d3b0 legacy=0x7fe9d40d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:41.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.552+0000 7fe9d90e7640 1 -- 172.21.15.64:0/3510995151 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40c6e40 con 0x7fe9d40d1d90 2023-11-24T08:07:41.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.552+0000 7fe9da0e9640 1 --2- 172.21.15.64:0/3510995151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d40b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:07:41.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.553+0000 7fe9d90e7640 1 --2- 172.21.15.64:0/3510995151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 0x7fe9d40ce6c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009d90 tx=0x7fe9c802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d8cc9e5960f7bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.553+0000 7fe9d88e6640 1 -- 172.21.15.64:0/3510995151 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c802cbd0 con 0x7fe9d40d1d90 2023-11-24T08:07:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.553+0000 7fe9d88e6640 1 -- 172.21.15.64:0/3510995151 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c802cd30 con 0x7fe9d40d1d90 2023-11-24T08:07:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.553+0000 7fe9d88e6640 1 -- 172.21.15.64:0/3510995151 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c802cea0 con 0x7fe9d40d1d90 2023-11-24T08:07:41.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.554+0000 7fe9da8ea640 1 -- 172.21.15.64:0/3510995151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 msgr2=0x7fe9d40ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:41.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.554+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/3510995151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 0x7fe9d40ce6c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009d90 tx=0x7fe9c802be20 comp rx=0 tx=0).stop 2023-11-24T08:07:41.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.554+0000 7fe9da8ea640 1 -- 172.21.15.64:0/3510995151 shutdown_connections 2023-11-24T08:07:41.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.554+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/3510995151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d40b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.554+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/3510995151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 0x7fe9d40ce6c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.554+0000 7fe9da8ea640 1 -- 172.21.15.64:0/3510995151 >> 172.21.15.64:0/3510995151 conn(0x7fe9d40d30a0 msgr2=0x7fe9d40d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:41.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.554+0000 7fe9da8ea640 1 -- 172.21.15.64:0/3510995151 shutdown_connections 2023-11-24T08:07:41.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.554+0000 7fe9da8ea640 1 -- 172.21.15.64:0/3510995151 wait complete. 2023-11-24T08:07:41.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.557+0000 7fe9da8ea640 1 Processor -- start 2023-11-24T08:07:41.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.557+0000 7fe9da8ea640 1 -- start start 2023-11-24T08:07:41.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.558+0000 7fe9da8ea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:41.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.558+0000 7fe9da8ea640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 0x7fe9d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:41.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.558+0000 7fe9da8ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d415d3b0 0x7fe9d4152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:41.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da8ea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d415fdc0 con 0x7fe9d415d3b0 2023-11-24T08:07:41.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da8ea640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d415faf0 con 0x7fe9d40d1d90 2023-11-24T08:07:41.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da8ea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40ca2a0 con 0x7fe9d40d0fa0 2023-11-24T08:07:41.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da0e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d415d3b0 0x7fe9d4152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da0e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d415d3b0 0x7fe9d4152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54754/0 (socket says 172.21.15.64:54754) 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da0e9640 1 -- 172.21.15.64:0/2066898273 learned_addr learned my addr 172.21.15.64:0/2066898273 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9d90e7640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 0x7fe9d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9d98e8640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da0e9640 1 -- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 msgr2=0x7fe9d415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da0e9640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d415f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da0e9640 1 -- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 msgr2=0x7fe9d40b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da0e9640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 0x7fe9d40b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:41.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.559+0000 7fe9da0e9640 1 -- 172.21.15.64:0/2066898273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0094d0 con 0x7fe9d415d3b0 2023-11-24T08:07:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.560+0000 7fe9da0e9640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d415d3b0 0x7fe9d4152650 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e930 tx=0x7fe9cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.560+0000 7fe9c67fc640 1 -- 172.21.15.64:0/2066898273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc009720 con 0x7fe9d415d3b0 2023-11-24T08:07:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.560+0000 7fe9c67fc640 1 -- 172.21.15.64:0/2066898273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc00ca00 con 0x7fe9d415d3b0 2023-11-24T08:07:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.560+0000 7fe9c67fc640 1 -- 172.21.15.64:0/2066898273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00cd90 con 0x7fe9d415d3b0 2023-11-24T08:07:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.560+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c8009a70 con 0x7fe9d415d3b0 2023-11-24T08:07:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.560+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d40ca860 con 0x7fe9d415d3b0 2023-11-24T08:07:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.561+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d404e840 con 0x7fe9d415d3b0 2023-11-24T08:07:41.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.564+0000 7fe9c67fc640 1 -- 172.21.15.64:0/2066898273 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe9cc004020 con 0x7fe9d415d3b0 2023-11-24T08:07:41.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.565+0000 7fe9c67fc640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe9b806b230 0x7fe9b806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:41.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.565+0000 7fe9c67fc640 1 -- 172.21.15.64:0/2066898273 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe9cc089ac0 con 0x7fe9d415d3b0 2023-11-24T08:07:41.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.566+0000 7fe9d98e8640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe9b806b230 0x7fe9b806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:41.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.567+0000 7fe9d98e8640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe9b806b230 0x7fe9b806d6f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0000f60 tx=0x7fe9d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:41.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.569+0000 7fe9c67fc640 1 -- 172.21.15.64:0/2066898273 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9cc054780 con 0x7fe9d415d3b0 2023-11-24T08:07:41.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:41.803+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe9d40c66d0 con 0x7fe9d415d3b0 2023-11-24T08:07:42.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.320+0000 7fe9c67fc640 1 -- 172.21.15.64:0/2066898273 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v39) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe9cc057da0 con 0x7fe9d415d3b0 2023-11-24T08:07:42.322 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 10 and data pool 11 2023-11-24T08:07:42.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.323+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe9b806b230 msgr2=0x7fe9b806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.323+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe9b806b230 0x7fe9b806d6f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0000f60 tx=0x7fe9d0009040 comp rx=0 tx=0).stop 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d415d3b0 msgr2=0x7fe9d4152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d415d3b0 0x7fe9d4152650 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e930 tx=0x7fe9cc00edf0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 shutdown_connections 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe9b806b230 0x7fe9b806d6f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d415d3b0 0x7fe9d4152650 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe9d40d1d90 0x7fe9d40b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 --2- 172.21.15.64:0/2066898273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe9d40d0fa0 0x7fe9d415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 >> 172.21.15.64:0/2066898273 conn(0x7fe9d40d30a0 msgr2=0x7fe9d40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:42.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 shutdown_connections 2023-11-24T08:07:42.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.324+0000 7fe9da8ea640 1 -- 172.21.15.64:0/2066898273 wait complete. 2023-11-24T08:07:42.369 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.552+0000 7f53a5583640 1 Processor -- start 2023-11-24T08:07:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.552+0000 7f53a5583640 1 -- start start 2023-11-24T08:07:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.553+0000 7f53a5583640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a015a100 0x7f53a00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.553+0000 7f53a5583640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f53a00abff0 0x7f53a00b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f53a5583640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a013da30 con 0x7f53a00ae7e0 2023-11-24T08:07:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f53a5583640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a00ac840 con 0x7f53a00abff0 2023-11-24T08:07:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f53a5583640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00ac560 con 0x7f53a015a100 2023-11-24T08:07:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f539f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f53a00ae7e0 0x7f53a006d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47528/0 (socket says 172.21.15.64:47528) 2023-11-24T08:07:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f539f7fe640 1 -- 172.21.15.64:0/3152225326 learned_addr learned my addr 172.21.15.64:0/3152225326 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f539effd640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a015a100 0x7f53a00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f539e7fc640 1 --2- 172.21.15.64:0/3152225326 >> v2:172.21.15.91:3300/0 conn(0x7f53a00abff0 0x7f53a00b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809423252 0 0) 0x7f53a013da30 con 0x7f53a00ae7e0 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.554+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5388005730 con 0x7f53a00ae7e0 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1160624896 0 0) 0x7f5388005730 con 0x7f53a00ae7e0 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a015a100 msgr2=0x7f53a00b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a015a100 0x7f53a00b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 >> v2:172.21.15.91:3300/0 conn(0x7f53a00abff0 msgr2=0x7f53a00b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 --2- 172.21.15.64:0/3152225326 >> v2:172.21.15.91:3300/0 conn(0x7f53a00abff0 0x7f53a00b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a0070650 con 0x7f53a00ae7e0 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f53980019f0 con 0x7f53a00ae7e0 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 >> v1:172.21.15.64:6789/0 conn(0x7f53a00ae7e0 legacy=0x7f53a006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5388005b80 0x7f5388005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53880069d0 0x7f5388006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a015a100 0x7f5388007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00ac840 con 0x7f5388005b80 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00ac560 con 0x7f53880069d0 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538800dbe0 con 0x7f53a015a100 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.555+0000 7f539effd640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5388005b80 0x7f5388005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.556+0000 7f539e7fc640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53880069d0 0x7f5388006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.556+0000 7f539effd640 1 -- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a015a100 msgr2=0x7f5388007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.556+0000 7f539effd640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a015a100 0x7f5388007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.556+0000 7f539effd640 1 -- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53880069d0 msgr2=0x7f5388006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.556+0000 7f539effd640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53880069d0 0x7f5388006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.556+0000 7f539effd640 1 -- 172.21.15.64:0/3152225326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5388005730 con 0x7f5388005b80 2023-11-24T08:07:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.556+0000 7f539e7fc640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f53880069d0 0x7f5388006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.557+0000 7f539effd640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5388005b80 0x7f5388005fa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5390009af0 tx=0x7f5388007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f2b4845889b7e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.557+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539002c710 con 0x7f5388005b80 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.557+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539002c870 con 0x7f5388005b80 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.557+0000 7f539dffb640 1 -- 172.21.15.64:0/3152225326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539002cc00 con 0x7f5388005b80 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.558+0000 7f53a5583640 1 -- 172.21.15.64:0/3152225326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5388005b80 msgr2=0x7f5388005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.558+0000 7f53a5583640 1 --2- 172.21.15.64:0/3152225326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5388005b80 0x7f5388005fa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5390009af0 tx=0x7f5388007a00 comp rx=0 tx=0).stop 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.558+0000 7f539f7fe640 1 -- 172.21.15.64:0/3152225326 reap_dead start 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.558+0000 7f53a5583640 1 -- 172.21.15.64:0/3152225326 shutdown_connections 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.558+0000 7f53a5583640 1 -- 172.21.15.64:0/3152225326 >> 172.21.15.64:0/3152225326 conn(0x7f53a00b0d80 msgr2=0x7f53a00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.558+0000 7f53a5583640 1 -- 172.21.15.64:0/3152225326 shutdown_connections 2023-11-24T08:07:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.558+0000 7f53a5583640 1 -- 172.21.15.64:0/3152225326 wait complete. 2023-11-24T08:07:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.560+0000 7f53a5583640 1 Processor -- start 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.560+0000 7f53a5583640 1 -- start start 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f53a5583640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5388005b80 0x7f53a00ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f53a5583640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a0153550 0x7f53a01557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f53a5583640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0153d20 0x7f53a01541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f53a5583640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00ce740 con 0x7f53a0153d20 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f53a5583640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00cd6c0 con 0x7f5388005b80 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f53a5583640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00cd9a0 con 0x7f53a0153550 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0153d20 0x7f53a01541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0153d20 0x7f53a01541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54770/0 (socket says 172.21.15.64:54770) 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f539f7fe640 1 -- 172.21.15.64:0/3232175792 learned_addr learned my addr 172.21.15.64:0/3232175792 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f539effd640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5388005b80 0x7f53a00ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.561+0000 7f539e7fc640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a0153550 0x7f53a01557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f539f7fe640 1 -- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a0153550 msgr2=0x7f53a01557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f539f7fe640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a0153550 0x7f53a01557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f539f7fe640 1 -- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5388005b80 msgr2=0x7f53a00ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f539f7fe640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5388005b80 0x7f53a00ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f539f7fe640 1 -- 172.21.15.64:0/3232175792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a00cdc80 con 0x7f53a0153d20 2023-11-24T08:07:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f539f7fe640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0153d20 0x7f53a01541d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f53a006bc90 tx=0x7f539800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f53877fe640 1 -- 172.21.15.64:0/3232175792 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539800ceb0 con 0x7f53a0153d20 2023-11-24T08:07:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f53877fe640 1 -- 172.21.15.64:0/3232175792 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539800a610 con 0x7f53a0153d20 2023-11-24T08:07:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f53877fe640 1 -- 172.21.15.64:0/3232175792 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539800a9c0 con 0x7f53a0153d20 2023-11-24T08:07:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a00cdf60 con 0x7f53a0153d20 2023-11-24T08:07:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.562+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a00cd2d0 con 0x7f53a0153d20 2023-11-24T08:07:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.563+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a00cd490 con 0x7f53a0153d20 2023-11-24T08:07:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.564+0000 7f53877fe640 1 -- 172.21.15.64:0/3232175792 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f539800ab20 con 0x7f53a0153d20 2023-11-24T08:07:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.565+0000 7f53877fe640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f538c06b230 0x7f538c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.565+0000 7f53877fe640 1 -- 172.21.15.64:0/3232175792 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5398089130 con 0x7f53a0153d20 2023-11-24T08:07:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.566+0000 7f539effd640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f538c06b230 0x7f538c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:42.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.567+0000 7f539effd640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f538c06b230 0x7f538c06d6f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5390036000 tx=0x7f5390030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:42.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.570+0000 7f53877fe640 1 -- 172.21.15.64:0/3232175792 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5398053c00 con 0x7f53a0153d20 2023-11-24T08:07:42.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.759+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53a0074770 con 0x7f53a0153d20 2023-11-24T08:07:42.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.762+0000 7f53877fe640 1 -- 172.21.15.64:0/3232175792 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f5398057220 con 0x7f53a0153d20 2023-11-24T08:07:42.764 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:42.764 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":73,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:41.804916+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:07:39.474762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:07:40.829840+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f538c06b230 msgr2=0x7f538c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f538c06b230 0x7f538c06d6f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5390036000 tx=0x7f5390030000 comp rx=0 tx=0).stop 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0153d20 msgr2=0x7f53a01541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0153d20 0x7f53a01541d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f53a006bc90 tx=0x7f539800cc00 comp rx=0 tx=0).stop 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 shutdown_connections 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f538c06b230 0x7f538c06d6f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0153d20 0x7f53a01541d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f53a0153550 0x7f53a01557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 --2- 172.21.15.64:0/3232175792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5388005b80 0x7f53a00ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.764+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 >> 172.21.15.64:0/3232175792 conn(0x7f53a00b0d80 msgr2=0x7f53a0071740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.765+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 shutdown_connections 2023-11-24T08:07:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:42.765+0000 7f53a5583640 1 -- 172.21.15.64:0/3232175792 wait complete. 2023-11-24T08:07:42.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.003+0000 7f130359e640 1 Processor -- start 2023-11-24T08:07:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.003+0000 7f130359e640 1 -- start start 2023-11-24T08:07:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.005+0000 7f130359e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f130415d460 0x7f13040b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.005+0000 7f130359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130406be70 0x7f13040cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.005+0000 7f130359e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13040cbf60 con 0x7f130415d840 2023-11-24T08:07:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.005+0000 7f130359e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13040cc520 con 0x7f130415d460 2023-11-24T08:07:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.005+0000 7f130359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13040d03b0 con 0x7f130406be70 2023-11-24T08:07:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f12fbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f130415d840 0x7f130406b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47544/0 (socket says 172.21.15.64:47544) 2023-11-24T08:07:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f12fbfff640 1 -- 172.21.15.64:0/1867624314 learned_addr learned my addr 172.21.15.64:0/1867624314 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f1302d9d640 1 --2- 172.21.15.64:0/1867624314 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130406be70 0x7f13040cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f130259c640 1 --2- 172.21.15.64:0/1867624314 >> v2:172.21.15.91:3300/0 conn(0x7f130415d460 0x7f13040b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f1301d9b640 1 -- 172.21.15.64:0/1867624314 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292407673 0 0) 0x7f13040cbf60 con 0x7f130415d840 2023-11-24T08:07:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f1301d9b640 1 -- 172.21.15.64:0/1867624314 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f0003520 con 0x7f130415d840 2023-11-24T08:07:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f1302d9d640 1 -- 172.21.15.64:0/1867624314 >> v2:172.21.15.91:3300/0 conn(0x7f130415d460 msgr2=0x7f13040b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f1302d9d640 1 --2- 172.21.15.64:0/1867624314 >> v2:172.21.15.91:3300/0 conn(0x7f130415d460 0x7f13040b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f1302d9d640 1 -- 172.21.15.64:0/1867624314 >> v1:172.21.15.64:6789/0 conn(0x7f130415d840 legacy=0x7f130406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.006+0000 7f1302d9d640 1 -- 172.21.15.64:0/1867624314 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130406c3c0 con 0x7f130406be70 2023-11-24T08:07:43.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.008+0000 7f1302d9d640 1 --2- 172.21.15.64:0/1867624314 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130406be70 0x7f13040cc9b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009d90 tx=0x7f12fc02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcf243299669bdbd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:43.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.008+0000 7f1301d9b640 1 -- 172.21.15.64:0/1867624314 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc02cc60 con 0x7f130406be70 2023-11-24T08:07:43.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.008+0000 7f1301d9b640 1 -- 172.21.15.64:0/1867624314 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc02cdc0 con 0x7f130406be70 2023-11-24T08:07:43.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.008+0000 7f1301d9b640 1 -- 172.21.15.64:0/1867624314 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc034660 con 0x7f130406be70 2023-11-24T08:07:43.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.009+0000 7f130359e640 1 -- 172.21.15.64:0/1867624314 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130406be70 msgr2=0x7f13040cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.009+0000 7f130359e640 1 --2- 172.21.15.64:0/1867624314 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130406be70 0x7f13040cc9b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009d90 tx=0x7f12fc02beb0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.009+0000 7f130359e640 1 -- 172.21.15.64:0/1867624314 shutdown_connections 2023-11-24T08:07:43.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.009+0000 7f130359e640 1 --2- 172.21.15.64:0/1867624314 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130406be70 0x7f13040cc9b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.009+0000 7f130359e640 1 --2- 172.21.15.64:0/1867624314 >> v2:172.21.15.91:3300/0 conn(0x7f130415d460 0x7f13040b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.009+0000 7f130359e640 1 -- 172.21.15.64:0/1867624314 >> 172.21.15.64:0/1867624314 conn(0x7f13040d0f00 msgr2=0x7f13040d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:43.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.009+0000 7f130359e640 1 -- 172.21.15.64:0/1867624314 shutdown_connections 2023-11-24T08:07:43.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.009+0000 7f130359e640 1 -- 172.21.15.64:0/1867624314 wait complete. 2023-11-24T08:07:43.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.012+0000 7f130359e640 1 Processor -- start 2023-11-24T08:07:43.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.012+0000 7f130359e640 1 -- start start 2023-11-24T08:07:43.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f130406be70 0x7f13040b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130359e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f130415d460 0x7f13040b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130415d840 0x7f13040b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13040b45f0 con 0x7f130406be70 2023-11-24T08:07:43.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130359e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13040b3a00 con 0x7f130415d460 2023-11-24T08:07:43.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1304150e70 con 0x7f130415d840 2023-11-24T08:07:43.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f130406be70 0x7f13040b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f130406be70 0x7f13040b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54774/0 (socket says 172.21.15.64:54774) 2023-11-24T08:07:43.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.013+0000 7f130259c640 1 -- 172.21.15.64:0/542093469 learned_addr learned my addr 172.21.15.64:0/542093469 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:43.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f12fbfff640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f130415d460 0x7f13040b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f1302d9d640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130415d840 0x7f13040b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f130259c640 1 -- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130415d840 msgr2=0x7f13040b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f130259c640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130415d840 0x7f13040b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f130259c640 1 -- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f130415d460 msgr2=0x7f13040b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f130259c640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f130415d460 0x7f13040b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f130259c640 1 -- 172.21.15.64:0/542093469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc009a70 con 0x7f130406be70 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f130259c640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f130406be70 0x7f13040b4a10 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f12f400e8a0 tx=0x7f12f400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.014+0000 7f12faffd640 1 -- 172.21.15.64:0/542093469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40094f0 con 0x7f130406be70 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.015+0000 7f12faffd640 1 -- 172.21.15.64:0/542093469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f400c940 con 0x7f130406be70 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.015+0000 7f12faffd640 1 -- 172.21.15.64:0/542093469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400ccf0 con 0x7f130406be70 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.015+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13040b3c50 con 0x7f130406be70 2023-11-24T08:07:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.015+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13040b4210 con 0x7f130406be70 2023-11-24T08:07:43.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.016+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130404e840 con 0x7f130406be70 2023-11-24T08:07:43.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.017+0000 7f12faffd640 1 -- 172.21.15.64:0/542093469 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f12f4013470 con 0x7f130406be70 2023-11-24T08:07:43.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.018+0000 7f12faffd640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12e406b230 0x7f12e406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.019+0000 7f12faffd640 1 -- 172.21.15.64:0/542093469 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12f4089d70 con 0x7f130406be70 2023-11-24T08:07:43.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.019+0000 7f12fbfff640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12e406b230 0x7f12e406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.020+0000 7f12fbfff640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12e406b230 0x7f12e406d6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f13040b34e0 tx=0x7f12ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:43.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.023+0000 7f12faffd640 1 -- 172.21.15.64:0/542093469 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12f40548d0 con 0x7f130406be70 2023-11-24T08:07:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.226+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f130406e170 con 0x7f130406be70 2023-11-24T08:07:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.230+0000 7f12faffd640 1 -- 172.21.15.64:0/542093469 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f12f4021090 con 0x7f130406be70 2023-11-24T08:07:43.232 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:43.232 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":73,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:41.804916+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:07:39.474762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:07:40.829840+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12e406b230 msgr2=0x7f12e406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12e406b230 0x7f12e406d6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f13040b34e0 tx=0x7f12ec009040 comp rx=0 tx=0).stop 2023-11-24T08:07:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f130406be70 msgr2=0x7f13040b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f130406be70 0x7f13040b4a10 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f12f400e8a0 tx=0x7f12f400ed60 comp rx=0 tx=0).stop 2023-11-24T08:07:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 shutdown_connections 2023-11-24T08:07:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f12e406b230 0x7f12e406d6f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f130415d840 0x7f13040b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f130415d460 0x7f13040b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 --2- 172.21.15.64:0/542093469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f130406be70 0x7f13040b4a10 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 >> 172.21.15.64:0/542093469 conn(0x7f13040d0f00 msgr2=0x7f130415df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 shutdown_connections 2023-11-24T08:07:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.233+0000 7f130359e640 1 -- 172.21.15.64:0/542093469 wait complete. 2023-11-24T08:07:43.255 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:07:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.487+0000 7fcd2ffff640 1 Processor -- start 2023-11-24T08:07:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.487+0000 7fcd2ffff640 1 -- start start 2023-11-24T08:07:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.489+0000 7fcd2ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcd300d1d90 0x7fcd300ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.489+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 0x7fcd300b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.489+0000 7fcd2ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300c6bf0 con 0x7fcd3015d3b0 2023-11-24T08:07:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.489+0000 7fcd2ffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd300d13f0 con 0x7fcd300d1d90 2023-11-24T08:07:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.489+0000 7fcd2ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300cedc0 con 0x7fcd300d0fa0 2023-11-24T08:07:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.489+0000 7fcd2effd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcd3015d3b0 0x7fcd300d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47552/0 (socket says 172.21.15.64:47552) 2023-11-24T08:07:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.489+0000 7fcd2effd640 1 -- 172.21.15.64:0/1752642280 learned_addr learned my addr 172.21.15.64:0/1752642280 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:43.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022574235 0 0) 0x7fcd300c6bf0 con 0x7fcd3015d3b0 2023-11-24T08:07:43.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd1c003520 con 0x7fcd3015d3b0 2023-11-24T08:07:43.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 0x7fcd300b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd27fff640 1 --2- 172.21.15.64:0/1752642280 >> v2:172.21.15.91:3300/0 conn(0x7fcd300d1d90 0x7fcd300ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2885764351 0 0) 0x7fcd1c003520 con 0x7fcd3015d3b0 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 msgr2=0x7fcd300b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 0x7fcd300b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 >> v2:172.21.15.91:3300/0 conn(0x7fcd300d1d90 msgr2=0x7fcd300ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 --2- 172.21.15.64:0/1752642280 >> v2:172.21.15.91:3300/0 conn(0x7fcd300d1d90 0x7fcd300ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300c6e40 con 0x7fcd3015d3b0 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fcd20001670 con 0x7fcd3015d3b0 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 0x7fcd300b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 >> v1:172.21.15.64:6789/0 conn(0x7fcd3015d3b0 legacy=0x7fcd300d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 0x7fcd1c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd1c004020 0x7fcd1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1c004d00 0x7fcd1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300d13f0 con 0x7fcd1c004d00 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300cedc0 con 0x7fcd1c004020 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.490+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c00bbc0 con 0x7fcd300d0fa0 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd27fff640 1 --2- 172.21.15.64:0/1752642280 >> v2:172.21.15.91:3300/0 conn(0x7fcd300d1d90 0x7fcd300ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1c004d00 0x7fcd1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd2effd640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 0x7fcd1c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd27fff640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd1c004020 0x7fcd1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd2f7fe640 1 -- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 msgr2=0x7fcd1c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd300d0fa0 0x7fcd1c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd2f7fe640 1 -- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd1c004020 msgr2=0x7fcd1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd1c004020 0x7fcd1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.491+0000 7fcd2f7fe640 1 -- 172.21.15.64:0/1752642280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c003520 con 0x7fcd1c004d00 2023-11-24T08:07:43.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.492+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1c004d00 0x7fcd1c005180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcd280073e0 tx=0x7fcd1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227c424a0dd7ae10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:43.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.493+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2802cd30 con 0x7fcd1c004d00 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.493+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2802ce90 con 0x7fcd1c004d00 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.493+0000 7fcd2e7fc640 1 -- 172.21.15.64:0/1752642280 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28034640 con 0x7fcd1c004d00 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.493+0000 7fcd2ffff640 1 -- 172.21.15.64:0/1752642280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1c004d00 msgr2=0x7fcd1c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.493+0000 7fcd2ffff640 1 --2- 172.21.15.64:0/1752642280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1c004d00 0x7fcd1c005180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcd280073e0 tx=0x7fcd1c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.493+0000 7fcd2f7fe640 1 -- 172.21.15.64:0/1752642280 reap_dead start 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.493+0000 7fcd2ffff640 1 -- 172.21.15.64:0/1752642280 shutdown_connections 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.493+0000 7fcd2ffff640 1 -- 172.21.15.64:0/1752642280 >> 172.21.15.64:0/1752642280 conn(0x7fcd300d30a0 msgr2=0x7fcd300d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.494+0000 7fcd2ffff640 1 -- 172.21.15.64:0/1752642280 shutdown_connections 2023-11-24T08:07:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.494+0000 7fcd2ffff640 1 -- 172.21.15.64:0/1752642280 wait complete. 2023-11-24T08:07:43.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.495+0000 7fcd2ffff640 1 Processor -- start 2023-11-24T08:07:43.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.496+0000 7fcd2ffff640 1 -- start start 2023-11-24T08:07:43.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.496+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd1c004d00 0x7fcd300b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.496+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd3015e190 0x7fcd301f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.496+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3006c2a0 0x7fcd3006c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.496+0000 7fcd2ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd301f3900 con 0x7fcd3006c2a0 2023-11-24T08:07:43.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.496+0000 7fcd2ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd301f3d50 con 0x7fcd3015e190 2023-11-24T08:07:43.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.496+0000 7fcd2ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b3990 con 0x7fcd1c004d00 2023-11-24T08:07:43.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3006c2a0 0x7fcd3006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3006c2a0 0x7fcd3006c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54788/0 (socket says 172.21.15.64:54788) 2023-11-24T08:07:43.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2f7fe640 1 -- 172.21.15.64:0/2921391870 learned_addr learned my addr 172.21.15.64:0/2921391870 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:43.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd27fff640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd3015e190 0x7fcd301f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2effd640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd1c004d00 0x7fcd300b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2f7fe640 1 -- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd1c004d00 msgr2=0x7fcd300b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd1c004d00 0x7fcd300b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2f7fe640 1 -- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd3015e190 msgr2=0x7fcd301f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd3015e190 0x7fcd301f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd2f7fe640 1 -- 172.21.15.64:0/2921391870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3006cc60 con 0x7fcd3006c2a0 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.497+0000 7fcd27fff640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd3015e190 0x7fcd301f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.498+0000 7fcd2f7fe640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3006c2a0 0x7fcd3006c720 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fcd28037040 tx=0x7fcd28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.498+0000 7fcd277fe640 1 -- 172.21.15.64:0/2921391870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28046070 con 0x7fcd3006c2a0 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.498+0000 7fcd277fe640 1 -- 172.21.15.64:0/2921391870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28035e10 con 0x7fcd3006c2a0 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.498+0000 7fcd277fe640 1 -- 172.21.15.64:0/2921391870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28003b90 con 0x7fcd3006c2a0 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.498+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300b58b0 con 0x7fcd3006c2a0 2023-11-24T08:07:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.498+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd301f3060 con 0x7fcd3006c2a0 2023-11-24T08:07:43.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.499+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd301f3ab0 con 0x7fcd3006c2a0 2023-11-24T08:07:43.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.500+0000 7fcd277fe640 1 -- 172.21.15.64:0/2921391870 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcd2803b050 con 0x7fcd3006c2a0 2023-11-24T08:07:43.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.501+0000 7fcd277fe640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd1006b260 0x7fcd1006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:43.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.501+0000 7fcd277fe640 1 -- 172.21.15.64:0/2921391870 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd280ab5e0 con 0x7fcd3006c2a0 2023-11-24T08:07:43.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.502+0000 7fcd2effd640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd1006b260 0x7fcd1006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:43.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.503+0000 7fcd2effd640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd1006b260 0x7fcd1006d720 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcd20001050 tx=0x7fcd20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:43.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.506+0000 7fcd277fe640 1 -- 172.21.15.64:0/2921391870 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd301f3ab0 con 0x7fcd3006c2a0 2023-11-24T08:07:43.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:43.737+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcd301f3e90 con 0x7fcd3006c2a0 2023-11-24T08:07:44.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.366+0000 7fcd277fe640 1 -- 172.21.15.64:0/2921391870 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcd301f3e90 con 0x7fcd3006c2a0 2023-11-24T08:07:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd1006b260 msgr2=0x7fcd1006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd1006b260 0x7fcd1006d720 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcd20001050 tx=0x7fcd20009040 comp rx=0 tx=0).stop 2023-11-24T08:07:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3006c2a0 msgr2=0x7fcd3006c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3006c2a0 0x7fcd3006c720 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fcd28037040 tx=0x7fcd28004ab0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 shutdown_connections 2023-11-24T08:07:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd1006b260 0x7fcd1006d720 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3006c2a0 0x7fcd3006c720 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcd3015e190 0x7fcd301f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 --2- 172.21.15.64:0/2921391870 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcd1c004d00 0x7fcd300b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 >> 172.21.15.64:0/2921391870 conn(0x7fcd300d30a0 msgr2=0x7fcd300c8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 shutdown_connections 2023-11-24T08:07:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.370+0000 7fcd2ffff640 1 -- 172.21.15.64:0/2921391870 wait complete. 2023-11-24T08:07:44.392 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:07:44.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:07:44.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.614+0000 7fa378bff640 1 Processor -- start 2023-11-24T08:07:44.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.614+0000 7fa378bff640 1 -- start start 2023-11-24T08:07:44.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.615+0000 7fa378bff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa37415d3b0 0x7fa3740ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:44.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.616+0000 7fa378bff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3740d1d90 0x7fa3740b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:44.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.616+0000 7fa378bff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3740c6bf0 con 0x7fa3740d0fa0 2023-11-24T08:07:44.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.616+0000 7fa378bff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3740d1400 con 0x7fa3740d1d90 2023-11-24T08:07:44.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.616+0000 7fa378bff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740cedc0 con 0x7fa37415d3b0 2023-11-24T08:07:44.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.616+0000 7fa373fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3740d1d90 0x7fa3740b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:44.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.616+0000 7fa372ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa3740d0fa0 0x7fa3740d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47554/0 (socket says 172.21.15.64:47554) 2023-11-24T08:07:44.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.616+0000 7fa372ffd640 1 -- 172.21.15.64:0/3498824427 learned_addr learned my addr 172.21.15.64:0/3498824427 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:44.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.616+0000 7fa3737fe640 1 --2- 172.21.15.64:0/3498824427 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa37415d3b0 0x7fa3740ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:44.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.617+0000 7fa3727fc640 1 -- 172.21.15.64:0/3498824427 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226847497 0 0) 0x7fa3740c6bf0 con 0x7fa3740d0fa0 2023-11-24T08:07:44.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.617+0000 7fa373fff640 1 -- 172.21.15.64:0/3498824427 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa37415d3b0 msgr2=0x7fa3740ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:44.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.617+0000 7fa373fff640 1 --2- 172.21.15.64:0/3498824427 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa37415d3b0 0x7fa3740ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.617+0000 7fa373fff640 1 -- 172.21.15.64:0/3498824427 >> v1:172.21.15.64:6789/0 conn(0x7fa3740d0fa0 legacy=0x7fa3740d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:44.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.617+0000 7fa373fff640 1 -- 172.21.15.64:0/3498824427 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740c6e40 con 0x7fa3740d1d90 2023-11-24T08:07:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.618+0000 7fa373fff640 1 --2- 172.21.15.64:0/3498824427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3740d1d90 0x7fa3740b3ca0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0097a0 tx=0x7fa36c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807a5c3740911c3a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.618+0000 7fa3727fc640 1 -- 172.21.15.64:0/3498824427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c031780 con 0x7fa3740d1d90 2023-11-24T08:07:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.618+0000 7fa3727fc640 1 -- 172.21.15.64:0/3498824427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c0318e0 con 0x7fa3740d1d90 2023-11-24T08:07:44.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.618+0000 7fa3727fc640 1 -- 172.21.15.64:0/3498824427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c031c70 con 0x7fa3740d1d90 2023-11-24T08:07:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.619+0000 7fa378bff640 1 -- 172.21.15.64:0/3498824427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3740d1d90 msgr2=0x7fa3740b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.619+0000 7fa378bff640 1 --2- 172.21.15.64:0/3498824427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3740d1d90 0x7fa3740b3ca0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0097a0 tx=0x7fa36c02b820 comp rx=0 tx=0).stop 2023-11-24T08:07:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.619+0000 7fa378bff640 1 -- 172.21.15.64:0/3498824427 shutdown_connections 2023-11-24T08:07:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.619+0000 7fa378bff640 1 --2- 172.21.15.64:0/3498824427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3740d1d90 0x7fa3740b3ca0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.619+0000 7fa378bff640 1 --2- 172.21.15.64:0/3498824427 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa37415d3b0 0x7fa3740ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.619+0000 7fa378bff640 1 -- 172.21.15.64:0/3498824427 >> 172.21.15.64:0/3498824427 conn(0x7fa3740d30a0 msgr2=0x7fa3740d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.619+0000 7fa378bff640 1 -- 172.21.15.64:0/3498824427 shutdown_connections 2023-11-24T08:07:44.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.619+0000 7fa378bff640 1 -- 172.21.15.64:0/3498824427 wait complete. 2023-11-24T08:07:44.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.621+0000 7fa378bff640 1 Processor -- start 2023-11-24T08:07:44.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.621+0000 7fa378bff640 1 -- start start 2023-11-24T08:07:44.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa378bff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3740d0fa0 0x7fa37414d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:44.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa378bff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3740d1d90 0x7fa3740b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa378bff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37415d3b0 0x7fa3740b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa378bff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37415e870 con 0x7fa37415d3b0 2023-11-24T08:07:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa378bff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3741f2bc0 con 0x7fa3740d0fa0 2023-11-24T08:07:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa378bff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37415d910 con 0x7fa3740d1d90 2023-11-24T08:07:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa373fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37415d3b0 0x7fa3740b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa373fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37415d3b0 0x7fa3740b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54818/0 (socket says 172.21.15.64:54818) 2023-11-24T08:07:44.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.622+0000 7fa373fff640 1 -- 172.21.15.64:0/3006313145 learned_addr learned my addr 172.21.15.64:0/3006313145 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa372ffd640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3740d1d90 0x7fa3740b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa3737fe640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3740d0fa0 0x7fa37414d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa373fff640 1 -- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3740d1d90 msgr2=0x7fa3740b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa373fff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3740d1d90 0x7fa3740b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa373fff640 1 -- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3740d0fa0 msgr2=0x7fa37414d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa373fff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3740d0fa0 0x7fa37414d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:44.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa373fff640 1 -- 172.21.15.64:0/3006313145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c009400 con 0x7fa37415d3b0 2023-11-24T08:07:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa373fff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37415d3b0 0x7fa3740b61a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa36c034040 tx=0x7fa36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.623+0000 7fa36bfff640 1 -- 172.21.15.64:0/3006313145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c032820 con 0x7fa37415d3b0 2023-11-24T08:07:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.624+0000 7fa36bfff640 1 -- 172.21.15.64:0/3006313145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c032980 con 0x7fa37415d3b0 2023-11-24T08:07:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.624+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37415dbf0 con 0x7fa37415d3b0 2023-11-24T08:07:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.624+0000 7fa36bfff640 1 -- 172.21.15.64:0/3006313145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c032d30 con 0x7fa37415d3b0 2023-11-24T08:07:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.624+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37415e1b0 con 0x7fa37415d3b0 2023-11-24T08:07:44.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.625+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37415e9b0 con 0x7fa37415d3b0 2023-11-24T08:07:44.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.626+0000 7fa36bfff640 1 -- 172.21.15.64:0/3006313145 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa36c04e450 con 0x7fa37415d3b0 2023-11-24T08:07:44.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.627+0000 7fa36bfff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa35406b260 0x7fa35406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:44.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.627+0000 7fa36bfff640 1 -- 172.21.15.64:0/3006313145 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa36c0abe00 con 0x7fa37415d3b0 2023-11-24T08:07:44.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.627+0000 7fa3737fe640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa35406b260 0x7fa35406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:44.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.628+0000 7fa3737fe640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa35406b260 0x7fa35406d720 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa364009970 tx=0x7fa364008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:44.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.631+0000 7fa36bfff640 1 -- 172.21.15.64:0/3006313145 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa37415e9b0 con 0x7fa37415d3b0 2023-11-24T08:07:44.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:44.863+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa37415e9b0 con 0x7fa37415d3b0 2023-11-24T08:07:45.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.370+0000 7fa36bfff640 1 -- 172.21.15.64:0/3006313145 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa37415e9b0 con 0x7fa37415d3b0 2023-11-24T08:07:45.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa35406b260 msgr2=0x7fa35406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:45.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa35406b260 0x7fa35406d720 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa364009970 tx=0x7fa364008040 comp rx=0 tx=0).stop 2023-11-24T08:07:45.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37415d3b0 msgr2=0x7fa3740b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:45.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37415d3b0 0x7fa3740b61a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa36c034040 tx=0x7fa36c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 shutdown_connections 2023-11-24T08:07:45.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa35406b260 0x7fa35406d720 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37415d3b0 0x7fa3740b61a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3740d1d90 0x7fa3740b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.374+0000 7fa378bff640 1 --2- 172.21.15.64:0/3006313145 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3740d0fa0 0x7fa37414d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.375+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 >> 172.21.15.64:0/3006313145 conn(0x7fa3740d30a0 msgr2=0x7fa3740d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:45.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.375+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 shutdown_connections 2023-11-24T08:07:45.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.375+0000 7fa378bff640 1 -- 172.21.15.64:0/3006313145 wait complete. 2023-11-24T08:07:45.400 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:07:45.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.630+0000 7f423214e640 1 Processor -- start 2023-11-24T08:07:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.630+0000 7f423214e640 1 -- start start 2023-11-24T08:07:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.632+0000 7f423214e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f422c13dfd0 0x7f422c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.632+0000 7f423214e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.632+0000 7f423214e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0ce7c0 con 0x7f422c0cf050 2023-11-24T08:07:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.632+0000 7f423214e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c140900 con 0x7f422c13dfd0 2023-11-24T08:07:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.632+0000 7f423214e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c15fa70 con 0x7f422c0cf500 2023-11-24T08:07:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.632+0000 7f423114c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f422c13dfd0 0x7f422c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.632+0000 7f423114c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f422c13dfd0 0x7f422c15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54830/0 (socket says 172.21.15.64:54830) 2023-11-24T08:07:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.632+0000 7f423114c640 1 -- 172.21.15.64:0/2941954848 learned_addr learned my addr 172.21.15.64:0/2941954848 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.633+0000 7f423194d640 1 --2- 172.21.15.64:0/2941954848 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.633+0000 7f423114c640 1 -- 172.21.15.64:0/2941954848 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 msgr2=0x7f422c06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.633+0000 7f423114c640 1 --2- 172.21.15.64:0/2941954848 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c06c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.633+0000 7f423114c640 1 -- 172.21.15.64:0/2941954848 >> v1:172.21.15.91:6789/0 conn(0x7f422c0cf050 legacy=0x7f422c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.633+0000 7f423114c640 1 -- 172.21.15.64:0/2941954848 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0cf950 con 0x7f422c13dfd0 2023-11-24T08:07:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.634+0000 7f423114c640 1 --2- 172.21.15.64:0/2941954848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c13dfd0 0x7f422c15be70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f422c0cfc10 tx=0x7f422802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d366c63cb33e71e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.634+0000 7f4223fff640 1 -- 172.21.15.64:0/2941954848 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228031830 con 0x7f422c13dfd0 2023-11-24T08:07:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.634+0000 7f4223fff640 1 -- 172.21.15.64:0/2941954848 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4228031990 con 0x7f422c13dfd0 2023-11-24T08:07:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.634+0000 7f4223fff640 1 -- 172.21.15.64:0/2941954848 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228031d20 con 0x7f422c13dfd0 2023-11-24T08:07:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.635+0000 7f423214e640 1 -- 172.21.15.64:0/2941954848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c13dfd0 msgr2=0x7f422c15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.635+0000 7f423214e640 1 --2- 172.21.15.64:0/2941954848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c13dfd0 0x7f422c15be70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f422c0cfc10 tx=0x7f422802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.635+0000 7f423214e640 1 -- 172.21.15.64:0/2941954848 shutdown_connections 2023-11-24T08:07:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.635+0000 7f423214e640 1 --2- 172.21.15.64:0/2941954848 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.635+0000 7f423214e640 1 --2- 172.21.15.64:0/2941954848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c13dfd0 0x7f422c15be70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.635+0000 7f423214e640 1 -- 172.21.15.64:0/2941954848 >> 172.21.15.64:0/2941954848 conn(0x7f422c06a860 msgr2=0x7f422c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.635+0000 7f423214e640 1 -- 172.21.15.64:0/2941954848 shutdown_connections 2023-11-24T08:07:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.635+0000 7f423214e640 1 -- 172.21.15.64:0/2941954848 wait complete. 2023-11-24T08:07:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.637+0000 7f423214e640 1 Processor -- start 2023-11-24T08:07:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.637+0000 7f423214e640 1 -- start start 2023-11-24T08:07:45.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.638+0000 7f423214e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c0cf050 0x7f422c154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:45.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423214e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:45.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423214e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f422c13dfd0 0x7f422c0b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:45.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423214e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c137760 con 0x7f422c0cf050 2023-11-24T08:07:45.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423214e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c137a40 con 0x7f422c13dfd0 2023-11-24T08:07:45.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423214e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c158320 con 0x7f422c0cf500 2023-11-24T08:07:45.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423114c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c0cf050 0x7f422c154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:45.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423114c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c0cf050 0x7f422c154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54844/0 (socket says 172.21.15.64:54844) 2023-11-24T08:07:45.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423114c640 1 -- 172.21.15.64:0/1014042151 learned_addr learned my addr 172.21.15.64:0/1014042151 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:45.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423114c640 1 -- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 msgr2=0x7f422c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423094b640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423194d640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f422c13dfd0 0x7f422c0b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423114c640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c15dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423114c640 1 -- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f422c13dfd0 msgr2=0x7f422c0b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423114c640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f422c13dfd0 0x7f422c0b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423114c640 1 -- 172.21.15.64:0/1014042151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42280094d0 con 0x7f422c0cf050 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.639+0000 7f423094b640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.640+0000 7f423114c640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c0cf050 0x7f422c154f10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f422802b910 tx=0x7f42280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.640+0000 7f4221ffb640 1 -- 172.21.15.64:0/1014042151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42280317d0 con 0x7f422c0cf050 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.640+0000 7f4221ffb640 1 -- 172.21.15.64:0/1014042151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422803aa60 con 0x7f422c0cf050 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.640+0000 7f4221ffb640 1 -- 172.21.15.64:0/1014042151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422803ae10 con 0x7f422c0cf050 2023-11-24T08:07:45.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.640+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c158600 con 0x7f422c0cf050 2023-11-24T08:07:45.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.640+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c155450 con 0x7f422c0cf050 2023-11-24T08:07:45.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.641+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c04e840 con 0x7f422c0cf050 2023-11-24T08:07:45.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.642+0000 7f4221ffb640 1 -- 172.21.15.64:0/1014042151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f42280443f0 con 0x7f422c0cf050 2023-11-24T08:07:45.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.643+0000 7f4221ffb640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f421006b230 0x7f421006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:45.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.643+0000 7f4221ffb640 1 -- 172.21.15.64:0/1014042151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42280ac0b0 con 0x7f422c0cf050 2023-11-24T08:07:45.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.644+0000 7f423094b640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f421006b230 0x7f421006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:45.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.645+0000 7f423094b640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f421006b230 0x7f421006d6f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f422c0b1020 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:45.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.648+0000 7f4221ffb640 1 -- 172.21.15.64:0/1014042151 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4228076b90 con 0x7f422c0cf050 2023-11-24T08:07:45.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:07:45.768 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:07:45.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.880+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f422c0ce9a0 con 0x7f422c0cf050 2023-11-24T08:07:45.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.881+0000 7f4221ffb640 1 -- 172.21.15.64:0/1014042151 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4909 (secure 0 0 0) 0x7f4228035090 con 0x7f422c0cf050 2023-11-24T08:07:45.883 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:45.883 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3149637441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3149637441},{"type":"v1","addr":"172.21.15.91:6836","nonce":3149637441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2159744284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2159744284},{"type":"v1","addr":"172.21.15.91:6837","nonce":2159744284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5764,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1010090226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1010090226},{"type":"v1","addr":"172.21.15.64:6835","nonce":1010090226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:07:41.805482+0000","modified":"2023-11-24T08:07:44.865599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6684},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6684":{"gid":6684,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/45948764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":45948764},{"type":"v1","addr":"172.21.15.64:6833","nonce":45948764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:07:45.884 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 43 2023-11-24T08:07:45.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f421006b230 msgr2=0x7f421006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:45.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f421006b230 0x7f421006d6f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f422c0b1020 tx=0x7f421c008040 comp rx=0 tx=0).stop 2023-11-24T08:07:45.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c0cf050 msgr2=0x7f422c154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:45.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c0cf050 0x7f422c154f10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f422802b910 tx=0x7f42280046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 shutdown_connections 2023-11-24T08:07:45.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f421006b230 0x7f421006d6f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f422c13dfd0 0x7f422c0b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f422c0cf500 0x7f422c15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 --2- 172.21.15.64:0/1014042151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f422c0cf050 0x7f422c154f10 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:45.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 >> 172.21.15.64:0/1014042151 conn(0x7f422c06a860 msgr2=0x7f422c1598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:45.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 shutdown_connections 2023-11-24T08:07:45.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:45.884+0000 7f423214e640 1 -- 172.21.15.64:0/1014042151 wait complete. 2023-11-24T08:07:45.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.179+0000 7f3d2bfff640 1 Processor -- start 2023-11-24T08:07:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.179+0000 7f3d2bfff640 1 -- start start 2023-11-24T08:07:46.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.181+0000 7f3d2bfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f3d2c0cf500 0x7f3d2c0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.181+0000 7f3d2bfff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c140900 con 0x7f3d2c0cf050 2023-11-24T08:07:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.181+0000 7f3d2bfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c15fa70 con 0x7f3d2c13dfd0 2023-11-24T08:07:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.181+0000 7f3d2bfff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2c15c580 con 0x7f3d2c0cf500 2023-11-24T08:07:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.181+0000 7f3d2a7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3d2c0cf050 0x7f3d2c0cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47570/0 (socket says 172.21.15.64:47570) 2023-11-24T08:07:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.181+0000 7f3d2a7fc640 1 -- 172.21.15.64:0/1219586379 learned_addr learned my addr 172.21.15.64:0/1219586379 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:46.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.181+0000 7f3d2b7fe640 1 --2- 172.21.15.64:0/1219586379 >> v2:172.21.15.91:3300/0 conn(0x7f3d2c0cf500 0x7f3d2c0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508145016 0 0) 0x7f3d2c140900 con 0x7f3d2c0cf050 2023-11-24T08:07:46.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d14003520 con 0x7f3d2c0cf050 2023-11-24T08:07:46.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1555726717 0 0) 0x7f3d14003520 con 0x7f3d2c0cf050 2023-11-24T08:07:46.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 >> v2:172.21.15.91:3300/0 conn(0x7f3d2c0cf500 msgr2=0x7f3d2c0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:46.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 --2- 172.21.15.64:0/1219586379 >> v2:172.21.15.91:3300/0 conn(0x7f3d2c0cf500 0x7f3d2c0cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 >> v1:172.21.15.91:6789/0 conn(0x7f3d2c13dfd0 legacy=0x7f3d2c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c15ee60 con 0x7f3d2c0cf050 2023-11-24T08:07:46.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f3d20001670 con 0x7f3d2c0cf050 2023-11-24T08:07:46.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 >> v1:172.21.15.64:6789/0 conn(0x7f3d2c0cf050 legacy=0x7f3d2c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d140039e0 0x7f3d14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d140047c0 0x7f3d14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d14005180 0x7f3d1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c15c580 con 0x7f3d140047c0 2023-11-24T08:07:46.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1400c0b0 con 0x7f3d140039e0 2023-11-24T08:07:46.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1400c270 con 0x7f3d14005180 2023-11-24T08:07:46.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.182+0000 7f3d2a7fc640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d140047c0 0x7f3d14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.183+0000 7f3d2affd640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d140039e0 0x7f3d14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.183+0000 7f3d2a7fc640 1 -- 172.21.15.64:0/1219586379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d14005180 msgr2=0x7f3d1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:46.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.183+0000 7f3d2a7fc640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d14005180 0x7f3d1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.183+0000 7f3d2a7fc640 1 -- 172.21.15.64:0/1219586379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d140039e0 msgr2=0x7f3d14003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:46.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.183+0000 7f3d2a7fc640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d140039e0 0x7f3d14003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.183+0000 7f3d2a7fc640 1 -- 172.21.15.64:0/1219586379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14003520 con 0x7f3d140047c0 2023-11-24T08:07:46.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.183+0000 7f3d2b7fe640 1 -- 172.21.15.64:0/1219586379 reap_dead start 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.184+0000 7f3d2a7fc640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d140047c0 0x7f3d14004c40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000c190 tx=0x7f3d140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c81a27e93d1d2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.184+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20005ad0 con 0x7f3d140047c0 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.184+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d20005c30 con 0x7f3d140047c0 2023-11-24T08:07:46.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.184+0000 7f3d29ffb640 1 -- 172.21.15.64:0/1219586379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2002cab0 con 0x7f3d140047c0 2023-11-24T08:07:46.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.185+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1219586379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d140047c0 msgr2=0x7f3d14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.185+0000 7f3d2bfff640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d140047c0 0x7f3d14004c40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000c190 tx=0x7f3d140057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.185+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1219586379 shutdown_connections 2023-11-24T08:07:46.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.185+0000 7f3d2bfff640 1 --2- 172.21.15.64:0/1219586379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d140047c0 0x7f3d14004c40 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.185+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1219586379 >> 172.21.15.64:0/1219586379 conn(0x7f3d2c06a860 msgr2=0x7f3d2c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:46.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.185+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1219586379 shutdown_connections 2023-11-24T08:07:46.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.185+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1219586379 wait complete. 2023-11-24T08:07:46.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.187+0000 7f3d2bfff640 1 Processor -- start 2023-11-24T08:07:46.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.187+0000 7f3d2bfff640 1 -- start start 2023-11-24T08:07:46.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.188+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d140047c0 0x7f3d2c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.188+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d2c1553a0 0x7f3d2c13edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.188+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d2c1f2a10 0x7f3d2c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.188+0000 7f3d2bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c13ae10 con 0x7f3d2c1553a0 2023-11-24T08:07:46.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.188+0000 7f3d2bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c1f2e90 con 0x7f3d2c1f2a10 2023-11-24T08:07:46.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.188+0000 7f3d2bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c13a200 con 0x7f3d140047c0 2023-11-24T08:07:46.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d2c1553a0 0x7f3d2c13edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d2c1553a0 0x7f3d2c13edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54868/0 (socket says 172.21.15.64:54868) 2023-11-24T08:07:46.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 -- 172.21.15.64:0/1153865228 learned_addr learned my addr 172.21.15.64:0/1153865228 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:46.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2affd640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d140047c0 0x7f3d2c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2b7fe640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d2c1f2a10 0x7f3d2c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 -- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d140047c0 msgr2=0x7f3d2c158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d140047c0 0x7f3d2c158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 -- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d2c1f2a10 msgr2=0x7f3d2c0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d2c1f2a10 0x7f3d2c0af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 -- 172.21.15.64:0/1153865228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c13a4e0 con 0x7f3d2c1553a0 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.189+0000 7f3d2a7fc640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d2c1553a0 0x7f3d2c13edc0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3d20001930 tx=0x7f3d2002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.190+0000 7f3d137fe640 1 -- 172.21.15.64:0/1153865228 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2003c040 con 0x7f3d2c1553a0 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.190+0000 7f3d137fe640 1 -- 172.21.15.64:0/1153865228 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2002cd40 con 0x7f3d2c1553a0 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.190+0000 7f3d137fe640 1 -- 172.21.15.64:0/1153865228 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20007290 con 0x7f3d2c1553a0 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.190+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c13a7c0 con 0x7f3d2c1553a0 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.190+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c139eb0 con 0x7f3d2c1553a0 2023-11-24T08:07:46.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.191+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c04e840 con 0x7f3d2c1553a0 2023-11-24T08:07:46.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.192+0000 7f3d137fe640 1 -- 172.21.15.64:0/1153865228 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3d20032070 con 0x7f3d2c1553a0 2023-11-24T08:07:46.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.193+0000 7f3d137fe640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3d1806b230 0x7f3d1806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.193+0000 7f3d137fe640 1 -- 172.21.15.64:0/1153865228 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d200ab240 con 0x7f3d2c1553a0 2023-11-24T08:07:46.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.194+0000 7f3d2affd640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3d1806b230 0x7f3d1806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.195+0000 7f3d2affd640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3d1806b230 0x7f3d1806d6f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c000c00 tx=0x7f3d1c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:46.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.198+0000 7f3d137fe640 1 -- 172.21.15.64:0/1153865228 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d20075d10 con 0x7f3d2c1553a0 2023-11-24T08:07:46.277 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-11-24T08:07:46.275+0000 7f6d66f38640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:07:46.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.392+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d2c06e0a0 con 0x7f3d2c1553a0 2023-11-24T08:07:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.395+0000 7f3d137fe640 1 -- 172.21.15.64:0/1153865228 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16520 (secure 0 0 0) 0x7f3d20079330 con 0x7f3d2c1553a0 2023-11-24T08:07:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:46.397 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":74,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:43.341796+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:07:39.474762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:07:40.829840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3d1806b230 msgr2=0x7f3d1806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3d1806b230 0x7f3d1806d6f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c000c00 tx=0x7f3d1c009040 comp rx=0 tx=0).stop 2023-11-24T08:07:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d2c1553a0 msgr2=0x7f3d2c13edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d2c1553a0 0x7f3d2c13edc0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3d20001930 tx=0x7f3d2002e070 comp rx=0 tx=0).stop 2023-11-24T08:07:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 shutdown_connections 2023-11-24T08:07:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3d1806b230 0x7f3d1806d6f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3d2c1f2a10 0x7f3d2c0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d2c1553a0 0x7f3d2c13edc0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 --2- 172.21.15.64:0/1153865228 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3d140047c0 0x7f3d2c158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 >> 172.21.15.64:0/1153865228 conn(0x7f3d2c06a860 msgr2=0x7f3d2c13cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 shutdown_connections 2023-11-24T08:07:46.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.398+0000 7f3d2bfff640 1 -- 172.21.15.64:0/1153865228 wait complete. 2023-11-24T08:07:46.420 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:07:46.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:07:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.637+0000 7f3f9edec640 1 Processor -- start 2023-11-24T08:07:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.637+0000 7f3f9edec640 1 -- start start 2023-11-24T08:07:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.638+0000 7f3f9edec640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f9806d520 0x7f3f9806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.638+0000 7f3f9edec640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 0x7f3f9815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.638+0000 7f3f9edec640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f9815c360 con 0x7f3f980ad720 2023-11-24T08:07:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.638+0000 7f3f9edec640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980b41a0 con 0x7f3f9806d520 2023-11-24T08:07:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.638+0000 7f3f9edec640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9806de40 con 0x7f3f980abff0 2023-11-24T08:07:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.639+0000 7f3f9cb88640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f9806d520 0x7f3f9806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.639+0000 7f3f9cb88640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f9806d520 0x7f3f9806d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54880/0 (socket says 172.21.15.64:54880) 2023-11-24T08:07:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.639+0000 7f3f9cb88640 1 -- 172.21.15.64:0/2133202291 learned_addr learned my addr 172.21.15.64:0/2133202291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.639+0000 7f3f9d389640 1 --2- 172.21.15.64:0/2133202291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 0x7f3f9815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.639+0000 7f3f9cb88640 1 -- 172.21.15.64:0/2133202291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 msgr2=0x7f3f9815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.639+0000 7f3f9cb88640 1 --2- 172.21.15.64:0/2133202291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 0x7f3f9815f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.639+0000 7f3f9cb88640 1 -- 172.21.15.64:0/2133202291 >> v1:172.21.15.91:6789/0 conn(0x7f3f980ad720 legacy=0x7f3f9815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.639+0000 7f3f9cb88640 1 -- 172.21.15.64:0/2133202291 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9815ba30 con 0x7f3f9806d520 2023-11-24T08:07:46.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.641+0000 7f3f9cb88640 1 --2- 172.21.15.64:0/2133202291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f9806d520 0x7f3f9806d900 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009870 tx=0x7f3f9402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d2e4f43c215c44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:46.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.641+0000 7f3f8f7fe640 1 -- 172.21.15.64:0/2133202291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94031640 con 0x7f3f9806d520 2023-11-24T08:07:46.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.641+0000 7f3f8f7fe640 1 -- 172.21.15.64:0/2133202291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f940317a0 con 0x7f3f9806d520 2023-11-24T08:07:46.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.641+0000 7f3f8f7fe640 1 -- 172.21.15.64:0/2133202291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94031b30 con 0x7f3f9806d520 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.642+0000 7f3f9edec640 1 -- 172.21.15.64:0/2133202291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f9806d520 msgr2=0x7f3f9806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.642+0000 7f3f9edec640 1 --2- 172.21.15.64:0/2133202291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f9806d520 0x7f3f9806d900 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009870 tx=0x7f3f9402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.642+0000 7f3f9edec640 1 -- 172.21.15.64:0/2133202291 shutdown_connections 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.642+0000 7f3f9edec640 1 --2- 172.21.15.64:0/2133202291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 0x7f3f9815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.642+0000 7f3f9edec640 1 --2- 172.21.15.64:0/2133202291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f9806d520 0x7f3f9806d900 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.642+0000 7f3f9edec640 1 -- 172.21.15.64:0/2133202291 >> 172.21.15.64:0/2133202291 conn(0x7f3f980b0d80 msgr2=0x7f3f980b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.642+0000 7f3f9edec640 1 -- 172.21.15.64:0/2133202291 shutdown_connections 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.642+0000 7f3f9edec640 1 -- 172.21.15.64:0/2133202291 wait complete. 2023-11-24T08:07:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.644+0000 7f3f9edec640 1 Processor -- start 2023-11-24T08:07:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.644+0000 7f3f9edec640 1 -- start start 2023-11-24T08:07:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.645+0000 7f3f9edec640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f9806d520 0x7f3f9814a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.645+0000 7f3f9edec640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 0x7f3f980cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.645+0000 7f3f9edec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f980ad720 0x7f3f980cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.645+0000 7f3f9edec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980cbe10 con 0x7f3f980ad720 2023-11-24T08:07:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.645+0000 7f3f9edec640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980cc0f0 con 0x7f3f9806d520 2023-11-24T08:07:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.645+0000 7f3f9edec640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980cc3d0 con 0x7f3f980abff0 2023-11-24T08:07:46.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.645+0000 7f3f9d389640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f980ad720 0x7f3f980cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.645+0000 7f3f9d389640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f980ad720 0x7f3f980cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54892/0 (socket says 172.21.15.64:54892) 2023-11-24T08:07:46.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9d389640 1 -- 172.21.15.64:0/4052360462 learned_addr learned my addr 172.21.15.64:0/4052360462 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:46.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f8ffff640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 0x7f3f980cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9cb88640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f9806d520 0x7f3f9814a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9d389640 1 -- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 msgr2=0x7f3f980cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9d389640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 0x7f3f980cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9d389640 1 -- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f9806d520 msgr2=0x7f3f9814a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9d389640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f9806d520 0x7f3f9814a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9d389640 1 -- 172.21.15.64:0/4052360462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940094d0 con 0x7f3f980ad720 2023-11-24T08:07:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9cb88640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f9806d520 0x7f3f9814a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f9d389640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f980ad720 0x7f3f980cdb80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007e30 tx=0x7f3f9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f8d7fa640 1 -- 172.21.15.64:0/4052360462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90014620 con 0x7f3f980ad720 2023-11-24T08:07:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.646+0000 7f3f8d7fa640 1 -- 172.21.15.64:0/4052360462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90014780 con 0x7f3f980ad720 2023-11-24T08:07:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.647+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f980cc6b0 con 0x7f3f980ad720 2023-11-24T08:07:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.647+0000 7f3f8d7fa640 1 -- 172.21.15.64:0/4052360462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90014b30 con 0x7f3f980ad720 2023-11-24T08:07:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.647+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9814a680 con 0x7f3f980ad720 2023-11-24T08:07:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.648+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f980ce8e0 con 0x7f3f980ad720 2023-11-24T08:07:46.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.649+0000 7f3f8d7fa640 1 -- 172.21.15.64:0/4052360462 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3f90014c90 con 0x7f3f980ad720 2023-11-24T08:07:46.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.650+0000 7f3f8d7fa640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f8406b260 0x7f3f8406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:46.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.650+0000 7f3f8d7fa640 1 -- 172.21.15.64:0/4052360462 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f9008a630 con 0x7f3f980ad720 2023-11-24T08:07:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.650+0000 7f3f9cb88640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f8406b260 0x7f3f8406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:46.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.651+0000 7f3f9cb88640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f8406b260 0x7f3f8406d720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3f94036000 tx=0x7f3f9403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:46.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.654+0000 7f3f8d7fa640 1 -- 172.21.15.64:0/4052360462 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f90055150 con 0x7f3f980ad720 2023-11-24T08:07:46.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.846+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f9815b6e0 con 0x7f3f8406b260 2023-11-24T08:07:46.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.872+0000 7f3f8d7fa640 1 -- 172.21.15.64:0/4052360462 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287009 (secure 0 0 0) 0x7f3f94031a10 con 0x7f3f8406b260 2023-11-24T08:07:46.877 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:46.879 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:07:46.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f8406b260 msgr2=0x7f3f8406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f8406b260 0x7f3f8406d720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3f94036000 tx=0x7f3f9403c040 comp rx=0 tx=0).stop 2023-11-24T08:07:46.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f980ad720 msgr2=0x7f3f980cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:46.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f980ad720 0x7f3f980cdb80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007e30 tx=0x7f3f9000e4b0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 shutdown_connections 2023-11-24T08:07:46.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3f8406b260 0x7f3f8406d720 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f980ad720 0x7f3f980cdb80 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3f980abff0 0x7f3f980cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 --2- 172.21.15.64:0/4052360462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3f9806d520 0x7f3f9814a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:46.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.878+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 >> 172.21.15.64:0/4052360462 conn(0x7f3f980b0d80 msgr2=0x7f3f980b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:46.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.879+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 shutdown_connections 2023-11-24T08:07:46.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:46.879+0000 7f3f9edec640 1 -- 172.21.15.64:0/4052360462 wait complete. 2023-11-24T08:07:46.900 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2023-11-24T08:07:46.097645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249308,"kb_used_data":8176,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725412,"statfs":{"total":773094113280,"available":772838821888,"internally_reserved":0,"allocated":8372224,"data_stored":4209637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.792864"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.371700+0000","last_change":"2023-11-24T08:07:42.367144+0000","last_active":"2023-11-24T08:07:42.371700+0000","last_peered":"2023-11-24T08:07:42.371700+0000","last_clean":"2023-11-24T08:07:42.371700+0000","last_became_active":"2023-11-24T08:07:42.366553+0000","last_became_peered":"2023-11-24T08:07:42.366553+0000","last_unstale":"2023-11-24T08:07:42.371700+0000","last_undegraded":"2023-11-24T08:07:42.371700+0000","last_fullsized":"2023-11-24T08:07:42.371700+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:07.735220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363557+0000","last_change":"2023-11-24T08:07:42.374234+0000","last_active":"2023-11-24T08:07:43.363557+0000","last_peered":"2023-11-24T08:07:43.363557+0000","last_clean":"2023-11-24T08:07:43.363557+0000","last_became_active":"2023-11-24T08:07:42.373895+0000","last_became_peered":"2023-11-24T08:07:42.373895+0000","last_unstale":"2023-11-24T08:07:43.363557+0000","last_undegraded":"2023-11-24T08:07:43.363557+0000","last_fullsized":"2023-11-24T08:07:43.363557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:00:05.987484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.363723+0000","last_change":"2023-11-24T08:07:42.356987+0000","last_active":"2023-11-24T08:07:42.363723+0000","last_peered":"2023-11-24T08:07:42.363723+0000","last_clean":"2023-11-24T08:07:42.363723+0000","last_became_active":"2023-11-24T08:07:42.356692+0000","last_became_peered":"2023-11-24T08:07:42.356692+0000","last_unstale":"2023-11-24T08:07:42.363723+0000","last_undegraded":"2023-11-24T08:07:42.363723+0000","last_fullsized":"2023-11-24T08:07:42.363723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:08:41.203736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.355661+0000","last_change":"2023-11-24T08:07:42.352914+0000","last_active":"2023-11-24T08:07:42.355661+0000","last_peered":"2023-11-24T08:07:42.355661+0000","last_clean":"2023-11-24T08:07:42.355661+0000","last_became_active":"2023-11-24T08:07:42.352564+0000","last_became_peered":"2023-11-24T08:07:42.352564+0000","last_unstale":"2023-11-24T08:07:42.355661+0000","last_undegraded":"2023-11-24T08:07:42.355661+0000","last_fullsized":"2023-11-24T08:07:42.355661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:19.261764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358186+0000","last_change":"2023-11-24T08:07:42.362583+0000","last_active":"2023-11-24T08:07:43.358186+0000","last_peered":"2023-11-24T08:07:43.358186+0000","last_clean":"2023-11-24T08:07:43.358186+0000","last_became_active":"2023-11-24T08:07:42.362237+0000","last_became_peered":"2023-11-24T08:07:42.362237+0000","last_unstale":"2023-11-24T08:07:43.358186+0000","last_undegraded":"2023-11-24T08:07:43.358186+0000","last_fullsized":"2023-11-24T08:07:43.358186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:47:20.878860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364311+0000","last_change":"2023-11-24T08:07:42.369124+0000","last_active":"2023-11-24T08:07:43.364311+0000","last_peered":"2023-11-24T08:07:43.364311+0000","last_clean":"2023-11-24T08:07:43.364311+0000","last_became_active":"2023-11-24T08:07:42.368858+0000","last_became_peered":"2023-11-24T08:07:42.368858+0000","last_unstale":"2023-11-24T08:07:43.364311+0000","last_undegraded":"2023-11-24T08:07:43.364311+0000","last_fullsized":"2023-11-24T08:07:43.364311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:28.325330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.744796+0000","last_change":"2023-11-24T08:07:42.347669+0000","last_active":"2023-11-24T08:07:43.744796+0000","last_peered":"2023-11-24T08:07:43.744796+0000","last_clean":"2023-11-24T08:07:43.744796+0000","last_became_active":"2023-11-24T08:07:42.341994+0000","last_became_peered":"2023-11-24T08:07:42.341994+0000","last_unstale":"2023-11-24T08:07:43.744796+0000","last_undegraded":"2023-11-24T08:07:43.744796+0000","last_fullsized":"2023-11-24T08:07:43.744796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:56:18.150787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362982+0000","last_change":"2023-11-24T08:07:42.355035+0000","last_active":"2023-11-24T08:07:43.362982+0000","last_peered":"2023-11-24T08:07:43.362982+0000","last_clean":"2023-11-24T08:07:43.362982+0000","last_became_active":"2023-11-24T08:07:42.354745+0000","last_became_peered":"2023-11-24T08:07:42.354745+0000","last_unstale":"2023-11-24T08:07:43.362982+0000","last_undegraded":"2023-11-24T08:07:43.362982+0000","last_fullsized":"2023-11-24T08:07:43.362982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:53:07.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358895+0000","last_change":"2023-11-24T08:07:42.362317+0000","last_active":"2023-11-24T08:07:43.358895+0000","last_peered":"2023-11-24T08:07:43.358895+0000","last_clean":"2023-11-24T08:07:43.358895+0000","last_became_active":"2023-11-24T08:07:42.361138+0000","last_became_peered":"2023-11-24T08:07:42.361138+0000","last_unstale":"2023-11-24T08:07:43.358895+0000","last_undegraded":"2023-11-24T08:07:43.358895+0000","last_fullsized":"2023-11-24T08:07:43.358895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:19.261764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365695+0000","last_change":"2023-11-24T08:07:42.366130+0000","last_active":"2023-11-24T08:07:43.365695+0000","last_peered":"2023-11-24T08:07:43.365695+0000","last_clean":"2023-11-24T08:07:43.365695+0000","last_became_active":"2023-11-24T08:07:42.365841+0000","last_became_peered":"2023-11-24T08:07:42.365841+0000","last_unstale":"2023-11-24T08:07:43.365695+0000","last_undegraded":"2023-11-24T08:07:43.365695+0000","last_fullsized":"2023-11-24T08:07:43.365695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:28.325330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.370833+0000","last_change":"2023-11-24T08:07:42.365927+0000","last_active":"2023-11-24T08:07:42.370833+0000","last_peered":"2023-11-24T08:07:42.370833+0000","last_clean":"2023-11-24T08:07:42.370833+0000","last_became_active":"2023-11-24T08:07:42.365619+0000","last_became_peered":"2023-11-24T08:07:42.365619+0000","last_unstale":"2023-11-24T08:07:42.370833+0000","last_undegraded":"2023-11-24T08:07:42.370833+0000","last_fullsized":"2023-11-24T08:07:42.370833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:07.490591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.351875+0000","last_change":"2023-11-24T08:07:42.340355+0000","last_active":"2023-11-24T08:07:42.351875+0000","last_peered":"2023-11-24T08:07:42.351875+0000","last_clean":"2023-11-24T08:07:42.351875+0000","last_became_active":"2023-11-24T08:07:42.340015+0000","last_became_peered":"2023-11-24T08:07:42.340015+0000","last_unstale":"2023-11-24T08:07:42.351875+0000","last_undegraded":"2023-11-24T08:07:42.351875+0000","last_fullsized":"2023-11-24T08:07:42.351875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:39.534095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.365945+0000","last_change":"2023-11-24T08:07:42.365334+0000","last_active":"2023-11-24T08:07:42.365945+0000","last_peered":"2023-11-24T08:07:42.365945+0000","last_clean":"2023-11-24T08:07:42.365945+0000","last_became_active":"2023-11-24T08:07:42.362607+0000","last_became_peered":"2023-11-24T08:07:42.362607+0000","last_unstale":"2023-11-24T08:07:42.365945+0000","last_undegraded":"2023-11-24T08:07:42.365945+0000","last_fullsized":"2023-11-24T08:07:42.365945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:14.975240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358219+0000","last_change":"2023-11-24T08:07:42.352402+0000","last_active":"2023-11-24T08:07:43.358219+0000","last_peered":"2023-11-24T08:07:43.358219+0000","last_clean":"2023-11-24T08:07:43.358219+0000","last_became_active":"2023-11-24T08:07:42.352065+0000","last_became_peered":"2023-11-24T08:07:42.352065+0000","last_unstale":"2023-11-24T08:07:43.358219+0000","last_undegraded":"2023-11-24T08:07:43.358219+0000","last_fullsized":"2023-11-24T08:07:43.358219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:32.193825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.349975+0000","last_change":"2023-11-24T08:07:42.344531+0000","last_active":"2023-11-24T08:07:42.349975+0000","last_peered":"2023-11-24T08:07:42.349975+0000","last_clean":"2023-11-24T08:07:42.349975+0000","last_became_active":"2023-11-24T08:07:42.344237+0000","last_became_peered":"2023-11-24T08:07:42.344237+0000","last_unstale":"2023-11-24T08:07:42.349975+0000","last_undegraded":"2023-11-24T08:07:42.349975+0000","last_fullsized":"2023-11-24T08:07:42.349975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:07.735220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361985+0000","last_change":"2023-11-24T08:07:42.367021+0000","last_active":"2023-11-24T08:07:43.361985+0000","last_peered":"2023-11-24T08:07:43.361985+0000","last_clean":"2023-11-24T08:07:43.361985+0000","last_became_active":"2023-11-24T08:07:42.366642+0000","last_became_peered":"2023-11-24T08:07:42.366642+0000","last_unstale":"2023-11-24T08:07:43.361985+0000","last_undegraded":"2023-11-24T08:07:43.361985+0000","last_fullsized":"2023-11-24T08:07:43.361985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:53:07.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358927+0000","last_change":"2023-11-24T08:07:42.361039+0000","last_active":"2023-11-24T08:07:43.358927+0000","last_peered":"2023-11-24T08:07:43.358927+0000","last_clean":"2023-11-24T08:07:43.358927+0000","last_became_active":"2023-11-24T08:07:42.360740+0000","last_became_peered":"2023-11-24T08:07:42.360740+0000","last_unstale":"2023-11-24T08:07:43.358927+0000","last_undegraded":"2023-11-24T08:07:43.358927+0000","last_fullsized":"2023-11-24T08:07:43.358927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:39.534095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.366138+0000","last_change":"2023-11-24T08:07:42.363424+0000","last_active":"2023-11-24T08:07:42.366138+0000","last_peered":"2023-11-24T08:07:42.366138+0000","last_clean":"2023-11-24T08:07:42.366138+0000","last_became_active":"2023-11-24T08:07:42.363071+0000","last_became_peered":"2023-11-24T08:07:42.363071+0000","last_unstale":"2023-11-24T08:07:42.366138+0000","last_undegraded":"2023-11-24T08:07:42.366138+0000","last_fullsized":"2023-11-24T08:07:42.366138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:38.089553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364306+0000","last_change":"2023-11-24T08:07:42.353530+0000","last_active":"2023-11-24T08:07:43.364306+0000","last_peered":"2023-11-24T08:07:43.364306+0000","last_clean":"2023-11-24T08:07:43.364306+0000","last_became_active":"2023-11-24T08:07:42.353047+0000","last_became_peered":"2023-11-24T08:07:42.353047+0000","last_unstale":"2023-11-24T08:07:43.364306+0000","last_undegraded":"2023-11-24T08:07:43.364306+0000","last_fullsized":"2023-11-24T08:07:43.364306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:50.994269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.359665+0000","last_change":"2023-11-24T08:07:42.352171+0000","last_active":"2023-11-24T08:07:42.359665+0000","last_peered":"2023-11-24T08:07:42.359665+0000","last_clean":"2023-11-24T08:07:42.359665+0000","last_became_active":"2023-11-24T08:07:42.351899+0000","last_became_peered":"2023-11-24T08:07:42.351899+0000","last_unstale":"2023-11-24T08:07:42.359665+0000","last_undegraded":"2023-11-24T08:07:42.359665+0000","last_fullsized":"2023-11-24T08:07:42.359665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:23.845233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364473+0000","last_change":"2023-11-24T08:07:42.347317+0000","last_active":"2023-11-24T08:07:43.364473+0000","last_peered":"2023-11-24T08:07:43.364473+0000","last_clean":"2023-11-24T08:07:43.364473+0000","last_became_active":"2023-11-24T08:07:42.346730+0000","last_became_peered":"2023-11-24T08:07:42.346730+0000","last_unstale":"2023-11-24T08:07:43.364473+0000","last_undegraded":"2023-11-24T08:07:43.364473+0000","last_fullsized":"2023-11-24T08:07:43.364473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:30.293229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363375+0000","last_change":"2023-11-24T08:07:42.353844+0000","last_active":"2023-11-24T08:07:43.363375+0000","last_peered":"2023-11-24T08:07:43.363375+0000","last_clean":"2023-11-24T08:07:43.363375+0000","last_became_active":"2023-11-24T08:07:42.352938+0000","last_became_peered":"2023-11-24T08:07:42.352938+0000","last_unstale":"2023-11-24T08:07:43.363375+0000","last_undegraded":"2023-11-24T08:07:43.363375+0000","last_fullsized":"2023-11-24T08:07:43.363375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:36.769677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.369777+0000","last_change":"2023-11-24T08:07:42.363647+0000","last_active":"2023-11-24T08:07:42.369777+0000","last_peered":"2023-11-24T08:07:42.369777+0000","last_clean":"2023-11-24T08:07:42.369777+0000","last_became_active":"2023-11-24T08:07:42.363388+0000","last_became_peered":"2023-11-24T08:07:42.363388+0000","last_unstale":"2023-11-24T08:07:42.369777+0000","last_undegraded":"2023-11-24T08:07:42.369777+0000","last_fullsized":"2023-11-24T08:07:42.369777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:31.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373781+0000","last_change":"2023-11-24T08:07:42.368789+0000","last_active":"2023-11-24T08:07:42.373781+0000","last_peered":"2023-11-24T08:07:42.373781+0000","last_clean":"2023-11-24T08:07:42.373781+0000","last_became_active":"2023-11-24T08:07:42.368523+0000","last_became_peered":"2023-11-24T08:07:42.368523+0000","last_unstale":"2023-11-24T08:07:42.373781+0000","last_undegraded":"2023-11-24T08:07:42.373781+0000","last_fullsized":"2023-11-24T08:07:42.373781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:17.571607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365884+0000","last_change":"2023-11-24T08:07:42.368813+0000","last_active":"2023-11-24T08:07:43.365884+0000","last_peered":"2023-11-24T08:07:43.365884+0000","last_clean":"2023-11-24T08:07:43.365884+0000","last_became_active":"2023-11-24T08:07:42.368558+0000","last_became_peered":"2023-11-24T08:07:42.368558+0000","last_unstale":"2023-11-24T08:07:43.365884+0000","last_undegraded":"2023-11-24T08:07:43.365884+0000","last_fullsized":"2023-11-24T08:07:43.365884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:14.069233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745685+0000","last_change":"2023-11-24T08:07:42.359797+0000","last_active":"2023-11-24T08:07:43.745685+0000","last_peered":"2023-11-24T08:07:43.745685+0000","last_clean":"2023-11-24T08:07:43.745685+0000","last_became_active":"2023-11-24T08:07:42.359485+0000","last_became_peered":"2023-11-24T08:07:42.359485+0000","last_unstale":"2023-11-24T08:07:43.745685+0000","last_undegraded":"2023-11-24T08:07:43.745685+0000","last_fullsized":"2023-11-24T08:07:43.745685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:59.024929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.369738+0000","last_change":"2023-11-24T08:07:42.367402+0000","last_active":"2023-11-24T08:07:42.369738+0000","last_peered":"2023-11-24T08:07:42.369738+0000","last_clean":"2023-11-24T08:07:42.369738+0000","last_became_active":"2023-11-24T08:07:42.367035+0000","last_became_peered":"2023-11-24T08:07:42.367035+0000","last_unstale":"2023-11-24T08:07:42.369738+0000","last_undegraded":"2023-11-24T08:07:42.369738+0000","last_fullsized":"2023-11-24T08:07:42.369738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:33.921127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364982+0000","last_change":"2023-11-24T08:07:42.351941+0000","last_active":"2023-11-24T08:07:43.364982+0000","last_peered":"2023-11-24T08:07:43.364982+0000","last_clean":"2023-11-24T08:07:43.364982+0000","last_became_active":"2023-11-24T08:07:42.351640+0000","last_became_peered":"2023-11-24T08:07:42.351640+0000","last_unstale":"2023-11-24T08:07:43.364982+0000","last_undegraded":"2023-11-24T08:07:43.364982+0000","last_fullsized":"2023-11-24T08:07:43.364982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:50.994269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.366170+0000","last_change":"2023-11-24T08:07:42.361604+0000","last_active":"2023-11-24T08:07:42.366170+0000","last_peered":"2023-11-24T08:07:42.366170+0000","last_clean":"2023-11-24T08:07:42.366170+0000","last_became_active":"2023-11-24T08:07:42.361347+0000","last_became_peered":"2023-11-24T08:07:42.361347+0000","last_unstale":"2023-11-24T08:07:42.366170+0000","last_undegraded":"2023-11-24T08:07:42.366170+0000","last_fullsized":"2023-11-24T08:07:42.366170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:49.714425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745025+0000","last_change":"2023-11-24T08:07:42.361686+0000","last_active":"2023-11-24T08:07:43.745025+0000","last_peered":"2023-11-24T08:07:43.745025+0000","last_clean":"2023-11-24T08:07:43.745025+0000","last_became_active":"2023-11-24T08:07:42.361137+0000","last_became_peered":"2023-11-24T08:07:42.361137+0000","last_unstale":"2023-11-24T08:07:43.745025+0000","last_undegraded":"2023-11-24T08:07:43.745025+0000","last_fullsized":"2023-11-24T08:07:43.745025+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:14.069233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745541+0000","last_change":"2023-11-24T08:07:42.348582+0000","last_active":"2023-11-24T08:07:43.745541+0000","last_peered":"2023-11-24T08:07:43.745541+0000","last_clean":"2023-11-24T08:07:43.745541+0000","last_became_active":"2023-11-24T08:07:42.348110+0000","last_became_peered":"2023-11-24T08:07:42.348110+0000","last_unstale":"2023-11-24T08:07:43.745541+0000","last_undegraded":"2023-11-24T08:07:43.745541+0000","last_fullsized":"2023-11-24T08:07:43.745541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:43.976124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363361+0000","last_change":"2023-11-24T08:07:42.359298+0000","last_active":"2023-11-24T08:07:43.363361+0000","last_peered":"2023-11-24T08:07:43.363361+0000","last_clean":"2023-11-24T08:07:43.363361+0000","last_became_active":"2023-11-24T08:07:42.358657+0000","last_became_peered":"2023-11-24T08:07:42.358657+0000","last_unstale":"2023-11-24T08:07:43.363361+0000","last_undegraded":"2023-11-24T08:07:43.363361+0000","last_fullsized":"2023-11-24T08:07:43.363361+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:30.293229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373819+0000","last_change":"2023-11-24T08:07:42.370318+0000","last_active":"2023-11-24T08:07:42.373819+0000","last_peered":"2023-11-24T08:07:42.373819+0000","last_clean":"2023-11-24T08:07:42.373819+0000","last_became_active":"2023-11-24T08:07:42.369414+0000","last_became_peered":"2023-11-24T08:07:42.369414+0000","last_unstale":"2023-11-24T08:07:42.373819+0000","last_undegraded":"2023-11-24T08:07:42.373819+0000","last_fullsized":"2023-11-24T08:07:42.373819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:44.749901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.369865+0000","last_change":"2023-11-24T08:07:41.356401+0000","last_active":"2023-11-24T08:07:42.369865+0000","last_peered":"2023-11-24T08:07:42.369865+0000","last_clean":"2023-11-24T08:07:42.369865+0000","last_became_active":"2023-11-24T08:07:41.355471+0000","last_became_peered":"2023-11-24T08:07:41.355471+0000","last_unstale":"2023-11-24T08:07:42.369865+0000","last_undegraded":"2023-11-24T08:07:42.369865+0000","last_fullsized":"2023-11-24T08:07:42.369865+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:01.313679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.359026+0000","last_change":"2023-11-24T08:07:42.360076+0000","last_active":"2023-11-24T08:07:43.359026+0000","last_peered":"2023-11-24T08:07:43.359026+0000","last_clean":"2023-11-24T08:07:43.359026+0000","last_became_active":"2023-11-24T08:07:42.358560+0000","last_became_peered":"2023-11-24T08:07:42.358560+0000","last_unstale":"2023-11-24T08:07:43.359026+0000","last_undegraded":"2023-11-24T08:07:43.359026+0000","last_fullsized":"2023-11-24T08:07:43.359026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:55.527459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.363276+0000","last_change":"2023-11-24T08:07:41.340910+0000","last_active":"2023-11-24T08:07:42.363276+0000","last_peered":"2023-11-24T08:07:42.363276+0000","last_clean":"2023-11-24T08:07:42.363276+0000","last_became_active":"2023-11-24T08:07:41.340557+0000","last_became_peered":"2023-11-24T08:07:41.340557+0000","last_unstale":"2023-11-24T08:07:42.363276+0000","last_undegraded":"2023-11-24T08:07:42.363276+0000","last_fullsized":"2023-11-24T08:07:42.363276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:21.924169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362110+0000","last_change":"2023-11-24T08:07:42.356921+0000","last_active":"2023-11-24T08:07:43.362110+0000","last_peered":"2023-11-24T08:07:43.362110+0000","last_clean":"2023-11-24T08:07:43.362110+0000","last_became_active":"2023-11-24T08:07:42.355455+0000","last_became_peered":"2023-11-24T08:07:42.355455+0000","last_unstale":"2023-11-24T08:07:43.362110+0000","last_undegraded":"2023-11-24T08:07:43.362110+0000","last_fullsized":"2023-11-24T08:07:43.362110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:39.983011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365537+0000","last_change":"2023-11-24T08:07:41.350994+0000","last_active":"2023-11-24T08:07:43.365537+0000","last_peered":"2023-11-24T08:07:43.365537+0000","last_clean":"2023-11-24T08:07:43.365537+0000","last_became_active":"2023-11-24T08:07:41.350697+0000","last_became_peered":"2023-11-24T08:07:41.350697+0000","last_unstale":"2023-11-24T08:07:43.365537+0000","last_undegraded":"2023-11-24T08:07:43.365537+0000","last_fullsized":"2023-11-24T08:07:43.365537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:38.880104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.371511+0000","last_change":"2023-11-24T08:07:42.367104+0000","last_active":"2023-11-24T08:07:42.371511+0000","last_peered":"2023-11-24T08:07:42.371511+0000","last_clean":"2023-11-24T08:07:42.371511+0000","last_became_active":"2023-11-24T08:07:42.366187+0000","last_became_peered":"2023-11-24T08:07:42.366187+0000","last_unstale":"2023-11-24T08:07:42.371511+0000","last_undegraded":"2023-11-24T08:07:42.371511+0000","last_fullsized":"2023-11-24T08:07:42.371511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:46.970844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362495+0000","last_change":"2023-11-24T08:07:41.356303+0000","last_active":"2023-11-24T08:07:43.362495+0000","last_peered":"2023-11-24T08:07:43.362495+0000","last_clean":"2023-11-24T08:07:43.362495+0000","last_became_active":"2023-11-24T08:07:41.354940+0000","last_became_peered":"2023-11-24T08:07:41.354940+0000","last_unstale":"2023-11-24T08:07:43.362495+0000","last_undegraded":"2023-11-24T08:07:43.362495+0000","last_fullsized":"2023-11-24T08:07:43.362495+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:17.956594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745287+0000","last_change":"2023-11-24T08:07:42.350542+0000","last_active":"2023-11-24T08:07:43.745287+0000","last_peered":"2023-11-24T08:07:43.745287+0000","last_clean":"2023-11-24T08:07:43.745287+0000","last_became_active":"2023-11-24T08:07:42.348205+0000","last_became_peered":"2023-11-24T08:07:42.348205+0000","last_unstale":"2023-11-24T08:07:43.745287+0000","last_undegraded":"2023-11-24T08:07:43.745287+0000","last_fullsized":"2023-11-24T08:07:43.745287+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:56.267643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.372725+0000","last_change":"2023-11-24T08:07:41.342300+0000","last_active":"2023-11-24T08:07:42.372725+0000","last_peered":"2023-11-24T08:07:42.372725+0000","last_clean":"2023-11-24T08:07:42.372725+0000","last_became_active":"2023-11-24T08:07:41.342008+0000","last_became_peered":"2023-11-24T08:07:41.342008+0000","last_unstale":"2023-11-24T08:07:42.372725+0000","last_undegraded":"2023-11-24T08:07:42.372725+0000","last_fullsized":"2023-11-24T08:07:42.372725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:58.391383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364581+0000","last_change":"2023-11-24T08:07:42.372248+0000","last_active":"2023-11-24T08:07:43.364581+0000","last_peered":"2023-11-24T08:07:43.364581+0000","last_clean":"2023-11-24T08:07:43.364581+0000","last_became_active":"2023-11-24T08:07:42.371948+0000","last_became_peered":"2023-11-24T08:07:42.371948+0000","last_unstale":"2023-11-24T08:07:43.364581+0000","last_undegraded":"2023-11-24T08:07:43.364581+0000","last_fullsized":"2023-11-24T08:07:43.364581+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:00:05.987484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361727+0000","last_change":"2023-11-24T08:07:41.355500+0000","last_active":"2023-11-24T08:07:43.361727+0000","last_peered":"2023-11-24T08:07:43.361727+0000","last_clean":"2023-11-24T08:07:43.361727+0000","last_became_active":"2023-11-24T08:07:41.355157+0000","last_became_peered":"2023-11-24T08:07:41.355157+0000","last_unstale":"2023-11-24T08:07:43.361727+0000","last_undegraded":"2023-11-24T08:07:43.361727+0000","last_fullsized":"2023-11-24T08:07:43.361727+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:39:48.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.357707+0000","last_change":"2023-11-24T08:07:42.369662+0000","last_active":"2023-11-24T08:07:43.357707+0000","last_peered":"2023-11-24T08:07:43.357707+0000","last_clean":"2023-11-24T08:07:43.357707+0000","last_became_active":"2023-11-24T08:07:42.368982+0000","last_became_peered":"2023-11-24T08:07:42.368982+0000","last_unstale":"2023-11-24T08:07:43.357707+0000","last_undegraded":"2023-11-24T08:07:43.357707+0000","last_fullsized":"2023-11-24T08:07:43.357707+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:25.159000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.326323+0000","last_change":"2023-11-24T08:07:41.339635+0000","last_active":"2023-11-24T08:07:42.326323+0000","last_peered":"2023-11-24T08:07:42.326323+0000","last_clean":"2023-11-24T08:07:42.326323+0000","last_became_active":"2023-11-24T08:07:41.339253+0000","last_became_peered":"2023-11-24T08:07:41.339253+0000","last_unstale":"2023-11-24T08:07:42.326323+0000","last_undegraded":"2023-11-24T08:07:42.326323+0000","last_fullsized":"2023-11-24T08:07:42.326323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:10.344531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.358078+0000","last_change":"2023-11-24T08:07:42.350087+0000","last_active":"2023-11-24T08:07:42.358078+0000","last_peered":"2023-11-24T08:07:42.358078+0000","last_clean":"2023-11-24T08:07:42.358078+0000","last_became_active":"2023-11-24T08:07:42.349771+0000","last_became_peered":"2023-11-24T08:07:42.349771+0000","last_unstale":"2023-11-24T08:07:42.358078+0000","last_undegraded":"2023-11-24T08:07:42.358078+0000","last_fullsized":"2023-11-24T08:07:42.358078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:44.749901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.357699+0000","last_change":"2023-11-24T08:07:41.343070+0000","last_active":"2023-11-24T08:07:43.357699+0000","last_peered":"2023-11-24T08:07:43.357699+0000","last_clean":"2023-11-24T08:07:43.357699+0000","last_became_active":"2023-11-24T08:07:41.342674+0000","last_became_peered":"2023-11-24T08:07:41.342674+0000","last_unstale":"2023-11-24T08:07:43.357699+0000","last_undegraded":"2023-11-24T08:07:43.357699+0000","last_fullsized":"2023-11-24T08:07:43.357699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:41.216815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745751+0000","last_change":"2023-11-24T08:07:42.359841+0000","last_active":"2023-11-24T08:07:43.745751+0000","last_peered":"2023-11-24T08:07:43.745751+0000","last_clean":"2023-11-24T08:07:43.745751+0000","last_became_active":"2023-11-24T08:07:42.359552+0000","last_became_peered":"2023-11-24T08:07:42.359552+0000","last_unstale":"2023-11-24T08:07:43.745751+0000","last_undegraded":"2023-11-24T08:07:43.745751+0000","last_fullsized":"2023-11-24T08:07:43.745751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:07.490591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365708+0000","last_change":"2023-11-24T08:07:41.341295+0000","last_active":"2023-11-24T08:07:43.365708+0000","last_peered":"2023-11-24T08:07:43.365708+0000","last_clean":"2023-11-24T08:07:43.365708+0000","last_became_active":"2023-11-24T08:07:41.339111+0000","last_became_peered":"2023-11-24T08:07:41.339111+0000","last_unstale":"2023-11-24T08:07:43.365708+0000","last_undegraded":"2023-11-24T08:07:43.365708+0000","last_fullsized":"2023-11-24T08:07:43.365708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:38.968606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373785+0000","last_change":"2023-11-24T08:07:42.369724+0000","last_active":"2023-11-24T08:07:42.373785+0000","last_peered":"2023-11-24T08:07:42.373785+0000","last_clean":"2023-11-24T08:07:42.373785+0000","last_became_active":"2023-11-24T08:07:42.369460+0000","last_became_peered":"2023-11-24T08:07:42.369460+0000","last_unstale":"2023-11-24T08:07:42.373785+0000","last_undegraded":"2023-11-24T08:07:42.373785+0000","last_fullsized":"2023-11-24T08:07:42.373785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:14.975240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363923+0000","last_change":"2023-11-24T08:07:41.357620+0000","last_active":"2023-11-24T08:07:43.363923+0000","last_peered":"2023-11-24T08:07:43.363923+0000","last_clean":"2023-11-24T08:07:43.363923+0000","last_became_active":"2023-11-24T08:07:41.357345+0000","last_became_peered":"2023-11-24T08:07:41.357345+0000","last_unstale":"2023-11-24T08:07:43.363923+0000","last_undegraded":"2023-11-24T08:07:43.363923+0000","last_fullsized":"2023-11-24T08:07:43.363923+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:19.383814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362440+0000","last_change":"2023-11-24T08:07:42.370971+0000","last_active":"2023-11-24T08:07:43.362440+0000","last_peered":"2023-11-24T08:07:43.362440+0000","last_clean":"2023-11-24T08:07:43.362440+0000","last_became_active":"2023-11-24T08:07:42.369160+0000","last_became_peered":"2023-11-24T08:07:42.369160+0000","last_unstale":"2023-11-24T08:07:43.362440+0000","last_undegraded":"2023-11-24T08:07:43.362440+0000","last_fullsized":"2023-11-24T08:07:43.362440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:52.334351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.370021+0000","last_change":"2023-11-24T08:07:41.351210+0000","last_active":"2023-11-24T08:07:42.370021+0000","last_peered":"2023-11-24T08:07:42.370021+0000","last_clean":"2023-11-24T08:07:42.370021+0000","last_became_active":"2023-11-24T08:07:41.350226+0000","last_became_peered":"2023-11-24T08:07:41.350226+0000","last_unstale":"2023-11-24T08:07:42.370021+0000","last_undegraded":"2023-11-24T08:07:42.370021+0000","last_fullsized":"2023-11-24T08:07:42.370021+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:19.929165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358453+0000","last_change":"2023-11-24T08:07:42.345516+0000","last_active":"2023-11-24T08:07:43.358453+0000","last_peered":"2023-11-24T08:07:43.358453+0000","last_clean":"2023-11-24T08:07:43.358453+0000","last_became_active":"2023-11-24T08:07:42.344592+0000","last_became_peered":"2023-11-24T08:07:42.344592+0000","last_unstale":"2023-11-24T08:07:43.358453+0000","last_undegraded":"2023-11-24T08:07:43.358453+0000","last_fullsized":"2023-11-24T08:07:43.358453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:57:09.164635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.374834+0000","last_change":"2023-11-24T08:07:41.336623+0000","last_active":"2023-11-24T08:07:42.374834+0000","last_peered":"2023-11-24T08:07:42.374834+0000","last_clean":"2023-11-24T08:07:42.374834+0000","last_became_active":"2023-11-24T08:07:41.336180+0000","last_became_peered":"2023-11-24T08:07:41.336180+0000","last_unstale":"2023-11-24T08:07:42.374834+0000","last_undegraded":"2023-11-24T08:07:42.374834+0000","last_fullsized":"2023-11-24T08:07:42.374834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:54.513055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.370685+0000","last_change":"2023-11-24T08:07:42.365859+0000","last_active":"2023-11-24T08:07:42.370685+0000","last_peered":"2023-11-24T08:07:42.370685+0000","last_clean":"2023-11-24T08:07:42.370685+0000","last_became_active":"2023-11-24T08:07:42.365524+0000","last_became_peered":"2023-11-24T08:07:42.365524+0000","last_unstale":"2023-11-24T08:07:42.370685+0000","last_undegraded":"2023-11-24T08:07:42.370685+0000","last_fullsized":"2023-11-24T08:07:42.370685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:43.976124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.370436+0000","last_change":"2023-11-24T08:07:41.348497+0000","last_active":"2023-11-24T08:07:42.370436+0000","last_peered":"2023-11-24T08:07:42.370436+0000","last_clean":"2023-11-24T08:07:42.370436+0000","last_became_active":"2023-11-24T08:07:41.348196+0000","last_became_peered":"2023-11-24T08:07:41.348196+0000","last_unstale":"2023-11-24T08:07:42.370436+0000","last_undegraded":"2023-11-24T08:07:42.370436+0000","last_fullsized":"2023-11-24T08:07:42.370436+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:15.746351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745387+0000","last_change":"2023-11-24T08:07:42.349890+0000","last_active":"2023-11-24T08:07:43.745387+0000","last_peered":"2023-11-24T08:07:43.745387+0000","last_clean":"2023-11-24T08:07:43.745387+0000","last_became_active":"2023-11-24T08:07:42.349520+0000","last_became_peered":"2023-11-24T08:07:42.349520+0000","last_unstale":"2023-11-24T08:07:43.745387+0000","last_undegraded":"2023-11-24T08:07:43.745387+0000","last_fullsized":"2023-11-24T08:07:43.745387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:17.571607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361954+0000","last_change":"2023-11-24T08:07:41.354093+0000","last_active":"2023-11-24T08:07:43.361954+0000","last_peered":"2023-11-24T08:07:43.361954+0000","last_clean":"2023-11-24T08:07:43.361954+0000","last_became_active":"2023-11-24T08:07:41.353814+0000","last_became_peered":"2023-11-24T08:07:41.353814+0000","last_unstale":"2023-11-24T08:07:43.361954+0000","last_undegraded":"2023-11-24T08:07:43.361954+0000","last_fullsized":"2023-11-24T08:07:43.361954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:36.286125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.364235+0000","last_change":"2023-11-24T08:07:42.362948+0000","last_active":"2023-11-24T08:07:42.364235+0000","last_peered":"2023-11-24T08:07:42.364235+0000","last_clean":"2023-11-24T08:07:42.364235+0000","last_became_active":"2023-11-24T08:07:42.359320+0000","last_became_peered":"2023-11-24T08:07:42.359320+0000","last_unstale":"2023-11-24T08:07:42.364235+0000","last_undegraded":"2023-11-24T08:07:42.364235+0000","last_fullsized":"2023-11-24T08:07:42.364235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:21.644216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.744969+0000","last_change":"2023-11-24T08:07:41.355775+0000","last_active":"2023-11-24T08:07:43.744969+0000","last_peered":"2023-11-24T08:07:43.744969+0000","last_clean":"2023-11-24T08:07:43.744969+0000","last_became_active":"2023-11-24T08:07:41.355174+0000","last_became_peered":"2023-11-24T08:07:41.355174+0000","last_unstale":"2023-11-24T08:07:43.744969+0000","last_undegraded":"2023-11-24T08:07:43.744969+0000","last_fullsized":"2023-11-24T08:07:43.744969+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:51.319947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358346+0000","last_change":"2023-11-24T08:07:42.348204+0000","last_active":"2023-11-24T08:07:43.358346+0000","last_peered":"2023-11-24T08:07:43.358346+0000","last_clean":"2023-11-24T08:07:43.358346+0000","last_became_active":"2023-11-24T08:07:42.347875+0000","last_became_peered":"2023-11-24T08:07:42.347875+0000","last_unstale":"2023-11-24T08:07:43.358346+0000","last_undegraded":"2023-11-24T08:07:43.358346+0000","last_fullsized":"2023-11-24T08:07:43.358346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:16.760756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362247+0000","last_change":"2023-11-24T08:07:41.345824+0000","last_active":"2023-11-24T08:07:43.362247+0000","last_peered":"2023-11-24T08:07:43.362247+0000","last_clean":"2023-11-24T08:07:43.362247+0000","last_became_active":"2023-11-24T08:07:41.345473+0000","last_became_peered":"2023-11-24T08:07:41.345473+0000","last_unstale":"2023-11-24T08:07:43.362247+0000","last_undegraded":"2023-11-24T08:07:43.362247+0000","last_fullsized":"2023-11-24T08:07:43.362247+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:39:48.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365762+0000","last_change":"2023-11-24T08:07:42.362434+0000","last_active":"2023-11-24T08:07:43.365762+0000","last_peered":"2023-11-24T08:07:43.365762+0000","last_clean":"2023-11-24T08:07:43.365762+0000","last_became_active":"2023-11-24T08:07:42.362110+0000","last_became_peered":"2023-11-24T08:07:42.362110+0000","last_unstale":"2023-11-24T08:07:43.365762+0000","last_undegraded":"2023-11-24T08:07:43.365762+0000","last_fullsized":"2023-11-24T08:07:43.365762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:56:18.150787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362120+0000","last_change":"2023-11-24T08:07:41.347678+0000","last_active":"2023-11-24T08:07:43.362120+0000","last_peered":"2023-11-24T08:07:43.362120+0000","last_clean":"2023-11-24T08:07:43.362120+0000","last_became_active":"2023-11-24T08:07:41.347132+0000","last_became_peered":"2023-11-24T08:07:41.347132+0000","last_unstale":"2023-11-24T08:07:43.362120+0000","last_undegraded":"2023-11-24T08:07:43.362120+0000","last_fullsized":"2023-11-24T08:07:43.362120+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:28.234377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373031+0000","last_change":"2023-11-24T08:07:42.367638+0000","last_active":"2023-11-24T08:07:42.373031+0000","last_peered":"2023-11-24T08:07:42.373031+0000","last_clean":"2023-11-24T08:07:42.373031+0000","last_became_active":"2023-11-24T08:07:42.367313+0000","last_became_peered":"2023-11-24T08:07:42.367313+0000","last_unstale":"2023-11-24T08:07:42.373031+0000","last_undegraded":"2023-11-24T08:07:42.373031+0000","last_fullsized":"2023-11-24T08:07:42.373031+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:23.845233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365904+0000","last_change":"2023-11-24T08:07:41.352151+0000","last_active":"2023-11-24T08:07:43.365904+0000","last_peered":"2023-11-24T08:07:43.365904+0000","last_clean":"2023-11-24T08:07:43.365904+0000","last_became_active":"2023-11-24T08:07:41.351233+0000","last_became_peered":"2023-11-24T08:07:41.351233+0000","last_unstale":"2023-11-24T08:07:43.365904+0000","last_undegraded":"2023-11-24T08:07:43.365904+0000","last_fullsized":"2023-11-24T08:07:43.365904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:15.862078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.351897+0000","last_change":"2023-11-24T08:07:42.342386+0000","last_active":"2023-11-24T08:07:42.351897+0000","last_peered":"2023-11-24T08:07:42.351897+0000","last_clean":"2023-11-24T08:07:42.351897+0000","last_became_active":"2023-11-24T08:07:42.340287+0000","last_became_peered":"2023-11-24T08:07:42.340287+0000","last_unstale":"2023-11-24T08:07:42.351897+0000","last_undegraded":"2023-11-24T08:07:42.351897+0000","last_fullsized":"2023-11-24T08:07:42.351897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:51:39.334989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364553+0000","last_change":"2023-11-24T08:07:41.355914+0000","last_active":"2023-11-24T08:07:43.364553+0000","last_peered":"2023-11-24T08:07:43.364553+0000","last_clean":"2023-11-24T08:07:43.364553+0000","last_became_active":"2023-11-24T08:07:41.355649+0000","last_became_peered":"2023-11-24T08:07:41.355649+0000","last_unstale":"2023-11-24T08:07:43.364553+0000","last_undegraded":"2023-11-24T08:07:43.364553+0000","last_fullsized":"2023-11-24T08:07:43.364553+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:16.721444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358666+0000","last_change":"2023-11-24T08:07:42.355643+0000","last_active":"2023-11-24T08:07:43.358666+0000","last_peered":"2023-11-24T08:07:43.358666+0000","last_clean":"2023-11-24T08:07:43.358666+0000","last_became_active":"2023-11-24T08:07:42.355355+0000","last_became_peered":"2023-11-24T08:07:42.355355+0000","last_unstale":"2023-11-24T08:07:43.358666+0000","last_undegraded":"2023-11-24T08:07:43.358666+0000","last_fullsized":"2023-11-24T08:07:43.358666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:38.089553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.348726+0000","last_change":"2023-11-24T08:07:41.357318+0000","last_active":"2023-11-24T08:07:42.348726+0000","last_peered":"2023-11-24T08:07:42.348726+0000","last_clean":"2023-11-24T08:07:42.348726+0000","last_became_active":"2023-11-24T08:07:41.356882+0000","last_became_peered":"2023-11-24T08:07:41.356882+0000","last_unstale":"2023-11-24T08:07:42.348726+0000","last_undegraded":"2023-11-24T08:07:42.348726+0000","last_fullsized":"2023-11-24T08:07:42.348726+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:20.629811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.363719+0000","last_change":"2023-11-24T08:07:42.356464+0000","last_active":"2023-11-24T08:07:42.363719+0000","last_peered":"2023-11-24T08:07:42.363719+0000","last_clean":"2023-11-24T08:07:42.363719+0000","last_became_active":"2023-11-24T08:07:42.356139+0000","last_became_peered":"2023-11-24T08:07:42.356139+0000","last_unstale":"2023-11-24T08:07:42.363719+0000","last_undegraded":"2023-11-24T08:07:42.363719+0000","last_fullsized":"2023-11-24T08:07:42.363719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:16.519158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362207+0000","last_change":"2023-11-24T08:07:41.351739+0000","last_active":"2023-11-24T08:07:43.362207+0000","last_peered":"2023-11-24T08:07:43.362207+0000","last_clean":"2023-11-24T08:07:43.362207+0000","last_became_active":"2023-11-24T08:07:41.351393+0000","last_became_peered":"2023-11-24T08:07:41.351393+0000","last_unstale":"2023-11-24T08:07:43.362207+0000","last_undegraded":"2023-11-24T08:07:43.362207+0000","last_fullsized":"2023-11-24T08:07:43.362207+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:08.824076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.366138+0000","last_change":"2023-11-24T08:07:42.360044+0000","last_active":"2023-11-24T08:07:42.366138+0000","last_peered":"2023-11-24T08:07:42.366138+0000","last_clean":"2023-11-24T08:07:42.366138+0000","last_became_active":"2023-11-24T08:07:42.359710+0000","last_became_peered":"2023-11-24T08:07:42.359710+0000","last_unstale":"2023-11-24T08:07:42.366138+0000","last_undegraded":"2023-11-24T08:07:42.366138+0000","last_fullsized":"2023-11-24T08:07:42.366138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:54.247656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745276+0000","last_change":"2023-11-24T08:07:41.351220+0000","last_active":"2023-11-24T08:07:43.745276+0000","last_peered":"2023-11-24T08:07:43.745276+0000","last_clean":"2023-11-24T08:07:43.745276+0000","last_became_active":"2023-11-24T08:07:41.350184+0000","last_became_peered":"2023-11-24T08:07:41.350184+0000","last_unstale":"2023-11-24T08:07:43.745276+0000","last_undegraded":"2023-11-24T08:07:43.745276+0000","last_fullsized":"2023-11-24T08:07:43.745276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:53:06.741852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.363760+0000","last_change":"2023-11-24T08:07:42.356190+0000","last_active":"2023-11-24T08:07:42.363760+0000","last_peered":"2023-11-24T08:07:42.363760+0000","last_clean":"2023-11-24T08:07:42.363760+0000","last_became_active":"2023-11-24T08:07:42.355861+0000","last_became_peered":"2023-11-24T08:07:42.355861+0000","last_unstale":"2023-11-24T08:07:42.363760+0000","last_undegraded":"2023-11-24T08:07:42.363760+0000","last_fullsized":"2023-11-24T08:07:42.363760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:50.065788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.334677+0000","last_change":"2023-11-24T08:07:41.345281+0000","last_active":"2023-11-24T08:07:42.334677+0000","last_peered":"2023-11-24T08:07:42.334677+0000","last_clean":"2023-11-24T08:07:42.334677+0000","last_became_active":"2023-11-24T08:07:41.344935+0000","last_became_peered":"2023-11-24T08:07:41.344935+0000","last_unstale":"2023-11-24T08:07:42.334677+0000","last_undegraded":"2023-11-24T08:07:42.334677+0000","last_fullsized":"2023-11-24T08:07:42.334677+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:45.956440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.348868+0000","last_change":"2023-11-24T08:07:42.343602+0000","last_active":"2023-11-24T08:07:42.348868+0000","last_peered":"2023-11-24T08:07:42.348868+0000","last_clean":"2023-11-24T08:07:42.348868+0000","last_became_active":"2023-11-24T08:07:42.343304+0000","last_became_peered":"2023-11-24T08:07:42.343304+0000","last_unstale":"2023-11-24T08:07:42.348868+0000","last_undegraded":"2023-11-24T08:07:42.348868+0000","last_fullsized":"2023-11-24T08:07:42.348868+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:44.851906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362370+0000","last_change":"2023-11-24T08:07:41.347454+0000","last_active":"2023-11-24T08:07:43.362370+0000","last_peered":"2023-11-24T08:07:43.362370+0000","last_clean":"2023-11-24T08:07:43.362370+0000","last_became_active":"2023-11-24T08:07:41.346759+0000","last_became_peered":"2023-11-24T08:07:41.346759+0000","last_unstale":"2023-11-24T08:07:43.362370+0000","last_undegraded":"2023-11-24T08:07:43.362370+0000","last_fullsized":"2023-11-24T08:07:43.362370+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:17.649797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745412+0000","last_change":"2023-11-24T08:07:42.345922+0000","last_active":"2023-11-24T08:07:43.745412+0000","last_peered":"2023-11-24T08:07:43.745412+0000","last_clean":"2023-11-24T08:07:43.745412+0000","last_became_active":"2023-11-24T08:07:42.345623+0000","last_became_peered":"2023-11-24T08:07:42.345623+0000","last_unstale":"2023-11-24T08:07:43.745412+0000","last_undegraded":"2023-11-24T08:07:43.745412+0000","last_fullsized":"2023-11-24T08:07:43.745412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:31.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745807+0000","last_change":"2023-11-24T08:07:41.352128+0000","last_active":"2023-11-24T08:07:43.745807+0000","last_peered":"2023-11-24T08:07:43.745807+0000","last_clean":"2023-11-24T08:07:43.745807+0000","last_became_active":"2023-11-24T08:07:41.351839+0000","last_became_peered":"2023-11-24T08:07:41.351839+0000","last_unstale":"2023-11-24T08:07:43.745807+0000","last_undegraded":"2023-11-24T08:07:43.745807+0000","last_fullsized":"2023-11-24T08:07:43.745807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.979865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373756+0000","last_change":"2023-11-24T08:07:42.368618+0000","last_active":"2023-11-24T08:07:42.373756+0000","last_peered":"2023-11-24T08:07:42.373756+0000","last_clean":"2023-11-24T08:07:42.373756+0000","last_became_active":"2023-11-24T08:07:42.368346+0000","last_became_peered":"2023-11-24T08:07:42.368346+0000","last_unstale":"2023-11-24T08:07:42.373756+0000","last_undegraded":"2023-11-24T08:07:42.373756+0000","last_fullsized":"2023-11-24T08:07:42.373756+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:59.024929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358618+0000","last_change":"2023-11-24T08:07:41.341274+0000","last_active":"2023-11-24T08:07:43.358618+0000","last_peered":"2023-11-24T08:07:43.358618+0000","last_clean":"2023-11-24T08:07:43.358618+0000","last_became_active":"2023-11-24T08:07:41.340971+0000","last_became_peered":"2023-11-24T08:07:41.340971+0000","last_unstale":"2023-11-24T08:07:43.358618+0000","last_undegraded":"2023-11-24T08:07:43.358618+0000","last_fullsized":"2023-11-24T08:07:43.358618+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:00.860722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.374080+0000","last_change":"2023-11-24T08:07:42.368422+0000","last_active":"2023-11-24T08:07:42.374080+0000","last_peered":"2023-11-24T08:07:42.374080+0000","last_clean":"2023-11-24T08:07:42.374080+0000","last_became_active":"2023-11-24T08:07:42.368017+0000","last_became_peered":"2023-11-24T08:07:42.368017+0000","last_unstale":"2023-11-24T08:07:42.374080+0000","last_undegraded":"2023-11-24T08:07:42.374080+0000","last_fullsized":"2023-11-24T08:07:42.374080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:44.851906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.336673+0000","last_change":"2023-11-24T08:07:41.349579+0000","last_active":"2023-11-24T08:07:42.336673+0000","last_peered":"2023-11-24T08:07:42.336673+0000","last_clean":"2023-11-24T08:07:42.336673+0000","last_became_active":"2023-11-24T08:07:41.349294+0000","last_became_peered":"2023-11-24T08:07:41.349294+0000","last_unstale":"2023-11-24T08:07:42.336673+0000","last_undegraded":"2023-11-24T08:07:42.336673+0000","last_fullsized":"2023-11-24T08:07:42.336673+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:01.313679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358195+0000","last_change":"2023-11-24T08:07:42.347859+0000","last_active":"2023-11-24T08:07:43.358195+0000","last_peered":"2023-11-24T08:07:43.358195+0000","last_clean":"2023-11-24T08:07:43.358195+0000","last_became_active":"2023-11-24T08:07:42.347560+0000","last_became_peered":"2023-11-24T08:07:42.347560+0000","last_unstale":"2023-11-24T08:07:43.358195+0000","last_undegraded":"2023-11-24T08:07:43.358195+0000","last_fullsized":"2023-11-24T08:07:43.358195+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:11.358935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358304+0000","last_change":"2023-11-24T08:07:41.349178+0000","last_active":"2023-11-24T08:07:43.358304+0000","last_peered":"2023-11-24T08:07:43.358304+0000","last_clean":"2023-11-24T08:07:43.358304+0000","last_became_active":"2023-11-24T08:07:41.348881+0000","last_became_peered":"2023-11-24T08:07:41.348881+0000","last_unstale":"2023-11-24T08:07:43.358304+0000","last_undegraded":"2023-11-24T08:07:43.358304+0000","last_fullsized":"2023-11-24T08:07:43.358304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:33.050522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361658+0000","last_change":"2023-11-24T08:07:42.373368+0000","last_active":"2023-11-24T08:07:43.361658+0000","last_peered":"2023-11-24T08:07:43.361658+0000","last_clean":"2023-11-24T08:07:43.361658+0000","last_became_active":"2023-11-24T08:07:42.372448+0000","last_became_peered":"2023-11-24T08:07:42.372448+0000","last_unstale":"2023-11-24T08:07:43.361658+0000","last_undegraded":"2023-11-24T08:07:43.361658+0000","last_fullsized":"2023-11-24T08:07:43.361658+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:39.894508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.326440+0000","last_change":"2023-11-24T08:07:41.342093+0000","last_active":"2023-11-24T08:07:42.326440+0000","last_peered":"2023-11-24T08:07:42.326440+0000","last_clean":"2023-11-24T08:07:42.326440+0000","last_became_active":"2023-11-24T08:07:41.341739+0000","last_became_peered":"2023-11-24T08:07:41.341739+0000","last_unstale":"2023-11-24T08:07:42.326440+0000","last_undegraded":"2023-11-24T08:07:42.326440+0000","last_fullsized":"2023-11-24T08:07:42.326440+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:31.179421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.349882+0000","last_change":"2023-11-24T08:07:42.346376+0000","last_active":"2023-11-24T08:07:42.349882+0000","last_peered":"2023-11-24T08:07:42.349882+0000","last_clean":"2023-11-24T08:07:42.349882+0000","last_became_active":"2023-11-24T08:07:42.345973+0000","last_became_peered":"2023-11-24T08:07:42.345973+0000","last_unstale":"2023-11-24T08:07:42.349882+0000","last_undegraded":"2023-11-24T08:07:42.349882+0000","last_fullsized":"2023-11-24T08:07:42.349882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:57:09.164635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362519+0000","last_change":"2023-11-24T08:07:41.345970+0000","last_active":"2023-11-24T08:07:43.362519+0000","last_peered":"2023-11-24T08:07:43.362519+0000","last_clean":"2023-11-24T08:07:43.362519+0000","last_became_active":"2023-11-24T08:07:41.345591+0000","last_became_peered":"2023-11-24T08:07:41.345591+0000","last_unstale":"2023-11-24T08:07:43.362519+0000","last_undegraded":"2023-11-24T08:07:43.362519+0000","last_fullsized":"2023-11-24T08:07:43.362519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:08.824076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363607+0000","last_change":"2023-11-24T08:07:42.353886+0000","last_active":"2023-11-24T08:07:43.363607+0000","last_peered":"2023-11-24T08:07:43.363607+0000","last_clean":"2023-11-24T08:07:43.363607+0000","last_became_active":"2023-11-24T08:07:42.353544+0000","last_became_peered":"2023-11-24T08:07:42.353544+0000","last_unstale":"2023-11-24T08:07:43.363607+0000","last_undegraded":"2023-11-24T08:07:43.363607+0000","last_fullsized":"2023-11-24T08:07:43.363607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:36.769677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363516+0000","last_change":"2023-11-24T08:07:41.352559+0000","last_active":"2023-11-24T08:07:43.363516+0000","last_peered":"2023-11-24T08:07:43.363516+0000","last_clean":"2023-11-24T08:07:43.363516+0000","last_became_active":"2023-11-24T08:07:41.352196+0000","last_became_peered":"2023-11-24T08:07:41.352196+0000","last_unstale":"2023-11-24T08:07:43.363516+0000","last_undegraded":"2023-11-24T08:07:43.363516+0000","last_fullsized":"2023-11-24T08:07:43.363516+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:10.411816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.366370+0000","last_change":"2023-11-24T08:07:42.364684+0000","last_active":"2023-11-24T08:07:42.366370+0000","last_peered":"2023-11-24T08:07:42.366370+0000","last_clean":"2023-11-24T08:07:42.366370+0000","last_became_active":"2023-11-24T08:07:42.364067+0000","last_became_peered":"2023-11-24T08:07:42.364067+0000","last_unstale":"2023-11-24T08:07:42.366370+0000","last_undegraded":"2023-11-24T08:07:42.366370+0000","last_fullsized":"2023-11-24T08:07:42.366370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:11:44.930805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361499+0000","last_change":"2023-11-24T08:07:41.352560+0000","last_active":"2023-11-24T08:07:43.361499+0000","last_peered":"2023-11-24T08:07:43.361499+0000","last_clean":"2023-11-24T08:07:43.361499+0000","last_became_active":"2023-11-24T08:07:41.351621+0000","last_became_peered":"2023-11-24T08:07:41.351621+0000","last_unstale":"2023-11-24T08:07:43.361499+0000","last_undegraded":"2023-11-24T08:07:43.361499+0000","last_fullsized":"2023-11-24T08:07:43.361499+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:48.251836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T08:07:41.342701+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:07:41.342701+0000","last_peered":"2023-11-24T08:07:41.342701+0000","last_clean":"2023-11-24T08:07:41.342701+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:07:41.342701+0000","last_undegraded":"2023-11-24T08:07:41.342701+0000","last_fullsized":"2023-11-24T08:07:41.342701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1490944,"data_stored":968334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0860000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.484,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.85199999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2789999999999999,"15min":1.3009999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6639999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3420000000000001,"15min":1.3839999999999999},"min":{"1min":0.97299999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5569999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.712}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.151,"15min":1.1479999999999999},"min":{"1min":0.70099999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0840000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.3069999999999999,"15min":1.3160000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.575}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2829999999999999,"15min":1.321},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.966,"5min":2.524,"15min":2.524},"last":1.117},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.454,"15min":1.5169999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8939999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.7889999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4550000000000001,"15min":1.532},"min":{"1min":0.88400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6419999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.526},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.379},"min":{"1min":0.84499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.484}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.581,"15min":1.619},"min":{"1min":1.0229999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.173,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.601,"15min":1.661},"min":{"1min":1.069,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7350000000000001,"5min":2.944,"15min":2.944},"last":1.8240000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.702,"15min":1.7589999999999999},"min":{"1min":1.252,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.649,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.8779999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6060000000000001,"15min":1.665},"min":{"1min":1.0509999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.841,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.75}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":909312,"data_stored":383742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.276,"15min":1.3300000000000001},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.113,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.274,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.2709999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1539999999999999,"15min":1.159},"min":{"1min":0.68400000000000005,"5min":0.625,"15min":0.625},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2370000000000001,"15min":1.3069999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6830000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.633}]},{"osd":2,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.232,"15min":1.296},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.508,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.419,"15min":1.5069999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.752,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.3879999999999999,"15min":1.444},"min":{"1min":0.79200000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.931,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.206},{"interface":"front","average":{"1min":1.407,"5min":1.4690000000000001,"15min":1.498},"min":{"1min":0.94899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.968,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.105}]},{"osd":4,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.6279999999999999,"15min":1.758},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2120000000000002,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":0.995},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.4390000000000001,"15min":1.472},"min":{"1min":0.86199999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.855,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.488}]},{"osd":5,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6679999999999999,"15min":1.8080000000000001},"min":{"1min":1.0029999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.893,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.806,"15min":1.9490000000000001},"min":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.901,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6379999999999999,"15min":1.6890000000000001},"min":{"1min":0.75800000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.379,"5min":2.98,"15min":2.98},"last":1.373},{"interface":"front","average":{"1min":1.482,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.1659999999999999,"5min":2.29,"15min":2.29},"last":1.0549999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":892928,"data_stored":377440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.048},"min":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2949999999999999,"5min":1.764,"15min":1.764},"last":1.048},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0329999999999999,"15min":1.006},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.5369999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6950000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.874},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0780000000000001,"15min":1.0509999999999999},"min":{"1min":0.70899999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.458,"5min":1.635,"15min":1.635},"last":1.1519999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2090000000000001,"15min":1.2130000000000001},"min":{"1min":0.77100000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.673,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.256},{"interface":"front","average":{"1min":1.226,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.745,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6259999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.012}]},{"osd":3,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.427},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.575,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2470000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5669999999999999,"5min":1.889,"15min":1.889},"last":1.218}]},{"osd":4,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.284,"15min":1.298},"min":{"1min":0.79100000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.871,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.3},{"interface":"front","average":{"1min":1.429,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.68400000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.766,"5min":2.27,"15min":2.27},"last":1.101}]},{"osd":6,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.448,"15min":1.47},"min":{"1min":0.97799999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7909999999999999,"5min":3.806,"15min":3.806},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.409,"15min":1.4079999999999999},"min":{"1min":0.84699999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0169999999999999,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5189999999999999,"15min":1.5269999999999999},"min":{"1min":0.63,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.974,"5min":2.923,"15min":2.923},"last":1.377},{"interface":"front","average":{"1min":1.569,"5min":1.54,"15min":1.53},"min":{"1min":1.0640000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.214,"5min":2.859,"15min":2.859},"last":1.4099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":892928,"data_stored":377420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0409999999999999,"15min":1.042},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4399999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.371},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.1279999999999999},"min":{"1min":0.77900000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.595,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.258}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.079,"15min":1.0840000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.377,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.79000000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5640000000000001,"5min":1.903,"15min":1.903},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1890000000000001,"15min":1.2},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.589,"5min":1.823,"15min":1.823},"last":1.151},{"interface":"front","average":{"1min":1.155,"5min":1.2410000000000001,"15min":1.258},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.29}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.2969999999999999},"min":{"1min":0.81399999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7330000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3109999999999999,"15min":1.3149999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9530000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3480000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.282,"15min":1.296},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.617,"5min":1.944,"15min":1.944},"last":1.319},{"interface":"front","average":{"1min":1.26,"5min":1.3400000000000001,"15min":1.3580000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":2.012,"15min":2.012},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.377,"15min":1.3680000000000001},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4890000000000001,"15min":1.5069999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.927,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.5740000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.4330000000000001,"15min":1.431},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.768,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.554,"15min":1.5589999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.411,"5min":2.722,"15min":2.722},"last":1.2430000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":897024,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.82099999999999995},"min":{"1min":0.50800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.569,"5min":1.585,"15min":1.585},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.92300000000000004,"15min":0.94799999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4930000000000001,"5min":1.51,"15min":1.51},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.075,"15min":1.151},"min":{"1min":0.63100000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.835,"5min":3.246,"15min":3.246},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97399999999999998,"15min":0.96699999999999997},"min":{"1min":0.54900000000000004,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4179999999999999,"5min":1.665,"15min":1.665},"last":1.5369999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1399999999999999,"15min":1.1819999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7410000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1160000000000001,"15min":1.155},"min":{"1min":0.66800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.399,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.4139999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2450000000000001,"15min":1.264},"min":{"1min":0.73299999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.671,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.355,"15min":1.3899999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.665,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.3799999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.4199999999999999,"15min":1.4590000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9430000000000001,"5min":2.395,"15min":2.395},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.306,"15min":1.3},"min":{"1min":0.90400000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9399999999999999,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.3380000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.421,"15min":1.4319999999999999},"min":{"1min":0.94099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8899999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.609},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3979999999999999,"15min":1.4179999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.1200000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4219999999999999,"15min":1.444},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.994,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.528,"15min":1.581},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.895,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.492}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":901120,"data_stored":378962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.417,"5min":1.524,"15min":1.524},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":1.016},"min":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5740000000000001,"5min":2.133,"15min":2.133},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1419999999999999,"15min":1.2250000000000001},"min":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8280000000000001,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0029999999999999,"15min":1.069},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4550000000000001,"5min":1.998,"15min":1.998},"last":1.425}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.321},"min":{"1min":0.73299999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6879999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.147,"15min":1.1950000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7809999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.2509999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.754,"5min":0.442,"15min":0.442},"max":{"1min":1.7969999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.048},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.4359999999999999,"15min":1.502},"min":{"1min":1.018,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.758,"5min":3.161,"15min":3.161},"last":1.3879999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.345,"15min":1.393},"min":{"1min":0.77000000000000002,"5min":0.623,"15min":0.623},"max":{"1min":1.752,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":2.1629999999999998},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.355,"15min":1.411},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.96,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.149}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.5169999999999999,"15min":1.593},"min":{"1min":0.96199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8979999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.5649999999999999},"min":{"1min":0.80900000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8520000000000001,"5min":2.702,"15min":2.702},"last":2.0510000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.448,"15min":1.514},"min":{"1min":0.78200000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0419999999999998,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.5660000000000001,"15min":1.6439999999999999},"min":{"1min":0.89300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.952,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.52}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":379578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.878,"15min":0.89200000000000002},"min":{"1min":0.70799999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.028},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0029999999999999,"15min":1.0580000000000001},"min":{"1min":0.56599999999999995,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.397,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.387}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.03},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.746,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.081,"5min":1.179,"15min":1.2290000000000001},"min":{"1min":0.76400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.794,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.2729999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.2729999999999999},"min":{"1min":0.72299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.897,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.091,"5min":1.177,"15min":1.232},"min":{"1min":0.66500000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.6890000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.1679999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.5620000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4199999999999999,"15min":1.454},"min":{"1min":0.95799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.5169999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4259999999999999,"15min":1.4570000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0089999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.452,"15min":1.478},"min":{"1min":0.91000000000000003,"5min":0.871,"15min":0.871},"max":{"1min":1.903,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.629}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.4319999999999999,"15min":1.458},"min":{"1min":0.83099999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.774,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.766},{"interface":"front","average":{"1min":1.373,"5min":1.4430000000000001,"15min":1.4830000000000001},"min":{"1min":0.79200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8420000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.5640000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":0.996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8700000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.591},{"interface":"front","average":{"1min":1.55,"5min":1.5680000000000001,"15min":1.595},"min":{"1min":0.96599999999999997,"5min":0.872,"15min":0.872},"max":{"1min":1.8500000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.8560000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":966699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.52500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.97799999999999998,"15min":0.94799999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.111,"15min":1.075},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.899,"5min":3.899,"15min":3.899},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1,"15min":0.95999999999999996},"min":{"1min":0.498,"5min":0.443,"15min":0.443},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.0700000000000001,"15min":1.0409999999999999},"min":{"1min":0.627,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.631,"5min":1.2729999999999999,"15min":1.2270000000000001},"min":{"1min":0.57299999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":6.3570000000000002,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":0.89600000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.3069999999999999,"15min":1.2709999999999999},"min":{"1min":0.92000000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.137},{"interface":"front","average":{"1min":1.7070000000000001,"5min":1.5,"15min":1.482},"min":{"1min":1.022,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":3.399,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.0620000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.4399999999999999,"15min":1.3899999999999999},"min":{"1min":0.88100000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":5.9939999999999998,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":1.026,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.391,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.2110000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.083},{"interface":"front","average":{"1min":1.744,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":1.0840000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.879,"5min":3.879,"15min":3.879},"last":1.333}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.8029999999999999,"5min":1.5660000000000001,"15min":1.548},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":3.4540000000000002,"5min":3.823,"15min":3.823},"last":0.999},{"interface":"front","average":{"1min":1.75,"5min":1.5209999999999999,"15min":1.4970000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.1930000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:07:46.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:07:47.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.169+0000 7fa95759e640 1 Processor -- start 2023-11-24T08:07:47.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.169+0000 7fa95759e640 1 -- start start 2023-11-24T08:07:47.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.171+0000 7fa95759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa958157770 0x7fa95806cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.171+0000 7fa95759e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa95815df80 0x7fa958070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.171+0000 7fa95759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa958161670 con 0x7fa95806d080 2023-11-24T08:07:47.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.171+0000 7fa95759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95815e410 con 0x7fa958157770 2023-11-24T08:07:47.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.171+0000 7fa95759e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95806d500 con 0x7fa95815df80 2023-11-24T08:07:47.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.171+0000 7fa95659c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa958157770 0x7fa95806cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.171+0000 7fa94ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa95806d080 0x7fa95815d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37800/0 (socket says 172.21.15.64:37800) 2023-11-24T08:07:47.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.171+0000 7fa94ffff640 1 -- 172.21.15.64:0/1431312462 learned_addr learned my addr 172.21.15.64:0/1431312462 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:47.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.172+0000 7fa95659c640 1 -- 172.21.15.64:0/1431312462 >> v2:172.21.15.91:3300/0 conn(0x7fa95815df80 msgr2=0x7fa958070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:47.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.172+0000 7fa956d9d640 1 --2- 172.21.15.64:0/1431312462 >> v2:172.21.15.91:3300/0 conn(0x7fa95815df80 0x7fa958070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.172+0000 7fa95659c640 1 --2- 172.21.15.64:0/1431312462 >> v2:172.21.15.91:3300/0 conn(0x7fa95815df80 0x7fa958070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.172+0000 7fa95659c640 1 -- 172.21.15.64:0/1431312462 >> v1:172.21.15.64:6789/0 conn(0x7fa95806d080 legacy=0x7fa95815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.172+0000 7fa955d9b640 1 -- 172.21.15.64:0/1431312462 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407431372 0 0) 0x7fa958161670 con 0x7fa95806d080 2023-11-24T08:07:47.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.172+0000 7fa95659c640 1 -- 172.21.15.64:0/1431312462 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95815e610 con 0x7fa958157770 2023-11-24T08:07:47.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.172+0000 7fa956d9d640 1 --2- 172.21.15.64:0/1431312462 >> v2:172.21.15.91:3300/0 conn(0x7fa95815df80 0x7fa958070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:47.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.173+0000 7fa95659c640 1 --2- 172.21.15.64:0/1431312462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958157770 0x7fa95806cb40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa95806ca90 tx=0x7fa94802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8084e40d294ec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:47.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.173+0000 7fa955d9b640 1 -- 172.21.15.64:0/1431312462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948031830 con 0x7fa958157770 2023-11-24T08:07:47.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.173+0000 7fa955d9b640 1 -- 172.21.15.64:0/1431312462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa948031990 con 0x7fa958157770 2023-11-24T08:07:47.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.173+0000 7fa955d9b640 1 -- 172.21.15.64:0/1431312462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948031d20 con 0x7fa958157770 2023-11-24T08:07:47.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.174+0000 7fa95759e640 1 -- 172.21.15.64:0/1431312462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958157770 msgr2=0x7fa95806cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.174+0000 7fa95759e640 1 --2- 172.21.15.64:0/1431312462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958157770 0x7fa95806cb40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa95806ca90 tx=0x7fa94802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.174+0000 7fa95759e640 1 -- 172.21.15.64:0/1431312462 shutdown_connections 2023-11-24T08:07:47.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.174+0000 7fa95759e640 1 --2- 172.21.15.64:0/1431312462 >> v2:172.21.15.91:3300/0 conn(0x7fa95815df80 0x7fa958070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.174+0000 7fa95759e640 1 --2- 172.21.15.64:0/1431312462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958157770 0x7fa95806cb40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.174+0000 7fa95759e640 1 -- 172.21.15.64:0/1431312462 >> 172.21.15.64:0/1431312462 conn(0x7fa95806a860 msgr2=0x7fa95806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:47.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.174+0000 7fa95759e640 1 -- 172.21.15.64:0/1431312462 shutdown_connections 2023-11-24T08:07:47.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.174+0000 7fa95759e640 1 -- 172.21.15.64:0/1431312462 wait complete. 2023-11-24T08:07:47.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.176+0000 7fa95759e640 1 Processor -- start 2023-11-24T08:07:47.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.176+0000 7fa95759e640 1 -- start start 2023-11-24T08:07:47.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa95759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa95806d080 0x7fa9581505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa95759e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa958157770 0x7fa95814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa95759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa95815df80 0x7fa958152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa95759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581503f0 con 0x7fa95815df80 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa95759e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958150b30 con 0x7fa958157770 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa95759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95814fe30 con 0x7fa95806d080 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa956d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa95815df80 0x7fa958152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa956d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa95815df80 0x7fa958152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48924/0 (socket says 172.21.15.64:48924) 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa956d9d640 1 -- 172.21.15.64:0/3125067676 learned_addr learned my addr 172.21.15.64:0/3125067676 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa956d9d640 1 -- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa95806d080 msgr2=0x7fa9581505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa95659c640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa95806d080 0x7fa9581505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa956d9d640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa95806d080 0x7fa9581505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa956d9d640 1 -- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa958157770 msgr2=0x7fa95814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa956d9d640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa958157770 0x7fa95814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.177+0000 7fa956d9d640 1 -- 172.21.15.64:0/3125067676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009400 con 0x7fa95815df80 2023-11-24T08:07:47.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.178+0000 7fa956d9d640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa95815df80 0x7fa958152af0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa95000cef0 tx=0x7fa95000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:47.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.178+0000 7fa94effd640 1 -- 172.21.15.64:0/3125067676 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000d840 con 0x7fa95815df80 2023-11-24T08:07:47.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.178+0000 7fa94effd640 1 -- 172.21.15.64:0/3125067676 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95000d9c0 con 0x7fa95815df80 2023-11-24T08:07:47.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.178+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa958150110 con 0x7fa95815df80 2023-11-24T08:07:47.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.178+0000 7fa94effd640 1 -- 172.21.15.64:0/3125067676 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000dd70 con 0x7fa95815df80 2023-11-24T08:07:47.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.178+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa958151280 con 0x7fa95815df80 2023-11-24T08:07:47.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.179+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9581514a0 con 0x7fa95815df80 2023-11-24T08:07:47.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.180+0000 7fa94effd640 1 -- 172.21.15.64:0/3125067676 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa95001e3b0 con 0x7fa95815df80 2023-11-24T08:07:47.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.182+0000 7fa94effd640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa93806b230 0x7fa93806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.182+0000 7fa94effd640 1 -- 172.21.15.64:0/3125067676 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa95008a7f0 con 0x7fa95815df80 2023-11-24T08:07:47.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.183+0000 7fa95659c640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa93806b230 0x7fa93806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.184+0000 7fa95659c640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa93806b230 0x7fa93806d6f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa948036000 tx=0x7fa948008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:47.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.186+0000 7fa94effd640 1 -- 172.21.15.64:0/3125067676 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9500553d0 con 0x7fa95815df80 2023-11-24T08:07:47.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.378+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa958161300 con 0x7fa93806b230 2023-11-24T08:07:47.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.402+0000 7fa94effd640 1 -- 172.21.15.64:0/3125067676 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287009 (secure 0 0 0) 0x7fa948031b00 con 0x7fa93806b230 2023-11-24T08:07:47.408 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:47.409 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:07:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa93806b230 msgr2=0x7fa93806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa93806b230 0x7fa93806d6f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa948036000 tx=0x7fa948008030 comp rx=0 tx=0).stop 2023-11-24T08:07:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa95815df80 msgr2=0x7fa958152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa95815df80 0x7fa958152af0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa95000cef0 tx=0x7fa95000a710 comp rx=0 tx=0).stop 2023-11-24T08:07:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 shutdown_connections 2023-11-24T08:07:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa93806b230 0x7fa93806d6f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa95815df80 0x7fa958152af0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa958157770 0x7fa95814c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 --2- 172.21.15.64:0/3125067676 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa95806d080 0x7fa9581505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 >> 172.21.15.64:0/3125067676 conn(0x7fa95806a860 msgr2=0x7fa958156940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 shutdown_connections 2023-11-24T08:07:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.408+0000 7fa95759e640 1 -- 172.21.15.64:0/3125067676 wait complete. 2023-11-24T08:07:47.429 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2023-11-24T08:07:46.097645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249308,"kb_used_data":8176,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725412,"statfs":{"total":773094113280,"available":772838821888,"internally_reserved":0,"allocated":8372224,"data_stored":4209637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.792864"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.371700+0000","last_change":"2023-11-24T08:07:42.367144+0000","last_active":"2023-11-24T08:07:42.371700+0000","last_peered":"2023-11-24T08:07:42.371700+0000","last_clean":"2023-11-24T08:07:42.371700+0000","last_became_active":"2023-11-24T08:07:42.366553+0000","last_became_peered":"2023-11-24T08:07:42.366553+0000","last_unstale":"2023-11-24T08:07:42.371700+0000","last_undegraded":"2023-11-24T08:07:42.371700+0000","last_fullsized":"2023-11-24T08:07:42.371700+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:07.735220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363557+0000","last_change":"2023-11-24T08:07:42.374234+0000","last_active":"2023-11-24T08:07:43.363557+0000","last_peered":"2023-11-24T08:07:43.363557+0000","last_clean":"2023-11-24T08:07:43.363557+0000","last_became_active":"2023-11-24T08:07:42.373895+0000","last_became_peered":"2023-11-24T08:07:42.373895+0000","last_unstale":"2023-11-24T08:07:43.363557+0000","last_undegraded":"2023-11-24T08:07:43.363557+0000","last_fullsized":"2023-11-24T08:07:43.363557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:00:05.987484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.363723+0000","last_change":"2023-11-24T08:07:42.356987+0000","last_active":"2023-11-24T08:07:42.363723+0000","last_peered":"2023-11-24T08:07:42.363723+0000","last_clean":"2023-11-24T08:07:42.363723+0000","last_became_active":"2023-11-24T08:07:42.356692+0000","last_became_peered":"2023-11-24T08:07:42.356692+0000","last_unstale":"2023-11-24T08:07:42.363723+0000","last_undegraded":"2023-11-24T08:07:42.363723+0000","last_fullsized":"2023-11-24T08:07:42.363723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:08:41.203736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.355661+0000","last_change":"2023-11-24T08:07:42.352914+0000","last_active":"2023-11-24T08:07:42.355661+0000","last_peered":"2023-11-24T08:07:42.355661+0000","last_clean":"2023-11-24T08:07:42.355661+0000","last_became_active":"2023-11-24T08:07:42.352564+0000","last_became_peered":"2023-11-24T08:07:42.352564+0000","last_unstale":"2023-11-24T08:07:42.355661+0000","last_undegraded":"2023-11-24T08:07:42.355661+0000","last_fullsized":"2023-11-24T08:07:42.355661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:19.261764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358186+0000","last_change":"2023-11-24T08:07:42.362583+0000","last_active":"2023-11-24T08:07:43.358186+0000","last_peered":"2023-11-24T08:07:43.358186+0000","last_clean":"2023-11-24T08:07:43.358186+0000","last_became_active":"2023-11-24T08:07:42.362237+0000","last_became_peered":"2023-11-24T08:07:42.362237+0000","last_unstale":"2023-11-24T08:07:43.358186+0000","last_undegraded":"2023-11-24T08:07:43.358186+0000","last_fullsized":"2023-11-24T08:07:43.358186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:47:20.878860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364311+0000","last_change":"2023-11-24T08:07:42.369124+0000","last_active":"2023-11-24T08:07:43.364311+0000","last_peered":"2023-11-24T08:07:43.364311+0000","last_clean":"2023-11-24T08:07:43.364311+0000","last_became_active":"2023-11-24T08:07:42.368858+0000","last_became_peered":"2023-11-24T08:07:42.368858+0000","last_unstale":"2023-11-24T08:07:43.364311+0000","last_undegraded":"2023-11-24T08:07:43.364311+0000","last_fullsized":"2023-11-24T08:07:43.364311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:28.325330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.744796+0000","last_change":"2023-11-24T08:07:42.347669+0000","last_active":"2023-11-24T08:07:43.744796+0000","last_peered":"2023-11-24T08:07:43.744796+0000","last_clean":"2023-11-24T08:07:43.744796+0000","last_became_active":"2023-11-24T08:07:42.341994+0000","last_became_peered":"2023-11-24T08:07:42.341994+0000","last_unstale":"2023-11-24T08:07:43.744796+0000","last_undegraded":"2023-11-24T08:07:43.744796+0000","last_fullsized":"2023-11-24T08:07:43.744796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:56:18.150787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362982+0000","last_change":"2023-11-24T08:07:42.355035+0000","last_active":"2023-11-24T08:07:43.362982+0000","last_peered":"2023-11-24T08:07:43.362982+0000","last_clean":"2023-11-24T08:07:43.362982+0000","last_became_active":"2023-11-24T08:07:42.354745+0000","last_became_peered":"2023-11-24T08:07:42.354745+0000","last_unstale":"2023-11-24T08:07:43.362982+0000","last_undegraded":"2023-11-24T08:07:43.362982+0000","last_fullsized":"2023-11-24T08:07:43.362982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:53:07.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358895+0000","last_change":"2023-11-24T08:07:42.362317+0000","last_active":"2023-11-24T08:07:43.358895+0000","last_peered":"2023-11-24T08:07:43.358895+0000","last_clean":"2023-11-24T08:07:43.358895+0000","last_became_active":"2023-11-24T08:07:42.361138+0000","last_became_peered":"2023-11-24T08:07:42.361138+0000","last_unstale":"2023-11-24T08:07:43.358895+0000","last_undegraded":"2023-11-24T08:07:43.358895+0000","last_fullsized":"2023-11-24T08:07:43.358895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:19.261764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365695+0000","last_change":"2023-11-24T08:07:42.366130+0000","last_active":"2023-11-24T08:07:43.365695+0000","last_peered":"2023-11-24T08:07:43.365695+0000","last_clean":"2023-11-24T08:07:43.365695+0000","last_became_active":"2023-11-24T08:07:42.365841+0000","last_became_peered":"2023-11-24T08:07:42.365841+0000","last_unstale":"2023-11-24T08:07:43.365695+0000","last_undegraded":"2023-11-24T08:07:43.365695+0000","last_fullsized":"2023-11-24T08:07:43.365695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:28.325330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.370833+0000","last_change":"2023-11-24T08:07:42.365927+0000","last_active":"2023-11-24T08:07:42.370833+0000","last_peered":"2023-11-24T08:07:42.370833+0000","last_clean":"2023-11-24T08:07:42.370833+0000","last_became_active":"2023-11-24T08:07:42.365619+0000","last_became_peered":"2023-11-24T08:07:42.365619+0000","last_unstale":"2023-11-24T08:07:42.370833+0000","last_undegraded":"2023-11-24T08:07:42.370833+0000","last_fullsized":"2023-11-24T08:07:42.370833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:07.490591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.351875+0000","last_change":"2023-11-24T08:07:42.340355+0000","last_active":"2023-11-24T08:07:42.351875+0000","last_peered":"2023-11-24T08:07:42.351875+0000","last_clean":"2023-11-24T08:07:42.351875+0000","last_became_active":"2023-11-24T08:07:42.340015+0000","last_became_peered":"2023-11-24T08:07:42.340015+0000","last_unstale":"2023-11-24T08:07:42.351875+0000","last_undegraded":"2023-11-24T08:07:42.351875+0000","last_fullsized":"2023-11-24T08:07:42.351875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:39.534095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.365945+0000","last_change":"2023-11-24T08:07:42.365334+0000","last_active":"2023-11-24T08:07:42.365945+0000","last_peered":"2023-11-24T08:07:42.365945+0000","last_clean":"2023-11-24T08:07:42.365945+0000","last_became_active":"2023-11-24T08:07:42.362607+0000","last_became_peered":"2023-11-24T08:07:42.362607+0000","last_unstale":"2023-11-24T08:07:42.365945+0000","last_undegraded":"2023-11-24T08:07:42.365945+0000","last_fullsized":"2023-11-24T08:07:42.365945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:14.975240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358219+0000","last_change":"2023-11-24T08:07:42.352402+0000","last_active":"2023-11-24T08:07:43.358219+0000","last_peered":"2023-11-24T08:07:43.358219+0000","last_clean":"2023-11-24T08:07:43.358219+0000","last_became_active":"2023-11-24T08:07:42.352065+0000","last_became_peered":"2023-11-24T08:07:42.352065+0000","last_unstale":"2023-11-24T08:07:43.358219+0000","last_undegraded":"2023-11-24T08:07:43.358219+0000","last_fullsized":"2023-11-24T08:07:43.358219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:32.193825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.349975+0000","last_change":"2023-11-24T08:07:42.344531+0000","last_active":"2023-11-24T08:07:42.349975+0000","last_peered":"2023-11-24T08:07:42.349975+0000","last_clean":"2023-11-24T08:07:42.349975+0000","last_became_active":"2023-11-24T08:07:42.344237+0000","last_became_peered":"2023-11-24T08:07:42.344237+0000","last_unstale":"2023-11-24T08:07:42.349975+0000","last_undegraded":"2023-11-24T08:07:42.349975+0000","last_fullsized":"2023-11-24T08:07:42.349975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:07.735220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361985+0000","last_change":"2023-11-24T08:07:42.367021+0000","last_active":"2023-11-24T08:07:43.361985+0000","last_peered":"2023-11-24T08:07:43.361985+0000","last_clean":"2023-11-24T08:07:43.361985+0000","last_became_active":"2023-11-24T08:07:42.366642+0000","last_became_peered":"2023-11-24T08:07:42.366642+0000","last_unstale":"2023-11-24T08:07:43.361985+0000","last_undegraded":"2023-11-24T08:07:43.361985+0000","last_fullsized":"2023-11-24T08:07:43.361985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:53:07.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358927+0000","last_change":"2023-11-24T08:07:42.361039+0000","last_active":"2023-11-24T08:07:43.358927+0000","last_peered":"2023-11-24T08:07:43.358927+0000","last_clean":"2023-11-24T08:07:43.358927+0000","last_became_active":"2023-11-24T08:07:42.360740+0000","last_became_peered":"2023-11-24T08:07:42.360740+0000","last_unstale":"2023-11-24T08:07:43.358927+0000","last_undegraded":"2023-11-24T08:07:43.358927+0000","last_fullsized":"2023-11-24T08:07:43.358927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:39.534095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.366138+0000","last_change":"2023-11-24T08:07:42.363424+0000","last_active":"2023-11-24T08:07:42.366138+0000","last_peered":"2023-11-24T08:07:42.366138+0000","last_clean":"2023-11-24T08:07:42.366138+0000","last_became_active":"2023-11-24T08:07:42.363071+0000","last_became_peered":"2023-11-24T08:07:42.363071+0000","last_unstale":"2023-11-24T08:07:42.366138+0000","last_undegraded":"2023-11-24T08:07:42.366138+0000","last_fullsized":"2023-11-24T08:07:42.366138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:38.089553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364306+0000","last_change":"2023-11-24T08:07:42.353530+0000","last_active":"2023-11-24T08:07:43.364306+0000","last_peered":"2023-11-24T08:07:43.364306+0000","last_clean":"2023-11-24T08:07:43.364306+0000","last_became_active":"2023-11-24T08:07:42.353047+0000","last_became_peered":"2023-11-24T08:07:42.353047+0000","last_unstale":"2023-11-24T08:07:43.364306+0000","last_undegraded":"2023-11-24T08:07:43.364306+0000","last_fullsized":"2023-11-24T08:07:43.364306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:50.994269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.359665+0000","last_change":"2023-11-24T08:07:42.352171+0000","last_active":"2023-11-24T08:07:42.359665+0000","last_peered":"2023-11-24T08:07:42.359665+0000","last_clean":"2023-11-24T08:07:42.359665+0000","last_became_active":"2023-11-24T08:07:42.351899+0000","last_became_peered":"2023-11-24T08:07:42.351899+0000","last_unstale":"2023-11-24T08:07:42.359665+0000","last_undegraded":"2023-11-24T08:07:42.359665+0000","last_fullsized":"2023-11-24T08:07:42.359665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:23.845233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364473+0000","last_change":"2023-11-24T08:07:42.347317+0000","last_active":"2023-11-24T08:07:43.364473+0000","last_peered":"2023-11-24T08:07:43.364473+0000","last_clean":"2023-11-24T08:07:43.364473+0000","last_became_active":"2023-11-24T08:07:42.346730+0000","last_became_peered":"2023-11-24T08:07:42.346730+0000","last_unstale":"2023-11-24T08:07:43.364473+0000","last_undegraded":"2023-11-24T08:07:43.364473+0000","last_fullsized":"2023-11-24T08:07:43.364473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:30.293229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363375+0000","last_change":"2023-11-24T08:07:42.353844+0000","last_active":"2023-11-24T08:07:43.363375+0000","last_peered":"2023-11-24T08:07:43.363375+0000","last_clean":"2023-11-24T08:07:43.363375+0000","last_became_active":"2023-11-24T08:07:42.352938+0000","last_became_peered":"2023-11-24T08:07:42.352938+0000","last_unstale":"2023-11-24T08:07:43.363375+0000","last_undegraded":"2023-11-24T08:07:43.363375+0000","last_fullsized":"2023-11-24T08:07:43.363375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:36.769677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.369777+0000","last_change":"2023-11-24T08:07:42.363647+0000","last_active":"2023-11-24T08:07:42.369777+0000","last_peered":"2023-11-24T08:07:42.369777+0000","last_clean":"2023-11-24T08:07:42.369777+0000","last_became_active":"2023-11-24T08:07:42.363388+0000","last_became_peered":"2023-11-24T08:07:42.363388+0000","last_unstale":"2023-11-24T08:07:42.369777+0000","last_undegraded":"2023-11-24T08:07:42.369777+0000","last_fullsized":"2023-11-24T08:07:42.369777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:31.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373781+0000","last_change":"2023-11-24T08:07:42.368789+0000","last_active":"2023-11-24T08:07:42.373781+0000","last_peered":"2023-11-24T08:07:42.373781+0000","last_clean":"2023-11-24T08:07:42.373781+0000","last_became_active":"2023-11-24T08:07:42.368523+0000","last_became_peered":"2023-11-24T08:07:42.368523+0000","last_unstale":"2023-11-24T08:07:42.373781+0000","last_undegraded":"2023-11-24T08:07:42.373781+0000","last_fullsized":"2023-11-24T08:07:42.373781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:17.571607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365884+0000","last_change":"2023-11-24T08:07:42.368813+0000","last_active":"2023-11-24T08:07:43.365884+0000","last_peered":"2023-11-24T08:07:43.365884+0000","last_clean":"2023-11-24T08:07:43.365884+0000","last_became_active":"2023-11-24T08:07:42.368558+0000","last_became_peered":"2023-11-24T08:07:42.368558+0000","last_unstale":"2023-11-24T08:07:43.365884+0000","last_undegraded":"2023-11-24T08:07:43.365884+0000","last_fullsized":"2023-11-24T08:07:43.365884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:14.069233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745685+0000","last_change":"2023-11-24T08:07:42.359797+0000","last_active":"2023-11-24T08:07:43.745685+0000","last_peered":"2023-11-24T08:07:43.745685+0000","last_clean":"2023-11-24T08:07:43.745685+0000","last_became_active":"2023-11-24T08:07:42.359485+0000","last_became_peered":"2023-11-24T08:07:42.359485+0000","last_unstale":"2023-11-24T08:07:43.745685+0000","last_undegraded":"2023-11-24T08:07:43.745685+0000","last_fullsized":"2023-11-24T08:07:43.745685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:59.024929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.369738+0000","last_change":"2023-11-24T08:07:42.367402+0000","last_active":"2023-11-24T08:07:42.369738+0000","last_peered":"2023-11-24T08:07:42.369738+0000","last_clean":"2023-11-24T08:07:42.369738+0000","last_became_active":"2023-11-24T08:07:42.367035+0000","last_became_peered":"2023-11-24T08:07:42.367035+0000","last_unstale":"2023-11-24T08:07:42.369738+0000","last_undegraded":"2023-11-24T08:07:42.369738+0000","last_fullsized":"2023-11-24T08:07:42.369738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:33.921127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364982+0000","last_change":"2023-11-24T08:07:42.351941+0000","last_active":"2023-11-24T08:07:43.364982+0000","last_peered":"2023-11-24T08:07:43.364982+0000","last_clean":"2023-11-24T08:07:43.364982+0000","last_became_active":"2023-11-24T08:07:42.351640+0000","last_became_peered":"2023-11-24T08:07:42.351640+0000","last_unstale":"2023-11-24T08:07:43.364982+0000","last_undegraded":"2023-11-24T08:07:43.364982+0000","last_fullsized":"2023-11-24T08:07:43.364982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:50.994269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.366170+0000","last_change":"2023-11-24T08:07:42.361604+0000","last_active":"2023-11-24T08:07:42.366170+0000","last_peered":"2023-11-24T08:07:42.366170+0000","last_clean":"2023-11-24T08:07:42.366170+0000","last_became_active":"2023-11-24T08:07:42.361347+0000","last_became_peered":"2023-11-24T08:07:42.361347+0000","last_unstale":"2023-11-24T08:07:42.366170+0000","last_undegraded":"2023-11-24T08:07:42.366170+0000","last_fullsized":"2023-11-24T08:07:42.366170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:49.714425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745025+0000","last_change":"2023-11-24T08:07:42.361686+0000","last_active":"2023-11-24T08:07:43.745025+0000","last_peered":"2023-11-24T08:07:43.745025+0000","last_clean":"2023-11-24T08:07:43.745025+0000","last_became_active":"2023-11-24T08:07:42.361137+0000","last_became_peered":"2023-11-24T08:07:42.361137+0000","last_unstale":"2023-11-24T08:07:43.745025+0000","last_undegraded":"2023-11-24T08:07:43.745025+0000","last_fullsized":"2023-11-24T08:07:43.745025+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:14.069233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745541+0000","last_change":"2023-11-24T08:07:42.348582+0000","last_active":"2023-11-24T08:07:43.745541+0000","last_peered":"2023-11-24T08:07:43.745541+0000","last_clean":"2023-11-24T08:07:43.745541+0000","last_became_active":"2023-11-24T08:07:42.348110+0000","last_became_peered":"2023-11-24T08:07:42.348110+0000","last_unstale":"2023-11-24T08:07:43.745541+0000","last_undegraded":"2023-11-24T08:07:43.745541+0000","last_fullsized":"2023-11-24T08:07:43.745541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:43.976124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363361+0000","last_change":"2023-11-24T08:07:42.359298+0000","last_active":"2023-11-24T08:07:43.363361+0000","last_peered":"2023-11-24T08:07:43.363361+0000","last_clean":"2023-11-24T08:07:43.363361+0000","last_became_active":"2023-11-24T08:07:42.358657+0000","last_became_peered":"2023-11-24T08:07:42.358657+0000","last_unstale":"2023-11-24T08:07:43.363361+0000","last_undegraded":"2023-11-24T08:07:43.363361+0000","last_fullsized":"2023-11-24T08:07:43.363361+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:30.293229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373819+0000","last_change":"2023-11-24T08:07:42.370318+0000","last_active":"2023-11-24T08:07:42.373819+0000","last_peered":"2023-11-24T08:07:42.373819+0000","last_clean":"2023-11-24T08:07:42.373819+0000","last_became_active":"2023-11-24T08:07:42.369414+0000","last_became_peered":"2023-11-24T08:07:42.369414+0000","last_unstale":"2023-11-24T08:07:42.373819+0000","last_undegraded":"2023-11-24T08:07:42.373819+0000","last_fullsized":"2023-11-24T08:07:42.373819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:44.749901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.369865+0000","last_change":"2023-11-24T08:07:41.356401+0000","last_active":"2023-11-24T08:07:42.369865+0000","last_peered":"2023-11-24T08:07:42.369865+0000","last_clean":"2023-11-24T08:07:42.369865+0000","last_became_active":"2023-11-24T08:07:41.355471+0000","last_became_peered":"2023-11-24T08:07:41.355471+0000","last_unstale":"2023-11-24T08:07:42.369865+0000","last_undegraded":"2023-11-24T08:07:42.369865+0000","last_fullsized":"2023-11-24T08:07:42.369865+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:01.313679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.359026+0000","last_change":"2023-11-24T08:07:42.360076+0000","last_active":"2023-11-24T08:07:43.359026+0000","last_peered":"2023-11-24T08:07:43.359026+0000","last_clean":"2023-11-24T08:07:43.359026+0000","last_became_active":"2023-11-24T08:07:42.358560+0000","last_became_peered":"2023-11-24T08:07:42.358560+0000","last_unstale":"2023-11-24T08:07:43.359026+0000","last_undegraded":"2023-11-24T08:07:43.359026+0000","last_fullsized":"2023-11-24T08:07:43.359026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:55.527459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.363276+0000","last_change":"2023-11-24T08:07:41.340910+0000","last_active":"2023-11-24T08:07:42.363276+0000","last_peered":"2023-11-24T08:07:42.363276+0000","last_clean":"2023-11-24T08:07:42.363276+0000","last_became_active":"2023-11-24T08:07:41.340557+0000","last_became_peered":"2023-11-24T08:07:41.340557+0000","last_unstale":"2023-11-24T08:07:42.363276+0000","last_undegraded":"2023-11-24T08:07:42.363276+0000","last_fullsized":"2023-11-24T08:07:42.363276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:21.924169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362110+0000","last_change":"2023-11-24T08:07:42.356921+0000","last_active":"2023-11-24T08:07:43.362110+0000","last_peered":"2023-11-24T08:07:43.362110+0000","last_clean":"2023-11-24T08:07:43.362110+0000","last_became_active":"2023-11-24T08:07:42.355455+0000","last_became_peered":"2023-11-24T08:07:42.355455+0000","last_unstale":"2023-11-24T08:07:43.362110+0000","last_undegraded":"2023-11-24T08:07:43.362110+0000","last_fullsized":"2023-11-24T08:07:43.362110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:39.983011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365537+0000","last_change":"2023-11-24T08:07:41.350994+0000","last_active":"2023-11-24T08:07:43.365537+0000","last_peered":"2023-11-24T08:07:43.365537+0000","last_clean":"2023-11-24T08:07:43.365537+0000","last_became_active":"2023-11-24T08:07:41.350697+0000","last_became_peered":"2023-11-24T08:07:41.350697+0000","last_unstale":"2023-11-24T08:07:43.365537+0000","last_undegraded":"2023-11-24T08:07:43.365537+0000","last_fullsized":"2023-11-24T08:07:43.365537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:38.880104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.371511+0000","last_change":"2023-11-24T08:07:42.367104+0000","last_active":"2023-11-24T08:07:42.371511+0000","last_peered":"2023-11-24T08:07:42.371511+0000","last_clean":"2023-11-24T08:07:42.371511+0000","last_became_active":"2023-11-24T08:07:42.366187+0000","last_became_peered":"2023-11-24T08:07:42.366187+0000","last_unstale":"2023-11-24T08:07:42.371511+0000","last_undegraded":"2023-11-24T08:07:42.371511+0000","last_fullsized":"2023-11-24T08:07:42.371511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:46.970844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362495+0000","last_change":"2023-11-24T08:07:41.356303+0000","last_active":"2023-11-24T08:07:43.362495+0000","last_peered":"2023-11-24T08:07:43.362495+0000","last_clean":"2023-11-24T08:07:43.362495+0000","last_became_active":"2023-11-24T08:07:41.354940+0000","last_became_peered":"2023-11-24T08:07:41.354940+0000","last_unstale":"2023-11-24T08:07:43.362495+0000","last_undegraded":"2023-11-24T08:07:43.362495+0000","last_fullsized":"2023-11-24T08:07:43.362495+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:17.956594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745287+0000","last_change":"2023-11-24T08:07:42.350542+0000","last_active":"2023-11-24T08:07:43.745287+0000","last_peered":"2023-11-24T08:07:43.745287+0000","last_clean":"2023-11-24T08:07:43.745287+0000","last_became_active":"2023-11-24T08:07:42.348205+0000","last_became_peered":"2023-11-24T08:07:42.348205+0000","last_unstale":"2023-11-24T08:07:43.745287+0000","last_undegraded":"2023-11-24T08:07:43.745287+0000","last_fullsized":"2023-11-24T08:07:43.745287+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:56.267643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.372725+0000","last_change":"2023-11-24T08:07:41.342300+0000","last_active":"2023-11-24T08:07:42.372725+0000","last_peered":"2023-11-24T08:07:42.372725+0000","last_clean":"2023-11-24T08:07:42.372725+0000","last_became_active":"2023-11-24T08:07:41.342008+0000","last_became_peered":"2023-11-24T08:07:41.342008+0000","last_unstale":"2023-11-24T08:07:42.372725+0000","last_undegraded":"2023-11-24T08:07:42.372725+0000","last_fullsized":"2023-11-24T08:07:42.372725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:58.391383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364581+0000","last_change":"2023-11-24T08:07:42.372248+0000","last_active":"2023-11-24T08:07:43.364581+0000","last_peered":"2023-11-24T08:07:43.364581+0000","last_clean":"2023-11-24T08:07:43.364581+0000","last_became_active":"2023-11-24T08:07:42.371948+0000","last_became_peered":"2023-11-24T08:07:42.371948+0000","last_unstale":"2023-11-24T08:07:43.364581+0000","last_undegraded":"2023-11-24T08:07:43.364581+0000","last_fullsized":"2023-11-24T08:07:43.364581+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:00:05.987484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361727+0000","last_change":"2023-11-24T08:07:41.355500+0000","last_active":"2023-11-24T08:07:43.361727+0000","last_peered":"2023-11-24T08:07:43.361727+0000","last_clean":"2023-11-24T08:07:43.361727+0000","last_became_active":"2023-11-24T08:07:41.355157+0000","last_became_peered":"2023-11-24T08:07:41.355157+0000","last_unstale":"2023-11-24T08:07:43.361727+0000","last_undegraded":"2023-11-24T08:07:43.361727+0000","last_fullsized":"2023-11-24T08:07:43.361727+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:39:48.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.357707+0000","last_change":"2023-11-24T08:07:42.369662+0000","last_active":"2023-11-24T08:07:43.357707+0000","last_peered":"2023-11-24T08:07:43.357707+0000","last_clean":"2023-11-24T08:07:43.357707+0000","last_became_active":"2023-11-24T08:07:42.368982+0000","last_became_peered":"2023-11-24T08:07:42.368982+0000","last_unstale":"2023-11-24T08:07:43.357707+0000","last_undegraded":"2023-11-24T08:07:43.357707+0000","last_fullsized":"2023-11-24T08:07:43.357707+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:25.159000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.326323+0000","last_change":"2023-11-24T08:07:41.339635+0000","last_active":"2023-11-24T08:07:42.326323+0000","last_peered":"2023-11-24T08:07:42.326323+0000","last_clean":"2023-11-24T08:07:42.326323+0000","last_became_active":"2023-11-24T08:07:41.339253+0000","last_became_peered":"2023-11-24T08:07:41.339253+0000","last_unstale":"2023-11-24T08:07:42.326323+0000","last_undegraded":"2023-11-24T08:07:42.326323+0000","last_fullsized":"2023-11-24T08:07:42.326323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:10.344531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.358078+0000","last_change":"2023-11-24T08:07:42.350087+0000","last_active":"2023-11-24T08:07:42.358078+0000","last_peered":"2023-11-24T08:07:42.358078+0000","last_clean":"2023-11-24T08:07:42.358078+0000","last_became_active":"2023-11-24T08:07:42.349771+0000","last_became_peered":"2023-11-24T08:07:42.349771+0000","last_unstale":"2023-11-24T08:07:42.358078+0000","last_undegraded":"2023-11-24T08:07:42.358078+0000","last_fullsized":"2023-11-24T08:07:42.358078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:44.749901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.357699+0000","last_change":"2023-11-24T08:07:41.343070+0000","last_active":"2023-11-24T08:07:43.357699+0000","last_peered":"2023-11-24T08:07:43.357699+0000","last_clean":"2023-11-24T08:07:43.357699+0000","last_became_active":"2023-11-24T08:07:41.342674+0000","last_became_peered":"2023-11-24T08:07:41.342674+0000","last_unstale":"2023-11-24T08:07:43.357699+0000","last_undegraded":"2023-11-24T08:07:43.357699+0000","last_fullsized":"2023-11-24T08:07:43.357699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:41.216815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745751+0000","last_change":"2023-11-24T08:07:42.359841+0000","last_active":"2023-11-24T08:07:43.745751+0000","last_peered":"2023-11-24T08:07:43.745751+0000","last_clean":"2023-11-24T08:07:43.745751+0000","last_became_active":"2023-11-24T08:07:42.359552+0000","last_became_peered":"2023-11-24T08:07:42.359552+0000","last_unstale":"2023-11-24T08:07:43.745751+0000","last_undegraded":"2023-11-24T08:07:43.745751+0000","last_fullsized":"2023-11-24T08:07:43.745751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:07.490591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365708+0000","last_change":"2023-11-24T08:07:41.341295+0000","last_active":"2023-11-24T08:07:43.365708+0000","last_peered":"2023-11-24T08:07:43.365708+0000","last_clean":"2023-11-24T08:07:43.365708+0000","last_became_active":"2023-11-24T08:07:41.339111+0000","last_became_peered":"2023-11-24T08:07:41.339111+0000","last_unstale":"2023-11-24T08:07:43.365708+0000","last_undegraded":"2023-11-24T08:07:43.365708+0000","last_fullsized":"2023-11-24T08:07:43.365708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:38.968606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373785+0000","last_change":"2023-11-24T08:07:42.369724+0000","last_active":"2023-11-24T08:07:42.373785+0000","last_peered":"2023-11-24T08:07:42.373785+0000","last_clean":"2023-11-24T08:07:42.373785+0000","last_became_active":"2023-11-24T08:07:42.369460+0000","last_became_peered":"2023-11-24T08:07:42.369460+0000","last_unstale":"2023-11-24T08:07:42.373785+0000","last_undegraded":"2023-11-24T08:07:42.373785+0000","last_fullsized":"2023-11-24T08:07:42.373785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:14.975240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363923+0000","last_change":"2023-11-24T08:07:41.357620+0000","last_active":"2023-11-24T08:07:43.363923+0000","last_peered":"2023-11-24T08:07:43.363923+0000","last_clean":"2023-11-24T08:07:43.363923+0000","last_became_active":"2023-11-24T08:07:41.357345+0000","last_became_peered":"2023-11-24T08:07:41.357345+0000","last_unstale":"2023-11-24T08:07:43.363923+0000","last_undegraded":"2023-11-24T08:07:43.363923+0000","last_fullsized":"2023-11-24T08:07:43.363923+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:19.383814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362440+0000","last_change":"2023-11-24T08:07:42.370971+0000","last_active":"2023-11-24T08:07:43.362440+0000","last_peered":"2023-11-24T08:07:43.362440+0000","last_clean":"2023-11-24T08:07:43.362440+0000","last_became_active":"2023-11-24T08:07:42.369160+0000","last_became_peered":"2023-11-24T08:07:42.369160+0000","last_unstale":"2023-11-24T08:07:43.362440+0000","last_undegraded":"2023-11-24T08:07:43.362440+0000","last_fullsized":"2023-11-24T08:07:43.362440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:52.334351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.370021+0000","last_change":"2023-11-24T08:07:41.351210+0000","last_active":"2023-11-24T08:07:42.370021+0000","last_peered":"2023-11-24T08:07:42.370021+0000","last_clean":"2023-11-24T08:07:42.370021+0000","last_became_active":"2023-11-24T08:07:41.350226+0000","last_became_peered":"2023-11-24T08:07:41.350226+0000","last_unstale":"2023-11-24T08:07:42.370021+0000","last_undegraded":"2023-11-24T08:07:42.370021+0000","last_fullsized":"2023-11-24T08:07:42.370021+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:19.929165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358453+0000","last_change":"2023-11-24T08:07:42.345516+0000","last_active":"2023-11-24T08:07:43.358453+0000","last_peered":"2023-11-24T08:07:43.358453+0000","last_clean":"2023-11-24T08:07:43.358453+0000","last_became_active":"2023-11-24T08:07:42.344592+0000","last_became_peered":"2023-11-24T08:07:42.344592+0000","last_unstale":"2023-11-24T08:07:43.358453+0000","last_undegraded":"2023-11-24T08:07:43.358453+0000","last_fullsized":"2023-11-24T08:07:43.358453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:57:09.164635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.374834+0000","last_change":"2023-11-24T08:07:41.336623+0000","last_active":"2023-11-24T08:07:42.374834+0000","last_peered":"2023-11-24T08:07:42.374834+0000","last_clean":"2023-11-24T08:07:42.374834+0000","last_became_active":"2023-11-24T08:07:41.336180+0000","last_became_peered":"2023-11-24T08:07:41.336180+0000","last_unstale":"2023-11-24T08:07:42.374834+0000","last_undegraded":"2023-11-24T08:07:42.374834+0000","last_fullsized":"2023-11-24T08:07:42.374834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:54.513055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.370685+0000","last_change":"2023-11-24T08:07:42.365859+0000","last_active":"2023-11-24T08:07:42.370685+0000","last_peered":"2023-11-24T08:07:42.370685+0000","last_clean":"2023-11-24T08:07:42.370685+0000","last_became_active":"2023-11-24T08:07:42.365524+0000","last_became_peered":"2023-11-24T08:07:42.365524+0000","last_unstale":"2023-11-24T08:07:42.370685+0000","last_undegraded":"2023-11-24T08:07:42.370685+0000","last_fullsized":"2023-11-24T08:07:42.370685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:43.976124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.370436+0000","last_change":"2023-11-24T08:07:41.348497+0000","last_active":"2023-11-24T08:07:42.370436+0000","last_peered":"2023-11-24T08:07:42.370436+0000","last_clean":"2023-11-24T08:07:42.370436+0000","last_became_active":"2023-11-24T08:07:41.348196+0000","last_became_peered":"2023-11-24T08:07:41.348196+0000","last_unstale":"2023-11-24T08:07:42.370436+0000","last_undegraded":"2023-11-24T08:07:42.370436+0000","last_fullsized":"2023-11-24T08:07:42.370436+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:15.746351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745387+0000","last_change":"2023-11-24T08:07:42.349890+0000","last_active":"2023-11-24T08:07:43.745387+0000","last_peered":"2023-11-24T08:07:43.745387+0000","last_clean":"2023-11-24T08:07:43.745387+0000","last_became_active":"2023-11-24T08:07:42.349520+0000","last_became_peered":"2023-11-24T08:07:42.349520+0000","last_unstale":"2023-11-24T08:07:43.745387+0000","last_undegraded":"2023-11-24T08:07:43.745387+0000","last_fullsized":"2023-11-24T08:07:43.745387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:17.571607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361954+0000","last_change":"2023-11-24T08:07:41.354093+0000","last_active":"2023-11-24T08:07:43.361954+0000","last_peered":"2023-11-24T08:07:43.361954+0000","last_clean":"2023-11-24T08:07:43.361954+0000","last_became_active":"2023-11-24T08:07:41.353814+0000","last_became_peered":"2023-11-24T08:07:41.353814+0000","last_unstale":"2023-11-24T08:07:43.361954+0000","last_undegraded":"2023-11-24T08:07:43.361954+0000","last_fullsized":"2023-11-24T08:07:43.361954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:36.286125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.364235+0000","last_change":"2023-11-24T08:07:42.362948+0000","last_active":"2023-11-24T08:07:42.364235+0000","last_peered":"2023-11-24T08:07:42.364235+0000","last_clean":"2023-11-24T08:07:42.364235+0000","last_became_active":"2023-11-24T08:07:42.359320+0000","last_became_peered":"2023-11-24T08:07:42.359320+0000","last_unstale":"2023-11-24T08:07:42.364235+0000","last_undegraded":"2023-11-24T08:07:42.364235+0000","last_fullsized":"2023-11-24T08:07:42.364235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:21.644216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.744969+0000","last_change":"2023-11-24T08:07:41.355775+0000","last_active":"2023-11-24T08:07:43.744969+0000","last_peered":"2023-11-24T08:07:43.744969+0000","last_clean":"2023-11-24T08:07:43.744969+0000","last_became_active":"2023-11-24T08:07:41.355174+0000","last_became_peered":"2023-11-24T08:07:41.355174+0000","last_unstale":"2023-11-24T08:07:43.744969+0000","last_undegraded":"2023-11-24T08:07:43.744969+0000","last_fullsized":"2023-11-24T08:07:43.744969+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:51.319947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358346+0000","last_change":"2023-11-24T08:07:42.348204+0000","last_active":"2023-11-24T08:07:43.358346+0000","last_peered":"2023-11-24T08:07:43.358346+0000","last_clean":"2023-11-24T08:07:43.358346+0000","last_became_active":"2023-11-24T08:07:42.347875+0000","last_became_peered":"2023-11-24T08:07:42.347875+0000","last_unstale":"2023-11-24T08:07:43.358346+0000","last_undegraded":"2023-11-24T08:07:43.358346+0000","last_fullsized":"2023-11-24T08:07:43.358346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:19:16.760756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362247+0000","last_change":"2023-11-24T08:07:41.345824+0000","last_active":"2023-11-24T08:07:43.362247+0000","last_peered":"2023-11-24T08:07:43.362247+0000","last_clean":"2023-11-24T08:07:43.362247+0000","last_became_active":"2023-11-24T08:07:41.345473+0000","last_became_peered":"2023-11-24T08:07:41.345473+0000","last_unstale":"2023-11-24T08:07:43.362247+0000","last_undegraded":"2023-11-24T08:07:43.362247+0000","last_fullsized":"2023-11-24T08:07:43.362247+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:39:48.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365762+0000","last_change":"2023-11-24T08:07:42.362434+0000","last_active":"2023-11-24T08:07:43.365762+0000","last_peered":"2023-11-24T08:07:43.365762+0000","last_clean":"2023-11-24T08:07:43.365762+0000","last_became_active":"2023-11-24T08:07:42.362110+0000","last_became_peered":"2023-11-24T08:07:42.362110+0000","last_unstale":"2023-11-24T08:07:43.365762+0000","last_undegraded":"2023-11-24T08:07:43.365762+0000","last_fullsized":"2023-11-24T08:07:43.365762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:56:18.150787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362120+0000","last_change":"2023-11-24T08:07:41.347678+0000","last_active":"2023-11-24T08:07:43.362120+0000","last_peered":"2023-11-24T08:07:43.362120+0000","last_clean":"2023-11-24T08:07:43.362120+0000","last_became_active":"2023-11-24T08:07:41.347132+0000","last_became_peered":"2023-11-24T08:07:41.347132+0000","last_unstale":"2023-11-24T08:07:43.362120+0000","last_undegraded":"2023-11-24T08:07:43.362120+0000","last_fullsized":"2023-11-24T08:07:43.362120+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:28.234377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373031+0000","last_change":"2023-11-24T08:07:42.367638+0000","last_active":"2023-11-24T08:07:42.373031+0000","last_peered":"2023-11-24T08:07:42.373031+0000","last_clean":"2023-11-24T08:07:42.373031+0000","last_became_active":"2023-11-24T08:07:42.367313+0000","last_became_peered":"2023-11-24T08:07:42.367313+0000","last_unstale":"2023-11-24T08:07:42.373031+0000","last_undegraded":"2023-11-24T08:07:42.373031+0000","last_fullsized":"2023-11-24T08:07:42.373031+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:23.845233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.365904+0000","last_change":"2023-11-24T08:07:41.352151+0000","last_active":"2023-11-24T08:07:43.365904+0000","last_peered":"2023-11-24T08:07:43.365904+0000","last_clean":"2023-11-24T08:07:43.365904+0000","last_became_active":"2023-11-24T08:07:41.351233+0000","last_became_peered":"2023-11-24T08:07:41.351233+0000","last_unstale":"2023-11-24T08:07:43.365904+0000","last_undegraded":"2023-11-24T08:07:43.365904+0000","last_fullsized":"2023-11-24T08:07:43.365904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:15.862078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.351897+0000","last_change":"2023-11-24T08:07:42.342386+0000","last_active":"2023-11-24T08:07:42.351897+0000","last_peered":"2023-11-24T08:07:42.351897+0000","last_clean":"2023-11-24T08:07:42.351897+0000","last_became_active":"2023-11-24T08:07:42.340287+0000","last_became_peered":"2023-11-24T08:07:42.340287+0000","last_unstale":"2023-11-24T08:07:42.351897+0000","last_undegraded":"2023-11-24T08:07:42.351897+0000","last_fullsized":"2023-11-24T08:07:42.351897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:51:39.334989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.364553+0000","last_change":"2023-11-24T08:07:41.355914+0000","last_active":"2023-11-24T08:07:43.364553+0000","last_peered":"2023-11-24T08:07:43.364553+0000","last_clean":"2023-11-24T08:07:43.364553+0000","last_became_active":"2023-11-24T08:07:41.355649+0000","last_became_peered":"2023-11-24T08:07:41.355649+0000","last_unstale":"2023-11-24T08:07:43.364553+0000","last_undegraded":"2023-11-24T08:07:43.364553+0000","last_fullsized":"2023-11-24T08:07:43.364553+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:16.721444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358666+0000","last_change":"2023-11-24T08:07:42.355643+0000","last_active":"2023-11-24T08:07:43.358666+0000","last_peered":"2023-11-24T08:07:43.358666+0000","last_clean":"2023-11-24T08:07:43.358666+0000","last_became_active":"2023-11-24T08:07:42.355355+0000","last_became_peered":"2023-11-24T08:07:42.355355+0000","last_unstale":"2023-11-24T08:07:43.358666+0000","last_undegraded":"2023-11-24T08:07:43.358666+0000","last_fullsized":"2023-11-24T08:07:43.358666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:38.089553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.348726+0000","last_change":"2023-11-24T08:07:41.357318+0000","last_active":"2023-11-24T08:07:42.348726+0000","last_peered":"2023-11-24T08:07:42.348726+0000","last_clean":"2023-11-24T08:07:42.348726+0000","last_became_active":"2023-11-24T08:07:41.356882+0000","last_became_peered":"2023-11-24T08:07:41.356882+0000","last_unstale":"2023-11-24T08:07:42.348726+0000","last_undegraded":"2023-11-24T08:07:42.348726+0000","last_fullsized":"2023-11-24T08:07:42.348726+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:20.629811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.363719+0000","last_change":"2023-11-24T08:07:42.356464+0000","last_active":"2023-11-24T08:07:42.363719+0000","last_peered":"2023-11-24T08:07:42.363719+0000","last_clean":"2023-11-24T08:07:42.363719+0000","last_became_active":"2023-11-24T08:07:42.356139+0000","last_became_peered":"2023-11-24T08:07:42.356139+0000","last_unstale":"2023-11-24T08:07:42.363719+0000","last_undegraded":"2023-11-24T08:07:42.363719+0000","last_fullsized":"2023-11-24T08:07:42.363719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:16.519158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362207+0000","last_change":"2023-11-24T08:07:41.351739+0000","last_active":"2023-11-24T08:07:43.362207+0000","last_peered":"2023-11-24T08:07:43.362207+0000","last_clean":"2023-11-24T08:07:43.362207+0000","last_became_active":"2023-11-24T08:07:41.351393+0000","last_became_peered":"2023-11-24T08:07:41.351393+0000","last_unstale":"2023-11-24T08:07:43.362207+0000","last_undegraded":"2023-11-24T08:07:43.362207+0000","last_fullsized":"2023-11-24T08:07:43.362207+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:08.824076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.366138+0000","last_change":"2023-11-24T08:07:42.360044+0000","last_active":"2023-11-24T08:07:42.366138+0000","last_peered":"2023-11-24T08:07:42.366138+0000","last_clean":"2023-11-24T08:07:42.366138+0000","last_became_active":"2023-11-24T08:07:42.359710+0000","last_became_peered":"2023-11-24T08:07:42.359710+0000","last_unstale":"2023-11-24T08:07:42.366138+0000","last_undegraded":"2023-11-24T08:07:42.366138+0000","last_fullsized":"2023-11-24T08:07:42.366138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:54.247656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745276+0000","last_change":"2023-11-24T08:07:41.351220+0000","last_active":"2023-11-24T08:07:43.745276+0000","last_peered":"2023-11-24T08:07:43.745276+0000","last_clean":"2023-11-24T08:07:43.745276+0000","last_became_active":"2023-11-24T08:07:41.350184+0000","last_became_peered":"2023-11-24T08:07:41.350184+0000","last_unstale":"2023-11-24T08:07:43.745276+0000","last_undegraded":"2023-11-24T08:07:43.745276+0000","last_fullsized":"2023-11-24T08:07:43.745276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:53:06.741852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.363760+0000","last_change":"2023-11-24T08:07:42.356190+0000","last_active":"2023-11-24T08:07:42.363760+0000","last_peered":"2023-11-24T08:07:42.363760+0000","last_clean":"2023-11-24T08:07:42.363760+0000","last_became_active":"2023-11-24T08:07:42.355861+0000","last_became_peered":"2023-11-24T08:07:42.355861+0000","last_unstale":"2023-11-24T08:07:42.363760+0000","last_undegraded":"2023-11-24T08:07:42.363760+0000","last_fullsized":"2023-11-24T08:07:42.363760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:50.065788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.334677+0000","last_change":"2023-11-24T08:07:41.345281+0000","last_active":"2023-11-24T08:07:42.334677+0000","last_peered":"2023-11-24T08:07:42.334677+0000","last_clean":"2023-11-24T08:07:42.334677+0000","last_became_active":"2023-11-24T08:07:41.344935+0000","last_became_peered":"2023-11-24T08:07:41.344935+0000","last_unstale":"2023-11-24T08:07:42.334677+0000","last_undegraded":"2023-11-24T08:07:42.334677+0000","last_fullsized":"2023-11-24T08:07:42.334677+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:45.956440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.348868+0000","last_change":"2023-11-24T08:07:42.343602+0000","last_active":"2023-11-24T08:07:42.348868+0000","last_peered":"2023-11-24T08:07:42.348868+0000","last_clean":"2023-11-24T08:07:42.348868+0000","last_became_active":"2023-11-24T08:07:42.343304+0000","last_became_peered":"2023-11-24T08:07:42.343304+0000","last_unstale":"2023-11-24T08:07:42.348868+0000","last_undegraded":"2023-11-24T08:07:42.348868+0000","last_fullsized":"2023-11-24T08:07:42.348868+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:44.851906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362370+0000","last_change":"2023-11-24T08:07:41.347454+0000","last_active":"2023-11-24T08:07:43.362370+0000","last_peered":"2023-11-24T08:07:43.362370+0000","last_clean":"2023-11-24T08:07:43.362370+0000","last_became_active":"2023-11-24T08:07:41.346759+0000","last_became_peered":"2023-11-24T08:07:41.346759+0000","last_unstale":"2023-11-24T08:07:43.362370+0000","last_undegraded":"2023-11-24T08:07:43.362370+0000","last_fullsized":"2023-11-24T08:07:43.362370+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:17.649797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745412+0000","last_change":"2023-11-24T08:07:42.345922+0000","last_active":"2023-11-24T08:07:43.745412+0000","last_peered":"2023-11-24T08:07:43.745412+0000","last_clean":"2023-11-24T08:07:43.745412+0000","last_became_active":"2023-11-24T08:07:42.345623+0000","last_became_peered":"2023-11-24T08:07:42.345623+0000","last_unstale":"2023-11-24T08:07:43.745412+0000","last_undegraded":"2023-11-24T08:07:43.745412+0000","last_fullsized":"2023-11-24T08:07:43.745412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:31.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.745807+0000","last_change":"2023-11-24T08:07:41.352128+0000","last_active":"2023-11-24T08:07:43.745807+0000","last_peered":"2023-11-24T08:07:43.745807+0000","last_clean":"2023-11-24T08:07:43.745807+0000","last_became_active":"2023-11-24T08:07:41.351839+0000","last_became_peered":"2023-11-24T08:07:41.351839+0000","last_unstale":"2023-11-24T08:07:43.745807+0000","last_undegraded":"2023-11-24T08:07:43.745807+0000","last_fullsized":"2023-11-24T08:07:43.745807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.979865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.373756+0000","last_change":"2023-11-24T08:07:42.368618+0000","last_active":"2023-11-24T08:07:42.373756+0000","last_peered":"2023-11-24T08:07:42.373756+0000","last_clean":"2023-11-24T08:07:42.373756+0000","last_became_active":"2023-11-24T08:07:42.368346+0000","last_became_peered":"2023-11-24T08:07:42.368346+0000","last_unstale":"2023-11-24T08:07:42.373756+0000","last_undegraded":"2023-11-24T08:07:42.373756+0000","last_fullsized":"2023-11-24T08:07:42.373756+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:59.024929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358618+0000","last_change":"2023-11-24T08:07:41.341274+0000","last_active":"2023-11-24T08:07:43.358618+0000","last_peered":"2023-11-24T08:07:43.358618+0000","last_clean":"2023-11-24T08:07:43.358618+0000","last_became_active":"2023-11-24T08:07:41.340971+0000","last_became_peered":"2023-11-24T08:07:41.340971+0000","last_unstale":"2023-11-24T08:07:43.358618+0000","last_undegraded":"2023-11-24T08:07:43.358618+0000","last_fullsized":"2023-11-24T08:07:43.358618+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:00.860722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.374080+0000","last_change":"2023-11-24T08:07:42.368422+0000","last_active":"2023-11-24T08:07:42.374080+0000","last_peered":"2023-11-24T08:07:42.374080+0000","last_clean":"2023-11-24T08:07:42.374080+0000","last_became_active":"2023-11-24T08:07:42.368017+0000","last_became_peered":"2023-11-24T08:07:42.368017+0000","last_unstale":"2023-11-24T08:07:42.374080+0000","last_undegraded":"2023-11-24T08:07:42.374080+0000","last_fullsized":"2023-11-24T08:07:42.374080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:44.851906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.336673+0000","last_change":"2023-11-24T08:07:41.349579+0000","last_active":"2023-11-24T08:07:42.336673+0000","last_peered":"2023-11-24T08:07:42.336673+0000","last_clean":"2023-11-24T08:07:42.336673+0000","last_became_active":"2023-11-24T08:07:41.349294+0000","last_became_peered":"2023-11-24T08:07:41.349294+0000","last_unstale":"2023-11-24T08:07:42.336673+0000","last_undegraded":"2023-11-24T08:07:42.336673+0000","last_fullsized":"2023-11-24T08:07:42.336673+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:01.313679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358195+0000","last_change":"2023-11-24T08:07:42.347859+0000","last_active":"2023-11-24T08:07:43.358195+0000","last_peered":"2023-11-24T08:07:43.358195+0000","last_clean":"2023-11-24T08:07:43.358195+0000","last_became_active":"2023-11-24T08:07:42.347560+0000","last_became_peered":"2023-11-24T08:07:42.347560+0000","last_unstale":"2023-11-24T08:07:43.358195+0000","last_undegraded":"2023-11-24T08:07:43.358195+0000","last_fullsized":"2023-11-24T08:07:43.358195+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:11.358935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.358304+0000","last_change":"2023-11-24T08:07:41.349178+0000","last_active":"2023-11-24T08:07:43.358304+0000","last_peered":"2023-11-24T08:07:43.358304+0000","last_clean":"2023-11-24T08:07:43.358304+0000","last_became_active":"2023-11-24T08:07:41.348881+0000","last_became_peered":"2023-11-24T08:07:41.348881+0000","last_unstale":"2023-11-24T08:07:43.358304+0000","last_undegraded":"2023-11-24T08:07:43.358304+0000","last_fullsized":"2023-11-24T08:07:43.358304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:33.050522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361658+0000","last_change":"2023-11-24T08:07:42.373368+0000","last_active":"2023-11-24T08:07:43.361658+0000","last_peered":"2023-11-24T08:07:43.361658+0000","last_clean":"2023-11-24T08:07:43.361658+0000","last_became_active":"2023-11-24T08:07:42.372448+0000","last_became_peered":"2023-11-24T08:07:42.372448+0000","last_unstale":"2023-11-24T08:07:43.361658+0000","last_undegraded":"2023-11-24T08:07:43.361658+0000","last_fullsized":"2023-11-24T08:07:43.361658+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:39.894508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.326440+0000","last_change":"2023-11-24T08:07:41.342093+0000","last_active":"2023-11-24T08:07:42.326440+0000","last_peered":"2023-11-24T08:07:42.326440+0000","last_clean":"2023-11-24T08:07:42.326440+0000","last_became_active":"2023-11-24T08:07:41.341739+0000","last_became_peered":"2023-11-24T08:07:41.341739+0000","last_unstale":"2023-11-24T08:07:42.326440+0000","last_undegraded":"2023-11-24T08:07:42.326440+0000","last_fullsized":"2023-11-24T08:07:42.326440+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:31.179421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.349882+0000","last_change":"2023-11-24T08:07:42.346376+0000","last_active":"2023-11-24T08:07:42.349882+0000","last_peered":"2023-11-24T08:07:42.349882+0000","last_clean":"2023-11-24T08:07:42.349882+0000","last_became_active":"2023-11-24T08:07:42.345973+0000","last_became_peered":"2023-11-24T08:07:42.345973+0000","last_unstale":"2023-11-24T08:07:42.349882+0000","last_undegraded":"2023-11-24T08:07:42.349882+0000","last_fullsized":"2023-11-24T08:07:42.349882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:57:09.164635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.362519+0000","last_change":"2023-11-24T08:07:41.345970+0000","last_active":"2023-11-24T08:07:43.362519+0000","last_peered":"2023-11-24T08:07:43.362519+0000","last_clean":"2023-11-24T08:07:43.362519+0000","last_became_active":"2023-11-24T08:07:41.345591+0000","last_became_peered":"2023-11-24T08:07:41.345591+0000","last_unstale":"2023-11-24T08:07:43.362519+0000","last_undegraded":"2023-11-24T08:07:43.362519+0000","last_fullsized":"2023-11-24T08:07:43.362519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:08.824076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363607+0000","last_change":"2023-11-24T08:07:42.353886+0000","last_active":"2023-11-24T08:07:43.363607+0000","last_peered":"2023-11-24T08:07:43.363607+0000","last_clean":"2023-11-24T08:07:43.363607+0000","last_became_active":"2023-11-24T08:07:42.353544+0000","last_became_peered":"2023-11-24T08:07:42.353544+0000","last_unstale":"2023-11-24T08:07:43.363607+0000","last_undegraded":"2023-11-24T08:07:43.363607+0000","last_fullsized":"2023-11-24T08:07:43.363607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:36.769677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.363516+0000","last_change":"2023-11-24T08:07:41.352559+0000","last_active":"2023-11-24T08:07:43.363516+0000","last_peered":"2023-11-24T08:07:43.363516+0000","last_clean":"2023-11-24T08:07:43.363516+0000","last_became_active":"2023-11-24T08:07:41.352196+0000","last_became_peered":"2023-11-24T08:07:41.352196+0000","last_unstale":"2023-11-24T08:07:43.363516+0000","last_undegraded":"2023-11-24T08:07:43.363516+0000","last_fullsized":"2023-11-24T08:07:43.363516+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:10.411816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:07:42.366370+0000","last_change":"2023-11-24T08:07:42.364684+0000","last_active":"2023-11-24T08:07:42.366370+0000","last_peered":"2023-11-24T08:07:42.366370+0000","last_clean":"2023-11-24T08:07:42.366370+0000","last_became_active":"2023-11-24T08:07:42.364067+0000","last_became_peered":"2023-11-24T08:07:42.364067+0000","last_unstale":"2023-11-24T08:07:42.366370+0000","last_undegraded":"2023-11-24T08:07:42.366370+0000","last_fullsized":"2023-11-24T08:07:42.366370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:41.302887+0000","last_clean_scrub_stamp":"2023-11-24T08:07:41.302887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:11:44.930805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:07:43.361499+0000","last_change":"2023-11-24T08:07:41.352560+0000","last_active":"2023-11-24T08:07:43.361499+0000","last_peered":"2023-11-24T08:07:43.361499+0000","last_clean":"2023-11-24T08:07:43.361499+0000","last_became_active":"2023-11-24T08:07:41.351621+0000","last_became_peered":"2023-11-24T08:07:41.351621+0000","last_unstale":"2023-11-24T08:07:43.361499+0000","last_undegraded":"2023-11-24T08:07:43.361499+0000","last_fullsized":"2023-11-24T08:07:43.361499+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:07:40.288482+0000","last_clean_scrub_stamp":"2023-11-24T08:07:40.288482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:53:48.251836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T08:07:41.342701+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:07:41.342701+0000","last_peered":"2023-11-24T08:07:41.342701+0000","last_clean":"2023-11-24T08:07:41.342701+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:07:41.342701+0000","last_undegraded":"2023-11-24T08:07:41.342701+0000","last_fullsized":"2023-11-24T08:07:41.342701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1490944,"data_stored":968334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0860000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.484,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.85199999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2789999999999999,"15min":1.3009999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6639999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3420000000000001,"15min":1.3839999999999999},"min":{"1min":0.97299999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5569999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.712}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.151,"15min":1.1479999999999999},"min":{"1min":0.70099999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0840000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.3069999999999999,"15min":1.3160000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.575}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2829999999999999,"15min":1.321},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.966,"5min":2.524,"15min":2.524},"last":1.117},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.454,"15min":1.5169999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8939999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.7889999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4550000000000001,"15min":1.532},"min":{"1min":0.88400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6419999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.526},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.379},"min":{"1min":0.84499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.484}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.581,"15min":1.619},"min":{"1min":1.0229999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.173,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.601,"15min":1.661},"min":{"1min":1.069,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7350000000000001,"5min":2.944,"15min":2.944},"last":1.8240000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.702,"15min":1.7589999999999999},"min":{"1min":1.252,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.649,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.8779999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6060000000000001,"15min":1.665},"min":{"1min":1.0509999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.841,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.75}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":909312,"data_stored":383742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.276,"15min":1.3300000000000001},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.113,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.274,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.2709999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1539999999999999,"15min":1.159},"min":{"1min":0.68400000000000005,"5min":0.625,"15min":0.625},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2370000000000001,"15min":1.3069999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6830000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.633}]},{"osd":2,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.232,"15min":1.296},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.508,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.419,"15min":1.5069999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.752,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.3879999999999999,"15min":1.444},"min":{"1min":0.79200000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.931,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.206},{"interface":"front","average":{"1min":1.407,"5min":1.4690000000000001,"15min":1.498},"min":{"1min":0.94899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.968,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.105}]},{"osd":4,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.6279999999999999,"15min":1.758},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2120000000000002,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":0.995},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.4390000000000001,"15min":1.472},"min":{"1min":0.86199999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.855,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.488}]},{"osd":5,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6679999999999999,"15min":1.8080000000000001},"min":{"1min":1.0029999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.893,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.806,"15min":1.9490000000000001},"min":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.901,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6379999999999999,"15min":1.6890000000000001},"min":{"1min":0.75800000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.379,"5min":2.98,"15min":2.98},"last":1.373},{"interface":"front","average":{"1min":1.482,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.1659999999999999,"5min":2.29,"15min":2.29},"last":1.0549999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":892928,"data_stored":377440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.048},"min":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2949999999999999,"5min":1.764,"15min":1.764},"last":1.048},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0329999999999999,"15min":1.006},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.5369999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6950000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.874},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0780000000000001,"15min":1.0509999999999999},"min":{"1min":0.70899999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.458,"5min":1.635,"15min":1.635},"last":1.1519999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2090000000000001,"15min":1.2130000000000001},"min":{"1min":0.77100000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.673,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.256},{"interface":"front","average":{"1min":1.226,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.745,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6259999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.012}]},{"osd":3,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.427},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.575,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2470000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5669999999999999,"5min":1.889,"15min":1.889},"last":1.218}]},{"osd":4,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.284,"15min":1.298},"min":{"1min":0.79100000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.871,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.3},{"interface":"front","average":{"1min":1.429,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.68400000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.766,"5min":2.27,"15min":2.27},"last":1.101}]},{"osd":6,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.448,"15min":1.47},"min":{"1min":0.97799999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7909999999999999,"5min":3.806,"15min":3.806},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.409,"15min":1.4079999999999999},"min":{"1min":0.84699999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0169999999999999,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5189999999999999,"15min":1.5269999999999999},"min":{"1min":0.63,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.974,"5min":2.923,"15min":2.923},"last":1.377},{"interface":"front","average":{"1min":1.569,"5min":1.54,"15min":1.53},"min":{"1min":1.0640000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.214,"5min":2.859,"15min":2.859},"last":1.4099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":892928,"data_stored":377420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0409999999999999,"15min":1.042},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4399999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.371},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.1279999999999999},"min":{"1min":0.77900000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.595,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.258}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.079,"15min":1.0840000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.377,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.79000000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5640000000000001,"5min":1.903,"15min":1.903},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1890000000000001,"15min":1.2},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.589,"5min":1.823,"15min":1.823},"last":1.151},{"interface":"front","average":{"1min":1.155,"5min":1.2410000000000001,"15min":1.258},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.29}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.2969999999999999},"min":{"1min":0.81399999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7330000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3109999999999999,"15min":1.3149999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9530000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3480000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.282,"15min":1.296},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.617,"5min":1.944,"15min":1.944},"last":1.319},{"interface":"front","average":{"1min":1.26,"5min":1.3400000000000001,"15min":1.3580000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":2.012,"15min":2.012},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.377,"15min":1.3680000000000001},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4890000000000001,"15min":1.5069999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.927,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.5740000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.4330000000000001,"15min":1.431},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.768,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.554,"15min":1.5589999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.411,"5min":2.722,"15min":2.722},"last":1.2430000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":897024,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.82099999999999995},"min":{"1min":0.50800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.569,"5min":1.585,"15min":1.585},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.92300000000000004,"15min":0.94799999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4930000000000001,"5min":1.51,"15min":1.51},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.075,"15min":1.151},"min":{"1min":0.63100000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.835,"5min":3.246,"15min":3.246},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97399999999999998,"15min":0.96699999999999997},"min":{"1min":0.54900000000000004,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4179999999999999,"5min":1.665,"15min":1.665},"last":1.5369999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1399999999999999,"15min":1.1819999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7410000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1160000000000001,"15min":1.155},"min":{"1min":0.66800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.399,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.4139999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2450000000000001,"15min":1.264},"min":{"1min":0.73299999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.671,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.355,"15min":1.3899999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.665,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.3799999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.4199999999999999,"15min":1.4590000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9430000000000001,"5min":2.395,"15min":2.395},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.306,"15min":1.3},"min":{"1min":0.90400000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9399999999999999,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.3380000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.421,"15min":1.4319999999999999},"min":{"1min":0.94099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8899999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.609},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3979999999999999,"15min":1.4179999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.1200000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:55 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4219999999999999,"15min":1.444},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.994,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.528,"15min":1.581},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.895,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.492}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":901120,"data_stored":378962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.417,"5min":1.524,"15min":1.524},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97399999999999998,"15min":1.016},"min":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5740000000000001,"5min":2.133,"15min":2.133},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1419999999999999,"15min":1.2250000000000001},"min":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8280000000000001,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0029999999999999,"15min":1.069},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4550000000000001,"5min":1.998,"15min":1.998},"last":1.425}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.321},"min":{"1min":0.73299999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6879999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.147,"15min":1.1950000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7809999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.2509999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.754,"5min":0.442,"15min":0.442},"max":{"1min":1.7969999999999999,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.048},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.4359999999999999,"15min":1.502},"min":{"1min":1.018,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.758,"5min":3.161,"15min":3.161},"last":1.3879999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.345,"15min":1.393},"min":{"1min":0.77000000000000002,"5min":0.623,"15min":0.623},"max":{"1min":1.752,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":2.1629999999999998},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.355,"15min":1.411},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.96,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.149}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.5169999999999999,"15min":1.593},"min":{"1min":0.96199999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8979999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.5649999999999999},"min":{"1min":0.80900000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8520000000000001,"5min":2.702,"15min":2.702},"last":2.0510000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.448,"15min":1.514},"min":{"1min":0.78200000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0419999999999998,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.5660000000000001,"15min":1.6439999999999999},"min":{"1min":0.89300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.952,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.52}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":379578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.878,"15min":0.89200000000000002},"min":{"1min":0.70799999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.028},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0029999999999999,"15min":1.0580000000000001},"min":{"1min":0.56599999999999995,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.397,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.387}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.03},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.746,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.081,"5min":1.179,"15min":1.2290000000000001},"min":{"1min":0.76400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.794,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.2729999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.2729999999999999},"min":{"1min":0.72299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.897,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.091,"5min":1.177,"15min":1.232},"min":{"1min":0.66500000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.6890000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.1679999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.5620000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4199999999999999,"15min":1.454},"min":{"1min":0.95799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.5169999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4259999999999999,"15min":1.4570000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0089999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.452,"15min":1.478},"min":{"1min":0.91000000000000003,"5min":0.871,"15min":0.871},"max":{"1min":1.903,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.629}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.4319999999999999,"15min":1.458},"min":{"1min":0.83099999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.774,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.766},{"interface":"front","average":{"1min":1.373,"5min":1.4430000000000001,"15min":1.4830000000000001},"min":{"1min":0.79200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8420000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.5640000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":0.996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8700000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.591},{"interface":"front","average":{"1min":1.55,"5min":1.5680000000000001,"15min":1.595},"min":{"1min":0.96599999999999997,"5min":0.872,"15min":0.872},"max":{"1min":1.8500000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.8560000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":966699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.52500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.97799999999999998,"15min":0.94799999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.111,"15min":1.075},"min":{"1min":0.52900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.899,"5min":3.899,"15min":3.899},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1,"15min":0.95999999999999996},"min":{"1min":0.498,"5min":0.443,"15min":0.443},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.0700000000000001,"15min":1.0409999999999999},"min":{"1min":0.627,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.631,"5min":1.2729999999999999,"15min":1.2270000000000001},"min":{"1min":0.57299999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":6.3570000000000002,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":0.89600000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.3069999999999999,"15min":1.2709999999999999},"min":{"1min":0.92000000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.137},{"interface":"front","average":{"1min":1.7070000000000001,"5min":1.5,"15min":1.482},"min":{"1min":1.022,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":3.399,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.0620000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.4399999999999999,"15min":1.3899999999999999},"min":{"1min":0.88100000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":5.9939999999999998,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":1.026,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.391,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.2110000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.083},{"interface":"front","average":{"1min":1.744,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":1.0840000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.879,"5min":3.879,"15min":3.879},"last":1.333}]},{"osd":7,"last update":"Fri Nov 24 08:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.8029999999999999,"5min":1.5660000000000001,"15min":1.548},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":3.4540000000000002,"5min":3.823,"15min":3.823},"last":0.999},{"interface":"front","average":{"1min":1.75,"5min":1.5209999999999999,"15min":1.4970000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.1930000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:07:47.439 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:07:47.439 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:07:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.709+0000 7f0066dac640 1 Processor -- start 2023-11-24T08:07:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.710+0000 7f0066dac640 1 -- start start 2023-11-24T08:07:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.711+0000 7f0066dac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00601448c0 0x7f006006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.711+0000 7f0066dac640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f006015bda0 0x7f0060070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.711+0000 7f0066dac640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006006dd50 con 0x7f006006d910 2023-11-24T08:07:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.711+0000 7f0066dac640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006015c230 con 0x7f006015bda0 2023-11-24T08:07:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.711+0000 7f0066dac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0060144ca0 con 0x7f00601448c0 2023-11-24T08:07:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.712+0000 7f00655a9640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f006006d910 0x7f006015b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37808/0 (socket says 172.21.15.64:37808) 2023-11-24T08:07:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.712+0000 7f00655a9640 1 -- 172.21.15.64:0/713471237 learned_addr learned my addr 172.21.15.64:0/713471237 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.712+0000 7f0065daa640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00601448c0 0x7f006006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.712+0000 7f00665ab640 1 --2- 172.21.15.64:0/713471237 >> v2:172.21.15.91:3300/0 conn(0x7f006015bda0 0x7f0060070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.712+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162784455 0 0) 0x7f006006dd50 con 0x7f006006d910 2023-11-24T08:07:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f004c003520 con 0x7f006006d910 2023-11-24T08:07:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2212088406 0 0) 0x7f004c003520 con 0x7f006006d910 2023-11-24T08:07:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00601448c0 msgr2=0x7f006006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00601448c0 0x7f006006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 >> v2:172.21.15.91:3300/0 conn(0x7f006015bda0 msgr2=0x7f0060070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 --2- 172.21.15.64:0/713471237 >> v2:172.21.15.91:3300/0 conn(0x7f006015bda0 0x7f0060070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006015c450 con 0x7f006006d910 2023-11-24T08:07:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f00500017f0 con 0x7f006006d910 2023-11-24T08:07:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 >> v1:172.21.15.64:6789/0 conn(0x7f006006d910 legacy=0x7f006015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c0039e0 0x7f004c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c0047c0 0x7f004c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00601448c0 0x7f004c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006015c230 con 0x7f004c0047c0 2023-11-24T08:07:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0060144ca0 con 0x7f004c0039e0 2023-11-24T08:07:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.713+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c00b980 con 0x7f00601448c0 2023-11-24T08:07:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.714+0000 7f00655a9640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c0047c0 0x7f004c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.714+0000 7f0065daa640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c0039e0 0x7f004c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.714+0000 7f00665ab640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00601448c0 0x7f004c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.714+0000 7f00655a9640 1 -- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00601448c0 msgr2=0x7f004c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.714+0000 7f00655a9640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f00601448c0 0x7f004c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.714+0000 7f00655a9640 1 -- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c0039e0 msgr2=0x7f004c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:47.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.714+0000 7f00655a9640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f004c0039e0 0x7f004c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.714+0000 7f00655a9640 1 -- 172.21.15.64:0/713471237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c003520 con 0x7f004c0047c0 2023-11-24T08:07:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.715+0000 7f00655a9640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c0047c0 0x7f004c004c40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f005000c240 tx=0x7f004c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65810ecf61e3e532 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.716+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050007270 con 0x7f004c0047c0 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.716+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00500073d0 con 0x7f004c0047c0 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.716+0000 7f0064da8640 1 -- 172.21.15.64:0/713471237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005002c8b0 con 0x7f004c0047c0 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.716+0000 7f0066dac640 1 -- 172.21.15.64:0/713471237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c0047c0 msgr2=0x7f004c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.716+0000 7f0066dac640 1 --2- 172.21.15.64:0/713471237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c0047c0 0x7f004c004c40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f005000c240 tx=0x7f004c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.716+0000 7f00665ab640 1 -- 172.21.15.64:0/713471237 reap_dead start 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.717+0000 7f0066dac640 1 -- 172.21.15.64:0/713471237 shutdown_connections 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.717+0000 7f0066dac640 1 -- 172.21.15.64:0/713471237 >> 172.21.15.64:0/713471237 conn(0x7f00600b0aa0 msgr2=0x7f00600b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.717+0000 7f0066dac640 1 -- 172.21.15.64:0/713471237 shutdown_connections 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.717+0000 7f0066dac640 1 -- 172.21.15.64:0/713471237 wait complete. 2023-11-24T08:07:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.719+0000 7f0066dac640 1 Processor -- start 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.719+0000 7f0066dac640 1 -- start start 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f0066dac640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f004c0047c0 0x7f006013c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f0066dac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b31c0 0x7f006013a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f0066dac640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0060136f40 0x7f00601373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f0066dac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006013afb0 con 0x7f00600b31c0 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f0066dac640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006013b290 con 0x7f0060136f40 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f0066dac640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006013b570 con 0x7f004c0047c0 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f00655a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b31c0 0x7f006013a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f00655a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b31c0 0x7f006013a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48942/0 (socket says 172.21.15.64:48942) 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f00655a9640 1 -- 172.21.15.64:0/808832505 learned_addr learned my addr 172.21.15.64:0/808832505 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:47.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f0065daa640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f004c0047c0 0x7f006013c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f00665ab640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0060136f40 0x7f00601373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f00655a9640 1 -- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f004c0047c0 msgr2=0x7f006013c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f00655a9640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f004c0047c0 0x7f006013c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f00655a9640 1 -- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0060136f40 msgr2=0x7f00601373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.720+0000 7f00655a9640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0060136f40 0x7f00601373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f00655a9640 1 -- 172.21.15.64:0/808832505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006013b850 con 0x7f00600b31c0 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f00665ab640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0060136f40 0x7f00601373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f0065daa640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f004c0047c0 0x7f006013c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f00655a9640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b31c0 0x7f006013a660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0050000ef0 tx=0x7f0050001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f00567fc640 1 -- 172.21.15.64:0/808832505 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050039070 con 0x7f00600b31c0 2023-11-24T08:07:47.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f00567fc640 1 -- 172.21.15.64:0/808832505 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005002ccc0 con 0x7f00600b31c0 2023-11-24T08:07:47.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f00567fc640 1 -- 172.21.15.64:0/808832505 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005003c040 con 0x7f00600b31c0 2023-11-24T08:07:47.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006013bb30 con 0x7f00600b31c0 2023-11-24T08:07:47.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.721+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006015ada0 con 0x7f00600b31c0 2023-11-24T08:07:47.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.723+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006004e840 con 0x7f00600b31c0 2023-11-24T08:07:47.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.724+0000 7f00567fc640 1 -- 172.21.15.64:0/808832505 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f0050032070 con 0x7f00600b31c0 2023-11-24T08:07:47.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.724+0000 7f00567fc640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f004406b230 0x7f004406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:47.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.725+0000 7f00567fc640 1 -- 172.21.15.64:0/808832505 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00500aa7f0 con 0x7f00600b31c0 2023-11-24T08:07:47.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.725+0000 7f0065daa640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f004406b230 0x7f004406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:47.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.726+0000 7f0065daa640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f004406b230 0x7f004406d6f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f005c006d40 tx=0x7f005c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:47.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.729+0000 7f00567fc640 1 -- 172.21.15.64:0/808832505 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0050075310 con 0x7f00600b31c0 2023-11-24T08:07:47.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.994+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f006015d170 con 0x7f00600b31c0 2023-11-24T08:07:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:47.999+0000 7f00567fc640 1 -- 172.21.15.64:0/808832505 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f005007de60 con 0x7f00600b31c0 2023-11-24T08:07:48.001 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-11-24T08:07:48.001 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA== 2023-11-24T08:07:48.001 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow" 2023-11-24T08:07:48.001 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow r" 2023-11-24T08:07:48.001 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:07:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-11-24T08:07:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f004406b230 msgr2=0x7f004406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:48.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f004406b230 0x7f004406d6f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f005c006d40 tx=0x7f005c008040 comp rx=0 tx=0).stop 2023-11-24T08:07:48.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b31c0 msgr2=0x7f006013a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:48.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b31c0 0x7f006013a660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0050000ef0 tx=0x7f0050001160 comp rx=0 tx=0).stop 2023-11-24T08:07:48.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 shutdown_connections 2023-11-24T08:07:48.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f004406b230 0x7f004406d6f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0060136f40 0x7f00601373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b31c0 0x7f006013a660 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 --2- 172.21.15.64:0/808832505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f004c0047c0 0x7f006013c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 >> 172.21.15.64:0/808832505 conn(0x7f00600b0aa0 msgr2=0x7f006015c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:48.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 shutdown_connections 2023-11-24T08:07:48.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.002+0000 7f0066dac640 1 -- 172.21.15.64:0/808832505 wait complete. 2023-11-24T08:07:48.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:07:48.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.240+0000 7efd6217a640 1 Processor -- start 2023-11-24T08:07:48.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.240+0000 7efd6217a640 1 -- start start 2023-11-24T08:07:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd6217a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd5c13dfd0 0x7efd5c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:48.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd6217a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7efd5c0cf050 0x7efd5c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:48.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd6217a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5c0ce7c0 con 0x7efd5c06c130 2023-11-24T08:07:48.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd6217a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0cfa40 con 0x7efd5c13dfd0 2023-11-24T08:07:48.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd6217a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c140900 con 0x7efd5c0cf050 2023-11-24T08:07:48.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd61178640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd5c13dfd0 0x7efd5c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:48.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd61979640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efd5c06c130 0x7efd5c06c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37820/0 (socket says 172.21.15.64:37820) 2023-11-24T08:07:48.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd61979640 1 -- 172.21.15.64:0/3127942583 learned_addr learned my addr 172.21.15.64:0/3127942583 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:48.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd60977640 1 --2- 172.21.15.64:0/3127942583 >> v2:172.21.15.91:3300/0 conn(0x7efd5c0cf050 0x7efd5c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:48.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd61178640 1 -- 172.21.15.64:0/3127942583 >> v2:172.21.15.91:3300/0 conn(0x7efd5c0cf050 msgr2=0x7efd5c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:48.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.242+0000 7efd61178640 1 --2- 172.21.15.64:0/3127942583 >> v2:172.21.15.91:3300/0 conn(0x7efd5c0cf050 0x7efd5c0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.243+0000 7efd61178640 1 -- 172.21.15.64:0/3127942583 >> v1:172.21.15.64:6789/0 conn(0x7efd5c06c130 legacy=0x7efd5c06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:48.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.243+0000 7efd61178640 1 -- 172.21.15.64:0/3127942583 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c15fa70 con 0x7efd5c13dfd0 2023-11-24T08:07:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.243+0000 7efd60977640 1 --2- 172.21.15.64:0/3127942583 >> v2:172.21.15.91:3300/0 conn(0x7efd5c0cf050 0x7efd5c0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.243+0000 7efd53fff640 1 -- 172.21.15.64:0/3127942583 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338905025 0 0) 0x7efd5c0ce7c0 con 0x7efd5c06c130 2023-11-24T08:07:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.244+0000 7efd61178640 1 --2- 172.21.15.64:0/3127942583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c15be70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efd5c06e070 tx=0x7efd5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7c035ebd9303b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.244+0000 7efd53fff640 1 -- 172.21.15.64:0/3127942583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58031830 con 0x7efd5c13dfd0 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.244+0000 7efd53fff640 1 -- 172.21.15.64:0/3127942583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd58031990 con 0x7efd5c13dfd0 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.244+0000 7efd53fff640 1 -- 172.21.15.64:0/3127942583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58031d20 con 0x7efd5c13dfd0 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.244+0000 7efd6217a640 1 -- 172.21.15.64:0/3127942583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 msgr2=0x7efd5c15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.244+0000 7efd6217a640 1 --2- 172.21.15.64:0/3127942583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c15be70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efd5c06e070 tx=0x7efd5802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.245+0000 7efd6217a640 1 -- 172.21.15.64:0/3127942583 shutdown_connections 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.245+0000 7efd6217a640 1 --2- 172.21.15.64:0/3127942583 >> v2:172.21.15.91:3300/0 conn(0x7efd5c0cf050 0x7efd5c0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.245+0000 7efd6217a640 1 --2- 172.21.15.64:0/3127942583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c15be70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.245+0000 7efd6217a640 1 -- 172.21.15.64:0/3127942583 >> 172.21.15.64:0/3127942583 conn(0x7efd5c06a860 msgr2=0x7efd5c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.245+0000 7efd6217a640 1 -- 172.21.15.64:0/3127942583 shutdown_connections 2023-11-24T08:07:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.245+0000 7efd6217a640 1 -- 172.21.15.64:0/3127942583 wait complete. 2023-11-24T08:07:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.246+0000 7efd6217a640 1 Processor -- start 2023-11-24T08:07:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.247+0000 7efd6217a640 1 -- start start 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd6217a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd5c06c130 0x7efd5c0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd6217a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd5c0cf050 0x7efd5c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd6217a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd6217a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c15c620 con 0x7efd5c13dfd0 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd6217a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0cb910 con 0x7efd5c0cf050 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd6217a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c15c860 con 0x7efd5c06c130 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61979640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61979640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48970/0 (socket says 172.21.15.64:48970) 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61979640 1 -- 172.21.15.64:0/4227243175 learned_addr learned my addr 172.21.15.64:0/4227243175 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:48.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61178640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd5c06c130 0x7efd5c0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd60977640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd5c0cf050 0x7efd5c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61979640 1 -- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd5c06c130 msgr2=0x7efd5c0cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61979640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd5c06c130 0x7efd5c0cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61979640 1 -- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd5c0cf050 msgr2=0x7efd5c15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61979640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd5c0cf050 0x7efd5c15d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61979640 1 -- 172.21.15.64:0/4227243175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd58009400 con 0x7efd5c13dfd0 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.248+0000 7efd61178640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd5c06c130 0x7efd5c0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.249+0000 7efd61979640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c157be0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c7e0 tx=0x7efd5400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.249+0000 7efd51ffb640 1 -- 172.21.15.64:0/4227243175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400a640 con 0x7efd5c13dfd0 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.249+0000 7efd51ffb640 1 -- 172.21.15.64:0/4227243175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5400a7c0 con 0x7efd5c13dfd0 2023-11-24T08:07:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.249+0000 7efd51ffb640 1 -- 172.21.15.64:0/4227243175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400ab70 con 0x7efd5c13dfd0 2023-11-24T08:07:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.249+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c15cb40 con 0x7efd5c13dfd0 2023-11-24T08:07:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.249+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c15d100 con 0x7efd5c13dfd0 2023-11-24T08:07:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.250+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c04e840 con 0x7efd5c13dfd0 2023-11-24T08:07:48.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.251+0000 7efd51ffb640 1 -- 172.21.15.64:0/4227243175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7efd5400acd0 con 0x7efd5c13dfd0 2023-11-24T08:07:48.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.252+0000 7efd51ffb640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd4006b230 0x7efd4006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:48.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.252+0000 7efd51ffb640 1 -- 172.21.15.64:0/4227243175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd54089330 con 0x7efd5c13dfd0 2023-11-24T08:07:48.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.253+0000 7efd61178640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd4006b230 0x7efd4006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.254+0000 7efd61178640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd4006b230 0x7efd4006d6f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efd58036000 tx=0x7efd5803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:48.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.256+0000 7efd51ffb640 1 -- 172.21.15.64:0/4227243175 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd54053e50 con 0x7efd5c13dfd0 2023-11-24T08:07:48.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.485+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd5c06e0a0 con 0x7efd5c13dfd0 2023-11-24T08:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.487+0000 7efd51ffb640 1 -- 172.21.15.64:0/4227243175 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4909 (secure 0 0 0) 0x7efd54018090 con 0x7efd5c13dfd0 2023-11-24T08:07:48.489 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:48.489 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3149637441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3149637441},{"type":"v1","addr":"172.21.15.91:6836","nonce":3149637441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2159744284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2159744284},{"type":"v1","addr":"172.21.15.91:6837","nonce":2159744284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5764,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1010090226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1010090226},{"type":"v1","addr":"172.21.15.64:6835","nonce":1010090226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:07:41.805482+0000","modified":"2023-11-24T08:07:44.865599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6684},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6684":{"gid":6684,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/45948764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":45948764},{"type":"v1","addr":"172.21.15.64:6833","nonce":45948764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 43 2023-11-24T08:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd4006b230 msgr2=0x7efd4006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd4006b230 0x7efd4006d6f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efd58036000 tx=0x7efd5803c040 comp rx=0 tx=0).stop 2023-11-24T08:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 msgr2=0x7efd5c157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c157be0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c7e0 tx=0x7efd5400cca0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 shutdown_connections 2023-11-24T08:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efd4006b230 0x7efd4006d6f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd5c13dfd0 0x7efd5c157be0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efd5c0cf050 0x7efd5c15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 --2- 172.21.15.64:0/4227243175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efd5c06c130 0x7efd5c0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.489+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 >> 172.21.15.64:0/4227243175 conn(0x7efd5c06a860 msgr2=0x7efd5c15b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.490+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 shutdown_connections 2023-11-24T08:07:48.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:48.490+0000 7efd6217a640 1 -- 172.21.15.64:0/4227243175 wait complete. 2023-11-24T08:07:48.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:07:41.805482+0000', 'modified': '2023-11-24T08:07:44.865599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6684}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6684': {'gid': 6684, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/45948764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 45948764}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 45948764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:07:48.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:07:48.913 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T08:07:49.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.148+0000 7fedcc448640 1 Processor -- start 2023-11-24T08:07:49.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.149+0000 7fedcc448640 1 -- start start 2023-11-24T08:07:49.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.150+0000 7fedcc448640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fedc415d4a0 0x7fedc40af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.150+0000 7fedcc448640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc40aeb50 con 0x7fedc40b4d20 2023-11-24T08:07:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.150+0000 7fedcc448640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc4161340 con 0x7fedc40af730 2023-11-24T08:07:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.150+0000 7fedcc448640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc4161fc0 con 0x7fedc415d4a0 2023-11-24T08:07:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.151+0000 7fedca1e4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fedc415d4a0 0x7fedc40af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.151+0000 7fedca9e5640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fedc40b4d20 0x7fedc415d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37830/0 (socket says 172.21.15.64:37830) 2023-11-24T08:07:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.151+0000 7fedca9e5640 1 -- 172.21.15.64:0/626756496 learned_addr learned my addr 172.21.15.64:0/626756496 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.151+0000 7fedca1e4640 1 -- 172.21.15.64:0/626756496 >> v1:172.21.15.91:6789/0 conn(0x7fedc40af730 legacy=0x7fedc40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.151+0000 7fedca1e4640 1 -- 172.21.15.64:0/626756496 >> v1:172.21.15.64:6789/0 conn(0x7fedc40b4d20 legacy=0x7fedc415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.151+0000 7fedca1e4640 1 -- 172.21.15.64:0/626756496 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4161620 con 0x7fedc415d4a0 2023-11-24T08:07:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.151+0000 7fedc91e2640 1 -- 172.21.15.64:0/626756496 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445946037 0 0) 0x7fedc40aeb50 con 0x7fedc40b4d20 2023-11-24T08:07:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.153+0000 7fedca1e4640 1 --2- 172.21.15.64:0/626756496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc415d4a0 0x7fedc40af1f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009d90 tx=0x7fedc002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef0c556469815bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.153+0000 7fedc91e2640 1 -- 172.21.15.64:0/626756496 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc002ccf0 con 0x7fedc415d4a0 2023-11-24T08:07:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.153+0000 7fedc91e2640 1 -- 172.21.15.64:0/626756496 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc002ce50 con 0x7fedc415d4a0 2023-11-24T08:07:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.153+0000 7fedc91e2640 1 -- 172.21.15.64:0/626756496 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0034660 con 0x7fedc415d4a0 2023-11-24T08:07:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.154+0000 7fedcc448640 1 -- 172.21.15.64:0/626756496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc415d4a0 msgr2=0x7fedc40af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.154+0000 7fedcc448640 1 --2- 172.21.15.64:0/626756496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc415d4a0 0x7fedc40af1f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009d90 tx=0x7fedc002bf40 comp rx=0 tx=0).stop 2023-11-24T08:07:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.154+0000 7fedcc448640 1 -- 172.21.15.64:0/626756496 shutdown_connections 2023-11-24T08:07:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.154+0000 7fedcc448640 1 --2- 172.21.15.64:0/626756496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc415d4a0 0x7fedc40af1f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.154+0000 7fedcc448640 1 -- 172.21.15.64:0/626756496 >> 172.21.15.64:0/626756496 conn(0x7fedc406d420 msgr2=0x7fedc406d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.154+0000 7fedcc448640 1 -- 172.21.15.64:0/626756496 shutdown_connections 2023-11-24T08:07:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.154+0000 7fedcc448640 1 -- 172.21.15.64:0/626756496 wait complete. 2023-11-24T08:07:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.156+0000 7fedcc448640 1 Processor -- start 2023-11-24T08:07:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.156+0000 7fedcc448640 1 -- start start 2023-11-24T08:07:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedcc448640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc40af730 0x7fedc4151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedcc448640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc40b4d20 0x7fedc41f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedcc448640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedc415d4a0 0x7fedc40cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedcc448640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40ccc60 con 0x7fedc40af730 2023-11-24T08:07:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedcc448640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc41f71d0 con 0x7fedc40b4d20 2023-11-24T08:07:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedcc448640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc41f6c10 con 0x7fedc415d4a0 2023-11-24T08:07:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedca1e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc40af730 0x7fedc4151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedca1e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc40af730 0x7fedc4151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48982/0 (socket says 172.21.15.64:48982) 2023-11-24T08:07:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedca1e4640 1 -- 172.21.15.64:0/638795126 learned_addr learned my addr 172.21.15.64:0/638795126 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedca9e5640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedc415d4a0 0x7fedc40cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedc99e3640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc40b4d20 0x7fedc41f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedca1e4640 1 -- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedc415d4a0 msgr2=0x7fedc40cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.157+0000 7fedca1e4640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedc415d4a0 0x7fedc40cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedca1e4640 1 -- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc40b4d20 msgr2=0x7fedc41f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedca1e4640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc40b4d20 0x7fedc41f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedca1e4640 1 -- 172.21.15.64:0/638795126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc0009a70 con 0x7fedc40af730 2023-11-24T08:07:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedca1e4640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc40af730 0x7fedc4151020 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fedc002dfd0 tx=0x7fedc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedbaffd640 1 -- 172.21.15.64:0/638795126 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0003b90 con 0x7fedc40af730 2023-11-24T08:07:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedbaffd640 1 -- 172.21.15.64:0/638795126 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0003cf0 con 0x7fedc40af730 2023-11-24T08:07:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedbaffd640 1 -- 172.21.15.64:0/638795126 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0004270 con 0x7fedc40af730 2023-11-24T08:07:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedcc448640 1 -- 172.21.15.64:0/638795126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc41f6ef0 con 0x7fedc40af730 2023-11-24T08:07:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.158+0000 7fedcc448640 1 -- 172.21.15.64:0/638795126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc41f67e0 con 0x7fedc40af730 2023-11-24T08:07:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.161+0000 7fedbaffd640 1 -- 172.21.15.64:0/638795126 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fedc003b040 con 0x7fedc40af730 2023-11-24T08:07:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.162+0000 7fedbaffd640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feda806b230 0x7feda806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.162+0000 7fedbaffd640 1 -- 172.21.15.64:0/638795126 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedc00acaf0 con 0x7fedc40af730 2023-11-24T08:07:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.162+0000 7fedc99e3640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feda806b230 0x7feda806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.162+0000 7fedcefdb740 1 Processor -- start 2023-11-24T08:07:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.163+0000 7fedcefdb740 1 -- start start 2023-11-24T08:07:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.163+0000 7fedc99e3640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feda806b230 0x7feda806d6f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fedc4151560 tx=0x7fedb4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.163+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6fea0 0x561461c70280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.163+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6aba0 0x561461c6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.163+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6f5e0 0x561461cb3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedcefdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561461bce880 con 0x561461c6fea0 2023-11-24T08:07:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedcefdb740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x561461bddae0 con 0x561461c6f5e0 2023-11-24T08:07:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedcefdb740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x561461bdb1d0 con 0x561461c6aba0 2023-11-24T08:07:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6fea0 0x561461c70280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6fea0 0x561461c70280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48986/0 (socket says 172.21.15.64:48986) 2023-11-24T08:07:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 -- 172.21.15.64:0/2209504778 learned_addr learned my addr 172.21.15.64:0/2209504778 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 -- 172.21.15.64:0/2209504778 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6aba0 msgr2=0x561461c6f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 --2- 172.21.15.64:0/2209504778 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6aba0 0x561461c6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 -- 172.21.15.64:0/2209504778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6f5e0 msgr2=0x561461cb3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca9e5640 1 --2- 172.21.15.64:0/2209504778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6f5e0 0x561461cb3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 --2- 172.21.15.64:0/2209504778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6f5e0 0x561461cb3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 -- 172.21.15.64:0/2209504778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561461b9e860 con 0x561461c6fea0 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.164+0000 7fedca1e4640 1 --2- 172.21.15.64:0/2209504778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6fea0 0x561461c70280 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fedc007b260 tx=0x7fedc004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de344fa70008d68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7feda7fff640 1 -- 172.21.15.64:0/2209504778 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc007b2d0 con 0x561461c6fea0 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7feda7fff640 1 -- 172.21.15.64:0/2209504778 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc007f420 con 0x561461c6fea0 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7feda7fff640 1 -- 172.21.15.64:0/2209504778 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc00835d0 con 0x561461c6fea0 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7fedcefdb740 1 -- 172.21.15.64:0/2209504778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6fea0 msgr2=0x561461c70280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7fedcefdb740 1 --2- 172.21.15.64:0/2209504778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6fea0 0x561461c70280 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fedc007b260 tx=0x7fedc004c000 comp rx=0 tx=0).stop 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7fedcefdb740 1 -- 172.21.15.64:0/2209504778 shutdown_connections 2023-11-24T08:07:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7fedcefdb740 1 --2- 172.21.15.64:0/2209504778 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6f5e0 0x561461cb3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7fedcefdb740 1 --2- 172.21.15.64:0/2209504778 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6aba0 0x561461c6f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7fedcefdb740 1 --2- 172.21.15.64:0/2209504778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6fea0 0x561461c70280 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.165+0000 7fedcefdb740 1 -- 172.21.15.64:0/2209504778 >> 172.21.15.64:0/2209504778 conn(0x561461c31c70 msgr2=0x561461cb5990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.166+0000 7fedcefdb740 1 -- 172.21.15.64:0/2209504778 shutdown_connections 2023-11-24T08:07:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.166+0000 7fedcefdb740 1 -- 172.21.15.64:0/2209504778 wait complete. 2023-11-24T08:07:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.166+0000 7fedcefdb740 1 -- start start 2023-11-24T08:07:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.167+0000 7fedcefdb740 1 Processor -- start 2023-11-24T08:07:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461cbdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 0x561461c95a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 0x561461c96ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedcefdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561461bd6ac0 con 0x561461c6aba0 2023-11-24T08:07:49.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedcefdb740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x561461b732f0 con 0x561461c6fea0 2023-11-24T08:07:49.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedcefdb740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x561461b90c70 con 0x561461c6f5e0 2023-11-24T08:07:49.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedcefdb740 10 client.0 _ll_register_callbacks cb 0x561461b953b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:07:49.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedca1e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461cbdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedca1e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461cbdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48992/0 (socket says 172.21.15.64:48992) 2023-11-24T08:07:49.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedca1e4640 1 -- 172.21.15.64:0/870459732 learned_addr learned my addr 172.21.15.64:0/870459732 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:49.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedca1e4640 1 -- 172.21.15.64:0/870459732 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 msgr2=0x561461c95a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedca1e4640 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 0x561461c95a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedca1e4640 1 -- 172.21.15.64:0/870459732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 msgr2=0x561461c96ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedca1e4640 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 0x561461c96ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.168+0000 7fedca1e4640 1 -- 172.21.15.64:0/870459732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc41f67e0 con 0x561461c6aba0 2023-11-24T08:07:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7fedca1e4640 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461cbdc10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fedc0079620 tx=0x7fedc007d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad45e078d83c116 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7feda57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:07:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7feda57fa640 1 -- 172.21.15.64:0/870459732 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0097070 con 0x561461c6aba0 2023-11-24T08:07:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7feda57fa640 1 -- 172.21.15.64:0/870459732 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedc007b2d0 con 0x561461c6aba0 2023-11-24T08:07:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7fedcefdb740 1 -- 172.21.15.64:0/870459732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561461bcfaa0 con 0x561461c6aba0 2023-11-24T08:07:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7feda57fa640 1 -- 172.21.15.64:0/870459732 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedc0079670 con 0x561461c6aba0 2023-11-24T08:07:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7fedcefdb740 10 client.6828 fetch_fsmap learned FSMap version 43 2023-11-24T08:07:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7fedcefdb740 1 -- 172.21.15.64:0/870459732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x561461bd5140 con 0x561461c6aba0 2023-11-24T08:07:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7feda57fa640 1 -- 172.21.15.64:0/870459732 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561461bcfaa0 con 0x561461c6aba0 2023-11-24T08:07:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7feda57fa640 1 -- 172.21.15.64:0/870459732 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fedc007a7c0 con 0x561461c6aba0 2023-11-24T08:07:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7fedcefdb740 10 client.6828 fetch_fsmap finished waiting for FSMap version 43 2023-11-24T08:07:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.169+0000 7fedcefdb740 10 client.6828 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:07:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.170+0000 7fedcefdb740 20 client.6828 populate_metadata read hostname 'smithi064' 2023-11-24T08:07:49.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.170+0000 7fedcefdb740 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] conn(0x561461cbbc30 0x561461cb8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.170+0000 7fedcefdb740 4 client.6828 mds_command: new command op to 6684 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:07:49.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.170+0000 7fedcefdb740 1 -- 172.21.15.64:0/870459732 --> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561461bcf7c0 con 0x561461cbbc30 2023-11-24T08:07:49.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.170+0000 7fedc99e3640 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] conn(0x561461cbbc30 0x561461cb8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.170+0000 7fedc99e3640 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] conn(0x561461cbbc30 0x561461cb8530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39713aa98bfaa5f server_cookie=d5eb5313418c6a0e in_seq=0 out_seq=0 2023-11-24T08:07:49.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.170+0000 7feda57fa640 10 client.6828 ms_handle_connect on v2:172.21.15.64:6832/45948764 2023-11-24T08:07:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.171+0000 7feda57fa640 1 -- 172.21.15.64:0/870459732 <== mds.0 v2:172.21.15.64:6832/45948764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561461bcf7c0 con 0x561461cbbc30 2023-11-24T08:07:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.171+0000 7feda57fa640 10 client.6828 handle_command_reply: tid=0 2023-11-24T08:07:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.171+0000 7fedcefdb740 1 client.6828 shutdown 2023-11-24T08:07:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.171+0000 7fedcefdb740 1 -- 172.21.15.64:0/870459732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 msgr2=0x561461cbdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.171+0000 7fedcefdb740 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461cbdc10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fedc0079620 tx=0x7fedc007d000 comp rx=0 tx=0).stop 2023-11-24T08:07:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 1 -- 172.21.15.64:0/870459732 shutdown_connections 2023-11-24T08:07:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] conn(0x561461cbbc30 0x561461cb8530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 0x561461c96ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 0x561461c95a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 1 --2- 172.21.15.64:0/870459732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461cbdc10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7feda57fa640 0 client.6828 ms_handle_reset on v2:172.21.15.64:6832/45948764 2023-11-24T08:07:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 1 -- 172.21.15.64:0/870459732 >> 172.21.15.64:0/870459732 conn(0x561461c31c70 msgr2=0x561461cb1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 1 -- 172.21.15.64:0/870459732 shutdown_connections 2023-11-24T08:07:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 1 -- 172.21.15.64:0/870459732 wait complete. 2023-11-24T08:07:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.172+0000 7fedcefdb740 20 client.6828 trim_cache size 0 max 16384 2023-11-24T08:07:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.189+0000 7fedcefdb740 1 Processor -- start 2023-11-24T08:07:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.189+0000 7fedcefdb740 1 -- start start 2023-11-24T08:07:49.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.190+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6aba0 0x561461c96ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.190+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6f5e0 0x561461d21c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.190+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6fea0 0x561461c69b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.190+0000 7fedcefdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561461939f20 con 0x561461c6f5e0 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.190+0000 7fedcefdb740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x561461c96140 con 0x561461c6aba0 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.190+0000 7fedcefdb740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x561461c6bdd0 con 0x561461c6fea0 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6f5e0 0x561461d21c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6f5e0 0x561461d21c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48998/0 (socket says 172.21.15.64:48998) 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 -- 172.21.15.64:0/3698032001 learned_addr learned my addr 172.21.15.64:0/3698032001 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedca9e5640 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6fea0 0x561461c69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 -- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6fea0 msgr2=0x561461c69b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedca1e4640 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6aba0 0x561461c96ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6fea0 0x561461c69b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 -- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6aba0 msgr2=0x561461c96ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6aba0 0x561461c96ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 -- 172.21.15.64:0/3698032001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561461b72b80 con 0x561461c6f5e0 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedca1e4640 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6aba0 0x561461c96ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.191+0000 7fedc99e3640 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6f5e0 0x561461d21c60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fedc41551b0 tx=0x7fedb4003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc07de1d0106f11 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedb8dff640 1 -- 172.21.15.64:0/3698032001 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb401c070 con 0x561461c6f5e0 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedb8dff640 1 -- 172.21.15.64:0/3698032001 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb4024070 con 0x561461c6f5e0 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedb8dff640 1 -- 172.21.15.64:0/3698032001 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4020a90 con 0x561461c6f5e0 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 -- 172.21.15.64:0/3698032001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6f5e0 msgr2=0x561461d21c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6f5e0 0x561461d21c60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fedc41551b0 tx=0x7fedb4003760 comp rx=0 tx=0).stop 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 -- 172.21.15.64:0/3698032001 shutdown_connections 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6fea0 0x561461c69b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6f5e0 0x561461d21c60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 --2- 172.21.15.64:0/3698032001 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6aba0 0x561461c96ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 -- 172.21.15.64:0/3698032001 >> 172.21.15.64:0/3698032001 conn(0x561461c31c70 msgr2=0x561461c9e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:49.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 -- 172.21.15.64:0/3698032001 shutdown_connections 2023-11-24T08:07:49.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.192+0000 7fedcefdb740 1 -- 172.21.15.64:0/3698032001 wait complete. 2023-11-24T08:07:49.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.193+0000 7fedcefdb740 1 -- start start 2023-11-24T08:07:49.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.193+0000 7fedcefdb740 1 Processor -- start 2023-11-24T08:07:49.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.194+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461d1de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.194+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 0x561461cd2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.194+0000 7fedcefdb740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 0x561461d1e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.194+0000 7fedcefdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561461cb3f00 con 0x561461c6aba0 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.194+0000 7fedcefdb740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x561461cd38d0 con 0x561461c6fea0 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.194+0000 7fedcefdb740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x561461c70fc0 con 0x561461c6f5e0 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedcefdb740 10 client.0 _ll_register_callbacks cb 0x7fedc00836c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461d1de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461d1de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49006/0 (socket says 172.21.15.64:49006) 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 -- 172.21.15.64:0/203580014 learned_addr learned my addr 172.21.15.64:0/203580014 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedc99e3640 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 0x561461cd2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca9e5640 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 0x561461d1e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 -- 172.21.15.64:0/203580014 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 msgr2=0x561461cd2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 0x561461cd2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 -- 172.21.15.64:0/203580014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 msgr2=0x561461d1e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 0x561461d1e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 -- 172.21.15.64:0/203580014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb40052e0 con 0x561461c6aba0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7fedca1e4640 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461d1de10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fedc0083790 tx=0x7fedc007b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6745935c7486a06a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.195+0000 7feda57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7feda57fa640 1 -- 172.21.15.64:0/203580014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0097070 con 0x561461c6aba0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7feda57fa640 1 -- 172.21.15.64:0/203580014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedc007b2d0 con 0x561461c6aba0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 1 -- 172.21.15.64:0/203580014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561461c99a00 con 0x561461c6aba0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7feda57fa640 1 -- 172.21.15.64:0/203580014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedc0077610 con 0x561461c6aba0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 10 client.6834 fetch_fsmap learned FSMap version 43 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 1 -- 172.21.15.64:0/203580014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x561461b98170 con 0x561461c6aba0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7feda57fa640 1 -- 172.21.15.64:0/203580014 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561461c99a00 con 0x561461c6aba0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7feda57fa640 1 -- 172.21.15.64:0/203580014 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fedc00811e0 con 0x561461c6aba0 2023-11-24T08:07:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 10 client.6834 fetch_fsmap finished waiting for FSMap version 43 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 10 client.6834 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 20 client.6834 populate_metadata read hostname 'smithi064' 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] conn(0x561461cb14d0 0x561461cb18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 4 client.6834 mds_command: new command op to 6684 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.196+0000 7fedcefdb740 1 -- 172.21.15.64:0/203580014 --> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fedc007f420 con 0x561461cb14d0 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.197+0000 7fedc99e3640 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] conn(0x561461cb14d0 0x561461cb18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.197+0000 7fedc99e3640 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] conn(0x561461cb14d0 0x561461cb18b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dcc33507b4a5d2b server_cookie=6874db647e0d8bf1 in_seq=0 out_seq=0 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.197+0000 7feda57fa640 10 client.6834 ms_handle_connect on v2:172.21.15.64:6832/45948764 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.197+0000 7feda57fa640 1 -- 172.21.15.64:0/203580014 <== mds.0 v2:172.21.15.64:6832/45948764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7fedc007f420 con 0x561461cb14d0 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.197+0000 7feda57fa640 10 client.6834 handle_command_reply: tid=0 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.197+0000 7fedcefdb740 1 client.6834 shutdown 2023-11-24T08:07:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 -- 172.21.15.64:0/203580014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 msgr2=0x561461d1de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461d1de10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fedc0083790 tx=0x7fedc007b120 comp rx=0 tx=0).stop 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 -- 172.21.15.64:0/203580014 shutdown_connections 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.64:6832/45948764,v1:172.21.15.64:6833/45948764] conn(0x561461cb14d0 0x561461cb18b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x561461c6fea0 0x561461d1e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x561461c6f5e0 0x561461cd2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7feda57fa640 0 client.6834 ms_handle_reset on v2:172.21.15.64:6832/45948764 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 --2- 172.21.15.64:0/203580014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561461c6aba0 0x561461d1de10 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 -- 172.21.15.64:0/203580014 >> 172.21.15.64:0/203580014 conn(0x561461c31c70 msgr2=0x561461c9e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 -- 172.21.15.64:0/203580014 shutdown_connections 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 1 -- 172.21.15.64:0/203580014 wait complete. 2023-11-24T08:07:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.198+0000 7fedcefdb740 20 client.6834 trim_cache size 0 max 16384 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918", 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 6684, 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 6.8760261309999997, 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 43, 2023-11-24T08:07:49.205 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 74, 2023-11-24T08:07:49.206 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 74, 2023-11-24T08:07:49.206 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 13.207592246999999 2023-11-24T08:07:49.206 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:07:49.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 -- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feda806b230 msgr2=0x7feda806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feda806b230 0x7feda806d6f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fedc4151560 tx=0x7fedb4009040 comp rx=0 tx=0).stop 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 -- 172.21.15.64:0/638795126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc40af730 msgr2=0x7fedc4151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc40af730 0x7fedc4151020 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fedc002dfd0 tx=0x7fedc0004a40 comp rx=0 tx=0).stop 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 -- 172.21.15.64:0/638795126 shutdown_connections 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7feda806b230 0x7feda806d6f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fedc415d4a0 0x7fedc40cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fedc40b4d20 0x7fedc41f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 --2- 172.21.15.64:0/638795126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc40af730 0x7fedc4151020 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 -- 172.21.15.64:0/638795126 >> 172.21.15.64:0/638795126 conn(0x7fedc406d420 msgr2=0x7fedc415c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 -- 172.21.15.64:0/638795126 shutdown_connections 2023-11-24T08:07:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.199+0000 7fedcc448640 1 -- 172.21.15.64:0/638795126 wait complete. 2023-11-24T08:07:49.218 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:07:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.432+0000 7f5cee527640 1 Processor -- start 2023-11-24T08:07:49.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.433+0000 7f5cee527640 1 -- start start 2023-11-24T08:07:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.434+0000 7f5cee527640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5ce80d14a0 0x7f5ce80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.434+0000 7f5cee527640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce80b0640 0x7f5ce80b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.434+0000 7f5cee527640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce80cf0c0 con 0x7f5ce815d5b0 2023-11-24T08:07:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.434+0000 7f5cee527640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce80b01b0 con 0x7f5ce80d14a0 2023-11-24T08:07:49.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.434+0000 7f5cee527640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce815d990 con 0x7f5ce80b0640 2023-11-24T08:07:49.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.434+0000 7f5cecd24640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5ce80d14a0 0x7f5ce80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.435+0000 7f5cecd24640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5ce80d14a0 0x7f5ce80d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49010/0 (socket says 172.21.15.64:49010) 2023-11-24T08:07:49.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.435+0000 7f5cecd24640 1 -- 172.21.15.64:0/1401907708 learned_addr learned my addr 172.21.15.64:0/1401907708 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:49.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.435+0000 7f5cedd26640 1 --2- 172.21.15.64:0/1401907708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce80b0640 0x7f5ce80b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.435+0000 7f5cecd24640 1 -- 172.21.15.64:0/1401907708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce80b0640 msgr2=0x7f5ce80b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.435+0000 7f5cecd24640 1 --2- 172.21.15.64:0/1401907708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce80b0640 0x7f5ce80b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.435+0000 7f5cecd24640 1 -- 172.21.15.64:0/1401907708 >> v1:172.21.15.91:6789/0 conn(0x7f5ce815d5b0 legacy=0x7f5ce80d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:49.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.435+0000 7f5cecd24640 1 -- 172.21.15.64:0/1401907708 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce80d1e40 con 0x7f5ce80d14a0 2023-11-24T08:07:49.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.436+0000 7f5cecd24640 1 --2- 172.21.15.64:0/1401907708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80d14a0 0x7f5ce80d1900 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009870 tx=0x7f5cd802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528ddd6ef5532a5d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cdffff640 1 -- 172.21.15.64:0/1401907708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8031640 con 0x7f5ce80d14a0 2023-11-24T08:07:49.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cdffff640 1 -- 172.21.15.64:0/1401907708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd80317a0 con 0x7f5ce80d14a0 2023-11-24T08:07:49.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cdffff640 1 -- 172.21.15.64:0/1401907708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8031b30 con 0x7f5ce80d14a0 2023-11-24T08:07:49.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cee527640 1 -- 172.21.15.64:0/1401907708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80d14a0 msgr2=0x7f5ce80d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cee527640 1 --2- 172.21.15.64:0/1401907708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80d14a0 0x7f5ce80d1900 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009870 tx=0x7f5cd802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cee527640 1 -- 172.21.15.64:0/1401907708 shutdown_connections 2023-11-24T08:07:49.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cee527640 1 --2- 172.21.15.64:0/1401907708 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce80b0640 0x7f5ce80b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cee527640 1 --2- 172.21.15.64:0/1401907708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80d14a0 0x7f5ce80d1900 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.437+0000 7f5cee527640 1 -- 172.21.15.64:0/1401907708 >> 172.21.15.64:0/1401907708 conn(0x7f5ce809c3b0 msgr2=0x7f5ce80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:49.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.438+0000 7f5cee527640 1 -- 172.21.15.64:0/1401907708 shutdown_connections 2023-11-24T08:07:49.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.438+0000 7f5cee527640 1 -- 172.21.15.64:0/1401907708 wait complete. 2023-11-24T08:07:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.439+0000 7f5cee527640 1 Processor -- start 2023-11-24T08:07:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.439+0000 7f5cee527640 1 -- start start 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.440+0000 7f5cee527640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80b0640 0x7f5ce8152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.440+0000 7f5cee527640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ce80d14a0 0x7f5ce81f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.440+0000 7f5cee527640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce815d5b0 0x7f5ce815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.440+0000 7f5cee527640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80b8690 con 0x7f5ce80b0640 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.440+0000 7f5cee527640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce81f7430 con 0x7f5ce80d14a0 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.440+0000 7f5cee527640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80b7a80 con 0x7f5ce815d5b0 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5ced525640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80b0640 0x7f5ce8152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5ced525640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80b0640 0x7f5ce8152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49026/0 (socket says 172.21.15.64:49026) 2023-11-24T08:07:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5ced525640 1 -- 172.21.15.64:0/1841271008 learned_addr learned my addr 172.21.15.64:0/1841271008 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:49.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5cedd26640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce815d5b0 0x7f5ce815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5cecd24640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ce80d14a0 0x7f5ce81f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5ced525640 1 -- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce815d5b0 msgr2=0x7f5ce815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5ced525640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce815d5b0 0x7f5ce815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5ced525640 1 -- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ce80d14a0 msgr2=0x7f5ce81f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5ced525640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ce80d14a0 0x7f5ce81f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.441+0000 7f5ced525640 1 -- 172.21.15.64:0/1841271008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80094d0 con 0x7f5ce80b0640 2023-11-24T08:07:49.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.442+0000 7f5ced525640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80b0640 0x7f5ce8152a50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400c430 tx=0x7f5ce400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.442+0000 7f5cddffb640 1 -- 172.21.15.64:0/1841271008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4011790 con 0x7f5ce80b0640 2023-11-24T08:07:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.442+0000 7f5cddffb640 1 -- 172.21.15.64:0/1841271008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce40118f0 con 0x7f5ce80b0640 2023-11-24T08:07:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.442+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce80b7d60 con 0x7f5ce80b0640 2023-11-24T08:07:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.442+0000 7f5cddffb640 1 -- 172.21.15.64:0/1841271008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4011ce0 con 0x7f5ce80b0640 2023-11-24T08:07:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.442+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce80b8320 con 0x7f5ce80b0640 2023-11-24T08:07:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.443+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce804e840 con 0x7f5ce80b0640 2023-11-24T08:07:49.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.444+0000 7f5cddffb640 1 -- 172.21.15.64:0/1841271008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5ce401c460 con 0x7f5ce80b0640 2023-11-24T08:07:49.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.445+0000 7f5cddffb640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5ccc06b260 0x7f5ccc06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:49.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.446+0000 7f5cddffb640 1 -- 172.21.15.64:0/1841271008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ce4089c60 con 0x7f5ce80b0640 2023-11-24T08:07:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.446+0000 7f5cecd24640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5ccc06b260 0x7f5ccc06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:49.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.447+0000 7f5cecd24640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5ccc06b260 0x7f5ccc06d720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8036000 tx=0x7f5cd803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:49.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.450+0000 7f5cddffb640 1 -- 172.21.15.64:0/1841271008 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ce40547c0 con 0x7f5ce80b0640 2023-11-24T08:07:49.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.686+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5ce80678d0 con 0x7f5ce80b0640 2023-11-24T08:07:49.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.687+0000 7f5cddffb640 1 -- 172.21.15.64:0/1841271008 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5ce4057de0 con 0x7f5ce80b0640 2023-11-24T08:07:49.689 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:07:49.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.689+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5ccc06b260 msgr2=0x7f5ccc06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.689+0000 7f5cee527640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5ccc06b260 0x7f5ccc06d720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8036000 tx=0x7f5cd803c040 comp rx=0 tx=0).stop 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.689+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80b0640 msgr2=0x7f5ce8152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.689+0000 7f5cee527640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80b0640 0x7f5ce8152a50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400c430 tx=0x7f5ce400c8f0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.690+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 shutdown_connections 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.690+0000 7f5cee527640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5ccc06b260 0x7f5ccc06d720 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.690+0000 7f5cee527640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5ce815d5b0 0x7f5ce815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.690+0000 7f5cee527640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5ce80d14a0 0x7f5ce81f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.690+0000 7f5cee527640 1 --2- 172.21.15.64:0/1841271008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ce80b0640 0x7f5ce8152a50 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.690+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 >> 172.21.15.64:0/1841271008 conn(0x7f5ce809c3b0 msgr2=0x7f5ce80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.690+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 shutdown_connections 2023-11-24T08:07:49.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:49.690+0000 7f5cee527640 1 -- 172.21.15.64:0/1841271008 wait complete. 2023-11-24T08:07:49.711 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:07:49.711 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:07:49.712 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T08:07:49.712 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:07:49.712 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:49.712 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:07:49.712 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:07:49.712 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:07:49.712 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:07:49.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:49.712 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T08:07:49.770 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:49.771 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42113sec preferred_lft 42113sec 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:07:49.772 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3c83:ffff:fee3:836f/64 scope link 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::54d4:76ff:fe60:984a/64 scope link 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether ae:e6:20:dc:18:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ace6:20ff:fedc:18d9/64 scope link 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:49.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:49.774 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:07:49.870 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T08:07:49.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:49.875 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T08:07:49.936 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T08:07:49.936 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:07:49.936 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:07:49.936 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:07:49.936 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:07:49.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:49.937 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:07:49.937 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:07:49.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:07:49.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:07:49.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:07:49.937 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:07:50.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:50.134 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:07:50.186 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:07:50.186 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:07:50.187 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:07:50.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:50.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-11-24T08:07:50.278 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T08:07:50.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:07:50.278 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:50.337 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:07:50.337 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:07:50.337 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:50.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:07:50.493 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:50.554 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:07:50.554 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:07:50.554 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:07:50.554 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:07:50.554 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:07:50.554 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:07:50.555 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:07:50.556 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:50.557 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:50.557 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:50.557 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:50.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:50.558 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:07:50.613 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:07:50.614 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T08:07:50.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:07:50.704 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:50.749 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:07:50.749 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:07:50.749 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:07:50.749 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:07:50.749 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:07:50.749 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:07:50.750 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:07:50.750 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:07:50.751 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:07:50.751 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:07:50.752 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:07:50.752 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:50.752 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:07:50.752 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:07:50.753 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:07:50.753 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:07:50.754 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:50.754 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:07:50.754 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:50.754 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:50.755 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:50.755 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:50.755 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:07:50.755 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:07:50.756 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:07:50.756 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:07:50.756 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:07:50.756 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:07:50.757 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:07:50.757 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:50.757 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:07:50.758 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:50.758 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:07:50.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:50.760 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:07:50.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-11-24T08:07:50.826+0000 7f266604a480 -1 init, newargv = 0x55807eb97000 newargc=15 2023-11-24T08:07:50.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[112430]: starting ceph client 2023-11-24T08:07:50.836 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[112430]: starting fuse 2023-11-24T08:07:50.852 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T08:07:50.853 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T08:07:50.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:07:50.853 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T08:07:50.853 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T08:07:50.853 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T08:07:50.853 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T08:07:50.854 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:07:50.854 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T08:07:50.854 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T08:07:50.855 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T08:07:50.855 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T08:07:50.855 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:07:50.855 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T08:07:50.855 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T08:07:50.855 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:07:50.856 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-11-24T08:07:50.857 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T08:07:51.060 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.112430.asok 2023-11-24T08:07:51.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:07:51.078 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.112430.asok 2023-11-24T08:07:51.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:51.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.112430.asok status 2023-11-24T08:07:51.335 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "112430", 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:51.336 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 6846, 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 6846 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:07:51.337 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 696886227 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 696886227 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.6846 192.168.0.1:0/696886227", 2023-11-24T08:07:51.338 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/696886227", 2023-11-24T08:07:51.339 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T08:07:51.339 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 43, 2023-11-24T08:07:51.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 74, 2023-11-24T08:07:51.339 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:07:51.339 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T08:07:51.339 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T08:07:51.339 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:07:51.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:51.352 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T08:07:51.352 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:51.410 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T08:07:51.410 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:51.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:51.410 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:07:51.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:07:51.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.741+0000 7f1da5b68640 1 Processor -- start 2023-11-24T08:07:51.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.741+0000 7f1da5b68640 1 -- start start 2023-11-24T08:07:51.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.742+0000 7f1da5b68640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 0x7f1da00cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:51.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.743+0000 7f1da5b68640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da015d8b0 con 0x7f1da00ce260 2023-11-24T08:07:51.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.743+0000 7f1da5b68640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00cdfb0 con 0x7f1da00d1f60 2023-11-24T08:07:51.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.743+0000 7f1da5b68640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00b91b0 con 0x7f1da015d4d0 2023-11-24T08:07:51.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.743+0000 7f1d96ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1da00ce260 0x7f1da00d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37834/0 (socket says 172.21.15.64:37834) 2023-11-24T08:07:51.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.743+0000 7f1d96ffd640 1 -- 172.21.15.64:0/399217763 learned_addr learned my addr 172.21.15.64:0/399217763 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:51.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.743+0000 7f1d9f7fe640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 0x7f1da00cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:51.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.743+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575710915 0 0) 0x7f1da015d8b0 con 0x7f1da00ce260 2023-11-24T08:07:51.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.743+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d8c003520 con 0x7f1da00ce260 2023-11-24T08:07:51.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496252390 0 0) 0x7f1d8c003520 con 0x7f1da00ce260 2023-11-24T08:07:51.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 msgr2=0x7f1da00cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 0x7f1da00cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 >> v1:172.21.15.91:6789/0 conn(0x7f1da00d1f60 legacy=0x7f1da00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00b2400 con 0x7f1da00ce260 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f1d88001670 con 0x7f1da00ce260 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 >> v1:172.21.15.64:6789/0 conn(0x7f1da00ce260 legacy=0x7f1da00d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 0x7f1d8c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d8c003f20 0x7f1d8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d8c004d00 0x7f1d8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:51.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00b91b0 con 0x7f1d8c004d00 2023-11-24T08:07:51.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c00ba60 con 0x7f1d8c003f20 2023-11-24T08:07:51.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c00bc20 con 0x7f1da015d4d0 2023-11-24T08:07:51.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.744+0000 7f1d9ffff640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d8c004d00 0x7f1d8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:51.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d9f7fe640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 0x7f1d8c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:51.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d96ffd640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d8c003f20 0x7f1d8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:51.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d9ffff640 1 -- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 msgr2=0x7f1d8c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:51.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d9ffff640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 0x7f1d8c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d9ffff640 1 -- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d8c003f20 msgr2=0x7f1d8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:51.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d9ffff640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d8c003f20 0x7f1d8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d9ffff640 1 -- 172.21.15.64:0/399217763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c003520 con 0x7f1d8c004d00 2023-11-24T08:07:51.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d9f7fe640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1da015d4d0 0x7f1d8c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:51.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.745+0000 7f1d96ffd640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d8c003f20 0x7f1d8c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:51.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.746+0000 7f1d9ffff640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d8c004d00 0x7f1d8c005180 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1da00d0d20 tx=0x7f1d8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc90fee9d45ecff7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:51.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.746+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d980052d0 con 0x7f1d8c004d00 2023-11-24T08:07:51.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.746+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9802e050 con 0x7f1d8c004d00 2023-11-24T08:07:51.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.746+0000 7f1d9effd640 1 -- 172.21.15.64:0/399217763 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98003830 con 0x7f1d8c004d00 2023-11-24T08:07:51.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.747+0000 7f1da5b68640 1 -- 172.21.15.64:0/399217763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d8c004d00 msgr2=0x7f1d8c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:51.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.747+0000 7f1da5b68640 1 --2- 172.21.15.64:0/399217763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d8c004d00 0x7f1d8c005180 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1da00d0d20 tx=0x7f1d8c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.747+0000 7f1d9ffff640 1 -- 172.21.15.64:0/399217763 reap_dead start 2023-11-24T08:07:51.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.747+0000 7f1da5b68640 1 -- 172.21.15.64:0/399217763 shutdown_connections 2023-11-24T08:07:51.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.747+0000 7f1da5b68640 1 -- 172.21.15.64:0/399217763 >> 172.21.15.64:0/399217763 conn(0x7f1da0071f90 msgr2=0x7f1da00723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:51.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.747+0000 7f1da5b68640 1 -- 172.21.15.64:0/399217763 shutdown_connections 2023-11-24T08:07:51.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.747+0000 7f1da5b68640 1 -- 172.21.15.64:0/399217763 wait complete. 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.749+0000 7f1da5b68640 1 Processor -- start 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.749+0000 7f1da5b68640 1 -- start start 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1da5b68640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d8c004d00 0x7f1da01f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1da5b68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da00cc0e0 0x7f1da00b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1da5b68640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da01f1e10 0x7f1da01f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1da5b68640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00cd200 con 0x7f1da00cc0e0 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1da5b68640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da01f6f00 con 0x7f1da01f1e10 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1da5b68640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00cbb10 con 0x7f1d8c004d00 2023-11-24T08:07:51.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1d96ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da00cc0e0 0x7f1da00b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:51.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1d96ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da00cc0e0 0x7f1da00b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49032/0 (socket says 172.21.15.64:49032) 2023-11-24T08:07:51.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.750+0000 7f1d96ffd640 1 -- 172.21.15.64:0/3992431746 learned_addr learned my addr 172.21.15.64:0/3992431746 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:51.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d9f7fe640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d8c004d00 0x7f1da01f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:51.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d9ffff640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da01f1e10 0x7f1da01f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:51.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d96ffd640 1 -- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d8c004d00 msgr2=0x7f1da01f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:51.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d96ffd640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d8c004d00 0x7f1da01f69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d96ffd640 1 -- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da01f1e10 msgr2=0x7f1da01f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:51.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d96ffd640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da01f1e10 0x7f1da01f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d96ffd640 1 -- 172.21.15.64:0/3992431746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00cc5f0 con 0x7f1da00cc0e0 2023-11-24T08:07:51.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d96ffd640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da00cc0e0 0x7f1da00b3ec0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800d8e0 tx=0x7f1d8800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:51.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d9cff9640 1 -- 172.21.15.64:0/3992431746 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88007560 con 0x7f1da00cc0e0 2023-11-24T08:07:51.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d9cff9640 1 -- 172.21.15.64:0/3992431746 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d880076c0 con 0x7f1da00cc0e0 2023-11-24T08:07:51.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da00cc8d0 con 0x7f1da00cc0e0 2023-11-24T08:07:51.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1d9cff9640 1 -- 172.21.15.64:0/3992431746 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880049e0 con 0x7f1da00cc0e0 2023-11-24T08:07:51.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.751+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da00cce90 con 0x7f1da00cc0e0 2023-11-24T08:07:51.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.752+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da00c6dd0 con 0x7f1da00cc0e0 2023-11-24T08:07:51.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.753+0000 7f1d9cff9640 1 -- 172.21.15.64:0/3992431746 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1d88004b40 con 0x7f1da00cc0e0 2023-11-24T08:07:51.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.754+0000 7f1d9cff9640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d8006b230 0x7f1d8006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:51.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.754+0000 7f1d9cff9640 1 -- 172.21.15.64:0/3992431746 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d88089c70 con 0x7f1da00cc0e0 2023-11-24T08:07:51.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.755+0000 7f1d9f7fe640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d8006b230 0x7f1d8006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:51.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.756+0000 7f1d9f7fe640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d8006b230 0x7f1d8006d6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1d90007440 tx=0x7f1d9000d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:51.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.759+0000 7f1d9cff9640 1 -- 172.21.15.64:0/3992431746 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d88058110 con 0x7f1da00cc0e0 2023-11-24T08:07:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.990+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1da0074770 con 0x7f1da00cc0e0 2023-11-24T08:07:51.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.991+0000 7f1d9cff9640 1 -- 172.21.15.64:0/3992431746 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1d88019020 con 0x7f1da00cc0e0 2023-11-24T08:07:51.993 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:07:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d8006b230 msgr2=0x7f1d8006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d8006b230 0x7f1d8006d6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1d90007440 tx=0x7f1d9000d040 comp rx=0 tx=0).stop 2023-11-24T08:07:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da00cc0e0 msgr2=0x7f1da00b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da00cc0e0 0x7f1da00b3ec0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800d8e0 tx=0x7f1d8800dda0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 shutdown_connections 2023-11-24T08:07:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d8006b230 0x7f1d8006d6f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1da01f1e10 0x7f1da01f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1da00cc0e0 0x7f1da00b3ec0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 --2- 172.21.15.64:0/3992431746 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d8c004d00 0x7f1da01f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 >> 172.21.15.64:0/3992431746 conn(0x7f1da0071f90 msgr2=0x7f1da0157800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 shutdown_connections 2023-11-24T08:07:51.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:51.993+0000 7f1da5b68640 1 -- 172.21.15.64:0/3992431746 wait complete. 2023-11-24T08:07:52.014 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:07:52.014 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:07:52.014 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2023-11-24T08:07:52.014 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:07:52.015 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:52.015 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:07:52.015 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:07:52.015 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:07:52.015 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:07:52.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:52.015 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2023-11-24T08:07:52.076 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:07:52.076 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:07:52.076 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:07:52.076 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:52.077 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:07:52.077 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:52.077 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:52.077 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:52.077 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2023-11-24T08:07:52.077 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:52.078 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:52.078 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2023-11-24T08:07:52.078 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:07:52.078 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:52.078 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:07:52.078 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:52.079 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:07:52.079 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42100sec preferred_lft 42100sec 2023-11-24T08:07:52.079 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link 2023-11-24T08:07:52.079 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:52.079 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:07:52.080 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:07:52.080 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:07:52.080 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:52.080 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::5073:3fff:fef1:71bb/64 scope link 2023-11-24T08:07:52.080 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:52.081 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:07:52.081 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:52.081 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::2497:37ff:fe0c:852a/64 scope link 2023-11-24T08:07:52.081 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:52.081 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:07:52.081 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 32:d9:7a:07:6c:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:52.082 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::30d9:7aff:fe07:6ce0/64 scope link 2023-11-24T08:07:52.082 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:07:52.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:52.082 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:07:52.176 INFO:teuthology.orchestra.run.smithi091.stdout:1 2023-11-24T08:07:52.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:52.180 DEBUG:teuthology.orchestra.run.smithi091:> route 2023-11-24T08:07:52.234 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2023-11-24T08:07:52.234 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:07:52.234 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:07:52.234 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:07:52.235 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:07:52.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:52.235 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:07:52.235 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:07:52.236 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:07:52.236 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:07:52.236 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:07:52.236 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:07:52.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:52.422 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:07:52.474 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:07:52.474 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:07:52.475 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:07:52.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:52.476 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2023-11-24T08:07:52.570 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T08:07:52.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:07:52.571 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:52.627 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:07:52.627 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:07:52.628 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:52.679 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:07:52.774 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:52.830 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:07:52.830 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:07:52.832 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:52.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:52.835 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:07:52.890 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:07:52.890 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T08:07:52.895 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:07:52.975 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:07:53.018 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:07:53.018 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:07:53.019 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:07:53.019 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:07:53.019 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:07:53.019 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:07:53.019 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:07:53.019 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:07:53.020 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:07:53.020 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:07:53.020 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:07:53.020 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:07:53.020 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:07:53.021 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:07:53.021 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:07:53.021 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:07:53.021 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:07:53.021 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:07:53.021 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:07:53.022 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:07:53.022 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:07:53.022 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:07:53.022 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:07:53.022 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:07:53.023 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:07:53.023 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:07:53.023 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:07:53.023 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:07:53.023 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:07:53.023 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:53.024 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:07:53.024 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:53.024 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:07:53.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:53.025 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:07:53.080 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:2023-11-24T08:07:53.078+0000 7fb86023c480 -1 init, newargv = 0x564bd8614bd0 newargc=15 2023-11-24T08:07:53.080 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[97915]: starting ceph client 2023-11-24T08:07:53.093 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[97915]: starting fuse 2023-11-24T08:07:53.110 INFO:teuthology.orchestra.run.smithi091.stdout:41 2023-11-24T08:07:53.110 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T08:07:53.110 DEBUG:teuthology.orchestra.run.smithi091:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:07:53.110 DEBUG:teuthology.orchestra.run.smithi091:> import glob 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> import re 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> import os 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> def _find_admin_socket(client_name): 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> files = glob.glob(asok_path) 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T08:07:53.111 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:07:53.112 DEBUG:teuthology.orchestra.run.smithi091:> # Given a non-glob path, it better be there 2023-11-24T08:07:53.112 DEBUG:teuthology.orchestra.run.smithi091:> if "*" not in asok_path: 2023-11-24T08:07:53.112 DEBUG:teuthology.orchestra.run.smithi091:> assert(len(files) == 1) 2023-11-24T08:07:53.113 DEBUG:teuthology.orchestra.run.smithi091:> return files[0] 2023-11-24T08:07:53.113 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:07:53.113 DEBUG:teuthology.orchestra.run.smithi091:> for f in files: 2023-11-24T08:07:53.113 DEBUG:teuthology.orchestra.run.smithi091:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:07:53.113 DEBUG:teuthology.orchestra.run.smithi091:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:07:53.113 DEBUG:teuthology.orchestra.run.smithi091:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:07:53.114 DEBUG:teuthology.orchestra.run.smithi091:> contents = proc_f.read() 2023-11-24T08:07:53.114 DEBUG:teuthology.orchestra.run.smithi091:> if mountpoint in contents: 2023-11-24T08:07:53.114 DEBUG:teuthology.orchestra.run.smithi091:> return f 2023-11-24T08:07:53.114 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:07:53.114 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:07:53.115 DEBUG:teuthology.orchestra.run.smithi091:> print(_find_admin_socket("client.1")) 2023-11-24T08:07:53.115 DEBUG:teuthology.orchestra.run.smithi091:> ' 2023-11-24T08:07:53.307 INFO:teuthology.orchestra.run.smithi091.stdout:/var/run/ceph/ceph-client.1.97915.asok 2023-11-24T08:07:53.311 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:07:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:07:53.324 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.97915.asok 2023-11-24T08:07:53.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:53.324 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.97915.asok status 2023-11-24T08:07:53.558 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2023-11-24T08:07:53.558 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": { 2023-11-24T08:07:53.558 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:07:53.558 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:07:53.558 INFO:teuthology.orchestra.run.smithi091.stdout: "entity_id": "1", 2023-11-24T08:07:53.558 INFO:teuthology.orchestra.run.smithi091.stdout: "hostname": "smithi091", 2023-11-24T08:07:53.558 INFO:teuthology.orchestra.run.smithi091.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T08:07:53.558 INFO:teuthology.orchestra.run.smithi091.stdout: "pid": "97915", 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "root": "/" 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_count": 0, 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_pinned_count": 0, 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 5833, 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "inst": { 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "name": { 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "client", 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "num": 5833 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:07:53.559 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 4010934677 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: } 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 4010934677 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "inst_str": "client.5833 192.168.0.1:0/4010934677", 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "addr_str": "192.168.0.1:0/4010934677", 2023-11-24T08:07:53.560 INFO:teuthology.orchestra.run.smithi091.stdout: "inode_count": 1, 2023-11-24T08:07:53.561 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_epoch": 43, 2023-11-24T08:07:53.561 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch": 74, 2023-11-24T08:07:53.561 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:07:53.561 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklisted": false, 2023-11-24T08:07:53.561 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs" 2023-11-24T08:07:53.561 INFO:teuthology.orchestra.run.smithi091.stdout:} 2023-11-24T08:07:53.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:53.572 DEBUG:teuthology.orchestra.run.smithi091:> stat --file-system '--printf=%T 2023-11-24T08:07:53.572 DEBUG:teuthology.orchestra.run.smithi091:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:53.627 INFO:teuthology.orchestra.run.smithi091.stdout:fuseblk 2023-11-24T08:07:53.627 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:53.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:53.627 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:07:53.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:07:53.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.958+0000 7fdf0da39640 1 Processor -- start 2023-11-24T08:07:53.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.958+0000 7fdf0da39640 1 -- start start 2023-11-24T08:07:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.959+0000 7fdf0da39640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf080d1d90 0x7fdf080ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.959+0000 7fdf0da39640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdf080d0fa0 0x7fdf080b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.959+0000 7fdf0da39640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf080c6bf0 con 0x7fdf0815d3b0 2023-11-24T08:07:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.959+0000 7fdf0da39640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf080d13f0 con 0x7fdf080d1d90 2023-11-24T08:07:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf0da39640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf080cedc0 con 0x7fdf080d0fa0 2023-11-24T08:07:53.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf07fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf080d1d90 0x7fdf080ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf07fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf080d1d90 0x7fdf080ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49042/0 (socket says 172.21.15.64:49042) 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf07fff640 1 -- 172.21.15.64:0/3507820466 learned_addr learned my addr 172.21.15.64:0/3507820466 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf0d238640 1 --2- 172.21.15.64:0/3507820466 >> v2:172.21.15.91:3300/0 conn(0x7fdf080d0fa0 0x7fdf080b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf07fff640 1 -- 172.21.15.64:0/3507820466 >> v2:172.21.15.91:3300/0 conn(0x7fdf080d0fa0 msgr2=0x7fdf080b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf07fff640 1 --2- 172.21.15.64:0/3507820466 >> v2:172.21.15.91:3300/0 conn(0x7fdf080d0fa0 0x7fdf080b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf07fff640 1 -- 172.21.15.64:0/3507820466 >> v1:172.21.15.91:6789/0 conn(0x7fdf0815d3b0 legacy=0x7fdf080d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.960+0000 7fdf07fff640 1 -- 172.21.15.64:0/3507820466 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080c6e40 con 0x7fdf080d1d90 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.962+0000 7fdf07fff640 1 --2- 172.21.15.64:0/3507820466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080ce6c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdf08072b10 tx=0x7fdef002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f692e84e735e351 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:53.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.962+0000 7fdf077fe640 1 -- 172.21.15.64:0/3507820466 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00317e0 con 0x7fdf080d1d90 2023-11-24T08:07:53.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.962+0000 7fdf077fe640 1 -- 172.21.15.64:0/3507820466 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0031940 con 0x7fdf080d1d90 2023-11-24T08:07:53.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.962+0000 7fdf077fe640 1 -- 172.21.15.64:0/3507820466 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0031cd0 con 0x7fdf080d1d90 2023-11-24T08:07:53.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.962+0000 7fdf0da39640 1 -- 172.21.15.64:0/3507820466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 msgr2=0x7fdf080ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:53.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.962+0000 7fdf0da39640 1 --2- 172.21.15.64:0/3507820466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080ce6c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdf08072b10 tx=0x7fdef002b880 comp rx=0 tx=0).stop 2023-11-24T08:07:53.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.963+0000 7fdf0da39640 1 -- 172.21.15.64:0/3507820466 shutdown_connections 2023-11-24T08:07:53.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.963+0000 7fdf0da39640 1 --2- 172.21.15.64:0/3507820466 >> v2:172.21.15.91:3300/0 conn(0x7fdf080d0fa0 0x7fdf080b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:53.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.963+0000 7fdf0da39640 1 --2- 172.21.15.64:0/3507820466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080ce6c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:53.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.963+0000 7fdf0da39640 1 -- 172.21.15.64:0/3507820466 >> 172.21.15.64:0/3507820466 conn(0x7fdf080d30a0 msgr2=0x7fdf080d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:53.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.963+0000 7fdf0da39640 1 -- 172.21.15.64:0/3507820466 shutdown_connections 2023-11-24T08:07:53.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.963+0000 7fdf0da39640 1 -- 172.21.15.64:0/3507820466 wait complete. 2023-11-24T08:07:53.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.964+0000 7fdf0da39640 1 Processor -- start 2023-11-24T08:07:53.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.965+0000 7fdf0da39640 1 -- start start 2023-11-24T08:07:53.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0da39640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf080d0fa0 0x7fdf0815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0da39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080b8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0da39640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf0815d3b0 0x7fdf0815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0da39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0815f8a0 con 0x7fdf080d1d90 2023-11-24T08:07:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0da39640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080ccdd0 con 0x7fdf080d0fa0 2023-11-24T08:07:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0da39640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080cc810 con 0x7fdf0815d3b0 2023-11-24T08:07:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080b8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49052/0 (socket says 172.21.15.64:49052) 2023-11-24T08:07:53.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf07fff640 1 -- 172.21.15.64:0/2465844797 learned_addr learned my addr 172.21.15.64:0/2465844797 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf07fff640 1 -- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf0815d3b0 msgr2=0x7fdf0815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0d238640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf0815d3b0 0x7fdf0815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0ca37640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf080d0fa0 0x7fdf0815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf07fff640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf0815d3b0 0x7fdf0815dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf07fff640 1 -- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf080d0fa0 msgr2=0x7fdf0815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf07fff640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf080d0fa0 0x7fdf0815f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf07fff640 1 -- 172.21.15.64:0/2465844797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00094d0 con 0x7fdf080d1d90 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.966+0000 7fdf0ca37640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf080d0fa0 0x7fdf0815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:53.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.967+0000 7fdf07fff640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080b8860 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdef002b8c0 tx=0x7fdef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:53.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.967+0000 7fdf057fa640 1 -- 172.21.15.64:0/2465844797 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0032880 con 0x7fdf080d1d90 2023-11-24T08:07:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.967+0000 7fdf057fa640 1 -- 172.21.15.64:0/2465844797 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef00329e0 con 0x7fdf080d1d90 2023-11-24T08:07:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.967+0000 7fdf057fa640 1 -- 172.21.15.64:0/2465844797 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0032d90 con 0x7fdf080d1d90 2023-11-24T08:07:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.967+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf080ccaf0 con 0x7fdf080d1d90 2023-11-24T08:07:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.967+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf080b8da0 con 0x7fdf080d1d90 2023-11-24T08:07:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.968+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0804e840 con 0x7fdf080d1d90 2023-11-24T08:07:53.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.969+0000 7fdf057fa640 1 -- 172.21.15.64:0/2465844797 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fdef004e450 con 0x7fdf080d1d90 2023-11-24T08:07:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.970+0000 7fdf057fa640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdee806b230 0x7fdee806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.970+0000 7fdf057fa640 1 -- 172.21.15.64:0/2465844797 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdef00abf80 con 0x7fdf080d1d90 2023-11-24T08:07:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.971+0000 7fdf0ca37640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdee806b230 0x7fdee806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.972+0000 7fdf0ca37640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdee806b230 0x7fdee806d6f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdef8007e90 tx=0x7fdef80073a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:53.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:53.975+0000 7fdf057fa640 1 -- 172.21.15.64:0/2465844797 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdef00769b0 con 0x7fdf080d1d90 2023-11-24T08:07:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.208+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdf080b5090 con 0x7fdf080d1d90 2023-11-24T08:07:54.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.418+0000 7fdf057fa640 1 -- 172.21.15.64:0/2465844797 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 136+0+0 (secure 0 0 0) 0x7fdef0079fd0 con 0x7fdf080d1d90 2023-11-24T08:07:54.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdee806b230 msgr2=0x7fdee806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:54.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdee806b230 0x7fdee806d6f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdef8007e90 tx=0x7fdef80073a0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 msgr2=0x7fdf080b8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:54.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080b8860 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdef002b8c0 tx=0x7fdef00046c0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 shutdown_connections 2023-11-24T08:07:54.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdee806b230 0x7fdee806d6f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdf0815d3b0 0x7fdf0815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf080d1d90 0x7fdf080b8860 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 --2- 172.21.15.64:0/2465844797 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdf080d0fa0 0x7fdf0815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 >> 172.21.15.64:0/2465844797 conn(0x7fdf080d30a0 msgr2=0x7fdf080d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:54.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 shutdown_connections 2023-11-24T08:07:54.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.422+0000 7fdf0da39640 1 -- 172.21.15.64:0/2465844797 wait complete. 2023-11-24T08:07:54.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:07:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.674+0000 7fea2dc15640 1 Processor -- start 2023-11-24T08:07:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.674+0000 7fea2dc15640 1 -- start start 2023-11-24T08:07:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea2dc15640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fea2806e7b0 0x7fea2806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea2dc15640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fea2815efc0 0x7fea2815f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea2dc15640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2815ba40 con 0x7fea281558c0 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea2dc15640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2806f180 con 0x7fea2806e7b0 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea2dc15640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2815bd20 con 0x7fea2815efc0 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea26ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fea2806e7b0 0x7fea2806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea26ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fea2806e7b0 0x7fea2806ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49056/0 (socket says 172.21.15.64:49056) 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea26ffd640 1 -- 172.21.15.64:0/1703201431 learned_addr learned my addr 172.21.15.64:0/1703201431 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea277fe640 1 --1- 172.21.15.64:0/1703201431 >> v1:172.21.15.64:6789/0 conn(0x7fea281558c0 0x7fea281411d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37848/0 (socket says 172.21.15.64:37848) 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea26ffd640 1 -- 172.21.15.64:0/1703201431 >> v2:172.21.15.91:3300/0 conn(0x7fea2815efc0 msgr2=0x7fea2815f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea26ffd640 1 --2- 172.21.15.64:0/1703201431 >> v2:172.21.15.91:3300/0 conn(0x7fea2815efc0 0x7fea2815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea26ffd640 1 -- 172.21.15.64:0/1703201431 >> v1:172.21.15.64:6789/0 conn(0x7fea281558c0 legacy=0x7fea281411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.676+0000 7fea26ffd640 1 -- 172.21.15.64:0/1703201431 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2815c000 con 0x7fea2806e7b0 2023-11-24T08:07:54.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.678+0000 7fea26ffd640 1 --2- 172.21.15.64:0/1703201431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2806ec40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fea28072460 tx=0x7fea1c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbd78160339c08e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.678+0000 7fea267fc640 1 -- 172.21.15.64:0/1703201431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c030860 con 0x7fea2806e7b0 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.678+0000 7fea267fc640 1 -- 172.21.15.64:0/1703201431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c0309c0 con 0x7fea2806e7b0 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.678+0000 7fea267fc640 1 -- 172.21.15.64:0/1703201431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c030d50 con 0x7fea2806e7b0 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.678+0000 7fea2dc15640 1 -- 172.21.15.64:0/1703201431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 msgr2=0x7fea2806ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.678+0000 7fea2dc15640 1 --2- 172.21.15.64:0/1703201431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2806ec40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fea28072460 tx=0x7fea1c02b9a0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.679+0000 7fea2dc15640 1 -- 172.21.15.64:0/1703201431 shutdown_connections 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.679+0000 7fea2dc15640 1 --2- 172.21.15.64:0/1703201431 >> v2:172.21.15.91:3300/0 conn(0x7fea2815efc0 0x7fea2815f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.679+0000 7fea2dc15640 1 --2- 172.21.15.64:0/1703201431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2806ec40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.679+0000 7fea2dc15640 1 -- 172.21.15.64:0/1703201431 >> 172.21.15.64:0/1703201431 conn(0x7fea28070be0 msgr2=0x7fea2806a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.679+0000 7fea2dc15640 1 -- 172.21.15.64:0/1703201431 shutdown_connections 2023-11-24T08:07:54.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.679+0000 7fea2dc15640 1 -- 172.21.15.64:0/1703201431 wait complete. 2023-11-24T08:07:54.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.680+0000 7fea2dc15640 1 Processor -- start 2023-11-24T08:07:54.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.681+0000 7fea2dc15640 1 -- start start 2023-11-24T08:07:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea2dc15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2813ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea2dc15640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea281558c0 0x7fea2806bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:54.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea2dc15640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea2815efc0 0x7fea2806c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:54.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea2dc15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2806b550 con 0x7fea2806e7b0 2023-11-24T08:07:54.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea2dc15640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2813cfa0 con 0x7fea2815efc0 2023-11-24T08:07:54.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea2dc15640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2806d110 con 0x7fea281558c0 2023-11-24T08:07:54.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2813ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:54.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2813ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49058/0 (socket says 172.21.15.64:49058) 2023-11-24T08:07:54.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea277fe640 1 -- 172.21.15.64:0/3221579208 learned_addr learned my addr 172.21.15.64:0/3221579208 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:54.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea26ffd640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea281558c0 0x7fea2806bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:54.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea27fff640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea2815efc0 0x7fea2806c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:54.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea277fe640 1 -- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea281558c0 msgr2=0x7fea2806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:54.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea277fe640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea281558c0 0x7fea2806bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea277fe640 1 -- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea2815efc0 msgr2=0x7fea2806c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:54.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea277fe640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea2815efc0 0x7fea2806c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.682+0000 7fea277fe640 1 -- 172.21.15.64:0/3221579208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0094d0 con 0x7fea2806e7b0 2023-11-24T08:07:54.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.683+0000 7fea277fe640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2813ca60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c800 tx=0x7fea1800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:54.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.683+0000 7fea0ffff640 1 -- 172.21.15.64:0/3221579208 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800a640 con 0x7fea2806e7b0 2023-11-24T08:07:54.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.683+0000 7fea0ffff640 1 -- 172.21.15.64:0/3221579208 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800a7a0 con 0x7fea2806e7b0 2023-11-24T08:07:54.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.683+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2806d3f0 con 0x7fea2806e7b0 2023-11-24T08:07:54.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.683+0000 7fea0ffff640 1 -- 172.21.15.64:0/3221579208 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800ab50 con 0x7fea2806e7b0 2023-11-24T08:07:54.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.683+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2806ba80 con 0x7fea2806e7b0 2023-11-24T08:07:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.684+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea28074770 con 0x7fea2806e7b0 2023-11-24T08:07:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.685+0000 7fea0ffff640 1 -- 172.21.15.64:0/3221579208 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fea1800acb0 con 0x7fea2806e7b0 2023-11-24T08:07:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.686+0000 7fea0ffff640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea1406b230 0x7fea1406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.686+0000 7fea0ffff640 1 -- 172.21.15.64:0/3221579208 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea180894d0 con 0x7fea2806e7b0 2023-11-24T08:07:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.687+0000 7fea26ffd640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea1406b230 0x7fea1406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.688+0000 7fea26ffd640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea1406b230 0x7fea1406d6f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fea1c036000 tx=0x7fea1c03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.691+0000 7fea0ffff640 1 -- 172.21.15.64:0/3221579208 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea18057b30 con 0x7fea2806e7b0 2023-11-24T08:07:54.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.909+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fea2815c960 con 0x7fea1406b230 2023-11-24T08:07:54.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.913+0000 7fea0ffff640 1 -- 172.21.15.64:0/3221579208 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fea2815c960 con 0x7fea1406b230 2023-11-24T08:07:54.914 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:07:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:07:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-11-24T08:07:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:07:54.915 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:07:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.914+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea1406b230 msgr2=0x7fea1406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.914+0000 7fea2dc15640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea1406b230 0x7fea1406d6f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fea1c036000 tx=0x7fea1c03b040 comp rx=0 tx=0).stop 2023-11-24T08:07:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.914+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 msgr2=0x7fea2813ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.914+0000 7fea2dc15640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2813ca60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c800 tx=0x7fea1800ccc0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.915+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 shutdown_connections 2023-11-24T08:07:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.915+0000 7fea2dc15640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fea1406b230 0x7fea1406d6f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.915+0000 7fea2dc15640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fea2815efc0 0x7fea2806c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.915+0000 7fea2dc15640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fea281558c0 0x7fea2806bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.915+0000 7fea2dc15640 1 --2- 172.21.15.64:0/3221579208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea2806e7b0 0x7fea2813ca60 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.915+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 >> 172.21.15.64:0/3221579208 conn(0x7fea28070be0 msgr2=0x7fea2813d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.915+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 shutdown_connections 2023-11-24T08:07:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:54.915+0000 7fea2dc15640 1 -- 172.21.15.64:0/3221579208 wait complete. 2023-11-24T08:07:54.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:07:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.168+0000 7f5f5ab83640 1 Processor -- start 2023-11-24T08:07:55.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.169+0000 7f5f5ab83640 1 -- start start 2023-11-24T08:07:55.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.170+0000 7f5f5ab83640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f5415d340 0x7f5f540d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.170+0000 7f5f5ab83640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5f540d1d90 0x7f5f540b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.170+0000 7f5f5ab83640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f540cef30 con 0x7f5f540cdbb0 2023-11-24T08:07:55.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.170+0000 7f5f5ab83640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f540d11a0 con 0x7f5f540d1d90 2023-11-24T08:07:55.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.170+0000 7f5f5ab83640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540ce320 con 0x7f5f5415d340 2023-11-24T08:07:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.171+0000 7f5f5a382640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5f540d1d90 0x7f5f540b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.171+0000 7f5f5a382640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5f540d1d90 0x7f5f540b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:57950/0 (socket says 172.21.15.64:57950) 2023-11-24T08:07:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.171+0000 7f5f5a382640 1 -- 172.21.15.64:0/3717486822 learned_addr learned my addr 172.21.15.64:0/3717486822 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.171+0000 7f5f59b81640 1 --2- 172.21.15.64:0/3717486822 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f5415d340 0x7f5f540d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.172+0000 7f5f5a382640 1 -- 172.21.15.64:0/3717486822 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f5415d340 msgr2=0x7f5f540d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.172+0000 7f5f5a382640 1 --2- 172.21.15.64:0/3717486822 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f5415d340 0x7f5f540d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.172+0000 7f5f5a382640 1 -- 172.21.15.64:0/3717486822 >> v1:172.21.15.91:6789/0 conn(0x7f5f540cdbb0 legacy=0x7f5f540d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.172+0000 7f5f5a382640 1 -- 172.21.15.64:0/3717486822 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f540ce600 con 0x7f5f540d1d90 2023-11-24T08:07:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.173+0000 7f5f5a382640 1 --2- 172.21.15.64:0/3717486822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540d1d90 0x7f5f540b3dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009870 tx=0x7f5f4c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a44dd03d5014bc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:55.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.174+0000 7f5f58b7f640 1 -- 172.21.15.64:0/3717486822 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c030840 con 0x7f5f540d1d90 2023-11-24T08:07:55.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.174+0000 7f5f58b7f640 1 -- 172.21.15.64:0/3717486822 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c0309a0 con 0x7f5f540d1d90 2023-11-24T08:07:55.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.174+0000 7f5f58b7f640 1 -- 172.21.15.64:0/3717486822 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c030d30 con 0x7f5f540d1d90 2023-11-24T08:07:55.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.175+0000 7f5f5ab83640 1 -- 172.21.15.64:0/3717486822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540d1d90 msgr2=0x7f5f540b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.175+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/3717486822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540d1d90 0x7f5f540b3dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009870 tx=0x7f5f4c02b980 comp rx=0 tx=0).stop 2023-11-24T08:07:55.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.175+0000 7f5f5ab83640 1 -- 172.21.15.64:0/3717486822 shutdown_connections 2023-11-24T08:07:55.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.175+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/3717486822 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540d1d90 0x7f5f540b3dc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.175+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/3717486822 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f5415d340 0x7f5f540d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.175+0000 7f5f5ab83640 1 -- 172.21.15.64:0/3717486822 >> 172.21.15.64:0/3717486822 conn(0x7f5f5409c370 msgr2=0x7f5f540d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:55.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.175+0000 7f5f5ab83640 1 -- 172.21.15.64:0/3717486822 shutdown_connections 2023-11-24T08:07:55.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.175+0000 7f5f5ab83640 1 -- 172.21.15.64:0/3717486822 wait complete. 2023-11-24T08:07:55.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.177+0000 7f5f5ab83640 1 Processor -- start 2023-11-24T08:07:55.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.177+0000 7f5f5ab83640 1 -- start start 2023-11-24T08:07:55.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5ab83640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540cdbb0 0x7f5f540b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5ab83640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f540d1d90 0x7f5f540b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5ab83640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5415d340 0x7f5f5415fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5ab83640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540b4180 con 0x7f5f5415d340 2023-11-24T08:07:55.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5ab83640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540b2ba0 con 0x7f5f540cdbb0 2023-11-24T08:07:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5ab83640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540b2e80 con 0x7f5f540d1d90 2023-11-24T08:07:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5a382640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5415d340 0x7f5f5415fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5a382640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5415d340 0x7f5f5415fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49074/0 (socket says 172.21.15.64:49074) 2023-11-24T08:07:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f5a382640 1 -- 172.21.15.64:0/4017252249 learned_addr learned my addr 172.21.15.64:0/4017252249 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f59b81640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540cdbb0 0x7f5f540b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.179+0000 7f5f59380640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f540d1d90 0x7f5f540b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.180+0000 7f5f5a382640 1 -- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f540d1d90 msgr2=0x7f5f540b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.180+0000 7f5f5a382640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f540d1d90 0x7f5f540b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.180+0000 7f5f5a382640 1 -- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540cdbb0 msgr2=0x7f5f540b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.180+0000 7f5f5a382640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540cdbb0 0x7f5f540b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.180+0000 7f5f5a382640 1 -- 172.21.15.64:0/4017252249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0094d0 con 0x7f5f5415d340 2023-11-24T08:07:55.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.180+0000 7f5f5a382640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5415d340 0x7f5f5415fa10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009870 tx=0x7f5f4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.180+0000 7f5f4a7fc640 1 -- 172.21.15.64:0/4017252249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c039c90 con 0x7f5f5415d340 2023-11-24T08:07:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.181+0000 7f5f4a7fc640 1 -- 172.21.15.64:0/4017252249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c039e10 con 0x7f5f5415d340 2023-11-24T08:07:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.181+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f540b3160 con 0x7f5f5415d340 2023-11-24T08:07:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.181+0000 7f5f4a7fc640 1 -- 172.21.15.64:0/4017252249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c043640 con 0x7f5f5415d340 2023-11-24T08:07:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.181+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f540b3720 con 0x7f5f5415d340 2023-11-24T08:07:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.182+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f54067d70 con 0x7f5f5415d340 2023-11-24T08:07:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.183+0000 7f5f4a7fc640 1 -- 172.21.15.64:0/4017252249 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5f4c0437a0 con 0x7f5f5415d340 2023-11-24T08:07:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.184+0000 7f5f4a7fc640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5f3806b260 0x7f5f3806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.184+0000 7f5f4a7fc640 1 -- 172.21.15.64:0/4017252249 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f4c03e070 con 0x7f5f5415d340 2023-11-24T08:07:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.185+0000 7f5f59b81640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5f3806b260 0x7f5f3806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.186+0000 7f5f59b81640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5f3806b260 0x7f5f3806d720 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5f50011340 tx=0x7f5f5000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:55.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.189+0000 7f5f4a7fc640 1 -- 172.21.15.64:0/4017252249 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f4c077770 con 0x7f5f5415d340 2023-11-24T08:07:55.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.382+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5f540b2450 con 0x7f5f5415d340 2023-11-24T08:07:55.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.383+0000 7f5f4a7fc640 1 -- 172.21.15.64:0/4017252249 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5f4c0340c0 con 0x7f5f5415d340 2023-11-24T08:07:55.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.386+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5f3806b260 msgr2=0x7f5f3806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.386+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5f3806b260 0x7f5f3806d720 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5f50011340 tx=0x7f5f5000b040 comp rx=0 tx=0).stop 2023-11-24T08:07:55.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5415d340 msgr2=0x7f5f5415fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5415d340 0x7f5f5415fa10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009870 tx=0x7f5f4c004730 comp rx=0 tx=0).stop 2023-11-24T08:07:55.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 shutdown_connections 2023-11-24T08:07:55.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5f3806b260 0x7f5f3806d720 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5415d340 0x7f5f5415fa10 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5f540d1d90 0x7f5f540b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 --2- 172.21.15.64:0/4017252249 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5f540cdbb0 0x7f5f540b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 >> 172.21.15.64:0/4017252249 conn(0x7f5f5409c370 msgr2=0x7f5f5406bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:55.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 shutdown_connections 2023-11-24T08:07:55.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.387+0000 7f5f5ab83640 1 -- 172.21.15.64:0/4017252249 wait complete. 2023-11-24T08:07:55.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp5787 2023-11-24T08:07:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.631+0000 7ff8c0f35640 1 Processor -- start 2023-11-24T08:07:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.631+0000 7ff8c0f35640 1 -- start start 2023-11-24T08:07:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.633+0000 7ff8c0f35640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8bc144770 0x7ff8bc15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.633+0000 7ff8c0f35640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff8bc0b35a0 0x7ff8bc15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.633+0000 7ff8c0f35640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc15f930 con 0x7ff8bc06c9d0 2023-11-24T08:07:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.633+0000 7ff8c0f35640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc0b43b0 con 0x7ff8bc144770 2023-11-24T08:07:55.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.633+0000 7ff8c0f35640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc0b3b10 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.633+0000 7ff8bb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8bc144770 0x7ff8bc15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.633+0000 7ff8bb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8bc144770 0x7ff8bc15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49080/0 (socket says 172.21.15.64:49080) 2023-11-24T08:07:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.633+0000 7ff8bb7fe640 1 -- 172.21.15.64:0/1869500343 learned_addr learned my addr 172.21.15.64:0/1869500343 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.634+0000 7ff8b3fff640 1 --2- 172.21.15.64:0/1869500343 >> v2:172.21.15.91:3300/0 conn(0x7ff8bc0b35a0 0x7ff8bc15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.634+0000 7ff8bb7fe640 1 -- 172.21.15.64:0/1869500343 >> v2:172.21.15.91:3300/0 conn(0x7ff8bc0b35a0 msgr2=0x7ff8bc15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:55.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.634+0000 7ff8bb7fe640 1 --2- 172.21.15.64:0/1869500343 >> v2:172.21.15.91:3300/0 conn(0x7ff8bc0b35a0 0x7ff8bc15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.634+0000 7ff8bb7fe640 1 -- 172.21.15.64:0/1869500343 >> v1:172.21.15.91:6789/0 conn(0x7ff8bc06c9d0 legacy=0x7ff8bc06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.634+0000 7ff8bb7fe640 1 -- 172.21.15.64:0/1869500343 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc0b3df0 con 0x7ff8bc144770 2023-11-24T08:07:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.635+0000 7ff8bb7fe640 1 --2- 172.21.15.64:0/1869500343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc144770 0x7ff8bc15eb60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0ae900 tx=0x7ff8ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23b9c486637bf01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.635+0000 7ff8baffd640 1 -- 172.21.15.64:0/1869500343 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac031870 con 0x7ff8bc144770 2023-11-24T08:07:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.635+0000 7ff8baffd640 1 -- 172.21.15.64:0/1869500343 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0319d0 con 0x7ff8bc144770 2023-11-24T08:07:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.635+0000 7ff8baffd640 1 -- 172.21.15.64:0/1869500343 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac031d60 con 0x7ff8bc144770 2023-11-24T08:07:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.636+0000 7ff8c0f35640 1 -- 172.21.15.64:0/1869500343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc144770 msgr2=0x7ff8bc15eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.636+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/1869500343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc144770 0x7ff8bc15eb60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0ae900 tx=0x7ff8ac02b910 comp rx=0 tx=0).stop 2023-11-24T08:07:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.636+0000 7ff8c0f35640 1 -- 172.21.15.64:0/1869500343 shutdown_connections 2023-11-24T08:07:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.636+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/1869500343 >> v2:172.21.15.91:3300/0 conn(0x7ff8bc0b35a0 0x7ff8bc15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.636+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/1869500343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc144770 0x7ff8bc15eb60 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.636+0000 7ff8c0f35640 1 -- 172.21.15.64:0/1869500343 >> 172.21.15.64:0/1869500343 conn(0x7ff8bc0cec10 msgr2=0x7ff8bc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.636+0000 7ff8c0f35640 1 -- 172.21.15.64:0/1869500343 shutdown_connections 2023-11-24T08:07:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.636+0000 7ff8c0f35640 1 -- 172.21.15.64:0/1869500343 wait complete. 2023-11-24T08:07:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.638+0000 7ff8c0f35640 1 Processor -- start 2023-11-24T08:07:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.638+0000 7ff8c0f35640 1 -- start start 2023-11-24T08:07:55.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8c0f35640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc06c9d0 0x7ff8bc140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8c0f35640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0b35a0 0x7ff8bc141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8c0f35640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc144770 0x7ff8bc141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8c0f35640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc142740 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8c0f35640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc142110 con 0x7ff8bc06c9d0 2023-11-24T08:07:55.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8c0f35640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0715b0 con 0x7ff8bc144770 2023-11-24T08:07:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0b35a0 0x7ff8bc141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0b35a0 0x7ff8bc141690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49088/0 (socket says 172.21.15.64:49088) 2023-11-24T08:07:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.639+0000 7ff8b3fff640 1 -- 172.21.15.64:0/3087578168 learned_addr learned my addr 172.21.15.64:0/3087578168 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b3fff640 1 -- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc144770 msgr2=0x7ff8bc141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8bbfff640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc144770 0x7ff8bc141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b3fff640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc144770 0x7ff8bc141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b3fff640 1 -- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc06c9d0 msgr2=0x7ff8bc140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b3fff640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc06c9d0 0x7ff8bc140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b3fff640 1 -- 172.21.15.64:0/3087578168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac0094d0 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b3fff640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0b35a0 0x7ff8bc141690 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c7f0 tx=0x7ff8a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b8ff9640 1 -- 172.21.15.64:0/3087578168 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4004220 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b8ff9640 1 -- 172.21.15.64:0/3087578168 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a40043a0 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8b8ff9640 1 -- 172.21.15.64:0/3087578168 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4007860 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.640+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc071890 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.641+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc142b60 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.642+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc1429b0 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.643+0000 7ff8b8ff9640 1 -- 172.21.15.64:0/3087578168 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff8a4007bd0 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.643+0000 7ff8b8ff9640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff89c06b230 0x7ff89c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.644+0000 7ff8b8ff9640 1 -- 172.21.15.64:0/3087578168 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8a4089e50 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.644+0000 7ff8bb7fe640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff89c06b230 0x7ff89c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:55.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.645+0000 7ff8bb7fe640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff89c06b230 0x7ff89c06d6f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac036000 tx=0x7ff8ac030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.648+0000 7ff8b8ff9640 1 -- 172.21.15.64:0/3087578168 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff8a408d030 con 0x7ff8bc0b35a0 2023-11-24T08:07:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.869+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp5787", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8bc144b50 con 0x7ff89c06b230 2023-11-24T08:07:55.885 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:55.884+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:55.885 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:55.884+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:55.886 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:55.884+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:55.886 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:55.884+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:55.886 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:55.884+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:55.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.917+0000 7ff8b8ff9640 1 -- 172.21.15.64:0/3087578168 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff8ac031bc0 con 0x7ff89c06b230 2023-11-24T08:07:55.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.920+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff89c06b230 msgr2=0x7ff89c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.920+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff89c06b230 0x7ff89c06d6f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac036000 tx=0x7ff8ac030000 comp rx=0 tx=0).stop 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.920+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0b35a0 msgr2=0x7ff8bc141690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.920+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0b35a0 0x7ff8bc141690 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c7f0 tx=0x7ff8a400ccb0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.920+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 shutdown_connections 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.920+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff89c06b230 0x7ff89c06d6f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.921+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff8bc144770 0x7ff8bc141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.921+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0b35a0 0x7ff8bc141690 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.921+0000 7ff8c0f35640 1 --2- 172.21.15.64:0/3087578168 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff8bc06c9d0 0x7ff8bc140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.921+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 >> 172.21.15.64:0/3087578168 conn(0x7ff8bc0cec10 msgr2=0x7ff8bc0b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.921+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 shutdown_connections 2023-11-24T08:07:55.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:55.921+0000 7ff8c0f35640 1 -- 172.21.15.64:0/3087578168 wait complete. 2023-11-24T08:07:55.941 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol652 --group_name subvol_grp5787 2023-11-24T08:07:56.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.169+0000 7f9947367640 1 Processor -- start 2023-11-24T08:07:56.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.169+0000 7f9947367640 1 -- start start 2023-11-24T08:07:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.170+0000 7f9947367640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f994015d3b0 0x7f99400ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9947367640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d1d90 0x7f99400b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9947367640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99400c6bf0 con 0x7f99400d0fa0 2023-11-24T08:07:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9947367640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99400d1430 con 0x7f994015d3b0 2023-11-24T08:07:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9947367640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99400cedc0 con 0x7f99400d1d90 2023-11-24T08:07:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9946365640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f994015d3b0 0x7f99400ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9946365640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f994015d3b0 0x7f99400ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49096/0 (socket says 172.21.15.64:49096) 2023-11-24T08:07:56.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9946365640 1 -- 172.21.15.64:0/2826402121 learned_addr learned my addr 172.21.15.64:0/2826402121 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:56.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9946b66640 1 --2- 172.21.15.64:0/2826402121 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d1d90 0x7f99400b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9946365640 1 -- 172.21.15.64:0/2826402121 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d1d90 msgr2=0x7f99400b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:56.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9946365640 1 --2- 172.21.15.64:0/2826402121 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d1d90 0x7f99400b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9946365640 1 -- 172.21.15.64:0/2826402121 >> v1:172.21.15.91:6789/0 conn(0x7f99400d0fa0 legacy=0x7f99400d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:56.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.171+0000 7f9946365640 1 -- 172.21.15.64:0/2826402121 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400c6e40 con 0x7f994015d3b0 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.173+0000 7f9946365640 1 --2- 172.21.15.64:0/2826402121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f99400ce6c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f993c009e50 tx=0x7f993c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19698d4bdbd117b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.173+0000 7f9945363640 1 -- 172.21.15.64:0/2826402121 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c02c9e0 con 0x7f994015d3b0 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.173+0000 7f9945363640 1 -- 172.21.15.64:0/2826402121 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c02cb40 con 0x7f994015d3b0 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.173+0000 7f9945363640 1 -- 172.21.15.64:0/2826402121 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c0343f0 con 0x7f994015d3b0 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.173+0000 7f9947367640 1 -- 172.21.15.64:0/2826402121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 msgr2=0x7f99400ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.173+0000 7f9947367640 1 --2- 172.21.15.64:0/2826402121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f99400ce6c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f993c009e50 tx=0x7f993c02be40 comp rx=0 tx=0).stop 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.174+0000 7f9947367640 1 -- 172.21.15.64:0/2826402121 shutdown_connections 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.174+0000 7f9947367640 1 --2- 172.21.15.64:0/2826402121 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d1d90 0x7f99400b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.174+0000 7f9947367640 1 --2- 172.21.15.64:0/2826402121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f99400ce6c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.174+0000 7f9947367640 1 -- 172.21.15.64:0/2826402121 >> 172.21.15.64:0/2826402121 conn(0x7f99400d30a0 msgr2=0x7f99400d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:56.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.174+0000 7f9947367640 1 -- 172.21.15.64:0/2826402121 shutdown_connections 2023-11-24T08:07:56.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.174+0000 7f9947367640 1 -- 172.21.15.64:0/2826402121 wait complete. 2023-11-24T08:07:56.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.175+0000 7f9947367640 1 Processor -- start 2023-11-24T08:07:56.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.176+0000 7f9947367640 1 -- start start 2023-11-24T08:07:56.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.176+0000 7f9947367640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d0fa0 0x7f99400ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.176+0000 7f9947367640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99400d1d90 0x7f99400ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.176+0000 7f9947367640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f994015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9947367640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400cbde0 con 0x7f994015d3b0 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9947367640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400cc920 con 0x7f99400d1d90 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9947367640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99400ca6b0 con 0x7f99400d0fa0 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946b66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f994015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946b66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f994015f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49110/0 (socket says 172.21.15.64:49110) 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946b66640 1 -- 172.21.15.64:0/4202304370 learned_addr learned my addr 172.21.15.64:0/4202304370 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946365640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d0fa0 0x7f99400ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9945b64640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99400d1d90 0x7f99400ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946b66640 1 -- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d0fa0 msgr2=0x7f99400ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946b66640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d0fa0 0x7f99400ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946b66640 1 -- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99400d1d90 msgr2=0x7f99400ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946b66640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99400d1d90 0x7f99400ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.177+0000 7f9946b66640 1 -- 172.21.15.64:0/4202304370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c009b30 con 0x7f994015d3b0 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.178+0000 7f9946b66640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f994015f710 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f993800c7c0 tx=0x7f993800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.178+0000 7f9936ffd640 1 -- 172.21.15.64:0/4202304370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938004220 con 0x7f994015d3b0 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.178+0000 7f9936ffd640 1 -- 172.21.15.64:0/4202304370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9938004380 con 0x7f994015d3b0 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.178+0000 7f9936ffd640 1 -- 172.21.15.64:0/4202304370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938007840 con 0x7f994015d3b0 2023-11-24T08:07:56.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.178+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99400cbff0 con 0x7f994015d3b0 2023-11-24T08:07:56.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.178+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99400cc5b0 con 0x7f994015d3b0 2023-11-24T08:07:56.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.179+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99400cca90 con 0x7f994015d3b0 2023-11-24T08:07:56.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.180+0000 7f9936ffd640 1 -- 172.21.15.64:0/4202304370 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f99380079a0 con 0x7f994015d3b0 2023-11-24T08:07:56.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.181+0000 7f9936ffd640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992406b260 0x7f992406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.181+0000 7f9936ffd640 1 -- 172.21.15.64:0/4202304370 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9938089bd0 con 0x7f994015d3b0 2023-11-24T08:07:56.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.182+0000 7f9946365640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992406b260 0x7f992406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.183+0000 7f9946365640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992406b260 0x7f992406d720 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f993c035000 tx=0x7f993c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:56.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.186+0000 7f9936ffd640 1 -- 172.21.15.64:0/4202304370 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9938054650 con 0x7f994015d3b0 2023-11-24T08:07:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.407+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol652", "group_name": "subvol_grp5787", "target": ["mon-mgr", ""]}) v1 -- 0x7f99400d0bd0 con 0x7f992406b260 2023-11-24T08:07:56.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.464+0000 7f9936ffd640 1 -- 172.21.15.64:0/4202304370 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f99400d0bd0 con 0x7f992406b260 2023-11-24T08:07:56.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.467+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992406b260 msgr2=0x7f992406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.467+0000 7f9947367640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992406b260 0x7f992406d720 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f993c035000 tx=0x7f993c030000 comp rx=0 tx=0).stop 2023-11-24T08:07:56.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.467+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 msgr2=0x7f994015f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.467+0000 7f9947367640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f994015f710 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f993800c7c0 tx=0x7f993800cc80 comp rx=0 tx=0).stop 2023-11-24T08:07:56.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.468+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 shutdown_connections 2023-11-24T08:07:56.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.468+0000 7f9947367640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992406b260 0x7f992406d720 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.468+0000 7f9947367640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f994015d3b0 0x7f994015f710 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.468+0000 7f9947367640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f99400d1d90 0x7f99400ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.468+0000 7f9947367640 1 --2- 172.21.15.64:0/4202304370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99400d0fa0 0x7f99400ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.468+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 >> 172.21.15.64:0/4202304370 conn(0x7f99400d30a0 msgr2=0x7f99400b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:56.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.468+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 shutdown_connections 2023-11-24T08:07:56.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.468+0000 7f9947367640 1 -- 172.21.15.64:0/4202304370 wait complete. 2023-11-24T08:07:56.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:56.488 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp5787/.snap/ancestral_snap_1) 2023-11-24T08:07:56.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:56.608 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%i ./volumes/subvol_grp5787) 2023-11-24T08:07:56.663 INFO:teuthology.orchestra.run.smithi064.stdout:1099511627777 2023-11-24T08:07:56.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:56.664 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls ./volumes/subvol_grp5787/subvol652/.snap/_ancestral_snap_1_1099511627777) 2023-11-24T08:07:56.729 INFO:teuthology.orchestra.run.smithi064.stdout:fea576cb-1489-4a42-ad50-b9c03eafb9ef 2023-11-24T08:07:56.731 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol652 _ancestral_snap_1_1099511627777 subvol_grp5787 2023-11-24T08:07:56.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.965+0000 7fb23a2cf640 1 Processor -- start 2023-11-24T08:07:56.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.966+0000 7fb23a2cf640 1 -- start start 2023-11-24T08:07:56.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.967+0000 7fb23a2cf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb23415b350 0x7fb2340abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.967+0000 7fb23a2cf640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb23409cb30 0x7fb2340b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.967+0000 7fb23a2cf640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23415b730 con 0x7fb2340adb60 2023-11-24T08:07:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.967+0000 7fb23a2cf640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2340ac740 con 0x7fb23415b350 2023-11-24T08:07:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb23a2cf640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2340ac460 con 0x7fb23409cb30 2023-11-24T08:07:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb233fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb23415b350 0x7fb2340abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb23886c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb2340adb60 0x7fb2340adf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37864/0 (socket says 172.21.15.64:37864) 2023-11-24T08:07:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb23886c640 1 -- 172.21.15.64:0/1682629292 learned_addr learned my addr 172.21.15.64:0/1682629292 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb22b7fe640 1 --2- 172.21.15.64:0/1682629292 >> v2:172.21.15.91:3300/0 conn(0x7fb23409cb30 0x7fb2340b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb233fff640 1 -- 172.21.15.64:0/1682629292 >> v2:172.21.15.91:3300/0 conn(0x7fb23409cb30 msgr2=0x7fb2340b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb233fff640 1 --2- 172.21.15.64:0/1682629292 >> v2:172.21.15.91:3300/0 conn(0x7fb23409cb30 0x7fb2340b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb233fff640 1 -- 172.21.15.64:0/1682629292 >> v1:172.21.15.64:6789/0 conn(0x7fb2340adb60 legacy=0x7fb2340adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb233fff640 1 -- 172.21.15.64:0/1682629292 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2340b2a50 con 0x7fb23415b350 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb2337fe640 1 -- 172.21.15.64:0/1682629292 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629857114 0 0) 0x7fb22c001490 con 0x7fb2340adb60 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.968+0000 7fb22b7fe640 1 --2- 172.21.15.64:0/1682629292 >> v2:172.21.15.91:3300/0 conn(0x7fb23409cb30 0x7fb2340b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.969+0000 7fb233fff640 1 --2- 172.21.15.64:0/1682629292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb23415b350 0x7fb2340abe80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb2340d0060 tx=0x7fb22402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cf6db69f1cdfe2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.970+0000 7fb2337fe640 1 -- 172.21.15.64:0/1682629292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224031830 con 0x7fb23415b350 2023-11-24T08:07:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.970+0000 7fb2337fe640 1 -- 172.21.15.64:0/1682629292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb224031990 con 0x7fb23415b350 2023-11-24T08:07:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.970+0000 7fb2337fe640 1 -- 172.21.15.64:0/1682629292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224031d20 con 0x7fb23415b350 2023-11-24T08:07:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.970+0000 7fb23a2cf640 1 -- 172.21.15.64:0/1682629292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb23415b350 msgr2=0x7fb2340abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.970+0000 7fb23a2cf640 1 --2- 172.21.15.64:0/1682629292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb23415b350 0x7fb2340abe80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb2340d0060 tx=0x7fb22402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.970+0000 7fb23a2cf640 1 -- 172.21.15.64:0/1682629292 shutdown_connections 2023-11-24T08:07:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.970+0000 7fb23a2cf640 1 --2- 172.21.15.64:0/1682629292 >> v2:172.21.15.91:3300/0 conn(0x7fb23409cb30 0x7fb2340b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.970+0000 7fb23a2cf640 1 --2- 172.21.15.64:0/1682629292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb23415b350 0x7fb2340abe80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.971+0000 7fb23a2cf640 1 -- 172.21.15.64:0/1682629292 >> 172.21.15.64:0/1682629292 conn(0x7fb2340b0ba0 msgr2=0x7fb2340b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:56.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.971+0000 7fb23a2cf640 1 -- 172.21.15.64:0/1682629292 shutdown_connections 2023-11-24T08:07:56.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.971+0000 7fb23a2cf640 1 -- 172.21.15.64:0/1682629292 wait complete. 2023-11-24T08:07:56.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.972+0000 7fb23a2cf640 1 Processor -- start 2023-11-24T08:07:56.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.973+0000 7fb23a2cf640 1 -- start start 2023-11-24T08:07:56.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.973+0000 7fb23a2cf640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb23409cb30 0x7fb2341e2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.973+0000 7fb23a2cf640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2340adb60 0x7fb234141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.973+0000 7fb23a2cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb23415b350 0x7fb2341e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.973+0000 7fb23a2cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2341e10a0 con 0x7fb23415b350 2023-11-24T08:07:56.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.973+0000 7fb23a2cf640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2341e1380 con 0x7fb23409cb30 2023-11-24T08:07:56.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.973+0000 7fb23a2cf640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2341e1660 con 0x7fb2340adb60 2023-11-24T08:07:56.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb23886c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb23415b350 0x7fb2341e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb23886c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb23415b350 0x7fb2341e05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49138/0 (socket says 172.21.15.64:49138) 2023-11-24T08:07:56.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb23886c640 1 -- 172.21.15.64:0/3732296818 learned_addr learned my addr 172.21.15.64:0/3732296818 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:56.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb233fff640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb23409cb30 0x7fb2341e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb22b7fe640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2340adb60 0x7fb234141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb23886c640 1 -- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2340adb60 msgr2=0x7fb234141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb23886c640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2340adb60 0x7fb234141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb23886c640 1 -- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb23409cb30 msgr2=0x7fb2341e2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:56.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb23886c640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb23409cb30 0x7fb2341e2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:56.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb23886c640 1 -- 172.21.15.64:0/3732296818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb224009400 con 0x7fb23415b350 2023-11-24T08:07:56.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.974+0000 7fb22b7fe640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb2340adb60 0x7fb234141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:56.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.975+0000 7fb23886c640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb23415b350 0x7fb2341e05e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c7f0 tx=0x7fb22c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:56.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.975+0000 7fb2317fa640 1 -- 172.21.15.64:0/3732296818 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00a5e0 con 0x7fb23415b350 2023-11-24T08:07:56.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.975+0000 7fb2317fa640 1 -- 172.21.15.64:0/3732296818 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c00a760 con 0x7fb23415b350 2023-11-24T08:07:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.975+0000 7fb23a2cf640 1 -- 172.21.15.64:0/3732296818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2341e1940 con 0x7fb23415b350 2023-11-24T08:07:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.975+0000 7fb2317fa640 1 -- 172.21.15.64:0/3732296818 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00ab50 con 0x7fb23415b350 2023-11-24T08:07:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.975+0000 7fb23a2cf640 1 -- 172.21.15.64:0/3732296818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23414eb20 con 0x7fb23415b350 2023-11-24T08:07:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.976+0000 7fb23a2cf640 1 -- 172.21.15.64:0/3732296818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2341e1e60 con 0x7fb23415b350 2023-11-24T08:07:56.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.977+0000 7fb2317fa640 1 -- 172.21.15.64:0/3732296818 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb22c00acb0 con 0x7fb23415b350 2023-11-24T08:07:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.978+0000 7fb2317fa640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb21406b230 0x7fb21406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.978+0000 7fb2317fa640 1 -- 172.21.15.64:0/3732296818 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb22c0891a0 con 0x7fb23415b350 2023-11-24T08:07:56.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.979+0000 7fb233fff640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb21406b230 0x7fb21406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:56.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.980+0000 7fb233fff640 1 --2- 172.21.15.64:0/3732296818 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb21406b230 0x7fb21406d6f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb224036000 tx=0x7fb224008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:56.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:56.983+0000 7fb2317fa640 1 -- 172.21.15.64:0/3732296818 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2341e1e60 con 0x7fb23415b350 2023-11-24T08:07:57.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.037+0000 7fb2317fa640 1 -- 172.21.15.64:0/3732296818 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7fb22c054000 con 0x7fb23415b350 2023-11-24T08:07:57.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.213+0000 7fb23a2cf640 1 -- 172.21.15.64:0/3732296818 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol652", "snap_name": "_ancestral_snap_1_1099511627777", "group_name": "subvol_grp5787", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2340b3a70 con 0x7fb21406b230 2023-11-24T08:07:57.226 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.224+0000 7fe78a34d640 -1 mgr.server reply reply (22) Invalid argument snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T08:07:57.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.225+0000 7fb2317fa640 1 -- 172.21.15.64:0/3732296818 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: -22 snapshot name '_ancestral_snap_1_1099511627777' is invalid) v1 ==== 66+0+0 (secure 0 0 0) 0x7fb224031c30 con 0x7fb21406b230 2023-11-24T08:07:57.227 INFO:teuthology.orchestra.run.smithi064.stderr:Error EINVAL: snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T08:07:57.240 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-24T08:07:57.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:57.241 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/subvol_grp5787/.snap/ancestral_snap_1) 2023-11-24T08:07:57.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol652 --group_name subvol_grp5787 2023-11-24T08:07:57.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.580+0000 7f2396a7e640 1 Processor -- start 2023-11-24T08:07:57.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.580+0000 7f2396a7e640 1 -- start start 2023-11-24T08:07:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.581+0000 7f2396a7e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f239015ae80 0x7f2390070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.581+0000 7f2396a7e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2390071310 0x7f23900b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.581+0000 7f2396a7e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239015a850 con 0x7f23900b3280 2023-11-24T08:07:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.581+0000 7f2396a7e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23900b4330 con 0x7f239015ae80 2023-11-24T08:07:57.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.581+0000 7f2396a7e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23900b3a90 con 0x7f2390071310 2023-11-24T08:07:57.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.582+0000 7f239627d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f23900b3280 0x7f23900cb480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58568/0 (socket says 172.21.15.64:58568) 2023-11-24T08:07:57.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.582+0000 7f239627d640 1 -- 172.21.15.64:0/3228267678 learned_addr learned my addr 172.21.15.64:0/3228267678 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.582+0000 7f2395a7c640 1 --2- 172.21.15.64:0/3228267678 >> v2:172.21.15.91:3300/0 conn(0x7f239015ae80 0x7f2390070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.582+0000 7f239527b640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2390071310 0x7f23900b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.582+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376968379 0 0) 0x7f239015a850 con 0x7f23900b3280 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.582+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380003520 con 0x7f23900b3280 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610672091 0 0) 0x7f2380003520 con 0x7f23900b3280 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2390071310 msgr2=0x7f23900b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2390071310 0x7f23900b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 >> v2:172.21.15.91:3300/0 conn(0x7f239015ae80 msgr2=0x7f2390070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 --2- 172.21.15.64:0/3228267678 >> v2:172.21.15.91:3300/0 conn(0x7f239015ae80 0x7f2390070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390071770 con 0x7f23900b3280 2023-11-24T08:07:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2388001670 con 0x7f23900b3280 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 >> v1:172.21.15.64:6789/0 conn(0x7f23900b3280 legacy=0x7f23900cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2380003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23800047c0 0x7f2380004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2390071310 0x7f2380005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23900b4330 con 0x7f23800039e0 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23900b3a90 con 0x7f23800047c0 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238000db90 con 0x7f2390071310 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2395a7c640 1 --2- 172.21.15.64:0/3228267678 >> v2:172.21.15.91:3300/0 conn(0x7f239015ae80 0x7f2390070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.583+0000 7f2395a7c640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2380003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.584+0000 7f239527b640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23800047c0 0x7f2380004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.584+0000 7f239627d640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2390071310 0x7f2380005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.584+0000 7f2395a7c640 1 -- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2390071310 msgr2=0x7f2380005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:57.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.584+0000 7f2395a7c640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2390071310 0x7f2380005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.584+0000 7f2395a7c640 1 -- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23800047c0 msgr2=0x7f2380004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:57.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.584+0000 7f2395a7c640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23800047c0 0x7f2380004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.584+0000 7f2395a7c640 1 -- 172.21.15.64:0/3228267678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380003520 con 0x7f23800039e0 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.585+0000 7f2395a7c640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2380003e00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f238400e380 tx=0x7f23800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957e47e3d54cd8cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.585+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238402cde0 con 0x7f23800039e0 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.585+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238400f040 con 0x7f23800039e0 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.585+0000 7f2394a7a640 1 -- 172.21.15.64:0/3228267678 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384003b90 con 0x7f23800039e0 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.586+0000 7f2396a7e640 1 -- 172.21.15.64:0/3228267678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 msgr2=0x7f2380003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.586+0000 7f2396a7e640 1 --2- 172.21.15.64:0/3228267678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2380003e00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f238400e380 tx=0x7f23800057f0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.586+0000 7f239627d640 1 -- 172.21.15.64:0/3228267678 reap_dead start 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.586+0000 7f2396a7e640 1 -- 172.21.15.64:0/3228267678 shutdown_connections 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.586+0000 7f2396a7e640 1 -- 172.21.15.64:0/3228267678 >> 172.21.15.64:0/3228267678 conn(0x7f23900b15c0 msgr2=0x7f23900b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.586+0000 7f2396a7e640 1 -- 172.21.15.64:0/3228267678 shutdown_connections 2023-11-24T08:07:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.586+0000 7f2396a7e640 1 -- 172.21.15.64:0/3228267678 wait complete. 2023-11-24T08:07:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.588+0000 7f2396a7e640 1 Processor -- start 2023-11-24T08:07:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.588+0000 7f2396a7e640 1 -- start start 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2396a7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2390159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2396a7e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f23901582e0 0x7f239013ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2396a7e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23901e4a90 0x7f23900ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2396a7e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2390159dc0 con 0x7f23800039e0 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2396a7e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23901587e0 con 0x7f23901e4a90 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2396a7e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2390158ac0 con 0x7f23901582e0 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2395a7c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2390159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2395a7c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2390159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41742/0 (socket says 172.21.15.64:41742) 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.589+0000 7f2395a7c640 1 -- 172.21.15.64:0/1359189541 learned_addr learned my addr 172.21.15.64:0/1359189541 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f239527b640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f23901582e0 0x7f239013ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f239627d640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23901e4a90 0x7f23900ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f2395a7c640 1 -- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f23901582e0 msgr2=0x7f239013ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:57.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f2395a7c640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f23901582e0 0x7f239013ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f2395a7c640 1 -- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23901e4a90 msgr2=0x7f23900ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:57.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f2395a7c640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23901e4a90 0x7f23900ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f2395a7c640 1 -- 172.21.15.64:0/1359189541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390158da0 con 0x7f23800039e0 2023-11-24T08:07:57.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f2395a7c640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2390159880 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f238402c780 tx=0x7f2384009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:57.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f238e7fc640 1 -- 172.21.15.64:0/1359189541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384034e40 con 0x7f23800039e0 2023-11-24T08:07:57.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f238e7fc640 1 -- 172.21.15.64:0/1359189541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238402cbe0 con 0x7f23800039e0 2023-11-24T08:07:57.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.590+0000 7f238e7fc640 1 -- 172.21.15.64:0/1359189541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384003990 con 0x7f23800039e0 2023-11-24T08:07:57.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.591+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2390159080 con 0x7f23800039e0 2023-11-24T08:07:57.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.591+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239013f160 con 0x7f23800039e0 2023-11-24T08:07:57.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.592+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23901595a0 con 0x7f23800039e0 2023-11-24T08:07:57.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.593+0000 7f238e7fc640 1 -- 172.21.15.64:0/1359189541 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f2384040470 con 0x7f23800039e0 2023-11-24T08:07:57.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.593+0000 7f238e7fc640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f237406b2c0 0x7f237406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:57.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.594+0000 7f238e7fc640 1 -- 172.21.15.64:0/1359189541 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23840acae0 con 0x7f23800039e0 2023-11-24T08:07:57.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.594+0000 7f239527b640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f237406b2c0 0x7f237406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:57.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.595+0000 7f239527b640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f237406b2c0 0x7f237406d780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f23901574b0 tx=0x7f237c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:57.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.599+0000 7f238e7fc640 1 -- 172.21.15.64:0/1359189541 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23901595a0 con 0x7f23800039e0 2023-11-24T08:07:57.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.825+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol652", "group_name": "subvol_grp5787", "target": ["mon-mgr", ""]}) v1 -- 0x7f239006cb70 con 0x7f237406b2c0 2023-11-24T08:07:57.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.857+0000 7f238e7fc640 1 -- 172.21.15.64:0/1359189541 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f237c0095c0 con 0x7f237406b2c0 2023-11-24T08:07:57.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f237406b2c0 msgr2=0x7f237406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:57.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f237406b2c0 0x7f237406d780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f23901574b0 tx=0x7f237c008040 comp rx=0 tx=0).stop 2023-11-24T08:07:57.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 msgr2=0x7f2390159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:57.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2390159880 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f238402c780 tx=0x7f2384009750 comp rx=0 tx=0).stop 2023-11-24T08:07:57.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 shutdown_connections 2023-11-24T08:07:57.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f237406b2c0 0x7f237406d780 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f23901e4a90 0x7f23900ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f23901582e0 0x7f239013ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 --2- 172.21.15.64:0/1359189541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23800039e0 0x7f2390159880 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:57.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 >> 172.21.15.64:0/1359189541 conn(0x7f23900b15c0 msgr2=0x7f23900b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:57.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 shutdown_connections 2023-11-24T08:07:57.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:57.860+0000 7f2396a7e640 1 -- 172.21.15.64:0/1359189541 wait complete. 2023-11-24T08:07:57.871 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.869+0000 7fe78cb52640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.871 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.869+0000 7fe78cb52640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.871 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.869+0000 7fe78cb52640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.872 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.869+0000 7fe78cb52640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.872 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.869+0000 7fe78cb52640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:07:57.882 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T08:07:57.897 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.895+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.897 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.895+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.897 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.895+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.897 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.895+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.897 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:07:57.895+0000 7fe78bb50640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:07:57.941 INFO:teuthology.orchestra.run.smithi064.stdout:2 2023-11-24T08:07:57.942 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp5787 2023-11-24T08:07:58.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.157+0000 7f7a6eb73640 1 Processor -- start 2023-11-24T08:07:58.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.157+0000 7f7a6eb73640 1 -- start start 2023-11-24T08:07:58.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.159+0000 7f7a6eb73640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a6815efa0 0x7f7a68161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.159+0000 7f7a6eb73640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7a680b3080 0x7f7a6806c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.159+0000 7f7a6eb73640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6806c9a0 con 0x7f7a6806da40 2023-11-24T08:07:58.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.159+0000 7f7a6eb73640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6806cc80 con 0x7f7a6815efa0 2023-11-24T08:07:58.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.159+0000 7f7a6eb73640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6806cf60 con 0x7f7a680b3080 2023-11-24T08:07:58.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.159+0000 7f7a6db71640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a6815efa0 0x7f7a68161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.160+0000 7f7a6db71640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a6815efa0 0x7f7a68161ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41752/0 (socket says 172.21.15.64:41752) 2023-11-24T08:07:58.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.160+0000 7f7a6db71640 1 -- 172.21.15.64:0/1195476800 learned_addr learned my addr 172.21.15.64:0/1195476800 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:58.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.160+0000 7f7a6e372640 1 --2- 172.21.15.64:0/1195476800 >> v2:172.21.15.91:3300/0 conn(0x7f7a680b3080 0x7f7a6806c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.160+0000 7f7a6db71640 1 -- 172.21.15.64:0/1195476800 >> v2:172.21.15.91:3300/0 conn(0x7f7a680b3080 msgr2=0x7f7a6806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:58.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.160+0000 7f7a6db71640 1 --2- 172.21.15.64:0/1195476800 >> v2:172.21.15.91:3300/0 conn(0x7f7a680b3080 0x7f7a6806c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.160+0000 7f7a6db71640 1 -- 172.21.15.64:0/1195476800 >> v1:172.21.15.91:6789/0 conn(0x7f7a6806da40 legacy=0x7f7a680b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:58.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.160+0000 7f7a6db71640 1 -- 172.21.15.64:0/1195476800 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6806d240 con 0x7f7a6815efa0 2023-11-24T08:07:58.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.161+0000 7f7a6db71640 1 --2- 172.21.15.64:0/1195476800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a6815efa0 0x7f7a68161ac0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b4390 tx=0x7f7a6402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73208bf2ef078b94 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:58.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.161+0000 7f7a6cb6f640 1 -- 172.21.15.64:0/1195476800 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a640317e0 con 0x7f7a6815efa0 2023-11-24T08:07:58.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.161+0000 7f7a6cb6f640 1 -- 172.21.15.64:0/1195476800 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64031940 con 0x7f7a6815efa0 2023-11-24T08:07:58.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.161+0000 7f7a6cb6f640 1 -- 172.21.15.64:0/1195476800 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64031cd0 con 0x7f7a6815efa0 2023-11-24T08:07:58.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.162+0000 7f7a6eb73640 1 -- 172.21.15.64:0/1195476800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a6815efa0 msgr2=0x7f7a68161ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:58.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.162+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/1195476800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a6815efa0 0x7f7a68161ac0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b4390 tx=0x7f7a6402b880 comp rx=0 tx=0).stop 2023-11-24T08:07:58.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.162+0000 7f7a6eb73640 1 -- 172.21.15.64:0/1195476800 shutdown_connections 2023-11-24T08:07:58.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.162+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/1195476800 >> v2:172.21.15.91:3300/0 conn(0x7f7a680b3080 0x7f7a6806c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.162+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/1195476800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a6815efa0 0x7f7a68161ac0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.162+0000 7f7a6eb73640 1 -- 172.21.15.64:0/1195476800 >> 172.21.15.64:0/1195476800 conn(0x7f7a680cec10 msgr2=0x7f7a680cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:58.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.162+0000 7f7a6eb73640 1 -- 172.21.15.64:0/1195476800 shutdown_connections 2023-11-24T08:07:58.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.162+0000 7f7a6eb73640 1 -- 172.21.15.64:0/1195476800 wait complete. 2023-11-24T08:07:58.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.164+0000 7f7a6eb73640 1 Processor -- start 2023-11-24T08:07:58.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.164+0000 7f7a6eb73640 1 -- start start 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.165+0000 7f7a6eb73640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a6806da40 0x7f7a68152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.165+0000 7f7a6eb73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680b3080 0x7f7a680af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.165+0000 7f7a6eb73640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a6815efa0 0x7f7a68154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.165+0000 7f7a6eb73640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68164ae0 con 0x7f7a680b3080 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.165+0000 7f7a6eb73640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680aee50 con 0x7f7a6815efa0 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.165+0000 7f7a6eb73640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681f7d40 con 0x7f7a6806da40 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6d370640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680b3080 0x7f7a680af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6d370640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680b3080 0x7f7a680af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41756/0 (socket says 172.21.15.64:41756) 2023-11-24T08:07:58.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6d370640 1 -- 172.21.15.64:0/2805918175 learned_addr learned my addr 172.21.15.64:0/2805918175 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:58.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6e372640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a6815efa0 0x7f7a68154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6db71640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a6806da40 0x7f7a68152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6d370640 1 -- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a6806da40 msgr2=0x7f7a68152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:58.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6d370640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a6806da40 0x7f7a68152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6d370640 1 -- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a6815efa0 msgr2=0x7f7a68154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:58.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6d370640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a6815efa0 0x7f7a68154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.166+0000 7f7a6d370640 1 -- 172.21.15.64:0/2805918175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a640094d0 con 0x7f7a680b3080 2023-11-24T08:07:58.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.167+0000 7f7a6d370640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680b3080 0x7f7a680af2e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c3c0 tx=0x7f7a5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:58.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.167+0000 7f7a5e7fc640 1 -- 172.21.15.64:0/2805918175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a580035f0 con 0x7f7a680b3080 2023-11-24T08:07:58.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.167+0000 7f7a5e7fc640 1 -- 172.21.15.64:0/2805918175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a58003750 con 0x7f7a680b3080 2023-11-24T08:07:58.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.167+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a681f8020 con 0x7f7a680b3080 2023-11-24T08:07:58.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.167+0000 7f7a5e7fc640 1 -- 172.21.15.64:0/2805918175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58003b00 con 0x7f7a680b3080 2023-11-24T08:07:58.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.167+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a680aeae0 con 0x7f7a680b3080 2023-11-24T08:07:58.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.168+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a680b4d20 con 0x7f7a680b3080 2023-11-24T08:07:58.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.169+0000 7f7a5e7fc640 1 -- 172.21.15.64:0/2805918175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f7a58003c60 con 0x7f7a680b3080 2023-11-24T08:07:58.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.170+0000 7f7a5e7fc640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7a4c06b2c0 0x7f7a4c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.170+0000 7f7a5e7fc640 1 -- 172.21.15.64:0/2805918175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a58089290 con 0x7f7a680b3080 2023-11-24T08:07:58.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.171+0000 7f7a6db71640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7a4c06b2c0 0x7f7a4c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.172+0000 7f7a6db71640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7a4c06b2c0 0x7f7a4c06d780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7a64036000 tx=0x7f7a6403d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.175+0000 7f7a5e7fc640 1 -- 172.21.15.64:0/2805918175 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a58053d30 con 0x7f7a680b3080 2023-11-24T08:07:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.400+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp5787", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a6804e840 con 0x7f7a4c06b2c0 2023-11-24T08:07:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.412+0000 7f7a5e7fc640 1 -- 172.21.15.64:0/2805918175 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7a6804e840 con 0x7f7a4c06b2c0 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7a4c06b2c0 msgr2=0x7f7a4c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7a4c06b2c0 0x7f7a4c06d780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7a64036000 tx=0x7f7a6403d040 comp rx=0 tx=0).stop 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680b3080 msgr2=0x7f7a680af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680b3080 0x7f7a680af2e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c3c0 tx=0x7f7a5800c880 comp rx=0 tx=0).stop 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 shutdown_connections 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7a4c06b2c0 0x7f7a4c06d780 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7a6815efa0 0x7f7a68154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680b3080 0x7f7a680af2e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 --2- 172.21.15.64:0/2805918175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7a6806da40 0x7f7a68152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 >> 172.21.15.64:0/2805918175 conn(0x7f7a680cec10 msgr2=0x7f7a681617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:58.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 shutdown_connections 2023-11-24T08:07:58.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.415+0000 7f7a6eb73640 1 -- 172.21.15.64:0/2805918175 wait complete. 2023-11-24T08:07:58.439 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:07:58.439 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:07:58.559 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:07:58.677 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:07:58.677 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:07:58.677 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:07:58.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:07:58.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.964+0000 7fb5960d4640 1 Processor -- start 2023-11-24T08:07:58.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.964+0000 7fb5960d4640 1 -- start start 2023-11-24T08:07:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.966+0000 7fb5960d4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb59015d520 0x7fb5900cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.966+0000 7fb5960d4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb5900ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.966+0000 7fb5960d4640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb5900b2f00 0x7fb5900cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.966+0000 7fb5960d4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb590161f40 con 0x7fb59015d520 2023-11-24T08:07:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.966+0000 7fb5960d4640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5901613c0 con 0x7fb5900b2f00 2023-11-24T08:07:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.966+0000 7fb5960d4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5901616a0 con 0x7fb5900b2aa0 2023-11-24T08:07:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb59015d520 0x7fb5900cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb59015d520 0x7fb5900cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41766/0 (socket says 172.21.15.64:41766) 2023-11-24T08:07:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58f7fe640 1 -- 172.21.15.64:0/3389355177 learned_addr learned my addr 172.21.15.64:0/3389355177 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58effd640 1 --2- 172.21.15.64:0/3389355177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb5900ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58ffff640 1 --2- 172.21.15.64:0/3389355177 >> v2:172.21.15.91:3300/0 conn(0x7fb5900b2f00 0x7fb5900cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58f7fe640 1 -- 172.21.15.64:0/3389355177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 msgr2=0x7fb5900ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58f7fe640 1 --2- 172.21.15.64:0/3389355177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb5900ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58f7fe640 1 -- 172.21.15.64:0/3389355177 >> v2:172.21.15.91:3300/0 conn(0x7fb5900b2f00 msgr2=0x7fb5900cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58f7fe640 1 --2- 172.21.15.64:0/3389355177 >> v2:172.21.15.91:3300/0 conn(0x7fb5900b2f00 0x7fb5900cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58f7fe640 1 -- 172.21.15.64:0/3389355177 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb590161980 con 0x7fb59015d520 2023-11-24T08:07:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.967+0000 7fb58ffff640 1 --2- 172.21.15.64:0/3389355177 >> v2:172.21.15.91:3300/0 conn(0x7fb5900b2f00 0x7fb5900cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:07:58.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.968+0000 7fb58f7fe640 1 --2- 172.21.15.64:0/3389355177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59015d520 0x7fb5900cdec0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb59006bd50 tx=0x7fb58002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee28689e853b1fbe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:58.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb58e7fc640 1 -- 172.21.15.64:0/3389355177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580031870 con 0x7fb59015d520 2023-11-24T08:07:58.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb58e7fc640 1 -- 172.21.15.64:0/3389355177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5800319d0 con 0x7fb59015d520 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb58e7fc640 1 -- 172.21.15.64:0/3389355177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580031d60 con 0x7fb59015d520 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb5960d4640 1 -- 172.21.15.64:0/3389355177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59015d520 msgr2=0x7fb5900cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3389355177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59015d520 0x7fb5900cdec0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb59006bd50 tx=0x7fb58002b910 comp rx=0 tx=0).stop 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb5960d4640 1 -- 172.21.15.64:0/3389355177 shutdown_connections 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3389355177 >> v2:172.21.15.91:3300/0 conn(0x7fb5900b2f00 0x7fb5900cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3389355177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb5900ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3389355177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59015d520 0x7fb5900cdec0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.969+0000 7fb5960d4640 1 -- 172.21.15.64:0/3389355177 >> 172.21.15.64:0/3389355177 conn(0x7fb5900b23a0 msgr2=0x7fb5900cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.970+0000 7fb5960d4640 1 -- 172.21.15.64:0/3389355177 shutdown_connections 2023-11-24T08:07:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.970+0000 7fb5960d4640 1 -- 172.21.15.64:0/3389355177 wait complete. 2023-11-24T08:07:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.971+0000 7fb5960d4640 1 Processor -- start 2023-11-24T08:07:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.971+0000 7fb5960d4640 1 -- start start 2023-11-24T08:07:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.972+0000 7fb5960d4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb59015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.972+0000 7fb5960d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5900b2f00 0x7fb5900b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.972+0000 7fb5960d4640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb59015d520 0x7fb5900b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.972+0000 7fb5960d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590144ba0 con 0x7fb5900b2f00 2023-11-24T08:07:58.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.972+0000 7fb5960d4640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590143cd0 con 0x7fb59015d520 2023-11-24T08:07:58.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.972+0000 7fb5960d4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900ae620 con 0x7fb5900b2aa0 2023-11-24T08:07:58.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.973+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb59015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.973+0000 7fb58effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5900b2f00 0x7fb5900b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.973+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb59015e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:46270/0 (socket says 172.21.15.64:46270) 2023-11-24T08:07:58.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.973+0000 7fb58f7fe640 1 -- 172.21.15.64:0/3233350806 learned_addr learned my addr 172.21.15.64:0/3233350806 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.973+0000 7fb58ffff640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb59015d520 0x7fb5900b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb58effd640 1 -- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 msgr2=0x7fb59015e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb58effd640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb59015e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb58effd640 1 -- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb59015d520 msgr2=0x7fb5900b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb58effd640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb59015d520 0x7fb5900b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb58effd640 1 -- 172.21.15.64:0/3233350806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5800094d0 con 0x7fb5900b2f00 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb58ffff640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb59015d520 0x7fb5900b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb58effd640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5900b2f00 0x7fb5900b01f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d550 tx=0x7fb58400da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb577fff640 1 -- 172.21.15.64:0/3233350806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584014620 con 0x7fb5900b2f00 2023-11-24T08:07:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb577fff640 1 -- 172.21.15.64:0/3233350806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb584014780 con 0x7fb5900b2f00 2023-11-24T08:07:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb577fff640 1 -- 172.21.15.64:0/3233350806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584014b30 con 0x7fb5900b2f00 2023-11-24T08:07:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb590143f20 con 0x7fb5900b2f00 2023-11-24T08:07:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.974+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5901444e0 con 0x7fb5900b2f00 2023-11-24T08:07:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.976+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb590074770 con 0x7fb5900b2f00 2023-11-24T08:07:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.977+0000 7fb577fff640 1 -- 172.21.15.64:0/3233350806 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb584014c90 con 0x7fb5900b2f00 2023-11-24T08:07:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.977+0000 7fb577fff640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb57c06b410 0x7fb57c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.978+0000 7fb577fff640 1 -- 172.21.15.64:0/3233350806 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb58408a500 con 0x7fb5900b2f00 2023-11-24T08:07:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.978+0000 7fb58f7fe640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb57c06b410 0x7fb57c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.979+0000 7fb58f7fe640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb57c06b410 0x7fb57c06d8d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb5800090b0 tx=0x7fb58003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:58.982+0000 7fb577fff640 1 -- 172.21.15.64:0/3233350806 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb584054d50 con 0x7fb5900b2f00 2023-11-24T08:07:59.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.177+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5900cda10 con 0x7fb5900b2f00 2023-11-24T08:07:59.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.179+0000 7fb577fff640 1 -- 172.21.15.64:0/3233350806 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16520 (secure 0 0 0) 0x7fb584058370 con 0x7fb5900b2f00 2023-11-24T08:07:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:07:59.181 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":74,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:07:43.341796+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:07:39.474762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:07:40.829840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:07:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb57c06b410 msgr2=0x7fb57c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb57c06b410 0x7fb57c06d8d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb5800090b0 tx=0x7fb58003c040 comp rx=0 tx=0).stop 2023-11-24T08:07:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5900b2f00 msgr2=0x7fb5900b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5900b2f00 0x7fb5900b01f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d550 tx=0x7fb58400da10 comp rx=0 tx=0).stop 2023-11-24T08:07:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 shutdown_connections 2023-11-24T08:07:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb57c06b410 0x7fb57c06d8d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb59015d520 0x7fb5900b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5900b2f00 0x7fb5900b01f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 --2- 172.21.15.64:0/3233350806 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb5900b2aa0 0x7fb59015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 >> 172.21.15.64:0/3233350806 conn(0x7fb5900b23a0 msgr2=0x7fb5900b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 shutdown_connections 2023-11-24T08:07:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.182+0000 7fb5960d4640 1 -- 172.21.15.64:0/3233350806 wait complete. 2023-11-24T08:07:59.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:07:39.474762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:07:40.829840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:07:59.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:07:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.431+0000 7f7e7287c640 1 Processor -- start 2023-11-24T08:07:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.431+0000 7f7e7287c640 1 -- start start 2023-11-24T08:07:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.432+0000 7f7e7287c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7e6c15b350 0x7f7e6c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e7287c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7e6c0adb60 0x7f7e6c0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e7287c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c15b730 con 0x7f7e6c09cb30 2023-11-24T08:07:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e7287c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c0ac740 con 0x7f7e6c0adb60 2023-11-24T08:07:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e7287c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c0ac460 con 0x7f7e6c15b350 2023-11-24T08:07:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e70e19640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7e6c0adb60 0x7f7e6c0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e70e19640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7e6c0adb60 0x7f7e6c0adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41780/0 (socket says 172.21.15.64:41780) 2023-11-24T08:07:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e70e19640 1 -- 172.21.15.64:0/3268663468 learned_addr learned my addr 172.21.15.64:0/3268663468 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e6bfff640 1 --2- 172.21.15.64:0/3268663468 >> v2:172.21.15.91:3300/0 conn(0x7f7e6c15b350 0x7f7e6c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e70e19640 1 -- 172.21.15.64:0/3268663468 >> v2:172.21.15.91:3300/0 conn(0x7f7e6c15b350 msgr2=0x7f7e6c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e70e19640 1 --2- 172.21.15.64:0/3268663468 >> v2:172.21.15.91:3300/0 conn(0x7f7e6c15b350 0x7f7e6c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e70e19640 1 -- 172.21.15.64:0/3268663468 >> v1:172.21.15.91:6789/0 conn(0x7f7e6c09cb30 legacy=0x7f7e6c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.433+0000 7f7e70e19640 1 -- 172.21.15.64:0/3268663468 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0b2a50 con 0x7f7e6c0adb60 2023-11-24T08:07:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.435+0000 7f7e70e19640 1 --2- 172.21.15.64:0/3268663468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c0adf70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0b5100 tx=0x7f7e6402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef1e83c9bc67528 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.435+0000 7f7e6affd640 1 -- 172.21.15.64:0/3268663468 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64031870 con 0x7f7e6c0adb60 2023-11-24T08:07:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.435+0000 7f7e6affd640 1 -- 172.21.15.64:0/3268663468 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e640319d0 con 0x7f7e6c0adb60 2023-11-24T08:07:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.435+0000 7f7e6affd640 1 -- 172.21.15.64:0/3268663468 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64031d60 con 0x7f7e6c0adb60 2023-11-24T08:07:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.436+0000 7f7e7287c640 1 -- 172.21.15.64:0/3268663468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 msgr2=0x7f7e6c0adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.436+0000 7f7e7287c640 1 --2- 172.21.15.64:0/3268663468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c0adf70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0b5100 tx=0x7f7e6402b910 comp rx=0 tx=0).stop 2023-11-24T08:07:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.436+0000 7f7e7287c640 1 -- 172.21.15.64:0/3268663468 shutdown_connections 2023-11-24T08:07:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.436+0000 7f7e7287c640 1 --2- 172.21.15.64:0/3268663468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c0adf70 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.436+0000 7f7e7287c640 1 --2- 172.21.15.64:0/3268663468 >> v2:172.21.15.91:3300/0 conn(0x7f7e6c15b350 0x7f7e6c0abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.436+0000 7f7e7287c640 1 -- 172.21.15.64:0/3268663468 >> 172.21.15.64:0/3268663468 conn(0x7f7e6c0b0ba0 msgr2=0x7f7e6c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.436+0000 7f7e7287c640 1 -- 172.21.15.64:0/3268663468 shutdown_connections 2023-11-24T08:07:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.436+0000 7f7e7287c640 1 -- 172.21.15.64:0/3268663468 wait complete. 2023-11-24T08:07:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.438+0000 7f7e7287c640 1 Processor -- start 2023-11-24T08:07:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.438+0000 7f7e7287c640 1 -- start start 2023-11-24T08:07:59.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e7287c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e6c09cb30 0x7f7e6c09b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e7287c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e7287c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e6c15b350 0x7f7e6c13b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e7287c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c157650 con 0x7f7e6c0adb60 2023-11-24T08:07:59.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e7287c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c157930 con 0x7f7e6c09cb30 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e7287c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c157c10 con 0x7f7e6c15b350 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e6b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e6b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41784/0 (socket says 172.21.15.64:41784) 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e6b7fe640 1 -- 172.21.15.64:0/706997709 learned_addr learned my addr 172.21.15.64:0/706997709 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.439+0000 7f7e6bfff640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e6c09cb30 0x7f7e6c09b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.440+0000 7f7e70e19640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e6c15b350 0x7f7e6c13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.440+0000 7f7e6b7fe640 1 -- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e6c15b350 msgr2=0x7f7e6c13b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.440+0000 7f7e6b7fe640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e6c15b350 0x7f7e6c13b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.440+0000 7f7e6b7fe640 1 -- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e6c09cb30 msgr2=0x7f7e6c09b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.440+0000 7f7e6b7fe640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e6c09cb30 0x7f7e6c09b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.440+0000 7f7e6b7fe640 1 -- 172.21.15.64:0/706997709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e640094d0 con 0x7f7e6c0adb60 2023-11-24T08:07:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.440+0000 7f7e6b7fe640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c156de0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000c430 tx=0x7f7e6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.443+0000 7f7e68ff9640 1 -- 172.21.15.64:0/706997709 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60011790 con 0x7f7e6c0adb60 2023-11-24T08:07:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.443+0000 7f7e68ff9640 1 -- 172.21.15.64:0/706997709 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e600118f0 con 0x7f7e6c0adb60 2023-11-24T08:07:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.443+0000 7f7e68ff9640 1 -- 172.21.15.64:0/706997709 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60011ce0 con 0x7f7e6c0adb60 2023-11-24T08:07:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.443+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c1f33f0 con 0x7f7e6c0adb60 2023-11-24T08:07:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.443+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c1f30a0 con 0x7f7e6c0adb60 2023-11-24T08:07:59.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.444+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c09d200 con 0x7f7e6c0adb60 2023-11-24T08:07:59.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.445+0000 7f7e68ff9640 1 -- 172.21.15.64:0/706997709 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f7e6001c460 con 0x7f7e6c0adb60 2023-11-24T08:07:59.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.446+0000 7f7e68ff9640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e5806b3e0 0x7f7e5806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.446+0000 7f7e68ff9640 1 -- 172.21.15.64:0/706997709 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e6008a0d0 con 0x7f7e6c0adb60 2023-11-24T08:07:59.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.447+0000 7f7e6bfff640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e5806b3e0 0x7f7e5806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.448+0000 7f7e6bfff640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e5806b3e0 0x7f7e5806d8a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0099c0 tx=0x7f7e5c003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:59.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.450+0000 7f7e68ff9640 1 -- 172.21.15.64:0/706997709 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e600bce90 con 0x7f7e6c0adb60 2023-11-24T08:07:59.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.648+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e6c04e920 con 0x7f7e6c0adb60 2023-11-24T08:07:59.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.649+0000 7f7e68ff9640 1 -- 172.21.15.64:0/706997709 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e60054880 con 0x7f7e6c0adb60 2023-11-24T08:07:59.651 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:07:59.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e5806b3e0 msgr2=0x7f7e5806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e5806b3e0 0x7f7e5806d8a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0099c0 tx=0x7f7e5c003990 comp rx=0 tx=0).stop 2023-11-24T08:07:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 msgr2=0x7f7e6c156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c156de0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000c430 tx=0x7f7e6000c8f0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 shutdown_connections 2023-11-24T08:07:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f7e5806b3e0 0x7f7e5806d8a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7e6c15b350 0x7f7e6c13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c0adb60 0x7f7e6c156de0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 --2- 172.21.15.64:0/706997709 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7e6c09cb30 0x7f7e6c09b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 >> 172.21.15.64:0/706997709 conn(0x7f7e6c0b0ba0 msgr2=0x7f7e6c15a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:59.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 shutdown_connections 2023-11-24T08:07:59.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.652+0000 7f7e7287c640 1 -- 172.21.15.64:0/706997709 wait complete. 2023-11-24T08:07:59.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:07:59.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.892+0000 7f2b0d2d1640 1 Processor -- start 2023-11-24T08:07:59.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.892+0000 7f2b0d2d1640 1 -- start start 2023-11-24T08:07:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.894+0000 7f2b0d2d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2b080b3080 0x7f2b08161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.894+0000 7f2b0d2d1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0806c9a0 con 0x7f2b0815efa0 2023-11-24T08:07:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.894+0000 7f2b0d2d1640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0806cc80 con 0x7f2b0806da40 2023-11-24T08:07:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.894+0000 7f2b0d2d1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0806cf60 con 0x7f2b080b3080 2023-11-24T08:07:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.894+0000 7f2b07fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2b0815efa0 0x7f2b080b2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58574/0 (socket says 172.21.15.64:58574) 2023-11-24T08:07:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.894+0000 7f2b07fff640 1 -- 172.21.15.64:0/2182694274 learned_addr learned my addr 172.21.15.64:0/2182694274 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:07:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.894+0000 7f2b077fe640 1 --2- 172.21.15.64:0/2182694274 >> v2:172.21.15.64:3300/0 conn(0x7f2b080b3080 0x7f2b08161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.895+0000 7f2b077fe640 1 -- 172.21.15.64:0/2182694274 >> v1:172.21.15.91:6789/0 conn(0x7f2b0806da40 legacy=0x7f2b08070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:07:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.895+0000 7f2b077fe640 1 -- 172.21.15.64:0/2182694274 >> v1:172.21.15.64:6789/0 conn(0x7f2b0815efa0 legacy=0x7f2b080b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.895+0000 7f2b06ffd640 1 -- 172.21.15.64:0/2182694274 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645715449 0 0) 0x7f2b0806c9a0 con 0x7f2b0815efa0 2023-11-24T08:07:59.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.895+0000 7f2b077fe640 1 -- 172.21.15.64:0/2182694274 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0806d240 con 0x7f2b080b3080 2023-11-24T08:07:59.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.896+0000 7f2b077fe640 1 --2- 172.21.15.64:0/2182694274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b080b3080 0x7f2b08161ac0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0097d0 tx=0x7f2afc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ea165f151fdb39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:59.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.896+0000 7f2b06ffd640 1 -- 172.21.15.64:0/2182694274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc02e070 con 0x7f2b080b3080 2023-11-24T08:07:59.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.896+0000 7f2b06ffd640 1 -- 172.21.15.64:0/2182694274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc0315a0 con 0x7f2b080b3080 2023-11-24T08:07:59.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.896+0000 7f2b06ffd640 1 -- 172.21.15.64:0/2182694274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc031930 con 0x7f2b080b3080 2023-11-24T08:07:59.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.897+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/2182694274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b080b3080 msgr2=0x7f2b08161ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.897+0000 7f2b0d2d1640 1 --2- 172.21.15.64:0/2182694274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b080b3080 0x7f2b08161ac0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0097d0 tx=0x7f2afc02b840 comp rx=0 tx=0).stop 2023-11-24T08:07:59.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.897+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/2182694274 shutdown_connections 2023-11-24T08:07:59.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.897+0000 7f2b0d2d1640 1 --2- 172.21.15.64:0/2182694274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b080b3080 0x7f2b08161ac0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.897+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/2182694274 >> 172.21.15.64:0/2182694274 conn(0x7f2b080cec10 msgr2=0x7f2b080cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:07:59.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.897+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/2182694274 shutdown_connections 2023-11-24T08:07:59.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.897+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/2182694274 wait complete. 2023-11-24T08:07:59.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.899+0000 7f2b0d2d1640 1 Processor -- start 2023-11-24T08:07:59.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.899+0000 7f2b0d2d1640 1 -- start start 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.900+0000 7f2b0d2d1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b0806da40 0x7f2b08149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.900+0000 7f2b0d2d1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b080b3080 0x7f2b080b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.900+0000 7f2b0d2d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b0815efa0 0x7f2b081493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.900+0000 7f2b0d2d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080b1870 con 0x7f2b0815efa0 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.900+0000 7f2b0d2d1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080a88c0 con 0x7f2b080b3080 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.900+0000 7f2b0d2d1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080a85e0 con 0x7f2b0806da40 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.900+0000 7f2b0cad0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b0815efa0 0x7f2b081493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b0cad0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b0815efa0 0x7f2b081493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41808/0 (socket says 172.21.15.64:41808) 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b0cad0640 1 -- 172.21.15.64:0/3919038060 learned_addr learned my addr 172.21.15.64:0/3919038060 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b077fe640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b080b3080 0x7f2b080b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b07fff640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b0806da40 0x7f2b08149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b0cad0640 1 -- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b0806da40 msgr2=0x7f2b08149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b0cad0640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b0806da40 0x7f2b08149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b0cad0640 1 -- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b080b3080 msgr2=0x7f2b080b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:07:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b0cad0640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b080b3080 0x7f2b080b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:07:59.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b0cad0640 1 -- 172.21.15.64:0/3919038060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc009400 con 0x7f2b0815efa0 2023-11-24T08:07:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.901+0000 7f2b0cad0640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b0815efa0 0x7f2b081493a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000c700 tx=0x7f2b0000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.902+0000 7f2b04ff9640 1 -- 172.21.15.64:0/3919038060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000aa50 con 0x7f2b0815efa0 2023-11-24T08:07:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.902+0000 7f2b04ff9640 1 -- 172.21.15.64:0/3919038060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0000abb0 con 0x7f2b0815efa0 2023-11-24T08:07:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.902+0000 7f2b04ff9640 1 -- 172.21.15.64:0/3919038060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000ce10 con 0x7f2b0815efa0 2023-11-24T08:07:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.902+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b08147140 con 0x7f2b0815efa0 2023-11-24T08:07:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.902+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b08147700 con 0x7f2b0815efa0 2023-11-24T08:07:59.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.903+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b081478f0 con 0x7f2b0815efa0 2023-11-24T08:07:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.904+0000 7f2b04ff9640 1 -- 172.21.15.64:0/3919038060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2b00012420 con 0x7f2b0815efa0 2023-11-24T08:07:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.905+0000 7f2b04ff9640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2af406b410 0x7f2af406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:07:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.905+0000 7f2b04ff9640 1 -- 172.21.15.64:0/3919038060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b0008a5a0 con 0x7f2b0815efa0 2023-11-24T08:07:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.906+0000 7f2b07fff640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2af406b410 0x7f2af406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:07:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.907+0000 7f2b07fff640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2af406b410 0x7f2af406d8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000f60 tx=0x7f2af8007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:07:59.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:07:59.909+0000 7f2b04ff9640 1 -- 172.21.15.64:0/3919038060 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b00054de0 con 0x7f2b0815efa0 2023-11-24T08:08:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.105+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b0804e840 con 0x7f2b0815efa0 2023-11-24T08:08:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.106+0000 7f2b04ff9640 1 -- 172.21.15.64:0/3919038060 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b00003eb0 con 0x7f2b0815efa0 2023-11-24T08:08:00.108 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:08:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2af406b410 msgr2=0x7f2af406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2af406b410 0x7f2af406d8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000f60 tx=0x7f2af8007c00 comp rx=0 tx=0).stop 2023-11-24T08:08:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b0815efa0 msgr2=0x7f2b081493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b0815efa0 0x7f2b081493a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000c700 tx=0x7f2b0000cbc0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 shutdown_connections 2023-11-24T08:08:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2af406b410 0x7f2af406d8d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b0815efa0 0x7f2b081493a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2b080b3080 0x7f2b080b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 --2- 172.21.15.64:0/3919038060 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2b0806da40 0x7f2b08149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 >> 172.21.15.64:0/3919038060 conn(0x7f2b080cec10 msgr2=0x7f2b08162c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 shutdown_connections 2023-11-24T08:08:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.108+0000 7f2b0d2d1640 1 -- 172.21.15.64:0/3919038060 wait complete. 2023-11-24T08:08:00.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:08:00.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.349+0000 7fdde641a640 1 Processor -- start 2023-11-24T08:08:00.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.350+0000 7fdde641a640 1 -- start start 2023-11-24T08:08:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.351+0000 7fdde641a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 0x7fdde00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.351+0000 7fdde641a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fdde00d0fa0 0x7fdde00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.351+0000 7fdde641a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde00c6bf0 con 0x7fdde015d3b0 2023-11-24T08:08:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.351+0000 7fdde641a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde00d13d0 con 0x7fdde00d0fa0 2023-11-24T08:08:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.351+0000 7fdde641a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00cedc0 con 0x7fdde00d1d90 2023-11-24T08:08:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.351+0000 7fdde5418640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdde015d3b0 0x7fdde00d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58584/0 (socket says 172.21.15.64:58584) 2023-11-24T08:08:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.351+0000 7fdde5418640 1 -- 172.21.15.64:0/2952987135 learned_addr learned my addr 172.21.15.64:0/2952987135 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fdde5c19640 1 --2- 172.21.15.64:0/2952987135 >> v2:172.21.15.91:3300/0 conn(0x7fdde00d0fa0 0x7fdde00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fdde4c17640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 0x7fdde00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888184745 0 0) 0x7fdde00c6bf0 con 0x7fdde015d3b0 2023-11-24T08:08:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc003520 con 0x7fdde015d3b0 2023-11-24T08:08:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3602066337 0 0) 0x7fddcc003520 con 0x7fdde015d3b0 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 msgr2=0x7fdde00ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 0x7fdde00ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 >> v2:172.21.15.91:3300/0 conn(0x7fdde00d0fa0 msgr2=0x7fdde00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 --2- 172.21.15.64:0/2952987135 >> v2:172.21.15.91:3300/0 conn(0x7fdde00d0fa0 0x7fdde00b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00c6e40 con 0x7fdde015d3b0 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fdddc001670 con 0x7fdde015d3b0 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 >> v1:172.21.15.64:6789/0 conn(0x7fdde015d3b0 legacy=0x7fdde00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddcc0039e0 0x7fddcc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 0x7fddcc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fddcc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.352+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00d13d0 con 0x7fddcc004d00 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00cedc0 con 0x7fddcc0039e0 2023-11-24T08:08:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc00ba00 con 0x7fdde00d1d90 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde5c19640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fddcc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde5418640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddcc0039e0 0x7fddcc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde4c17640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 0x7fddcc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde5c19640 1 -- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 msgr2=0x7fddcc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde5c19640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 0x7fddcc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde5c19640 1 -- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddcc0039e0 msgr2=0x7fddcc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde5c19640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fddcc0039e0 0x7fddcc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde5c19640 1 -- 172.21.15.64:0/2952987135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc003520 con 0x7fddcc004d00 2023-11-24T08:08:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.353+0000 7fdde4c17640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00d1d90 0x7fddcc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.354+0000 7fdde5c19640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fddcc005180 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009fc0 tx=0x7fddcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287a76ecde425454 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd802cc30 con 0x7fddcc004d00 2023-11-24T08:08:00.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd800f040 con 0x7fddcc004d00 2023-11-24T08:08:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fddd7fff640 1 -- 172.21.15.64:0/2952987135 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8034440 con 0x7fddcc004d00 2023-11-24T08:08:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fdde641a640 1 -- 172.21.15.64:0/2952987135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 msgr2=0x7fddcc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fdde641a640 1 --2- 172.21.15.64:0/2952987135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fddcc005180 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009fc0 tx=0x7fddcc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fdde5c19640 1 -- 172.21.15.64:0/2952987135 reap_dead start 2023-11-24T08:08:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fdde641a640 1 -- 172.21.15.64:0/2952987135 shutdown_connections 2023-11-24T08:08:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fdde641a640 1 -- 172.21.15.64:0/2952987135 >> 172.21.15.64:0/2952987135 conn(0x7fdde00d30a0 msgr2=0x7fdde00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fdde641a640 1 -- 172.21.15.64:0/2952987135 shutdown_connections 2023-11-24T08:08:00.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.355+0000 7fdde641a640 1 -- 172.21.15.64:0/2952987135 wait complete. 2023-11-24T08:08:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.357+0000 7fdde641a640 1 Processor -- start 2023-11-24T08:08:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.357+0000 7fdde641a640 1 -- start start 2023-11-24T08:08:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.358+0000 7fdde641a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fdde00b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.358+0000 7fdde641a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdde015c340 0x7fdde0153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.358+0000 7fdde641a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00b74c0 0x7fdde009e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.358+0000 7fdde641a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00b7030 con 0x7fddcc004d00 2023-11-24T08:08:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.358+0000 7fdde641a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00b61a0 con 0x7fdde015c340 2023-11-24T08:08:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.358+0000 7fdde641a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00b6420 con 0x7fdde00b74c0 2023-11-24T08:08:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5418640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fdde00b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5418640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fdde00b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41826/0 (socket says 172.21.15.64:41826) 2023-11-24T08:08:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5418640 1 -- 172.21.15.64:0/359030905 learned_addr learned my addr 172.21.15.64:0/359030905 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5c19640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00b74c0 0x7fdde009e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5418640 1 -- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00b74c0 msgr2=0x7fdde009e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5418640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00b74c0 0x7fdde009e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde4c17640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdde015c340 0x7fdde0153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5418640 1 -- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdde015c340 msgr2=0x7fdde0153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5418640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdde015c340 0x7fdde0153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.359+0000 7fdde5418640 1 -- 172.21.15.64:0/359030905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00b6700 con 0x7fddcc004d00 2023-11-24T08:08:00.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.360+0000 7fdde5418640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fdde00b5c60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00e820 tx=0x7fdddc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:00.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.360+0000 7fddd5ffb640 1 -- 172.21.15.64:0/359030905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00eed0 con 0x7fddcc004d00 2023-11-24T08:08:00.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.360+0000 7fddd5ffb640 1 -- 172.21.15.64:0/359030905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc00c870 con 0x7fddcc004d00 2023-11-24T08:08:00.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.360+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde00b69e0 con 0x7fddcc004d00 2023-11-24T08:08:00.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.360+0000 7fddd5ffb640 1 -- 172.21.15.64:0/359030905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00cc20 con 0x7fddcc004d00 2023-11-24T08:08:00.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.360+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde0153850 con 0x7fddcc004d00 2023-11-24T08:08:00.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.361+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde004e840 con 0x7fddcc004d00 2023-11-24T08:08:00.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.362+0000 7fddd5ffb640 1 -- 172.21.15.64:0/359030905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fdddc01d460 con 0x7fddcc004d00 2023-11-24T08:08:00.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.364+0000 7fddd5ffb640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fddc406b410 0x7fddc406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.364+0000 7fddd5ffb640 1 -- 172.21.15.64:0/359030905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdddc08a020 con 0x7fddcc004d00 2023-11-24T08:08:00.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.364+0000 7fdde4c17640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fddc406b410 0x7fddc406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.365+0000 7fdde4c17640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fddc406b410 0x7fddc406d8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdde00cc9c0 tx=0x7fddd0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:00.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.369+0000 7fddd5ffb640 1 -- 172.21.15.64:0/359030905 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdddc054870 con 0x7fddcc004d00 2023-11-24T08:08:00.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.564+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdde0160620 con 0x7fddcc004d00 2023-11-24T08:08:00.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.566+0000 7fddd5ffb640 1 -- 172.21.15.64:0/359030905 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdddc057e90 con 0x7fddcc004d00 2023-11-24T08:08:00.568 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:08:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fddc406b410 msgr2=0x7fddc406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fddc406b410 0x7fddc406d8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdde00cc9c0 tx=0x7fddd0008040 comp rx=0 tx=0).stop 2023-11-24T08:08:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 msgr2=0x7fdde00b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fdde00b5c60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00e820 tx=0x7fdddc00ece0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 shutdown_connections 2023-11-24T08:08:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fddc406b410 0x7fddc406d8d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdde00b74c0 0x7fdde009e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdde015c340 0x7fdde0153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 --2- 172.21.15.64:0/359030905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddcc004d00 0x7fdde00b5c60 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 >> 172.21.15.64:0/359030905 conn(0x7fdde00d30a0 msgr2=0x7fdde00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:00.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.568+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 shutdown_connections 2023-11-24T08:08:00.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.569+0000 7fdde641a640 1 -- 172.21.15.64:0/359030905 wait complete. 2023-11-24T08:08:00.590 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:08:00.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.809+0000 7f5717783640 1 Processor -- start 2023-11-24T08:08:00.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.809+0000 7f5717783640 1 -- start start 2023-11-24T08:08:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.810+0000 7f5717783640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f571015d6d0 0x7f57100d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.810+0000 7f5717783640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f57100d2d80 0x7f57100b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5717783640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100d1300 con 0x7f57100b51c0 2023-11-24T08:08:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5717783640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100d21d0 con 0x7f57100d2d80 2023-11-24T08:08:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5717783640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100b5670 con 0x7f571015d6d0 2023-11-24T08:08:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5714d1e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f57100d2d80 0x7f57100b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5714d1e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f57100d2d80 0x7f57100b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41832/0 (socket says 172.21.15.64:41832) 2023-11-24T08:08:00.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5714d1e640 1 -- 172.21.15.64:0/648108294 learned_addr learned my addr 172.21.15.64:0/648108294 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:00.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f571551f640 1 --2- 172.21.15.64:0/648108294 >> v2:172.21.15.91:3300/0 conn(0x7f571015d6d0 0x7f57100d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5714d1e640 1 -- 172.21.15.64:0/648108294 >> v2:172.21.15.91:3300/0 conn(0x7f571015d6d0 msgr2=0x7f57100d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:00.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5714d1e640 1 --2- 172.21.15.64:0/648108294 >> v2:172.21.15.91:3300/0 conn(0x7f571015d6d0 0x7f57100d2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5714d1e640 1 -- 172.21.15.64:0/648108294 >> v1:172.21.15.91:6789/0 conn(0x7f57100b51c0 legacy=0x7f571006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:00.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.811+0000 7f5714d1e640 1 -- 172.21.15.64:0/648108294 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100b2830 con 0x7f57100d2d80 2023-11-24T08:08:00.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.812+0000 7f5714d1e640 1 --2- 172.21.15.64:0/648108294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100d2d80 0x7f57100b4c80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f571006c690 tx=0x7f570402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9149551505beb7a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:00.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5703fff640 1 -- 172.21.15.64:0/648108294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704030860 con 0x7f57100d2d80 2023-11-24T08:08:00.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5703fff640 1 -- 172.21.15.64:0/648108294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57040309c0 con 0x7f57100d2d80 2023-11-24T08:08:00.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5703fff640 1 -- 172.21.15.64:0/648108294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704030d50 con 0x7f57100d2d80 2023-11-24T08:08:00.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5717783640 1 -- 172.21.15.64:0/648108294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100d2d80 msgr2=0x7f57100b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5717783640 1 --2- 172.21.15.64:0/648108294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100d2d80 0x7f57100b4c80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f571006c690 tx=0x7f570402b9a0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5717783640 1 -- 172.21.15.64:0/648108294 shutdown_connections 2023-11-24T08:08:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5717783640 1 --2- 172.21.15.64:0/648108294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100d2d80 0x7f57100b4c80 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5717783640 1 --2- 172.21.15.64:0/648108294 >> v2:172.21.15.91:3300/0 conn(0x7f571015d6d0 0x7f57100d2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.813+0000 7f5717783640 1 -- 172.21.15.64:0/648108294 >> 172.21.15.64:0/648108294 conn(0x7f57100721a0 msgr2=0x7f57100cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.814+0000 7f5717783640 1 -- 172.21.15.64:0/648108294 shutdown_connections 2023-11-24T08:08:00.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.814+0000 7f5717783640 1 -- 172.21.15.64:0/648108294 wait complete. 2023-11-24T08:08:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.815+0000 7f5717783640 1 Processor -- start 2023-11-24T08:08:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.815+0000 7f5717783640 1 -- start start 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.816+0000 7f5717783640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100b51c0 0x7f5710159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.816+0000 7f5717783640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57100d2d80 0x7f57101f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.816+0000 7f5717783640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f571015d6d0 0x7f57101f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.816+0000 7f5717783640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100cbaf0 con 0x7f57100b51c0 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.816+0000 7f5717783640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100c9990 con 0x7f57100d2d80 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.816+0000 7f5717783640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57101f6e20 con 0x7f571015d6d0 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f571551f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100b51c0 0x7f5710159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f571551f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100b51c0 0x7f5710159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41846/0 (socket says 172.21.15.64:41846) 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f571551f640 1 -- 172.21.15.64:0/4232826613 learned_addr learned my addr 172.21.15.64:0/4232826613 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f571551f640 1 -- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f571015d6d0 msgr2=0x7f57101f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f5715d20640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f571015d6d0 0x7f57101f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f571551f640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f571015d6d0 0x7f57101f67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f571551f640 1 -- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57100d2d80 msgr2=0x7f57101f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:00.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f571551f640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57100d2d80 0x7f57101f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:00.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f571551f640 1 -- 172.21.15.64:0/4232826613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040094d0 con 0x7f57100b51c0 2023-11-24T08:08:00.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.817+0000 7f5715d20640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f571015d6d0 0x7f57101f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.818+0000 7f571551f640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100b51c0 0x7f5710159a90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f570c00d5f0 tx=0x7f570c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.818+0000 7f5701ffb640 1 -- 172.21.15.64:0/4232826613 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c0125f0 con 0x7f57100b51c0 2023-11-24T08:08:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.818+0000 7f5701ffb640 1 -- 172.21.15.64:0/4232826613 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c012750 con 0x7f57100b51c0 2023-11-24T08:08:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.818+0000 7f5701ffb640 1 -- 172.21.15.64:0/4232826613 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c012b00 con 0x7f57100b51c0 2023-11-24T08:08:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.818+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57100cb430 con 0x7f57100b51c0 2023-11-24T08:08:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.818+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57101f7100 con 0x7f57100b51c0 2023-11-24T08:08:00.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.819+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5710074770 con 0x7f57100b51c0 2023-11-24T08:08:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.820+0000 7f5701ffb640 1 -- 172.21.15.64:0/4232826613 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f570c012c60 con 0x7f57100b51c0 2023-11-24T08:08:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.821+0000 7f5701ffb640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56f406b3e0 0x7f56f406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.821+0000 7f5701ffb640 1 -- 172.21.15.64:0/4232826613 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f570c08a520 con 0x7f57100b51c0 2023-11-24T08:08:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.821+0000 7f5714d1e640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56f406b3e0 0x7f56f406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:00.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.822+0000 7f5714d1e640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56f406b3e0 0x7f56f406d8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5704036000 tx=0x7f570403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:00.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:00.825+0000 7f5701ffb640 1 -- 172.21.15.64:0/4232826613 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f570c054db0 con 0x7f57100b51c0 2023-11-24T08:08:01.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.036+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f57100b2d00 con 0x7f57100b51c0 2023-11-24T08:08:01.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.036+0000 7f5701ffb640 1 -- 172.21.15.64:0/4232826613 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f570c0583d0 con 0x7f57100b51c0 2023-11-24T08:08:01.038 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:08:01.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56f406b3e0 msgr2=0x7f56f406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:01.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56f406b3e0 0x7f56f406d8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5704036000 tx=0x7f570403b040 comp rx=0 tx=0).stop 2023-11-24T08:08:01.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100b51c0 msgr2=0x7f5710159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:01.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100b51c0 0x7f5710159a90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f570c00d5f0 tx=0x7f570c00dab0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 shutdown_connections 2023-11-24T08:08:01.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f56f406b3e0 0x7f56f406d8a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f571015d6d0 0x7f57101f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57100d2d80 0x7f57101f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 --2- 172.21.15.64:0/4232826613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57100b51c0 0x7f5710159a90 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.039+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 >> 172.21.15.64:0/4232826613 conn(0x7f57100721a0 msgr2=0x7f57100cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.040+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 shutdown_connections 2023-11-24T08:08:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.040+0000 7f5717783640 1 -- 172.21.15.64:0/4232826613 wait complete. 2023-11-24T08:08:01.059 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T08:08:01.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:01.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T08:08:01.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[112430]: fuse finished with error 0 and tester_r 0 2023-11-24T08:08:01.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:08:01.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:01.160 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:08:01.345 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:08:01.345 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:01.406 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.631+0000 7fc70d6ff640 1 Processor -- start 2023-11-24T08:08:01.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.631+0000 7fc70d6ff640 1 -- start start 2023-11-24T08:08:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.633+0000 7fc70d6ff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fc70813dfd0 0x7fc70815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.633+0000 7fc70d6ff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7080cf050 0x7fc7080cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.633+0000 7fc70d6ff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7080ce7c0 con 0x7fc70806c130 2023-11-24T08:08:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.633+0000 7fc70d6ff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7080cfa40 con 0x7fc7080cf050 2023-11-24T08:08:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.633+0000 7fc70d6ff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc708140900 con 0x7fc70813dfd0 2023-11-24T08:08:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.633+0000 7fc70cefe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc70806c130 0x7fc70806c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58590/0 (socket says 172.21.15.64:58590) 2023-11-24T08:08:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.633+0000 7fc70cefe640 1 -- 172.21.15.64:0/3070286893 learned_addr learned my addr 172.21.15.64:0/3070286893 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.633+0000 7fc6fffff640 1 --2- 172.21.15.64:0/3070286893 >> v2:172.21.15.64:3300/0 conn(0x7fc7080cf050 0x7fc7080cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.634+0000 7fc707fff640 1 --2- 172.21.15.64:0/3070286893 >> v2:172.21.15.91:3300/0 conn(0x7fc70813dfd0 0x7fc70815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.634+0000 7fc7077fe640 1 -- 172.21.15.64:0/3070286893 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134245556 0 0) 0x7fc7080ce7c0 con 0x7fc70806c130 2023-11-24T08:08:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.634+0000 7fc7077fe640 1 -- 172.21.15.64:0/3070286893 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f4003520 con 0x7fc70806c130 2023-11-24T08:08:01.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.634+0000 7fc6fffff640 1 -- 172.21.15.64:0/3070286893 >> v2:172.21.15.91:3300/0 conn(0x7fc70813dfd0 msgr2=0x7fc70815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:01.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.634+0000 7fc6fffff640 1 --2- 172.21.15.64:0/3070286893 >> v2:172.21.15.91:3300/0 conn(0x7fc70813dfd0 0x7fc70815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.634+0000 7fc6fffff640 1 -- 172.21.15.64:0/3070286893 >> v1:172.21.15.64:6789/0 conn(0x7fc70806c130 legacy=0x7fc70806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:01.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.634+0000 7fc6fffff640 1 -- 172.21.15.64:0/3070286893 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70815fa70 con 0x7fc7080cf050 2023-11-24T08:08:01.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.635+0000 7fc6fffff640 1 --2- 172.21.15.64:0/3070286893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc7080cf500 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc7080cfc10 tx=0x7fc6f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fe3e5ee36ab2ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:01.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc7077fe640 1 -- 172.21.15.64:0/3070286893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0031830 con 0x7fc7080cf050 2023-11-24T08:08:01.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc7077fe640 1 -- 172.21.15.64:0/3070286893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0031990 con 0x7fc7080cf050 2023-11-24T08:08:01.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc7077fe640 1 -- 172.21.15.64:0/3070286893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0031d20 con 0x7fc7080cf050 2023-11-24T08:08:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc70d6ff640 1 -- 172.21.15.64:0/3070286893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 msgr2=0x7fc7080cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/3070286893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc7080cf500 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc7080cfc10 tx=0x7fc6f002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc70d6ff640 1 -- 172.21.15.64:0/3070286893 shutdown_connections 2023-11-24T08:08:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/3070286893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc7080cf500 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/3070286893 >> v2:172.21.15.91:3300/0 conn(0x7fc70813dfd0 0x7fc70815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc70d6ff640 1 -- 172.21.15.64:0/3070286893 >> 172.21.15.64:0/3070286893 conn(0x7fc70806a860 msgr2=0x7fc70806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.636+0000 7fc70d6ff640 1 -- 172.21.15.64:0/3070286893 shutdown_connections 2023-11-24T08:08:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.637+0000 7fc70d6ff640 1 -- 172.21.15.64:0/3070286893 wait complete. 2023-11-24T08:08:01.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.638+0000 7fc70d6ff640 1 Processor -- start 2023-11-24T08:08:01.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.638+0000 7fc70d6ff640 1 -- start start 2023-11-24T08:08:01.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.639+0000 7fc70d6ff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc70806c130 0x7fc708158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:01.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.639+0000 7fc70d6ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc708156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:01.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.639+0000 7fc70d6ff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc70813dfd0 0x7fc7080b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:01.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.639+0000 7fc70d6ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081593a0 con 0x7fc7080cf050 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.639+0000 7fc70d6ff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70813ff70 con 0x7fc70806c130 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.639+0000 7fc70d6ff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc708071fb0 con 0x7fc70813dfd0 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc6fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc708156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc6fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc708156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41872/0 (socket says 172.21.15.64:41872) 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc6fffff640 1 -- 172.21.15.64:0/1609832217 learned_addr learned my addr 172.21.15.64:0/1609832217 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc707fff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc70806c130 0x7fc708158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc70cefe640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc70813dfd0 0x7fc7080b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc6fffff640 1 -- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc70813dfd0 msgr2=0x7fc7080b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc6fffff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc70813dfd0 0x7fc7080b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc6fffff640 1 -- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc70806c130 msgr2=0x7fc708158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:01.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc6fffff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc70806c130 0x7fc708158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc6fffff640 1 -- 172.21.15.64:0/1609832217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f0009400 con 0x7fc7080cf050 2023-11-24T08:08:01.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc707fff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc70806c130 0x7fc708158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:01.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.640+0000 7fc70cefe640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc70813dfd0 0x7fc7080b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:01.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.641+0000 7fc6fffff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc708156230 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009890 tx=0x7fc6f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:01.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.641+0000 7fc7057fa640 1 -- 172.21.15.64:0/1609832217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f003a630 con 0x7fc7080cf050 2023-11-24T08:08:01.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.641+0000 7fc7057fa640 1 -- 172.21.15.64:0/1609832217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f003a7b0 con 0x7fc7080cf050 2023-11-24T08:08:01.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.641+0000 7fc7057fa640 1 -- 172.21.15.64:0/1609832217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f003ab60 con 0x7fc7080cf050 2023-11-24T08:08:01.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.641+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc708140230 con 0x7fc7080cf050 2023-11-24T08:08:01.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.641+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc708156770 con 0x7fc7080cf050 2023-11-24T08:08:01.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.642+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70804e840 con 0x7fc7080cf050 2023-11-24T08:08:01.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.643+0000 7fc7057fa640 1 -- 172.21.15.64:0/1609832217 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fc6f003acc0 con 0x7fc7080cf050 2023-11-24T08:08:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.644+0000 7fc7057fa640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc6e806b3e0 0x7fc6e806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.644+0000 7fc7057fa640 1 -- 172.21.15.64:0/1609832217 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc6f00abdc0 con 0x7fc7080cf050 2023-11-24T08:08:01.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.645+0000 7fc707fff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc6e806b3e0 0x7fc6e806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:01.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.646+0000 7fc707fff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc6e806b3e0 0x7fc6e806d8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009a20 tx=0x7fc6f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:01.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.649+0000 7fc7057fa640 1 -- 172.21.15.64:0/1609832217 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6f0076590 con 0x7fc7080cf050 2023-11-24T08:08:01.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.845+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7080cc500 con 0x7fc7080cf050 2023-11-24T08:08:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.848+0000 7fc7057fa640 1 -- 172.21.15.64:0/1609832217 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16630 (secure 0 0 0) 0x7fc7080cc500 con 0x7fc7080cf050 2023-11-24T08:08:01.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:01.850 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:01.452465+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:07:39.474762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:07:40.829840+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.850+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc6e806b3e0 msgr2=0x7fc6e806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.850+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc6e806b3e0 0x7fc6e806d8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009a20 tx=0x7fc6f8008040 comp rx=0 tx=0).stop 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.850+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 msgr2=0x7fc708156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.850+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc708156230 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009890 tx=0x7fc6f00046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.851+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 shutdown_connections 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.851+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc6e806b3e0 0x7fc6e806d8a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.851+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc70813dfd0 0x7fc7080b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.851+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7080cf050 0x7fc708156230 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.851+0000 7fc70d6ff640 1 --2- 172.21.15.64:0/1609832217 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc70806c130 0x7fc708158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.851+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 >> 172.21.15.64:0/1609832217 conn(0x7fc70806a860 msgr2=0x7fc70806c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.851+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 shutdown_connections 2023-11-24T08:08:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:01.851+0000 7fc70d6ff640 1 -- 172.21.15.64:0/1609832217 wait complete. 2023-11-24T08:08:01.874 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:07:39.474762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:07:40.829840+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:01.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.088+0000 7febecf47640 1 Processor -- start 2023-11-24T08:08:02.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.089+0000 7febecf47640 1 -- start start 2023-11-24T08:08:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.091+0000 7febecf47640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7febe815ac80 0x7febe8071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.091+0000 7febecf47640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe8071670 con 0x7febe815d7a0 2023-11-24T08:08:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.091+0000 7febecf47640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe815e240 con 0x7febe806ab50 2023-11-24T08:08:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.091+0000 7febecf47640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe80710f0 con 0x7febe815ac80 2023-11-24T08:08:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.091+0000 7febe6ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7febe815d7a0 0x7febe806a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58604/0 (socket says 172.21.15.64:58604) 2023-11-24T08:08:02.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.091+0000 7febe6ffd640 1 -- 172.21.15.64:0/1689395173 learned_addr learned my addr 172.21.15.64:0/1689395173 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.092+0000 7febe77fe640 1 --2- 172.21.15.64:0/1689395173 >> v2:172.21.15.91:3300/0 conn(0x7febe815ac80 0x7febe8071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.092+0000 7febe67fc640 1 -- 172.21.15.64:0/1689395173 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388135815 0 0) 0x7febe8071670 con 0x7febe815d7a0 2023-11-24T08:08:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.092+0000 7febe67fc640 1 -- 172.21.15.64:0/1689395173 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd0003520 con 0x7febe815d7a0 2023-11-24T08:08:02.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.093+0000 7febe77fe640 1 -- 172.21.15.64:0/1689395173 >> v1:172.21.15.91:6789/0 conn(0x7febe806ab50 legacy=0x7febe806d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.093+0000 7febe77fe640 1 -- 172.21.15.64:0/1689395173 >> v1:172.21.15.64:6789/0 conn(0x7febe815d7a0 legacy=0x7febe806a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.093+0000 7febe77fe640 1 -- 172.21.15.64:0/1689395173 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe815e520 con 0x7febe815ac80 2023-11-24T08:08:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.094+0000 7febe77fe640 1 --2- 172.21.15.64:0/1689395173 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7febe815ac80 0x7febe8071b00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7febe0009d90 tx=0x7febe002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc2290a9524028d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.095+0000 7febe67fc640 1 -- 172.21.15.64:0/1689395173 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe002cd00 con 0x7febe815ac80 2023-11-24T08:08:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.095+0000 7febe67fc640 1 -- 172.21.15.64:0/1689395173 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe002ce60 con 0x7febe815ac80 2023-11-24T08:08:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.095+0000 7febe67fc640 1 -- 172.21.15.64:0/1689395173 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0034660 con 0x7febe815ac80 2023-11-24T08:08:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.095+0000 7febecf47640 1 -- 172.21.15.64:0/1689395173 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7febe815ac80 msgr2=0x7febe8071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.095+0000 7febecf47640 1 --2- 172.21.15.64:0/1689395173 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7febe815ac80 0x7febe8071b00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7febe0009d90 tx=0x7febe002bf50 comp rx=0 tx=0).stop 2023-11-24T08:08:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.096+0000 7febecf47640 1 -- 172.21.15.64:0/1689395173 shutdown_connections 2023-11-24T08:08:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.096+0000 7febecf47640 1 --2- 172.21.15.64:0/1689395173 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7febe815ac80 0x7febe8071b00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.096+0000 7febecf47640 1 -- 172.21.15.64:0/1689395173 >> 172.21.15.64:0/1689395173 conn(0x7febe80cfde0 msgr2=0x7febe80d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.096+0000 7febecf47640 1 -- 172.21.15.64:0/1689395173 shutdown_connections 2023-11-24T08:08:02.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.096+0000 7febecf47640 1 -- 172.21.15.64:0/1689395173 wait complete. 2023-11-24T08:08:02.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.098+0000 7febecf47640 1 Processor -- start 2023-11-24T08:08:02.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.098+0000 7febecf47640 1 -- start start 2023-11-24T08:08:02.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febecf47640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7febe806ab50 0x7febe815f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febecf47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe815ac80 0x7febe80b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febecf47640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7febe815d7a0 0x7febe80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febecf47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe815f2e0 con 0x7febe815ac80 2023-11-24T08:08:02.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febecf47640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80b57f0 con 0x7febe815d7a0 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febecf47640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe80b5a70 con 0x7febe806ab50 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febe6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe815ac80 0x7febe80b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febe6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe815ac80 0x7febe80b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41884/0 (socket says 172.21.15.64:41884) 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.099+0000 7febe6ffd640 1 -- 172.21.15.64:0/496868214 learned_addr learned my addr 172.21.15.64:0/496868214 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febe6ffd640 1 -- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7febe806ab50 msgr2=0x7febe815f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febe6ffd640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7febe806ab50 0x7febe815f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febe6ffd640 1 -- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7febe815d7a0 msgr2=0x7febe80b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febe6ffd640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7febe815d7a0 0x7febe80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febe6ffd640 1 -- 172.21.15.64:0/496868214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0009a70 con 0x7febe815ac80 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febe6ffd640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe815ac80 0x7febe80b4a30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c500 tx=0x7febdc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febcffff640 1 -- 172.21.15.64:0/496868214 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc005ce0 con 0x7febe815ac80 2023-11-24T08:08:02.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febcffff640 1 -- 172.21.15.64:0/496868214 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc005e40 con 0x7febe815ac80 2023-11-24T08:08:02.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febcffff640 1 -- 172.21.15.64:0/496868214 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc007330 con 0x7febe815ac80 2023-11-24T08:08:02.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe80721c0 con 0x7febe815ac80 2023-11-24T08:08:02.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.100+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe815ec20 con 0x7febe815ac80 2023-11-24T08:08:02.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.102+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe804e840 con 0x7febe815ac80 2023-11-24T08:08:02.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.103+0000 7febcffff640 1 -- 172.21.15.64:0/496868214 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7febdc007490 con 0x7febe815ac80 2023-11-24T08:08:02.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.104+0000 7febcffff640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febd406b3e0 0x7febd406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.104+0000 7febcffff640 1 -- 172.21.15.64:0/496868214 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7febdc0896a0 con 0x7febe815ac80 2023-11-24T08:08:02.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.105+0000 7febe77fe640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febd406b3e0 0x7febd406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.107+0000 7febe77fe640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febd406b3e0 0x7febd406d8a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7febe0036000 tx=0x7febe002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:02.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.109+0000 7febcffff640 1 -- 172.21.15.64:0/496868214 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febdc053ff0 con 0x7febe815ac80 2023-11-24T08:08:02.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.306+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febe815f420 con 0x7febe815ac80 2023-11-24T08:08:02.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.307+0000 7febcffff640 1 -- 172.21.15.64:0/496868214 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7febe815f420 con 0x7febe815ac80 2023-11-24T08:08:02.309 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:08:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febd406b3e0 msgr2=0x7febd406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febd406b3e0 0x7febd406d8a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7febe0036000 tx=0x7febe002e000 comp rx=0 tx=0).stop 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe815ac80 msgr2=0x7febe80b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe815ac80 0x7febe80b4a30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c500 tx=0x7febdc00c9c0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 shutdown_connections 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7febd406b3e0 0x7febd406d8a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7febe815d7a0 0x7febe80b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe815ac80 0x7febe80b4a30 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 --2- 172.21.15.64:0/496868214 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7febe806ab50 0x7febe815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 >> 172.21.15.64:0/496868214 conn(0x7febe80cfde0 msgr2=0x7febe80cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 shutdown_connections 2023-11-24T08:08:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.310+0000 7febecf47640 1 -- 172.21.15.64:0/496868214 wait complete. 2023-11-24T08:08:02.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:08:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.565+0000 7f441d86e640 1 Processor -- start 2023-11-24T08:08:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.565+0000 7f441d86e640 1 -- start start 2023-11-24T08:08:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.567+0000 7f441d86e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f441815af50 0x7f44180b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.567+0000 7f441d86e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44180cc830 con 0x7f44180d13b0 2023-11-24T08:08:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.567+0000 7f441d86e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44180b13b0 con 0x7f44180b2450 2023-11-24T08:08:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.567+0000 7f441d86e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44180b11c0 con 0x7f441815af50 2023-11-24T08:08:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.567+0000 7f441d06d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f44180d13b0 0x7f44180afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58618/0 (socket says 172.21.15.64:58618) 2023-11-24T08:08:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.567+0000 7f441d06d640 1 -- 172.21.15.64:0/2235108657 learned_addr learned my addr 172.21.15.64:0/2235108657 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.567+0000 7f441c86c640 1 --2- 172.21.15.64:0/2235108657 >> v2:172.21.15.91:3300/0 conn(0x7f441815af50 0x7f44180b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515448156 0 0) 0x7f44180cc830 con 0x7f44180d13b0 2023-11-24T08:08:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4400003520 con 0x7f44180d13b0 2023-11-24T08:08:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 73433547 0 0) 0x7f4400003520 con 0x7f44180d13b0 2023-11-24T08:08:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 >> v2:172.21.15.91:3300/0 conn(0x7f441815af50 msgr2=0x7f44180b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 --2- 172.21.15.64:0/2235108657 >> v2:172.21.15.91:3300/0 conn(0x7f441815af50 0x7f44180b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 >> v1:172.21.15.91:6789/0 conn(0x7f44180b2450 legacy=0x7f44180d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180b1690 con 0x7f44180d13b0 2023-11-24T08:08:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f44100019f0 con 0x7f44180d13b0 2023-11-24T08:08:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 >> v1:172.21.15.64:6789/0 conn(0x7f44180d13b0 legacy=0x7f44180afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4400003a30 0x7f4400003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f4400004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44000052a0 0x7f440000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440000c140 con 0x7f44000047c0 2023-11-24T08:08:02.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440000c2b0 con 0x7f4400003a30 2023-11-24T08:08:02.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.568+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440000c470 con 0x7f44000052a0 2023-11-24T08:08:02.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f440ffff640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f4400004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f441c86c640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4400003a30 0x7f4400003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f441d06d640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44000052a0 0x7f440000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f440ffff640 1 -- 172.21.15.64:0/2235108657 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44000052a0 msgr2=0x7f440000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:02.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f440ffff640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44000052a0 0x7f440000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f440ffff640 1 -- 172.21.15.64:0/2235108657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4400003a30 msgr2=0x7f4400003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:02.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f440ffff640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f4400003a30 0x7f4400003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f440ffff640 1 -- 172.21.15.64:0/2235108657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400003520 con 0x7f44000047c0 2023-11-24T08:08:02.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.569+0000 7f441d06d640 1 -- 172.21.15.64:0/2235108657 reap_dead start 2023-11-24T08:08:02.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.570+0000 7f440ffff640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f4400004c40 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4414001720 tx=0x7f44000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf9b2328660bcd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:02.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.571+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140052d0 con 0x7f44000047c0 2023-11-24T08:08:02.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.571+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441402e050 con 0x7f44000047c0 2023-11-24T08:08:02.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.571+0000 7f440f7fe640 1 -- 172.21.15.64:0/2235108657 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414003830 con 0x7f44000047c0 2023-11-24T08:08:02.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.571+0000 7f441d86e640 1 -- 172.21.15.64:0/2235108657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 msgr2=0x7f4400004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.571+0000 7f441d86e640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f4400004c40 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4414001720 tx=0x7f44000057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.571+0000 7f441d86e640 1 -- 172.21.15.64:0/2235108657 shutdown_connections 2023-11-24T08:08:02.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.571+0000 7f441d86e640 1 --2- 172.21.15.64:0/2235108657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f4400004c40 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.571+0000 7f441d86e640 1 -- 172.21.15.64:0/2235108657 >> 172.21.15.64:0/2235108657 conn(0x7f4418099a00 msgr2=0x7f4418099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:02.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.572+0000 7f441d86e640 1 -- 172.21.15.64:0/2235108657 shutdown_connections 2023-11-24T08:08:02.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.572+0000 7f441d86e640 1 -- 172.21.15.64:0/2235108657 wait complete. 2023-11-24T08:08:02.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.573+0000 7f441d86e640 1 Processor -- start 2023-11-24T08:08:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.574+0000 7f441d86e640 1 -- start start 2023-11-24T08:08:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.574+0000 7f441d86e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f44180c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.574+0000 7f441d86e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44180ca3f0 0x7f44181f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.574+0000 7f441d86e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44181f2e30 0x7f44181f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441d86e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44181f3820 con 0x7f44000047c0 2023-11-24T08:08:02.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441d86e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441815bd20 con 0x7f44180ca3f0 2023-11-24T08:08:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441d86e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44180b4a40 con 0x7f44181f2e30 2023-11-24T08:08:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441c86c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f44180c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441c86c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f44180c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41904/0 (socket says 172.21.15.64:41904) 2023-11-24T08:08:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441c86c640 1 -- 172.21.15.64:0/1292214929 learned_addr learned my addr 172.21.15.64:0/1292214929 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441c86c640 1 -- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44181f2e30 msgr2=0x7f44181f32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441c86c640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44181f2e30 0x7f44181f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f440ffff640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44180ca3f0 0x7f44181f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441c86c640 1 -- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44180ca3f0 msgr2=0x7f44181f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441c86c640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44180ca3f0 0x7f44181f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.575+0000 7f441c86c640 1 -- 172.21.15.64:0/1292214929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441815ba00 con 0x7f44000047c0 2023-11-24T08:08:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.576+0000 7f441c86c640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f44180c7f30 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f44080116d0 tx=0x7f4408011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.576+0000 7f440d7fa640 1 -- 172.21.15.64:0/1292214929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440800f9c0 con 0x7f44000047c0 2023-11-24T08:08:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.576+0000 7f440d7fa640 1 -- 172.21.15.64:0/1292214929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440800fb20 con 0x7f44000047c0 2023-11-24T08:08:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.576+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441815e440 con 0x7f44000047c0 2023-11-24T08:08:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.576+0000 7f440d7fa640 1 -- 172.21.15.64:0/1292214929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440800fe90 con 0x7f44000047c0 2023-11-24T08:08:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.576+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44180cabb0 con 0x7f44000047c0 2023-11-24T08:08:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.577+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441804e840 con 0x7f44000047c0 2023-11-24T08:08:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.578+0000 7f440d7fa640 1 -- 172.21.15.64:0/1292214929 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4408015460 con 0x7f44000047c0 2023-11-24T08:08:02.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.579+0000 7f440d7fa640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f440406b410 0x7f440406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:02.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.579+0000 7f440d7fa640 1 -- 172.21.15.64:0/1292214929 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f440808a9f0 con 0x7f44000047c0 2023-11-24T08:08:02.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.580+0000 7f440ffff640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f440406b410 0x7f440406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:02.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.581+0000 7f440ffff640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f440406b410 0x7f440406d8d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4414033000 tx=0x7f4414009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:02.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.583+0000 7f440d7fa640 1 -- 172.21.15.64:0/1292214929 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44080551b0 con 0x7f44000047c0 2023-11-24T08:08:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.777+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44180ce610 con 0x7f44000047c0 2023-11-24T08:08:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.778+0000 7f440d7fa640 1 -- 172.21.15.64:0/1292214929 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44080587d0 con 0x7f44000047c0 2023-11-24T08:08:02.780 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:08:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f440406b410 msgr2=0x7f440406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f440406b410 0x7f440406d8d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4414033000 tx=0x7f4414009030 comp rx=0 tx=0).stop 2023-11-24T08:08:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 msgr2=0x7f44180c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f44180c7f30 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f44080116d0 tx=0x7f4408011b90 comp rx=0 tx=0).stop 2023-11-24T08:08:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 shutdown_connections 2023-11-24T08:08:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f440406b410 0x7f440406d8d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f44181f2e30 0x7f44181f32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f44180ca3f0 0x7f44181f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 --2- 172.21.15.64:0/1292214929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44000047c0 0x7f44180c7f30 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 >> 172.21.15.64:0/1292214929 conn(0x7f4418099a00 msgr2=0x7f44180afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 shutdown_connections 2023-11-24T08:08:02.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:02.781+0000 7f441d86e640 1 -- 172.21.15.64:0/1292214929 wait complete. 2023-11-24T08:08:02.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:08:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.032+0000 7f0b0c852640 1 Processor -- start 2023-11-24T08:08:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.032+0000 7f0b0c852640 1 -- start start 2023-11-24T08:08:03.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.034+0000 7f0b0c852640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0b08144770 0x7f0b0815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.034+0000 7f0b0c852640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b080b35a0 0x7f0b0815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.034+0000 7f0b0c852640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0815f930 con 0x7f0b0806c9d0 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.034+0000 7f0b0c852640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b080b43b0 con 0x7f0b080b35a0 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.034+0000 7f0b0c852640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b080b3b10 con 0x7f0b08144770 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.034+0000 7f0afffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b080b35a0 0x7f0b0815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.034+0000 7f0afffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b080b35a0 0x7f0b0815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41910/0 (socket says 172.21.15.64:41910) 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.034+0000 7f0afffff640 1 -- 172.21.15.64:0/255231859 learned_addr learned my addr 172.21.15.64:0/255231859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.035+0000 7f0b077fe640 1 --2- 172.21.15.64:0/255231859 >> v2:172.21.15.91:3300/0 conn(0x7f0b08144770 0x7f0b0815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.035+0000 7f0afffff640 1 -- 172.21.15.64:0/255231859 >> v2:172.21.15.91:3300/0 conn(0x7f0b08144770 msgr2=0x7f0b0815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:03.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.035+0000 7f0afffff640 1 --2- 172.21.15.64:0/255231859 >> v2:172.21.15.91:3300/0 conn(0x7f0b08144770 0x7f0b0815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.035+0000 7f0afffff640 1 -- 172.21.15.64:0/255231859 >> v1:172.21.15.91:6789/0 conn(0x7f0b0806c9d0 legacy=0x7f0b0806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:03.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.035+0000 7f0afffff640 1 -- 172.21.15.64:0/255231859 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b080b3df0 con 0x7f0b080b35a0 2023-11-24T08:08:03.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.035+0000 7f0b077fe640 1 --2- 172.21.15.64:0/255231859 >> v2:172.21.15.91:3300/0 conn(0x7f0b08144770 0x7f0b0815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:03.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.036+0000 7f0afffff640 1 --2- 172.21.15.64:0/255231859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b080b35a0 0x7f0b0815f0a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0b0806df90 tx=0x7f0af002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24791c9284dc9c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:03.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.036+0000 7f0b06ffd640 1 -- 172.21.15.64:0/255231859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0031870 con 0x7f0b080b35a0 2023-11-24T08:08:03.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.036+0000 7f0b06ffd640 1 -- 172.21.15.64:0/255231859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af00319d0 con 0x7f0b080b35a0 2023-11-24T08:08:03.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.036+0000 7f0b06ffd640 1 -- 172.21.15.64:0/255231859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0031d60 con 0x7f0b080b35a0 2023-11-24T08:08:03.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.037+0000 7f0b0c852640 1 -- 172.21.15.64:0/255231859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b080b35a0 msgr2=0x7f0b0815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.037+0000 7f0b0c852640 1 --2- 172.21.15.64:0/255231859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b080b35a0 0x7f0b0815f0a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0b0806df90 tx=0x7f0af002b910 comp rx=0 tx=0).stop 2023-11-24T08:08:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.037+0000 7f0b0c852640 1 -- 172.21.15.64:0/255231859 shutdown_connections 2023-11-24T08:08:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.037+0000 7f0b0c852640 1 --2- 172.21.15.64:0/255231859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b080b35a0 0x7f0b0815f0a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.037+0000 7f0b0c852640 1 --2- 172.21.15.64:0/255231859 >> v2:172.21.15.91:3300/0 conn(0x7f0b08144770 0x7f0b0815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.037+0000 7f0b0c852640 1 -- 172.21.15.64:0/255231859 >> 172.21.15.64:0/255231859 conn(0x7f0b080cec10 msgr2=0x7f0b080cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.037+0000 7f0b0c852640 1 -- 172.21.15.64:0/255231859 shutdown_connections 2023-11-24T08:08:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.037+0000 7f0b0c852640 1 -- 172.21.15.64:0/255231859 wait complete. 2023-11-24T08:08:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.039+0000 7f0b0c852640 1 Processor -- start 2023-11-24T08:08:03.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.039+0000 7f0b0c852640 1 -- start start 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.040+0000 7f0b0c852640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b0806c9d0 0x7f0b08156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.040+0000 7f0b0c852640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b080b35a0 0x7f0b081f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.040+0000 7f0b0c852640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b08144770 0x7f0b08139f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.040+0000 7f0b0c852640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0815d5e0 con 0x7f0b08144770 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.040+0000 7f0b0c852640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08157020 con 0x7f0b0806c9d0 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.040+0000 7f0b0c852640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0813aa40 con 0x7f0b080b35a0 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b08144770 0x7f0b08139f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b08144770 0x7f0b08139f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41916/0 (socket says 172.21.15.64:41916) 2023-11-24T08:08:03.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b07fff640 1 -- 172.21.15.64:0/1069616662 learned_addr learned my addr 172.21.15.64:0/1069616662 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b077fe640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b0806c9d0 0x7f0b08156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0afffff640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b080b35a0 0x7f0b081f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b07fff640 1 -- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b080b35a0 msgr2=0x7f0b081f0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b07fff640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b080b35a0 0x7f0b081f0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b07fff640 1 -- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b0806c9d0 msgr2=0x7f0b08156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b07fff640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b0806c9d0 0x7f0b08156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.041+0000 7f0b07fff640 1 -- 172.21.15.64:0/1069616662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af00094d0 con 0x7f0b08144770 2023-11-24T08:08:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.042+0000 7f0b07fff640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b08144770 0x7f0b08139f80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000c780 tx=0x7f0b0000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.042+0000 7f0b04ff9640 1 -- 172.21.15.64:0/1069616662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00004b60 con 0x7f0b08144770 2023-11-24T08:08:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.042+0000 7f0b04ff9640 1 -- 172.21.15.64:0/1069616662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0000ce90 con 0x7f0b08144770 2023-11-24T08:08:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.042+0000 7f0b04ff9640 1 -- 172.21.15.64:0/1069616662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000a850 con 0x7f0b08144770 2023-11-24T08:08:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.042+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0815cc40 con 0x7f0b08144770 2023-11-24T08:08:03.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.042+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0815d200 con 0x7f0b08144770 2023-11-24T08:08:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.043+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0815d720 con 0x7f0b08144770 2023-11-24T08:08:03.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.044+0000 7f0b04ff9640 1 -- 172.21.15.64:0/1069616662 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0b0000a9b0 con 0x7f0b08144770 2023-11-24T08:08:03.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.046+0000 7f0b04ff9640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0ae806b3e0 0x7f0ae806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.046+0000 7f0b04ff9640 1 -- 172.21.15.64:0/1069616662 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0b0008a5c0 con 0x7f0b08144770 2023-11-24T08:08:03.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.046+0000 7f0b077fe640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0ae806b3e0 0x7f0ae806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.048+0000 7f0b077fe640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0ae806b3e0 0x7f0ae806d8a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0af800a950 tx=0x7f0af8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:03.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.050+0000 7f0b04ff9640 1 -- 172.21.15.64:0/1069616662 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b0815d720 con 0x7f0b08144770 2023-11-24T08:08:03.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.246+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b0815d720 con 0x7f0b08144770 2023-11-24T08:08:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.247+0000 7f0b04ff9640 1 -- 172.21.15.64:0/1069616662 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b0815d720 con 0x7f0b08144770 2023-11-24T08:08:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:08:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0ae806b3e0 msgr2=0x7f0ae806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0ae806b3e0 0x7f0ae806d8a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0af800a950 tx=0x7f0af8008040 comp rx=0 tx=0).stop 2023-11-24T08:08:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b08144770 msgr2=0x7f0b08139f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b08144770 0x7f0b08139f80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000c780 tx=0x7f0b0000cc40 comp rx=0 tx=0).stop 2023-11-24T08:08:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 shutdown_connections 2023-11-24T08:08:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0ae806b3e0 0x7f0ae806d8a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b08144770 0x7f0b08139f80 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b080b35a0 0x7f0b081f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 --2- 172.21.15.64:0/1069616662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b0806c9d0 0x7f0b08156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 >> 172.21.15.64:0/1069616662 conn(0x7f0b080cec10 msgr2=0x7f0b08143330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 shutdown_connections 2023-11-24T08:08:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.250+0000 7f0b0c852640 1 -- 172.21.15.64:0/1069616662 wait complete. 2023-11-24T08:08:03.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:08:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.495+0000 7f86440ad640 1 Processor -- start 2023-11-24T08:08:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.495+0000 7f86440ad640 1 -- start start 2023-11-24T08:08:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.497+0000 7f86440ad640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f863c15b340 0x7f863c0abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.497+0000 7f86440ad640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f863c06dc30 0x7f863c0b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.497+0000 7f86440ad640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c0ac530 con 0x7f863c0cbcb0 2023-11-24T08:08:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.497+0000 7f86440ad640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c0b2080 con 0x7f863c06dc30 2023-11-24T08:08:03.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.497+0000 7f86440ad640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c0b2360 con 0x7f863c15b340 2023-11-24T08:08:03.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.497+0000 7f864264a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f863c06dc30 0x7f863c0b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.498+0000 7f864264a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f863c06dc30 0x7f863c0b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41920/0 (socket says 172.21.15.64:41920) 2023-11-24T08:08:03.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.498+0000 7f864264a640 1 -- 172.21.15.64:0/3446027188 learned_addr learned my addr 172.21.15.64:0/3446027188 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:03.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.498+0000 7f8641e49640 1 --2- 172.21.15.64:0/3446027188 >> v2:172.21.15.91:3300/0 conn(0x7f863c15b340 0x7f863c0abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.498+0000 7f864264a640 1 -- 172.21.15.64:0/3446027188 >> v2:172.21.15.91:3300/0 conn(0x7f863c15b340 msgr2=0x7f863c0abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.498+0000 7f864264a640 1 --2- 172.21.15.64:0/3446027188 >> v2:172.21.15.91:3300/0 conn(0x7f863c15b340 0x7f863c0abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.498+0000 7f864264a640 1 -- 172.21.15.64:0/3446027188 >> v1:172.21.15.91:6789/0 conn(0x7f863c0cbcb0 legacy=0x7f863c06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.498+0000 7f864264a640 1 -- 172.21.15.64:0/3446027188 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0b2640 con 0x7f863c06dc30 2023-11-24T08:08:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.499+0000 7f864264a640 1 --2- 172.21.15.64:0/3446027188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c06dc30 0x7f863c0b3120 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f863c0b1e40 tx=0x7f863402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21190d566311821b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.499+0000 7f8640e47640 1 -- 172.21.15.64:0/3446027188 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863402ca80 con 0x7f863c06dc30 2023-11-24T08:08:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.499+0000 7f8640e47640 1 -- 172.21.15.64:0/3446027188 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863402cbe0 con 0x7f863c06dc30 2023-11-24T08:08:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.499+0000 7f8640e47640 1 -- 172.21.15.64:0/3446027188 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340343f0 con 0x7f863c06dc30 2023-11-24T08:08:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.500+0000 7f86440ad640 1 -- 172.21.15.64:0/3446027188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c06dc30 msgr2=0x7f863c0b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.500+0000 7f86440ad640 1 --2- 172.21.15.64:0/3446027188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c06dc30 0x7f863c0b3120 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f863c0b1e40 tx=0x7f863402bee0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.500+0000 7f86440ad640 1 -- 172.21.15.64:0/3446027188 shutdown_connections 2023-11-24T08:08:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.500+0000 7f86440ad640 1 --2- 172.21.15.64:0/3446027188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c06dc30 0x7f863c0b3120 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.500+0000 7f86440ad640 1 --2- 172.21.15.64:0/3446027188 >> v2:172.21.15.91:3300/0 conn(0x7f863c15b340 0x7f863c0abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.500+0000 7f86440ad640 1 -- 172.21.15.64:0/3446027188 >> 172.21.15.64:0/3446027188 conn(0x7f863c0b0d80 msgr2=0x7f863c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.500+0000 7f86440ad640 1 -- 172.21.15.64:0/3446027188 shutdown_connections 2023-11-24T08:08:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.500+0000 7f86440ad640 1 -- 172.21.15.64:0/3446027188 wait complete. 2023-11-24T08:08:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.502+0000 7f86440ad640 1 Processor -- start 2023-11-24T08:08:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.502+0000 7f86440ad640 1 -- start start 2023-11-24T08:08:03.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.503+0000 7f86440ad640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f863c06dc30 0x7f863c15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.503+0000 7f86440ad640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f863c0cbcb0 0x7f863c15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.503+0000 7f86440ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c15b340 0x7f863c151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.503+0000 7f86440ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c151400 con 0x7f863c15b340 2023-11-24T08:08:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.503+0000 7f86440ad640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c0cc0e0 con 0x7f863c0cbcb0 2023-11-24T08:08:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.503+0000 7f86440ad640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c0cc380 con 0x7f863c06dc30 2023-11-24T08:08:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c15b340 0x7f863c151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c15b340 0x7f863c151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41932/0 (socket says 172.21.15.64:41932) 2023-11-24T08:08:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 -- 172.21.15.64:0/1367392535 learned_addr learned my addr 172.21.15.64:0/1367392535 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 -- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f863c06dc30 msgr2=0x7f863c15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f863c06dc30 0x7f863c15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 -- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f863c0cbcb0 msgr2=0x7f863c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f863c0cbcb0 0x7f863c15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 -- 172.21.15.64:0/1367392535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634009b30 con 0x7f863c15b340 2023-11-24T08:08:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.504+0000 7f864264a640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c15b340 0x7f863c151ff0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8634036040 tx=0x7f8634004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.505+0000 7f86327fc640 1 -- 172.21.15.64:0/1367392535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634009510 con 0x7f863c15b340 2023-11-24T08:08:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.505+0000 7f86327fc640 1 -- 172.21.15.64:0/1367392535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863403e450 con 0x7f863c15b340 2023-11-24T08:08:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.505+0000 7f86327fc640 1 -- 172.21.15.64:0/1367392535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863403e7e0 con 0x7f863c15b340 2023-11-24T08:08:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.505+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c0cc660 con 0x7f863c15b340 2023-11-24T08:08:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.505+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c0ccc20 con 0x7f863c15b340 2023-11-24T08:08:03.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.506+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c074770 con 0x7f863c15b340 2023-11-24T08:08:03.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.507+0000 7f86327fc640 1 -- 172.21.15.64:0/1367392535 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f863402cd50 con 0x7f863c15b340 2023-11-24T08:08:03.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.508+0000 7f86327fc640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f862006b410 0x7f862006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:03.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.509+0000 7f86327fc640 1 -- 172.21.15.64:0/1367392535 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f86340ac8a0 con 0x7f863c15b340 2023-11-24T08:08:03.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.509+0000 7f8641e49640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f862006b410 0x7f862006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:03.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.510+0000 7f8641e49640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f862006b410 0x7f862006d8d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8638006fd0 tx=0x7f8638008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:03.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.513+0000 7f86327fc640 1 -- 172.21.15.64:0/1367392535 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8634077130 con 0x7f863c15b340 2023-11-24T08:08:03.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.719+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f863c06e050 con 0x7f863c15b340 2023-11-24T08:08:03.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.720+0000 7f86327fc640 1 -- 172.21.15.64:0/1367392535 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8634033e80 con 0x7f863c15b340 2023-11-24T08:08:03.721 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:08:03.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f862006b410 msgr2=0x7f862006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f862006b410 0x7f862006d8d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8638006fd0 tx=0x7f8638008040 comp rx=0 tx=0).stop 2023-11-24T08:08:03.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c15b340 msgr2=0x7f863c151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c15b340 0x7f863c151ff0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8634036040 tx=0x7f8634004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 shutdown_connections 2023-11-24T08:08:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f862006b410 0x7f862006d8d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863c15b340 0x7f863c151ff0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f863c0cbcb0 0x7f863c15dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 --2- 172.21.15.64:0/1367392535 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f863c06dc30 0x7f863c15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 >> 172.21.15.64:0/1367392535 conn(0x7f863c0b0d80 msgr2=0x7f863c06c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 shutdown_connections 2023-11-24T08:08:03.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:03.722+0000 7f86440ad640 1 -- 172.21.15.64:0/1367392535 wait complete. 2023-11-24T08:08:03.744 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi091.front.sepia.ceph.com... 2023-11-24T08:08:03.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:03.745 DEBUG:teuthology.orchestra.run.smithi091:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T08:08:03.846 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[97915]: fuse finished with error 0 and tester_r 0 2023-11-24T08:08:03.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:08:03.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:03.851 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:08:03.902 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:08:03.903 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:03.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:08:03.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:08:03.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:03.965 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T08:08:04.026 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T08:08:04.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:08:04.027 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:08:04.027 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:04.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:08:04.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:08:04.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:04.088 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T08:08:04.141 INFO:teuthology.orchestra.run.smithi091.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T08:08:04.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:08:04.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:04.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.375+0000 7f822b3b3640 1 Processor -- start 2023-11-24T08:08:04.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.375+0000 7f822b3b3640 1 -- start start 2023-11-24T08:08:04.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.377+0000 7f822b3b3640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f82240d14a0 0x7f82240d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.377+0000 7f822b3b3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82240b0640 0x7f82240b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.377+0000 7f822b3b3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82240cf0c0 con 0x7f822415d5b0 2023-11-24T08:08:04.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.377+0000 7f822b3b3640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82240b01b0 con 0x7f82240b0640 2023-11-24T08:08:04.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.377+0000 7f822b3b3640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822415d990 con 0x7f82240d14a0 2023-11-24T08:08:04.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.377+0000 7f822abb2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82240b0640 0x7f82240b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:04.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.377+0000 7f822abb2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82240b0640 0x7f82240b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41944/0 (socket says 172.21.15.64:41944) 2023-11-24T08:08:04.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.377+0000 7f822abb2640 1 -- 172.21.15.64:0/724686945 learned_addr learned my addr 172.21.15.64:0/724686945 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:04.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.378+0000 7f8229bb0640 1 --2- 172.21.15.64:0/724686945 >> v2:172.21.15.91:3300/0 conn(0x7f82240d14a0 0x7f82240d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:04.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.378+0000 7f822abb2640 1 -- 172.21.15.64:0/724686945 >> v2:172.21.15.91:3300/0 conn(0x7f82240d14a0 msgr2=0x7f82240d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:04.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.378+0000 7f822abb2640 1 --2- 172.21.15.64:0/724686945 >> v2:172.21.15.91:3300/0 conn(0x7f82240d14a0 0x7f82240d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.378+0000 7f822abb2640 1 -- 172.21.15.64:0/724686945 >> v1:172.21.15.91:6789/0 conn(0x7f822415d5b0 legacy=0x7f82240d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:04.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.378+0000 7f822abb2640 1 -- 172.21.15.64:0/724686945 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240d1e40 con 0x7f82240b0640 2023-11-24T08:08:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.379+0000 7f822abb2640 1 --2- 172.21.15.64:0/724686945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82240b0ad0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8224161000 tx=0x7f821c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcaead1c6d21b70 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.379+0000 7f82293af640 1 -- 172.21.15.64:0/724686945 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c02ca80 con 0x7f82240b0640 2023-11-24T08:08:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f82293af640 1 -- 172.21.15.64:0/724686945 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c02cbe0 con 0x7f82240b0640 2023-11-24T08:08:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f82293af640 1 -- 172.21.15.64:0/724686945 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c0343f0 con 0x7f82240b0640 2023-11-24T08:08:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f822b3b3640 1 -- 172.21.15.64:0/724686945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 msgr2=0x7f82240b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f822b3b3640 1 --2- 172.21.15.64:0/724686945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82240b0ad0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8224161000 tx=0x7f821c02bee0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f822b3b3640 1 -- 172.21.15.64:0/724686945 shutdown_connections 2023-11-24T08:08:04.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f822b3b3640 1 --2- 172.21.15.64:0/724686945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82240b0ad0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f822b3b3640 1 --2- 172.21.15.64:0/724686945 >> v2:172.21.15.91:3300/0 conn(0x7f82240d14a0 0x7f82240d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f822b3b3640 1 -- 172.21.15.64:0/724686945 >> 172.21.15.64:0/724686945 conn(0x7f822409c3b0 msgr2=0x7f82240d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:04.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f822b3b3640 1 -- 172.21.15.64:0/724686945 shutdown_connections 2023-11-24T08:08:04.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.380+0000 7f822b3b3640 1 -- 172.21.15.64:0/724686945 wait complete. 2023-11-24T08:08:04.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.382+0000 7f822b3b3640 1 Processor -- start 2023-11-24T08:08:04.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.382+0000 7f822b3b3640 1 -- start start 2023-11-24T08:08:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.383+0000 7f822b3b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82241f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.383+0000 7f822b3b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f82240d14a0 0x7f82240b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.383+0000 7f822b3b3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f822415d5b0 0x7f82240b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822b3b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240b7720 con 0x7f82240b0640 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822b3b3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240b6b10 con 0x7f822415d5b0 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822b3b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82240b6df0 con 0x7f82240d14a0 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822a3b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82241f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822a3b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82241f2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41960/0 (socket says 172.21.15.64:41960) 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822a3b1640 1 -- 172.21.15.64:0/4270882866 learned_addr learned my addr 172.21.15.64:0/4270882866 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822a3b1640 1 -- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f82240d14a0 msgr2=0x7f82240b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822a3b1640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f82240d14a0 0x7f82240b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822a3b1640 1 -- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f822415d5b0 msgr2=0x7f82240b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822a3b1640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f822415d5b0 0x7f82240b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.384+0000 7f822a3b1640 1 -- 172.21.15.64:0/4270882866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c009b30 con 0x7f82240b0640 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.385+0000 7f822a3b1640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82241f2300 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f822000b3f0 tx=0x7f822000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.385+0000 7f821affd640 1 -- 172.21.15.64:0/4270882866 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220011830 con 0x7f82240b0640 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.385+0000 7f821affd640 1 -- 172.21.15.64:0/4270882866 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8220011990 con 0x7f82240b0640 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.385+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82240b70d0 con 0x7f82240b0640 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.385+0000 7f821affd640 1 -- 172.21.15.64:0/4270882866 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220011d40 con 0x7f82240b0640 2023-11-24T08:08:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.385+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82241f2840 con 0x7f82240b0640 2023-11-24T08:08:04.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.386+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82241f2a60 con 0x7f82240b0640 2023-11-24T08:08:04.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.387+0000 7f821affd640 1 -- 172.21.15.64:0/4270882866 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f822001a400 con 0x7f82240b0640 2023-11-24T08:08:04.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.388+0000 7f821affd640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f820806b3e0 0x7f820806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.388+0000 7f821affd640 1 -- 172.21.15.64:0/4270882866 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8220089ec0 con 0x7f82240b0640 2023-11-24T08:08:04.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.389+0000 7f8229bb0640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f820806b3e0 0x7f820806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:04.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.390+0000 7f8229bb0640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f820806b3e0 0x7f820806d8a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8214006fd0 tx=0x7f8214008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:04.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.392+0000 7f821affd640 1 -- 172.21.15.64:0/4270882866 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8220054750 con 0x7f82240b0640 2023-11-24T08:08:04.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.627+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82240c76b0 con 0x7f82240b0640 2023-11-24T08:08:04.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.628+0000 7f821affd640 1 -- 172.21.15.64:0/4270882866 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4909 (secure 0 0 0) 0x7f8220015090 con 0x7f82240b0640 2023-11-24T08:08:04.631 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:04.631 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3149637441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3149637441},{"type":"v1","addr":"172.21.15.91:6836","nonce":3149637441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2159744284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2159744284},{"type":"v1","addr":"172.21.15.91:6837","nonce":2159744284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5764,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1010090226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1010090226},{"type":"v1","addr":"172.21.15.64:6835","nonce":1010090226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:07:41.805482+0000","modified":"2023-11-24T08:07:44.865599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6684},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6684":{"gid":6684,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/45948764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":45948764},{"type":"v1","addr":"172.21.15.64:6833","nonce":45948764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:08:04.631 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 44 2023-11-24T08:08:04.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f820806b3e0 msgr2=0x7f820806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f820806b3e0 0x7f820806d8a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8214006fd0 tx=0x7f8214008040 comp rx=0 tx=0).stop 2023-11-24T08:08:04.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 msgr2=0x7f82241f2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82241f2300 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f822000b3f0 tx=0x7f822000b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 shutdown_connections 2023-11-24T08:08:04.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f820806b3e0 0x7f820806d8a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f822415d5b0 0x7f82240b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f82240d14a0 0x7f82240b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 --2- 172.21.15.64:0/4270882866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82240b0640 0x7f82241f2300 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 >> 172.21.15.64:0/4270882866 conn(0x7f822409c3b0 msgr2=0x7f82240d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:04.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 shutdown_connections 2023-11-24T08:08:04.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.631+0000 7f822b3b3640 1 -- 172.21.15.64:0/4270882866 wait complete. 2023-11-24T08:08:04.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:04.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.880+0000 7f6f53fff640 1 Processor -- start 2023-11-24T08:08:04.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.880+0000 7f6f53fff640 1 -- start start 2023-11-24T08:08:04.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.882+0000 7f6f53fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6f5413dfd0 0x7f6f5415be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.882+0000 7f6f53fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6f540cf050 0x7f6f540cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.882+0000 7f6f53fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f540ce7c0 con 0x7f6f5406c130 2023-11-24T08:08:04.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.882+0000 7f6f53fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f540cfa40 con 0x7f6f5413dfd0 2023-11-24T08:08:04.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.882+0000 7f6f53fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f54140900 con 0x7f6f540cf050 2023-11-24T08:08:04.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.882+0000 7f6f52ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6f5413dfd0 0x7f6f5415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:04.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.882+0000 7f6f537fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6f5406c130 0x7f6f5406c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58624/0 (socket says 172.21.15.64:58624) 2023-11-24T08:08:04.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.882+0000 7f6f537fe640 1 -- 172.21.15.64:0/774152846 learned_addr learned my addr 172.21.15.64:0/774152846 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:04.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.883+0000 7f6f527fc640 1 --2- 172.21.15.64:0/774152846 >> v2:172.21.15.91:3300/0 conn(0x7f6f540cf050 0x7f6f540cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:04.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.883+0000 7f6f52ffd640 1 -- 172.21.15.64:0/774152846 >> v2:172.21.15.91:3300/0 conn(0x7f6f540cf050 msgr2=0x7f6f540cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:04.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.883+0000 7f6f52ffd640 1 --2- 172.21.15.64:0/774152846 >> v2:172.21.15.91:3300/0 conn(0x7f6f540cf050 0x7f6f540cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.883+0000 7f6f52ffd640 1 -- 172.21.15.64:0/774152846 >> v1:172.21.15.64:6789/0 conn(0x7f6f5406c130 legacy=0x7f6f5406c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.883+0000 7f6f52ffd640 1 -- 172.21.15.64:0/774152846 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5415fa70 con 0x7f6f5413dfd0 2023-11-24T08:08:04.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.883+0000 7f6f51ffb640 1 -- 172.21.15.64:0/774152846 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306510083 0 0) 0x7f6f540ce7c0 con 0x7f6f5406c130 2023-11-24T08:08:04.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.884+0000 7f6f52ffd640 1 --2- 172.21.15.64:0/774152846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5413dfd0 0x7f6f5415be70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6f5406e070 tx=0x7f6f4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea505956d8f51aa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:04.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.884+0000 7f6f51ffb640 1 -- 172.21.15.64:0/774152846 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44031830 con 0x7f6f5413dfd0 2023-11-24T08:08:04.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.884+0000 7f6f51ffb640 1 -- 172.21.15.64:0/774152846 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f44031990 con 0x7f6f5413dfd0 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.884+0000 7f6f51ffb640 1 -- 172.21.15.64:0/774152846 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44031d20 con 0x7f6f5413dfd0 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.885+0000 7f6f53fff640 1 -- 172.21.15.64:0/774152846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5413dfd0 msgr2=0x7f6f5415be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.885+0000 7f6f53fff640 1 --2- 172.21.15.64:0/774152846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5413dfd0 0x7f6f5415be70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6f5406e070 tx=0x7f6f4402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.885+0000 7f6f53fff640 1 -- 172.21.15.64:0/774152846 shutdown_connections 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.885+0000 7f6f53fff640 1 --2- 172.21.15.64:0/774152846 >> v2:172.21.15.91:3300/0 conn(0x7f6f540cf050 0x7f6f540cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.885+0000 7f6f53fff640 1 --2- 172.21.15.64:0/774152846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5413dfd0 0x7f6f5415be70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.885+0000 7f6f53fff640 1 -- 172.21.15.64:0/774152846 >> 172.21.15.64:0/774152846 conn(0x7f6f5406a860 msgr2=0x7f6f5406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.885+0000 7f6f53fff640 1 -- 172.21.15.64:0/774152846 shutdown_connections 2023-11-24T08:08:04.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.885+0000 7f6f53fff640 1 -- 172.21.15.64:0/774152846 wait complete. 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.887+0000 7f6f53fff640 1 Processor -- start 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.887+0000 7f6f53fff640 1 -- start start 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f53fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5406c130 0x7f6f54071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f53fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f540cf050 0x7f6f541551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f53fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f5413dfd0 0x7f6f541578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f53fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f54071230 con 0x7f6f5406c130 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f53fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f540705b0 con 0x7f6f5413dfd0 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f53fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f54070890 con 0x7f6f540cf050 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5406c130 0x7f6f54071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5406c130 0x7f6f54071650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41972/0 (socket says 172.21.15.64:41972) 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.888+0000 7f6f52ffd640 1 -- 172.21.15.64:0/2354501100 learned_addr learned my addr 172.21.15.64:0/2354501100 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:04.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f52ffd640 1 -- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f540cf050 msgr2=0x7f6f541551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f52ffd640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f540cf050 0x7f6f541551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f52ffd640 1 -- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f5413dfd0 msgr2=0x7f6f541578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f52ffd640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f5413dfd0 0x7f6f541578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f52ffd640 1 -- 172.21.15.64:0/2354501100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f44009400 con 0x7f6f5406c130 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f52ffd640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5406c130 0x7f6f54071650 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6f44000c00 tx=0x7f6f440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f3b7fe640 1 -- 172.21.15.64:0/2354501100 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4403aa60 con 0x7f6f5406c130 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f3b7fe640 1 -- 172.21.15.64:0/2354501100 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4403abc0 con 0x7f6f5406c130 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f3b7fe640 1 -- 172.21.15.64:0/2354501100 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440443f0 con 0x7f6f5406c130 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.889+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f54070b70 con 0x7f6f5406c130 2023-11-24T08:08:04.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.890+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f541556f0 con 0x7f6f5406c130 2023-11-24T08:08:04.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.891+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f54155880 con 0x7f6f5406c130 2023-11-24T08:08:04.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.891+0000 7f6f3b7fe640 1 -- 172.21.15.64:0/2354501100 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6f4404d450 con 0x7f6f5406c130 2023-11-24T08:08:04.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.892+0000 7f6f3b7fe640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b3e0 0x7f6f4006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:04.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.893+0000 7f6f3b7fe640 1 -- 172.21.15.64:0/2354501100 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6f440ac3e0 con 0x7f6f5406c130 2023-11-24T08:08:04.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.893+0000 7f6f527fc640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b3e0 0x7f6f4006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:04.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.894+0000 7f6f527fc640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b3e0 0x7f6f4006d8a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6f5413aba0 tx=0x7f6f48008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:04.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:04.897+0000 7f6f3b7fe640 1 -- 172.21.15.64:0/2354501100 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f44076a00 con 0x7f6f5406c130 2023-11-24T08:08:05.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.091+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f5404e840 con 0x7f6f5406c130 2023-11-24T08:08:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.093+0000 7f6f3b7fe640 1 -- 172.21.15.64:0/2354501100 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16630 (secure 0 0 0) 0x7f6f44035020 con 0x7f6f5406c130 2023-11-24T08:08:05.095 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:05.095 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:01.452465+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:07:39.474762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:07:40.829840+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.095+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b3e0 msgr2=0x7f6f4006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.095+0000 7f6f53fff640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b3e0 0x7f6f4006d8a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6f5413aba0 tx=0x7f6f48008040 comp rx=0 tx=0).stop 2023-11-24T08:08:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.095+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5406c130 msgr2=0x7f6f54071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.095+0000 7f6f53fff640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5406c130 0x7f6f54071650 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6f44000c00 tx=0x7f6f440046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.096+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 shutdown_connections 2023-11-24T08:08:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.096+0000 7f6f53fff640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b3e0 0x7f6f4006d8a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.096+0000 7f6f53fff640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f5413dfd0 0x7f6f541578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.096+0000 7f6f53fff640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f540cf050 0x7f6f541551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.096+0000 7f6f53fff640 1 --2- 172.21.15.64:0/2354501100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5406c130 0x7f6f54071650 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.096+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 >> 172.21.15.64:0/2354501100 conn(0x7f6f5406a860 msgr2=0x7f6f5413f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:05.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.096+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 shutdown_connections 2023-11-24T08:08:05.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.096+0000 7f6f53fff640 1 -- 172.21.15.64:0/2354501100 wait complete. 2023-11-24T08:08:05.117 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:07:39.474762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:07:40.829840+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:05.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.337+0000 7f3312abb640 1 Processor -- start 2023-11-24T08:08:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.337+0000 7f3312abb640 1 -- start start 2023-11-24T08:08:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.338+0000 7f3312abb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 0x7f330c06bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.338+0000 7f3312abb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f330c06c180 0x7f330c0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.338+0000 7f3312abb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c15dc90 con 0x7f330c15d300 2023-11-24T08:08:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.338+0000 7f3312abb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330c15e910 con 0x7f330c06c180 2023-11-24T08:08:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.338+0000 7f3312abb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c06c610 con 0x7f330c15d6e0 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.339+0000 7f33122ba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f330c06c180 0x7f330c0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.339+0000 7f3311ab9640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f330c15d300 0x7f330c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58630/0 (socket says 172.21.15.64:58630) 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.339+0000 7f3311ab9640 1 -- 172.21.15.64:0/3809594153 learned_addr learned my addr 172.21.15.64:0/3809594153 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.339+0000 7f33112b8640 1 --2- 172.21.15.64:0/3809594153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 0x7f330c06bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.340+0000 7f33122ba640 1 -- 172.21.15.64:0/3809594153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 msgr2=0x7f330c06bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.340+0000 7f33122ba640 1 --2- 172.21.15.64:0/3809594153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 0x7f330c06bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.340+0000 7f3310ab7640 1 -- 172.21.15.64:0/3809594153 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941312421 0 0) 0x7f330c15dc90 con 0x7f330c15d300 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.340+0000 7f33122ba640 1 -- 172.21.15.64:0/3809594153 >> v1:172.21.15.64:6789/0 conn(0x7f330c15d300 legacy=0x7f330c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.340+0000 7f33122ba640 1 -- 172.21.15.64:0/3809594153 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c15df70 con 0x7f330c06c180 2023-11-24T08:08:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.341+0000 7f33122ba640 1 --2- 172.21.15.64:0/3809594153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c06c180 0x7f330c0b0490 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f33040097a0 tx=0x7f330402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9ef06e9afaac19 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.341+0000 7f3310ab7640 1 -- 172.21.15.64:0/3809594153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304031780 con 0x7f330c06c180 2023-11-24T08:08:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.341+0000 7f3310ab7640 1 -- 172.21.15.64:0/3809594153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33040318e0 con 0x7f330c06c180 2023-11-24T08:08:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.341+0000 7f3310ab7640 1 -- 172.21.15.64:0/3809594153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304031c70 con 0x7f330c06c180 2023-11-24T08:08:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.342+0000 7f3312abb640 1 -- 172.21.15.64:0/3809594153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c06c180 msgr2=0x7f330c0b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.342+0000 7f3312abb640 1 --2- 172.21.15.64:0/3809594153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c06c180 0x7f330c0b0490 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f33040097a0 tx=0x7f330402b820 comp rx=0 tx=0).stop 2023-11-24T08:08:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.342+0000 7f3312abb640 1 -- 172.21.15.64:0/3809594153 shutdown_connections 2023-11-24T08:08:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.342+0000 7f3312abb640 1 --2- 172.21.15.64:0/3809594153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c06c180 0x7f330c0b0490 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.342+0000 7f3312abb640 1 --2- 172.21.15.64:0/3809594153 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 0x7f330c06bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.342+0000 7f3312abb640 1 -- 172.21.15.64:0/3809594153 >> 172.21.15.64:0/3809594153 conn(0x7f330c099a00 msgr2=0x7f330c099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.342+0000 7f3312abb640 1 -- 172.21.15.64:0/3809594153 shutdown_connections 2023-11-24T08:08:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.342+0000 7f3312abb640 1 -- 172.21.15.64:0/3809594153 wait complete. 2023-11-24T08:08:05.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.344+0000 7f3312abb640 1 Processor -- start 2023-11-24T08:08:05.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.344+0000 7f3312abb640 1 -- start start 2023-11-24T08:08:05.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.345+0000 7f3312abb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f330c06c180 0x7f330c09e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.345+0000 7f3312abb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c15d300 0x7f330c1f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.345+0000 7f3312abb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 0x7f330c09c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.345+0000 7f3312abb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c09b530 con 0x7f330c15d300 2023-11-24T08:08:05.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.345+0000 7f3312abb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c09e7f0 con 0x7f330c06c180 2023-11-24T08:08:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.345+0000 7f3312abb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c09b7b0 con 0x7f330c15d6e0 2023-11-24T08:08:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33112b8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c15d300 0x7f330c1f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33112b8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c15d300 0x7f330c1f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41986/0 (socket says 172.21.15.64:41986) 2023-11-24T08:08:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33112b8640 1 -- 172.21.15.64:0/1785429068 learned_addr learned my addr 172.21.15.64:0/1785429068 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f3311ab9640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f330c06c180 0x7f330c09e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33112b8640 1 -- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 msgr2=0x7f330c09c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33122ba640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 0x7f330c09c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33112b8640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 0x7f330c09c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33112b8640 1 -- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f330c06c180 msgr2=0x7f330c09e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33112b8640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f330c06c180 0x7f330c09e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f33112b8640 1 -- 172.21.15.64:0/1785429068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304009400 con 0x7f330c15d300 2023-11-24T08:08:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.346+0000 7f3311ab9640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f330c06c180 0x7f330c09e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.347+0000 7f33112b8640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c15d300 0x7f330c1f3c60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f330000e9b0 tx=0x7f330000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.347+0000 7f330a7fc640 1 -- 172.21.15.64:0/1785429068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300009670 con 0x7f330c15d300 2023-11-24T08:08:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.347+0000 7f330a7fc640 1 -- 172.21.15.64:0/1785429068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330000c9a0 con 0x7f330c15d300 2023-11-24T08:08:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.347+0000 7f330a7fc640 1 -- 172.21.15.64:0/1785429068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330000cd50 con 0x7f330c15d300 2023-11-24T08:08:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.347+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c09ba90 con 0x7f330c15d300 2023-11-24T08:08:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.347+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c09c050 con 0x7f330c15d300 2023-11-24T08:08:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.348+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c04e840 con 0x7f330c15d300 2023-11-24T08:08:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.349+0000 7f330a7fc640 1 -- 172.21.15.64:0/1785429068 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f3300013470 con 0x7f330c15d300 2023-11-24T08:08:05.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.350+0000 7f330a7fc640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f32f806b410 0x7f32f806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.350+0000 7f330a7fc640 1 -- 172.21.15.64:0/1785429068 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f330008a320 con 0x7f330c15d300 2023-11-24T08:08:05.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.351+0000 7f3311ab9640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f32f806b410 0x7f32f806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.353+0000 7f3311ab9640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f32f806b410 0x7f32f806d8d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0012e0 tx=0x7f32fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.355+0000 7f330a7fc640 1 -- 172.21.15.64:0/1785429068 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3300054b70 con 0x7f330c15d300 2023-11-24T08:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.548+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f330c0b1530 con 0x7f330c15d300 2023-11-24T08:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.549+0000 7f330a7fc640 1 -- 172.21.15.64:0/1785429068 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3300058190 con 0x7f330c15d300 2023-11-24T08:08:05.551 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:08:05.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f32f806b410 msgr2=0x7f32f806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f32f806b410 0x7f32f806d8d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0012e0 tx=0x7f32fc009040 comp rx=0 tx=0).stop 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c15d300 msgr2=0x7f330c1f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c15d300 0x7f330c1f3c60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f330000e9b0 tx=0x7f330000ee70 comp rx=0 tx=0).stop 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 shutdown_connections 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f32f806b410 0x7f32f806d8d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f330c15d6e0 0x7f330c09c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f330c15d300 0x7f330c1f3c60 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 --2- 172.21.15.64:0/1785429068 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f330c06c180 0x7f330c09e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.551+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 >> 172.21.15.64:0/1785429068 conn(0x7f330c099a00 msgr2=0x7f330c15c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.552+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 shutdown_connections 2023-11-24T08:08:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.552+0000 7f3312abb640 1 -- 172.21.15.64:0/1785429068 wait complete. 2023-11-24T08:08:05.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:08:05.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.796+0000 7ffa4f59e640 1 Processor -- start 2023-11-24T08:08:05.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.796+0000 7ffa4f59e640 1 -- start start 2023-11-24T08:08:05.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.798+0000 7ffa4f59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ffa5006d0e0 0x7ffa50071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.798+0000 7ffa4f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa500cdf70 con 0x7ffa50070dd0 2023-11-24T08:08:05.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.798+0000 7ffa4f59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa500ce230 con 0x7ffa5015c1c0 2023-11-24T08:08:05.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.798+0000 7ffa4f59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5006d4f0 con 0x7ffa5006d0e0 2023-11-24T08:08:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.799+0000 7ffa47fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffa50070dd0 0x7ffa5006c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58646/0 (socket says 172.21.15.64:58646) 2023-11-24T08:08:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.799+0000 7ffa47fff640 1 -- 172.21.15.64:0/3348310710 learned_addr learned my addr 172.21.15.64:0/3348310710 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.799+0000 7ffa4ed9d640 1 --2- 172.21.15.64:0/3348310710 >> v2:172.21.15.91:3300/0 conn(0x7ffa5006d0e0 0x7ffa50071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.799+0000 7ffa4dd9b640 1 -- 172.21.15.64:0/3348310710 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959766686 0 0) 0x7ffa500cdf70 con 0x7ffa50070dd0 2023-11-24T08:08:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.799+0000 7ffa4dd9b640 1 -- 172.21.15.64:0/3348310710 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa3c003520 con 0x7ffa50070dd0 2023-11-24T08:08:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.800+0000 7ffa4ed9d640 1 -- 172.21.15.64:0/3348310710 >> v1:172.21.15.91:6789/0 conn(0x7ffa5015c1c0 legacy=0x7ffa500ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.800+0000 7ffa4ed9d640 1 -- 172.21.15.64:0/3348310710 >> v1:172.21.15.64:6789/0 conn(0x7ffa50070dd0 legacy=0x7ffa5006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.800+0000 7ffa4ed9d640 1 -- 172.21.15.64:0/3348310710 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa500ce510 con 0x7ffa5006d0e0 2023-11-24T08:08:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.801+0000 7ffa4ed9d640 1 --2- 172.21.15.64:0/3348310710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5006d0e0 0x7ffa50071260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009d90 tx=0x7ffa4802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336a7c4e5a6513f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.801+0000 7ffa4dd9b640 1 -- 172.21.15.64:0/3348310710 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4802ccf0 con 0x7ffa5006d0e0 2023-11-24T08:08:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.801+0000 7ffa4dd9b640 1 -- 172.21.15.64:0/3348310710 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4802ce50 con 0x7ffa5006d0e0 2023-11-24T08:08:05.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.801+0000 7ffa4dd9b640 1 -- 172.21.15.64:0/3348310710 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48034660 con 0x7ffa5006d0e0 2023-11-24T08:08:05.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.802+0000 7ffa4f59e640 1 -- 172.21.15.64:0/3348310710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5006d0e0 msgr2=0x7ffa50071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.802+0000 7ffa4f59e640 1 --2- 172.21.15.64:0/3348310710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5006d0e0 0x7ffa50071260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009d90 tx=0x7ffa4802bf40 comp rx=0 tx=0).stop 2023-11-24T08:08:05.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.802+0000 7ffa4f59e640 1 -- 172.21.15.64:0/3348310710 shutdown_connections 2023-11-24T08:08:05.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.802+0000 7ffa4f59e640 1 --2- 172.21.15.64:0/3348310710 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5006d0e0 0x7ffa50071260 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.802+0000 7ffa4f59e640 1 -- 172.21.15.64:0/3348310710 >> 172.21.15.64:0/3348310710 conn(0x7ffa500b15c0 msgr2=0x7ffa500b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:05.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.802+0000 7ffa4f59e640 1 -- 172.21.15.64:0/3348310710 shutdown_connections 2023-11-24T08:08:05.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.802+0000 7ffa4f59e640 1 -- 172.21.15.64:0/3348310710 wait complete. 2023-11-24T08:08:05.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.804+0000 7ffa4f59e640 1 Processor -- start 2023-11-24T08:08:05.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.804+0000 7ffa4f59e640 1 -- start start 2023-11-24T08:08:05.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa4f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffa5006d0e0 0x7ffa5013a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa4f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa50070dd0 0x7ffa5013c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa4f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5015c1c0 0x7ffa5014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa4f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5013ad60 con 0x7ffa50070dd0 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa4f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5013b600 con 0x7ffa5015c1c0 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa4f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5013b8e0 con 0x7ffa5006d0e0 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa50070dd0 0x7ffa5013c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa50070dd0 0x7ffa5013c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41988/0 (socket says 172.21.15.64:41988) 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.805+0000 7ffa47fff640 1 -- 172.21.15.64:0/1789304726 learned_addr learned my addr 172.21.15.64:0/1789304726 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa4ed9d640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5015c1c0 0x7ffa5014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa47fff640 1 -- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffa5006d0e0 msgr2=0x7ffa5013a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa47fff640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffa5006d0e0 0x7ffa5013a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa47fff640 1 -- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5015c1c0 msgr2=0x7ffa5014ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa47fff640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5015c1c0 0x7ffa5014ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa47fff640 1 -- 172.21.15.64:0/1789304726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa48009a70 con 0x7ffa50070dd0 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa47fff640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa50070dd0 0x7ffa5013c6a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800c500 tx=0x7ffa3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa46ffd640 1 -- 172.21.15.64:0/1789304726 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38005ce0 con 0x7ffa50070dd0 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa46ffd640 1 -- 172.21.15.64:0/1789304726 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa38005e40 con 0x7ffa50070dd0 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.806+0000 7ffa46ffd640 1 -- 172.21.15.64:0/1789304726 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38007330 con 0x7ffa50070dd0 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.807+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5013bbc0 con 0x7ffa50070dd0 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.807+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5013c180 con 0x7ffa50070dd0 2023-11-24T08:08:05.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.808+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5006a5f0 con 0x7ffa50070dd0 2023-11-24T08:08:05.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.809+0000 7ffa46ffd640 1 -- 172.21.15.64:0/1789304726 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ffa38007550 con 0x7ffa50070dd0 2023-11-24T08:08:05.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.810+0000 7ffa46ffd640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa3006b3e0 0x7ffa3006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:05.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.810+0000 7ffa46ffd640 1 -- 172.21.15.64:0/1789304726 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ffa380896d0 con 0x7ffa50070dd0 2023-11-24T08:08:05.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.810+0000 7ffa4e59c640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa3006b3e0 0x7ffa3006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:05.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.812+0000 7ffa4e59c640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa3006b3e0 0x7ffa3006d8a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffa40000f60 tx=0x7ffa400079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:05.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:05.814+0000 7ffa46ffd640 1 -- 172.21.15.64:0/1789304726 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa38054020 con 0x7ffa50070dd0 2023-11-24T08:08:06.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.013+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa5004e840 con 0x7ffa50070dd0 2023-11-24T08:08:06.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.013+0000 7ffa46ffd640 1 -- 172.21.15.64:0/1789304726 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa38057640 con 0x7ffa50070dd0 2023-11-24T08:08:06.015 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:08:06.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa3006b3e0 msgr2=0x7ffa3006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa3006b3e0 0x7ffa3006d8a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffa40000f60 tx=0x7ffa400079b0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa50070dd0 msgr2=0x7ffa5013c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa50070dd0 0x7ffa5013c6a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800c500 tx=0x7ffa3800c9c0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 shutdown_connections 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa3006b3e0 0x7ffa3006d8a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa5015c1c0 0x7ffa5014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa50070dd0 0x7ffa5013c6a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 --2- 172.21.15.64:0/1789304726 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffa5006d0e0 0x7ffa5013a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 >> 172.21.15.64:0/1789304726 conn(0x7ffa500b15c0 msgr2=0x7ffa5006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 shutdown_connections 2023-11-24T08:08:06.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.016+0000 7ffa4f59e640 1 -- 172.21.15.64:0/1789304726 wait complete. 2023-11-24T08:08:06.044 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:08:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.245+0000 7fa2f1d8a640 1 Processor -- start 2023-11-24T08:08:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.246+0000 7fa2f1d8a640 1 -- start start 2023-11-24T08:08:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.247+0000 7fa2f1d8a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2ec144770 0x7fa2ec15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.247+0000 7fa2f1d8a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec15f0a0 0x7fa2ec0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.247+0000 7fa2f1d8a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec15f930 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.247+0000 7fa2f1d8a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec0b43b0 con 0x7fa2ec144770 2023-11-24T08:08:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.247+0000 7fa2f1d8a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec15f530 con 0x7fa2ec15f0a0 2023-11-24T08:08:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.248+0000 7fa2f0d88640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2ec144770 0x7fa2ec15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.248+0000 7fa2f0d88640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2ec144770 0x7fa2ec15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42000/0 (socket says 172.21.15.64:42000) 2023-11-24T08:08:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.248+0000 7fa2f0d88640 1 -- 172.21.15.64:0/577134747 learned_addr learned my addr 172.21.15.64:0/577134747 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.248+0000 7fa2f1589640 1 --2- 172.21.15.64:0/577134747 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec15f0a0 0x7fa2ec0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.248+0000 7fa2f0d88640 1 -- 172.21.15.64:0/577134747 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec15f0a0 msgr2=0x7fa2ec0ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.248+0000 7fa2f0d88640 1 --2- 172.21.15.64:0/577134747 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec15f0a0 0x7fa2ec0ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.248+0000 7fa2f0d88640 1 -- 172.21.15.64:0/577134747 >> v1:172.21.15.91:6789/0 conn(0x7fa2ec0b35a0 legacy=0x7fa2ec06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.248+0000 7fa2f0d88640 1 -- 172.21.15.64:0/577134747 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0ce9f0 con 0x7fa2ec144770 2023-11-24T08:08:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.249+0000 7fa2f0d88640 1 --2- 172.21.15.64:0/577134747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec144770 0x7fa2ec15eb60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009e50 tx=0x7fa2dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0f5e564fb8fceb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2eb7fe640 1 -- 172.21.15.64:0/577134747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc02ca70 con 0x7fa2ec144770 2023-11-24T08:08:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2eb7fe640 1 -- 172.21.15.64:0/577134747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc02cbd0 con 0x7fa2ec144770 2023-11-24T08:08:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2eb7fe640 1 -- 172.21.15.64:0/577134747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0343f0 con 0x7fa2ec144770 2023-11-24T08:08:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/577134747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec144770 msgr2=0x7fa2ec15eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/577134747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec144770 0x7fa2ec15eb60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009e50 tx=0x7fa2dc02bed0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/577134747 shutdown_connections 2023-11-24T08:08:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/577134747 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec15f0a0 0x7fa2ec0ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/577134747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec144770 0x7fa2ec15eb60 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.250+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/577134747 >> 172.21.15.64:0/577134747 conn(0x7fa2ec0cec10 msgr2=0x7fa2ec0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.251+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/577134747 shutdown_connections 2023-11-24T08:08:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.251+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/577134747 wait complete. 2023-11-24T08:08:06.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.252+0000 7fa2f1d8a640 1 Processor -- start 2023-11-24T08:08:06.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.253+0000 7fa2f1d8a640 1 -- start start 2023-11-24T08:08:06.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.253+0000 7fa2f1d8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec0b35a0 0x7fa2ec1f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.253+0000 7fa2f1d8a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec144770 0x7fa2ec157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f1d8a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec15f0a0 0x7fa2ec158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f1d8a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec155e80 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f1d8a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec156160 con 0x7fa2ec15f0a0 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f1d8a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec156440 con 0x7fa2ec144770 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f0d88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec0b35a0 0x7fa2ec1f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f0d88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec0b35a0 0x7fa2ec1f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42008/0 (socket says 172.21.15.64:42008) 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f0d88640 1 -- 172.21.15.64:0/4178301688 learned_addr learned my addr 172.21.15.64:0/4178301688 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2ebfff640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec144770 0x7fa2ec157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f1589640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec15f0a0 0x7fa2ec158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f0d88640 1 -- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec144770 msgr2=0x7fa2ec157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f0d88640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec144770 0x7fa2ec157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f0d88640 1 -- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec15f0a0 msgr2=0x7fa2ec158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f0d88640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec15f0a0 0x7fa2ec158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.254+0000 7fa2f0d88640 1 -- 172.21.15.64:0/4178301688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc009b30 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.255+0000 7fa2f0d88640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec0b35a0 0x7fa2ec1f3e90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc000f80 tx=0x7fa2dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.255+0000 7fa2e97fa640 1 -- 172.21.15.64:0/4178301688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc009510 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.255+0000 7fa2e97fa640 1 -- 172.21.15.64:0/4178301688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc03e450 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.255+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec156720 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.255+0000 7fa2e97fa640 1 -- 172.21.15.64:0/4178301688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc03e800 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.255+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec1f43d0 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.256+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec04e840 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.257+0000 7fa2e97fa640 1 -- 172.21.15.64:0/4178301688 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fa2dc03e960 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.258+0000 7fa2e97fa640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa2d806b430 0x7fa2d806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.258+0000 7fa2e97fa640 1 -- 172.21.15.64:0/4178301688 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa2dc0ace10 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.259+0000 7fa2ebfff640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa2d806b430 0x7fa2d806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.260+0000 7fa2ebfff640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa2d806b430 0x7fa2d806d8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0cf510 tx=0x7fa2e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.263+0000 7fa2e97fa640 1 -- 172.21.15.64:0/4178301688 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2dc077670 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.466+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2ec0b2460 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.467+0000 7fa2e97fa640 1 -- 172.21.15.64:0/4178301688 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2dc07ac90 con 0x7fa2ec0b35a0 2023-11-24T08:08:06.469 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:08:06.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa2d806b430 msgr2=0x7fa2d806d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa2d806b430 0x7fa2d806d8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0cf510 tx=0x7fa2e0009040 comp rx=0 tx=0).stop 2023-11-24T08:08:06.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec0b35a0 msgr2=0x7fa2ec1f3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec0b35a0 0x7fa2ec1f3e90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc000f80 tx=0x7fa2dc004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:06.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 shutdown_connections 2023-11-24T08:08:06.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa2d806b430 0x7fa2d806d8f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa2ec15f0a0 0x7fa2ec158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa2ec144770 0x7fa2ec157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 --2- 172.21.15.64:0/4178301688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2ec0b35a0 0x7fa2ec1f3e90 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 >> 172.21.15.64:0/4178301688 conn(0x7fa2ec0cec10 msgr2=0x7fa2ec0afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 shutdown_connections 2023-11-24T08:08:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.470+0000 7fa2f1d8a640 1 -- 172.21.15.64:0/4178301688 wait complete. 2023-11-24T08:08:06.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:06.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.716+0000 7f9941c72640 1 Processor -- start 2023-11-24T08:08:06.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.716+0000 7f9941c72640 1 -- start start 2023-11-24T08:08:06.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.717+0000 7f9941c72640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f993c13dfd0 0x7f993c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.718+0000 7f9941c72640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c0ce7c0 con 0x7f993c0cf050 2023-11-24T08:08:06.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.718+0000 7f9941c72640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c140900 con 0x7f993c0cf500 2023-11-24T08:08:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.718+0000 7f9941c72640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c06cf50 con 0x7f993c13dfd0 2023-11-24T08:08:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.718+0000 7f9933fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f993c0cf050 0x7f993c06c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58652/0 (socket says 172.21.15.64:58652) 2023-11-24T08:08:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.718+0000 7f9933fff640 1 -- 172.21.15.64:0/3579315464 learned_addr learned my addr 172.21.15.64:0/3579315464 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.718+0000 7f9940c70640 1 --2- 172.21.15.64:0/3579315464 >> v2:172.21.15.91:3300/0 conn(0x7f993c13dfd0 0x7f993c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.718+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253718866 0 0) 0x7f993c0ce7c0 con 0x7f993c0cf050 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.718+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9924003520 con 0x7f993c0cf050 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 873644566 0 0) 0x7f9924003520 con 0x7f993c0cf050 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 >> v2:172.21.15.91:3300/0 conn(0x7f993c13dfd0 msgr2=0x7f993c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 --2- 172.21.15.64:0/3579315464 >> v2:172.21.15.91:3300/0 conn(0x7f993c13dfd0 0x7f993c15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 >> v1:172.21.15.91:6789/0 conn(0x7f993c0cf500 legacy=0x7f993c06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c15fa70 con 0x7f993c0cf050 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f992c001670 con 0x7f993c0cf050 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 >> v1:172.21.15.64:6789/0 conn(0x7f993c0cf050 legacy=0x7f993c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9924003a30 0x7f9924003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f9924004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99240051d0 0x7f992400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c06cf50 con 0x7f99240047c0 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992400c100 con 0x7f9924003a30 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992400c2c0 con 0x7f99240051d0 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.719+0000 7f9933fff640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f9924004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.720+0000 7f9940c70640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9924003a30 0x7f9924003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.720+0000 7f9933fff640 1 -- 172.21.15.64:0/3579315464 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99240051d0 msgr2=0x7f992400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.720+0000 7f9933fff640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f99240051d0 0x7f992400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.720+0000 7f9933fff640 1 -- 172.21.15.64:0/3579315464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9924003a30 msgr2=0x7f9924003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.720+0000 7f9933fff640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9924003a30 0x7f9924003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.720+0000 7f9933fff640 1 -- 172.21.15.64:0/3579315464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924003520 con 0x7f99240047c0 2023-11-24T08:08:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.720+0000 7f9941471640 1 -- 172.21.15.64:0/3579315464 reap_dead start 2023-11-24T08:08:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.721+0000 7f9933fff640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f9924004c40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c0e0 tx=0x7f99240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa2d889e7e38fa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.721+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c005ad0 con 0x7f99240047c0 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.721+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c005c50 con 0x7f99240047c0 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.721+0000 7f99337fe640 1 -- 172.21.15.64:0/3579315464 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c02c7e0 con 0x7f99240047c0 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.722+0000 7f9941c72640 1 -- 172.21.15.64:0/3579315464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 msgr2=0x7f9924004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.722+0000 7f9941c72640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f9924004c40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c0e0 tx=0x7f99240057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.722+0000 7f9941c72640 1 -- 172.21.15.64:0/3579315464 shutdown_connections 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.722+0000 7f9941c72640 1 --2- 172.21.15.64:0/3579315464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f9924004c40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.722+0000 7f9941c72640 1 -- 172.21.15.64:0/3579315464 >> 172.21.15.64:0/3579315464 conn(0x7f993c06a860 msgr2=0x7f993c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.722+0000 7f9941c72640 1 -- 172.21.15.64:0/3579315464 shutdown_connections 2023-11-24T08:08:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.722+0000 7f9941c72640 1 -- 172.21.15.64:0/3579315464 wait complete. 2023-11-24T08:08:06.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.724+0000 7f9941c72640 1 Processor -- start 2023-11-24T08:08:06.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.724+0000 7f9941c72640 1 -- start start 2023-11-24T08:08:06.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9941c72640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f993c153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9941c72640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993c154ad0 0x7f993c154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9941c72640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f993c1e65f0 0x7f993c1e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9941c72640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c1e4990 con 0x7f99240047c0 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9941c72640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c155490 con 0x7f993c154ad0 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9941c72640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c1e4ba0 con 0x7f993c1e65f0 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9940c70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f993c153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9940c70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f993c153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42028/0 (socket says 172.21.15.64:42028) 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9940c70640 1 -- 172.21.15.64:0/3988589322 learned_addr learned my addr 172.21.15.64:0/3988589322 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9941471640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f993c1e65f0 0x7f993c1e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9940c70640 1 -- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f993c1e65f0 msgr2=0x7f993c1e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.725+0000 7f9933fff640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993c154ad0 0x7f993c154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f9940c70640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f993c1e65f0 0x7f993c1e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f9940c70640 1 -- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993c154ad0 msgr2=0x7f993c154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f9940c70640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993c154ad0 0x7f993c154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f9940c70640 1 -- 172.21.15.64:0/3988589322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c1e4e80 con 0x7f99240047c0 2023-11-24T08:08:06.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f9941471640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f993c1e65f0 0x7f993c1e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:06.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f9940c70640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f993c153020 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f993800f3d0 tx=0x7f993800f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:06.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f99317fa640 1 -- 172.21.15.64:0/3988589322 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938016620 con 0x7f99240047c0 2023-11-24T08:08:06.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f99317fa640 1 -- 172.21.15.64:0/3988589322 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99380167a0 con 0x7f99240047c0 2023-11-24T08:08:06.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f99317fa640 1 -- 172.21.15.64:0/3988589322 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938016b50 con 0x7f99240047c0 2023-11-24T08:08:06.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c1e5160 con 0x7f99240047c0 2023-11-24T08:08:06.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.726+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c1e6fb0 con 0x7f99240047c0 2023-11-24T08:08:06.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.728+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c04e840 con 0x7f99240047c0 2023-11-24T08:08:06.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.729+0000 7f99317fa640 1 -- 172.21.15.64:0/3988589322 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f9938016cb0 con 0x7f99240047c0 2023-11-24T08:08:06.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.729+0000 7f99317fa640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992806b3e0 0x7f992806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:06.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.730+0000 7f99317fa640 1 -- 172.21.15.64:0/3988589322 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f993808b300 con 0x7f99240047c0 2023-11-24T08:08:06.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.730+0000 7f9933fff640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992806b3e0 0x7f992806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:06.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.731+0000 7f9933fff640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992806b3e0 0x7f992806d8a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f992c034000 tx=0x7f992c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.734+0000 7f99317fa640 1 -- 172.21.15.64:0/3988589322 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9938055a60 con 0x7f99240047c0 2023-11-24T08:08:06.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.966+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f993c0ce9a0 con 0x7f99240047c0 2023-11-24T08:08:06.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.967+0000 7f99317fa640 1 -- 172.21.15.64:0/3988589322 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4909 (secure 0 0 0) 0x7f9938059080 con 0x7f99240047c0 2023-11-24T08:08:06.969 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:06.969 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3149637441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3149637441},{"type":"v1","addr":"172.21.15.91:6836","nonce":3149637441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2159744284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2159744284},{"type":"v1","addr":"172.21.15.91:6837","nonce":2159744284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5764,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1010090226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1010090226},{"type":"v1","addr":"172.21.15.64:6835","nonce":1010090226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:07:41.805482+0000","modified":"2023-11-24T08:07:44.865599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6684},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6684":{"gid":6684,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/45948764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":45948764},{"type":"v1","addr":"172.21.15.64:6833","nonce":45948764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:08:06.969 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 44 2023-11-24T08:08:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992806b3e0 msgr2=0x7f992806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992806b3e0 0x7f992806d8a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f992c034000 tx=0x7f992c009030 comp rx=0 tx=0).stop 2023-11-24T08:08:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 msgr2=0x7f993c153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f993c153020 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f993800f3d0 tx=0x7f993800f890 comp rx=0 tx=0).stop 2023-11-24T08:08:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 shutdown_connections 2023-11-24T08:08:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f992806b3e0 0x7f992806d8a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f993c1e65f0 0x7f993c1e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f993c154ad0 0x7f993c154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 --2- 172.21.15.64:0/3988589322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99240047c0 0x7f993c153020 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.969+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 >> 172.21.15.64:0/3988589322 conn(0x7f993c06a860 msgr2=0x7f993c0cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.970+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 shutdown_connections 2023-11-24T08:08:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:06.970+0000 7f9941c72640 1 -- 172.21.15.64:0/3988589322 wait complete. 2023-11-24T08:08:06.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:07.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.216+0000 7f6ddb59e640 1 Processor -- start 2023-11-24T08:08:07.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.217+0000 7f6ddb59e640 1 -- start start 2023-11-24T08:08:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6ddb59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc162670 0x7f6ddc0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6ddb59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ddc0cf590 0x7f6ddc06c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6ddb59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ddc0cf9a0 con 0x7f6ddc14b1c0 2023-11-24T08:08:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6ddb59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ddc165ca0 con 0x7f6ddc0cf590 2023-11-24T08:08:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6ddb59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc165f80 con 0x7f6ddc162670 2023-11-24T08:08:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6ddad9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ddc0cf590 0x7f6ddc06c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6ddad9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ddc0cf590 0x7f6ddc06c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35372/0 (socket says 172.21.15.64:35372) 2023-11-24T08:08:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6ddad9d640 1 -- 172.21.15.64:0/2113450592 learned_addr learned my addr 172.21.15.64:0/2113450592 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.219+0000 7f6dd3fff640 1 --2- 172.21.15.64:0/2113450592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc162670 0x7f6ddc0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.220+0000 7f6ddad9d640 1 -- 172.21.15.64:0/2113450592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc162670 msgr2=0x7f6ddc0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.220+0000 7f6ddad9d640 1 --2- 172.21.15.64:0/2113450592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc162670 0x7f6ddc0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.220+0000 7f6ddad9d640 1 -- 172.21.15.64:0/2113450592 >> v1:172.21.15.91:6789/0 conn(0x7f6ddc14b1c0 legacy=0x7f6ddc1651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:07.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.220+0000 7f6ddad9d640 1 -- 172.21.15.64:0/2113450592 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc161e70 con 0x7f6ddc0cf590 2023-11-24T08:08:07.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.220+0000 7f6dd3fff640 1 --2- 172.21.15.64:0/2113450592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc162670 0x7f6ddc0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:07.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.221+0000 7f6ddad9d640 1 --2- 172.21.15.64:0/2113450592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc0cf590 0x7f6ddc06c640 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009e50 tx=0x7f6dd402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7842edcb8309a2ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:07.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.221+0000 7f6dd9d9b640 1 -- 172.21.15.64:0/2113450592 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd402c9e0 con 0x7f6ddc0cf590 2023-11-24T08:08:07.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.221+0000 7f6dd9d9b640 1 -- 172.21.15.64:0/2113450592 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd402cb40 con 0x7f6ddc0cf590 2023-11-24T08:08:07.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.221+0000 7f6dd9d9b640 1 -- 172.21.15.64:0/2113450592 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40343f0 con 0x7f6ddc0cf590 2023-11-24T08:08:07.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.222+0000 7f6ddb59e640 1 -- 172.21.15.64:0/2113450592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc0cf590 msgr2=0x7f6ddc06c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.222+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/2113450592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc0cf590 0x7f6ddc06c640 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009e50 tx=0x7f6dd402be40 comp rx=0 tx=0).stop 2023-11-24T08:08:07.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.222+0000 7f6ddb59e640 1 -- 172.21.15.64:0/2113450592 shutdown_connections 2023-11-24T08:08:07.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.222+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/2113450592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc0cf590 0x7f6ddc06c640 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.222+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/2113450592 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc162670 0x7f6ddc0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.222+0000 7f6ddb59e640 1 -- 172.21.15.64:0/2113450592 >> 172.21.15.64:0/2113450592 conn(0x7f6ddc06a860 msgr2=0x7f6ddc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:07.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.222+0000 7f6ddb59e640 1 -- 172.21.15.64:0/2113450592 shutdown_connections 2023-11-24T08:08:07.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.222+0000 7f6ddb59e640 1 -- 172.21.15.64:0/2113450592 wait complete. 2023-11-24T08:08:07.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.224+0000 7f6ddb59e640 1 Processor -- start 2023-11-24T08:08:07.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.225+0000 7f6ddb59e640 1 -- start start 2023-11-24T08:08:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddb59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ddc0cf590 0x7f6ddc0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddb59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc14b1c0 0x7f6ddc1fa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddb59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc162670 0x7f6ddc145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddb59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1fad70 con 0x7f6ddc162670 2023-11-24T08:08:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddb59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1638c0 con 0x7f6ddc0cf590 2023-11-24T08:08:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddb59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1f7240 con 0x7f6ddc14b1c0 2023-11-24T08:08:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc162670 0x7f6ddc145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc162670 0x7f6ddc145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35384/0 (socket says 172.21.15.64:35384) 2023-11-24T08:08:07.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.226+0000 7f6ddad9d640 1 -- 172.21.15.64:0/4247318471 learned_addr learned my addr 172.21.15.64:0/4247318471 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6dd3fff640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc14b1c0 0x7f6ddc1fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6dda59c640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ddc0cf590 0x7f6ddc0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6ddad9d640 1 -- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc14b1c0 msgr2=0x7f6ddc1fa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6ddad9d640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc14b1c0 0x7f6ddc1fa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6ddad9d640 1 -- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ddc0cf590 msgr2=0x7f6ddc0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6ddad9d640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ddc0cf590 0x7f6ddc0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6ddad9d640 1 -- 172.21.15.64:0/4247318471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd4009b30 con 0x7f6ddc162670 2023-11-24T08:08:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6dda59c640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ddc0cf590 0x7f6ddc0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6ddad9d640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc162670 0x7f6ddc145d20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4036040 tx=0x7f6dd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.227+0000 7f6dd2ffd640 1 -- 172.21.15.64:0/4247318471 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd403e450 con 0x7f6ddc162670 2023-11-24T08:08:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.228+0000 7f6dd2ffd640 1 -- 172.21.15.64:0/4247318471 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd403e5d0 con 0x7f6ddc162670 2023-11-24T08:08:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.228+0000 7f6dd2ffd640 1 -- 172.21.15.64:0/4247318471 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd403e980 con 0x7f6ddc162670 2023-11-24T08:08:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.228+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc1faf60 con 0x7f6ddc162670 2023-11-24T08:08:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.228+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc1fb520 con 0x7f6ddc162670 2023-11-24T08:08:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.229+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc163a70 con 0x7f6ddc162670 2023-11-24T08:08:07.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.230+0000 7f6dd2ffd640 1 -- 172.21.15.64:0/4247318471 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6dd403eae0 con 0x7f6ddc162670 2023-11-24T08:08:07.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.231+0000 7f6dd2ffd640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dbc06b3e0 0x7f6dbc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.231+0000 7f6dd2ffd640 1 -- 172.21.15.64:0/4247318471 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6dd40acf20 con 0x7f6ddc162670 2023-11-24T08:08:07.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.232+0000 7f6dda59c640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dbc06b3e0 0x7f6dbc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.233+0000 7f6dda59c640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dbc06b3e0 0x7f6dbc06d8a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00a830 tx=0x7f6dcc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:07.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.236+0000 7f6dd2ffd640 1 -- 172.21.15.64:0/4247318471 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ddc163a70 con 0x7f6ddc162670 2023-11-24T08:08:07.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.435+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ddc163a30 con 0x7f6ddc162670 2023-11-24T08:08:07.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.437+0000 7f6dd2ffd640 1 -- 172.21.15.64:0/4247318471 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16586 (secure 0 0 0) 0x7f6dd4077740 con 0x7f6ddc162670 2023-11-24T08:08:07.439 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:07.440 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":76,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:06.019068+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:07:39.474762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:07:40.829840+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:07.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dbc06b3e0 msgr2=0x7f6dbc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dbc06b3e0 0x7f6dbc06d8a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00a830 tx=0x7f6dcc008040 comp rx=0 tx=0).stop 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc162670 msgr2=0x7f6ddc145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc162670 0x7f6ddc145d20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4036040 tx=0x7f6dd4004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 shutdown_connections 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6dbc06b3e0 0x7f6dbc06d8a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ddc162670 0x7f6ddc145d20 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6ddc14b1c0 0x7f6ddc1fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 --2- 172.21.15.64:0/4247318471 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6ddc0cf590 0x7f6ddc0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 >> 172.21.15.64:0/4247318471 conn(0x7f6ddc06a860 msgr2=0x7f6ddc0b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.440+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 shutdown_connections 2023-11-24T08:08:07.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.441+0000 7f6ddb59e640 1 -- 172.21.15.64:0/4247318471 wait complete. 2023-11-24T08:08:07.463 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:08:07.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:07.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.690+0000 7fef13fff640 1 Processor -- start 2023-11-24T08:08:07.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.690+0000 7fef13fff640 1 -- start start 2023-11-24T08:08:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.692+0000 7fef13fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef1415df80 0x7fef141680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.692+0000 7fef13fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef14161670 con 0x7fef14157770 2023-11-24T08:08:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.692+0000 7fef13fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef14071730 con 0x7fef1406d250 2023-11-24T08:08:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.692+0000 7fef13fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1415e430 con 0x7fef1415df80 2023-11-24T08:08:07.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.692+0000 7fef12ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fef14157770 0x7fef1406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50226/0 (socket says 172.21.15.64:50226) 2023-11-24T08:08:07.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.692+0000 7fef12ffd640 1 -- 172.21.15.64:0/3570685627 learned_addr learned my addr 172.21.15.64:0/3570685627 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:07.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.692+0000 7fef137fe640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef1415df80 0x7fef141680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210084090 0 0) 0x7fef14161670 con 0x7fef14157770 2023-11-24T08:08:07.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feefc003520 con 0x7fef14157770 2023-11-24T08:08:07.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496771123 0 0) 0x7feefc003520 con 0x7fef14157770 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef1415df80 msgr2=0x7fef141680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef1415df80 0x7fef141680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 >> v1:172.21.15.91:6789/0 conn(0x7fef1406d250 legacy=0x7fef1415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1415e610 con 0x7fef14157770 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fef04001670 con 0x7fef14157770 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 >> v1:172.21.15.64:6789/0 conn(0x7fef14157770 legacy=0x7fef1406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef1415df80 0x7feefc003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7feefc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feefc004d00 0x7feefc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1415e430 con 0x7feefc003f90 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc00bae0 con 0x7feefc004d00 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.693+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc00bca0 con 0x7fef1415df80 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.694+0000 7fef127fc640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7feefc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.694+0000 7fef12ffd640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef1415df80 0x7feefc003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.694+0000 7fef137fe640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feefc004d00 0x7feefc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.694+0000 7fef127fc640 1 -- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef1415df80 msgr2=0x7feefc003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.694+0000 7fef127fc640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef1415df80 0x7feefc003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.694+0000 7fef127fc640 1 -- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feefc004d00 msgr2=0x7feefc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.694+0000 7fef127fc640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7feefc004d00 0x7feefc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.694+0000 7fef127fc640 1 -- 172.21.15.64:0/3570685627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc003520 con 0x7feefc003f90 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.695+0000 7fef127fc640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7feefc0047c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fef08001750 tx=0x7feefc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826f6342b4000a0f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080052d0 con 0x7feefc003f90 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0802e050 con 0x7feefc003f90 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef11ffb640 1 -- 172.21.15.64:0/3570685627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08003830 con 0x7feefc003f90 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef13fff640 1 -- 172.21.15.64:0/3570685627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 msgr2=0x7feefc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef13fff640 1 --2- 172.21.15.64:0/3570685627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7feefc0047c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fef08001750 tx=0x7feefc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef137fe640 1 -- 172.21.15.64:0/3570685627 reap_dead start 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef13fff640 1 -- 172.21.15.64:0/3570685627 shutdown_connections 2023-11-24T08:08:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef13fff640 1 -- 172.21.15.64:0/3570685627 >> 172.21.15.64:0/3570685627 conn(0x7fef1406a860 msgr2=0x7fef1406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef13fff640 1 -- 172.21.15.64:0/3570685627 shutdown_connections 2023-11-24T08:08:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.696+0000 7fef13fff640 1 -- 172.21.15.64:0/3570685627 wait complete. 2023-11-24T08:08:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.698+0000 7fef13fff640 1 Processor -- start 2023-11-24T08:08:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.698+0000 7fef13fff640 1 -- start start 2023-11-24T08:08:07.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.699+0000 7fef13fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7fef14140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.699+0000 7fef13fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef1415f940 0x7fef1415fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.699+0000 7fef13fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef14153d90 0x7fef14154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.699+0000 7fef13fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef14160300 con 0x7feefc003f90 2023-11-24T08:08:07.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.699+0000 7fef13fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1414f230 con 0x7fef1415f940 2023-11-24T08:08:07.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.699+0000 7fef13fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1414f510 con 0x7fef14153d90 2023-11-24T08:08:07.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef12ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7fef14140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef12ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7fef14140800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35404/0 (socket says 172.21.15.64:35404) 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef12ffd640 1 -- 172.21.15.64:0/2866570042 learned_addr learned my addr 172.21.15.64:0/2866570042 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef137fe640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef14153d90 0x7fef14154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef127fc640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef1415f940 0x7fef1415fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef12ffd640 1 -- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef14153d90 msgr2=0x7fef14154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef12ffd640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef14153d90 0x7fef14154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef12ffd640 1 -- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef1415f940 msgr2=0x7fef1415fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef12ffd640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef1415f940 0x7fef1415fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.700+0000 7fef12ffd640 1 -- 172.21.15.64:0/2866570042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1414f7f0 con 0x7feefc003f90 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.701+0000 7fef12ffd640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7fef14140800 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fef0400e700 tx=0x7fef0400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.701+0000 7feefb7fe640 1 -- 172.21.15.64:0/2866570042 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0400ee10 con 0x7feefc003f90 2023-11-24T08:08:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.701+0000 7feefb7fe640 1 -- 172.21.15.64:0/2866570042 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0400c8d0 con 0x7feefc003f90 2023-11-24T08:08:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.701+0000 7feefb7fe640 1 -- 172.21.15.64:0/2866570042 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0400cc80 con 0x7feefc003f90 2023-11-24T08:08:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.701+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1414fad0 con 0x7feefc003f90 2023-11-24T08:08:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.701+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef14154780 con 0x7feefc003f90 2023-11-24T08:08:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.702+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1414fff0 con 0x7feefc003f90 2023-11-24T08:08:07.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.703+0000 7feefb7fe640 1 -- 172.21.15.64:0/2866570042 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fef0401d460 con 0x7feefc003f90 2023-11-24T08:08:07.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.704+0000 7feefb7fe640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef0006b410 0x7fef0006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:07.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.704+0000 7feefb7fe640 1 -- 172.21.15.64:0/2866570042 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fef0408a000 con 0x7feefc003f90 2023-11-24T08:08:07.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.705+0000 7fef127fc640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef0006b410 0x7fef0006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.706+0000 7fef127fc640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef0006b410 0x7fef0006d8d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef08033000 tx=0x7fef08009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.708+0000 7feefb7fe640 1 -- 172.21.15.64:0/2866570042 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef1414fff0 con 0x7feefc003f90 2023-11-24T08:08:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.944+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef1414fff0 con 0x7feefc003f90 2023-11-24T08:08:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.945+0000 7feefb7fe640 1 -- 172.21.15.64:0/2866570042 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4909 (secure 0 0 0) 0x7fef1414fff0 con 0x7feefc003f90 2023-11-24T08:08:07.948 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:07.948 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3149637441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3149637441},{"type":"v1","addr":"172.21.15.91:6836","nonce":3149637441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2159744284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2159744284},{"type":"v1","addr":"172.21.15.91:6837","nonce":2159744284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5764,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1010090226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1010090226},{"type":"v1","addr":"172.21.15.64:6835","nonce":1010090226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:07:41.805482+0000","modified":"2023-11-24T08:07:44.865599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6684},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6684":{"gid":6684,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/45948764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":45948764},{"type":"v1","addr":"172.21.15.64:6833","nonce":45948764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:08:07.949 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 44 2023-11-24T08:08:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef0006b410 msgr2=0x7fef0006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef0006b410 0x7fef0006d8d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef08033000 tx=0x7fef08009030 comp rx=0 tx=0).stop 2023-11-24T08:08:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 msgr2=0x7fef14140800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7fef14140800 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fef0400e700 tx=0x7fef0400ebc0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 shutdown_connections 2023-11-24T08:08:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fef0006b410 0x7fef0006d8d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fef14153d90 0x7fef14154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fef1415f940 0x7fef1415fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 --2- 172.21.15.64:0/2866570042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feefc003f90 0x7fef14140800 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.948+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 >> 172.21.15.64:0/2866570042 conn(0x7fef1406a860 msgr2=0x7fef140cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.949+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 shutdown_connections 2023-11-24T08:08:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:07.949+0000 7fef13fff640 1 -- 172.21.15.64:0/2866570042 wait complete. 2023-11-24T08:08:07.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:08.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.191+0000 7f2f4b59e640 1 Processor -- start 2023-11-24T08:08:08.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.191+0000 7f2f4b59e640 1 -- start start 2023-11-24T08:08:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.193+0000 7f2f4b59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2f4c13dfd0 0x7f2f4c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.193+0000 7f2f4b59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf500 0x7f2f4c06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.193+0000 7f2f4b59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c0ce7c0 con 0x7f2f4c0cf050 2023-11-24T08:08:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.193+0000 7f2f4b59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c140900 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.193+0000 7f2f4b59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c15fa70 con 0x7f2f4c0cf500 2023-11-24T08:08:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.193+0000 7f2f4a59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2f4c13dfd0 0x7f2f4c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.194+0000 7f2f4a59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2f4c13dfd0 0x7f2f4c15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35406/0 (socket says 172.21.15.64:35406) 2023-11-24T08:08:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.194+0000 7f2f4a59c640 1 -- 172.21.15.64:0/1821576177 learned_addr learned my addr 172.21.15.64:0/1821576177 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.194+0000 7f2f4ad9d640 1 --2- 172.21.15.64:0/1821576177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf500 0x7f2f4c06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.194+0000 7f2f4a59c640 1 -- 172.21.15.64:0/1821576177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf500 msgr2=0x7f2f4c06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.194+0000 7f2f4a59c640 1 --2- 172.21.15.64:0/1821576177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf500 0x7f2f4c06c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.194+0000 7f2f4a59c640 1 -- 172.21.15.64:0/1821576177 >> v1:172.21.15.91:6789/0 conn(0x7f2f4c0cf050 legacy=0x7f2f4c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.194+0000 7f2f4a59c640 1 -- 172.21.15.64:0/1821576177 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c0cf950 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.194+0000 7f2f4ad9d640 1 --2- 172.21.15.64:0/1821576177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf500 0x7f2f4c06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:08.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.196+0000 7f2f4a59c640 1 --2- 172.21.15.64:0/1821576177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c15be70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009870 tx=0x7f2f3c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6c712475454af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:08.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.196+0000 7f2f4959a640 1 -- 172.21.15.64:0/1821576177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c030840 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4959a640 1 -- 172.21.15.64:0/1821576177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c0309a0 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4959a640 1 -- 172.21.15.64:0/1821576177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c030d30 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1821576177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 msgr2=0x7f2f4c15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1821576177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c15be70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009870 tx=0x7f2f3c02b980 comp rx=0 tx=0).stop 2023-11-24T08:08:08.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1821576177 shutdown_connections 2023-11-24T08:08:08.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1821576177 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf500 0x7f2f4c06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1821576177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c15be70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1821576177 >> 172.21.15.64:0/1821576177 conn(0x7f2f4c06a860 msgr2=0x7f2f4c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:08.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1821576177 shutdown_connections 2023-11-24T08:08:08.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.197+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1821576177 wait complete. 2023-11-24T08:08:08.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.199+0000 7f2f4b59e640 1 Processor -- start 2023-11-24T08:08:08.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.199+0000 7f2f4b59e640 1 -- start start 2023-11-24T08:08:08.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.200+0000 7f2f4b59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf050 0x7f2f4c15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.200+0000 7f2f4b59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0cf500 0x7f2f4c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.200+0000 7f2f4b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.200+0000 7f2f4b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c15d8d0 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.200+0000 7f2f4b59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c158a40 con 0x7f2f4c0cf500 2023-11-24T08:08:08.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.200+0000 7f2f4b59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c15cc50 con 0x7f2f4c0cf050 2023-11-24T08:08:08.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35408/0 (socket says 172.21.15.64:35408) 2023-11-24T08:08:08.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 -- 172.21.15.64:0/1212473608 learned_addr learned my addr 172.21.15.64:0/1212473608 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:08.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4a59c640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf050 0x7f2f4c15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f49d9b640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0cf500 0x7f2f4c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 -- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf050 msgr2=0x7f2f4c15c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf050 0x7f2f4c15c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 -- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0cf500 msgr2=0x7f2f4c158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0cf500 0x7f2f4c158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 -- 172.21.15.64:0/1212473608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0094d0 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f49d9b640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0cf500 0x7f2f4c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.201+0000 7f2f4ad9d640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c155170 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009fa0 tx=0x7f2f4400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.202+0000 7f2f3affd640 1 -- 172.21.15.64:0/1212473608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44003b90 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.202+0000 7f2f3affd640 1 -- 172.21.15.64:0/1212473608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4400f040 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.202+0000 7f2f3affd640 1 -- 172.21.15.64:0/1212473608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44004030 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.202+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c15cf30 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.202+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c15d4f0 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.203+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c15da10 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.204+0000 7f2f3affd640 1 -- 172.21.15.64:0/1212473608 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2f44004190 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.205+0000 7f2f3affd640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2f2c06b3e0 0x7f2f2c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.205+0000 7f2f3affd640 1 -- 172.21.15.64:0/1212473608 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2f4408a4f0 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.206+0000 7f2f4a59c640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2f2c06b3e0 0x7f2f2c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.207+0000 7f2f4a59c640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2f2c06b3e0 0x7f2f2c06d8a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c036000 tx=0x7f2f3c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:08.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.210+0000 7f2f3affd640 1 -- 172.21.15.64:0/1212473608 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f4c15da10 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.444+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f4c15da10 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.446+0000 7f2f3affd640 1 -- 172.21.15.64:0/1212473608 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4909 (secure 0 0 0) 0x7f2f4c15da10 con 0x7f2f4c13dfd0 2023-11-24T08:08:08.448 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:08.448 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3149637441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3149637441},{"type":"v1","addr":"172.21.15.91:6836","nonce":3149637441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2159744284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2159744284},{"type":"v1","addr":"172.21.15.91:6837","nonce":2159744284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5764,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1010090226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1010090226},{"type":"v1","addr":"172.21.15.64:6835","nonce":1010090226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:07:41.805482+0000","modified":"2023-11-24T08:07:44.865599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6684},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6684":{"gid":6684,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/45948764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":45948764},{"type":"v1","addr":"172.21.15.64:6833","nonce":45948764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:08:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 44 2023-11-24T08:08:08.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2f2c06b3e0 msgr2=0x7f2f2c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2f2c06b3e0 0x7f2f2c06d8a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c036000 tx=0x7f2f3c008030 comp rx=0 tx=0).stop 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 msgr2=0x7f2f4c155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c155170 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009fa0 tx=0x7f2f4400d560 comp rx=0 tx=0).stop 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 shutdown_connections 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2f2c06b3e0 0x7f2f2c06d8a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f4c13dfd0 0x7f2f4c155170 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2f4c0cf500 0x7f2f4c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 --2- 172.21.15.64:0/1212473608 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2f4c0cf050 0x7f2f4c15c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 >> 172.21.15.64:0/1212473608 conn(0x7f2f4c06a860 msgr2=0x7f2f4c06ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 shutdown_connections 2023-11-24T08:08:08.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.448+0000 7f2f4b59e640 1 -- 172.21.15.64:0/1212473608 wait complete. 2023-11-24T08:08:08.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.686+0000 7f5793fff640 1 Processor -- start 2023-11-24T08:08:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.686+0000 7f5793fff640 1 -- start start 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.687+0000 7f5793fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 0x7f579415be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.687+0000 7f5793fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f57940cf4d0 0x7f579406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.687+0000 7f5793fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57940ce7c0 con 0x7f57940cf050 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.687+0000 7f5793fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5794140900 con 0x7f57940cf4d0 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.687+0000 7f5793fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579415fa70 con 0x7f579413dfd0 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.687+0000 7f57937fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f57940cf4d0 0x7f579406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f57927fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f57940cf050 0x7f579406c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50238/0 (socket says 172.21.15.64:50238) 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f57927fc640 1 -- 172.21.15.64:0/3914863279 learned_addr learned my addr 172.21.15.64:0/3914863279 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f5792ffd640 1 --2- 172.21.15.64:0/3914863279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 0x7f579415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f5791ffb640 1 -- 172.21.15.64:0/3914863279 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717913967 0 0) 0x7f57940ce7c0 con 0x7f57940cf050 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f5791ffb640 1 -- 172.21.15.64:0/3914863279 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f577c003520 con 0x7f57940cf050 2023-11-24T08:08:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f57937fe640 1 -- 172.21.15.64:0/3914863279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 msgr2=0x7f579415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f57937fe640 1 --2- 172.21.15.64:0/3914863279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 0x7f579415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f57937fe640 1 -- 172.21.15.64:0/3914863279 >> v1:172.21.15.64:6789/0 conn(0x7f57940cf050 legacy=0x7f579406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.688+0000 7f57937fe640 1 -- 172.21.15.64:0/3914863279 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579415ee60 con 0x7f57940cf4d0 2023-11-24T08:08:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.689+0000 7f57937fe640 1 --2- 172.21.15.64:0/3914863279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f579406c840 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f578c009d90 tx=0x7f578c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b54b1a79d85314 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5791ffb640 1 -- 172.21.15.64:0/3914863279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c02cc50 con 0x7f57940cf4d0 2023-11-24T08:08:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5791ffb640 1 -- 172.21.15.64:0/3914863279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c02cdb0 con 0x7f57940cf4d0 2023-11-24T08:08:08.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5791ffb640 1 -- 172.21.15.64:0/3914863279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c034660 con 0x7f57940cf4d0 2023-11-24T08:08:08.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5793fff640 1 -- 172.21.15.64:0/3914863279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 msgr2=0x7f579406c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5793fff640 1 --2- 172.21.15.64:0/3914863279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f579406c840 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f578c009d90 tx=0x7f578c02bea0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5793fff640 1 -- 172.21.15.64:0/3914863279 shutdown_connections 2023-11-24T08:08:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5793fff640 1 --2- 172.21.15.64:0/3914863279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f579406c840 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5793fff640 1 --2- 172.21.15.64:0/3914863279 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 0x7f579415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5793fff640 1 -- 172.21.15.64:0/3914863279 >> 172.21.15.64:0/3914863279 conn(0x7f579406a860 msgr2=0x7f579406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5793fff640 1 -- 172.21.15.64:0/3914863279 shutdown_connections 2023-11-24T08:08:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.690+0000 7f5793fff640 1 -- 172.21.15.64:0/3914863279 wait complete. 2023-11-24T08:08:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.692+0000 7f5793fff640 1 Processor -- start 2023-11-24T08:08:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.692+0000 7f5793fff640 1 -- start start 2023-11-24T08:08:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.693+0000 7f5793fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57940cf050 0x7f57941f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f5793fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f5794070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f5793fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 0x7f57941f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f5793fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57941f4460 con 0x7f57940cf4d0 2023-11-24T08:08:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f5793fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5794155be0 con 0x7f57940cf050 2023-11-24T08:08:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f5793fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5794155ec0 con 0x7f579413dfd0 2023-11-24T08:08:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f57927fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f5794070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f57927fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f5794070dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35432/0 (socket says 172.21.15.64:35432) 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f57927fc640 1 -- 172.21.15.64:0/2552529350 learned_addr learned my addr 172.21.15.64:0/2552529350 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f5792ffd640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57940cf050 0x7f57941f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f57937fe640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 0x7f57941f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f57927fc640 1 -- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 msgr2=0x7f57941f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f57927fc640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 0x7f57941f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.694+0000 7f57927fc640 1 -- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57940cf050 msgr2=0x7f57941f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f57927fc640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57940cf050 0x7f57941f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f57927fc640 1 -- 172.21.15.64:0/2552529350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009a70 con 0x7f57940cf4d0 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f5792ffd640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57940cf050 0x7f57941f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f57927fc640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f5794070dd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f578800c500 tx=0x7f578800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f577b7fe640 1 -- 172.21.15.64:0/2552529350 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788005ce0 con 0x7f57940cf4d0 2023-11-24T08:08:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f577b7fe640 1 -- 172.21.15.64:0/2552529350 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5788005e40 con 0x7f57940cf4d0 2023-11-24T08:08:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f577b7fe640 1 -- 172.21.15.64:0/2552529350 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788007330 con 0x7f57940cf4d0 2023-11-24T08:08:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57941561a0 con 0x7f57940cf4d0 2023-11-24T08:08:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.695+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5794156760 con 0x7f57940cf4d0 2023-11-24T08:08:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.696+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579404e840 con 0x7f57940cf4d0 2023-11-24T08:08:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.697+0000 7f577b7fe640 1 -- 172.21.15.64:0/2552529350 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5788007550 con 0x7f57940cf4d0 2023-11-24T08:08:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.698+0000 7f577b7fe640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f578006b3e0 0x7f578006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.698+0000 7f577b7fe640 1 -- 172.21.15.64:0/2552529350 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f57880896b0 con 0x7f57940cf4d0 2023-11-24T08:08:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.699+0000 7f5792ffd640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f578006b3e0 0x7f578006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.700+0000 7f5792ffd640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f578006b3e0 0x7f578006d8a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f579414f970 tx=0x7f5784008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.702+0000 7f577b7fe640 1 -- 172.21.15.64:0/2552529350 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5788053fe0 con 0x7f57940cf4d0 2023-11-24T08:08:08.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.896+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57940ce9a0 con 0x7f57940cf4d0 2023-11-24T08:08:08.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.898+0000 7f577b7fe640 1 -- 172.21.15.64:0/2552529350 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16586 (secure 0 0 0) 0x7f5788057600 con 0x7f57940cf4d0 2023-11-24T08:08:08.900 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:08.900 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":76,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:06.019068+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:07:39.474762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:07:40.829840+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.900+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f578006b3e0 msgr2=0x7f578006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.900+0000 7f5793fff640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f578006b3e0 0x7f578006d8a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f579414f970 tx=0x7f5784008040 comp rx=0 tx=0).stop 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.900+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 msgr2=0x7f5794070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.900+0000 7f5793fff640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f5794070dd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f578800c500 tx=0x7f578800c9c0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.901+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 shutdown_connections 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.901+0000 7f5793fff640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f578006b3e0 0x7f578006d8a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.901+0000 7f5793fff640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f579413dfd0 0x7f57941f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.901+0000 7f5793fff640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57940cf4d0 0x7f5794070dd0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.901+0000 7f5793fff640 1 --2- 172.21.15.64:0/2552529350 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f57940cf050 0x7f57941f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:08.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.901+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 >> 172.21.15.64:0/2552529350 conn(0x7f579406a860 msgr2=0x7f57941595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.901+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 shutdown_connections 2023-11-24T08:08:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:08.901+0000 7f5793fff640 1 -- 172.21.15.64:0/2552529350 wait complete. 2023-11-24T08:08:08.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:08:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.148+0000 7f1a56002640 1 Processor -- start 2023-11-24T08:08:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.149+0000 7f1a56002640 1 -- start start 2023-11-24T08:08:09.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a56002640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1a501460b0 0x7f1a500b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a56002640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a5006b760 0x7f1a500b3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a56002640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5006cd50 con 0x7f1a500b35b0 2023-11-24T08:08:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a56002640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a50146a90 con 0x7f1a501460b0 2023-11-24T08:08:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a56002640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500b3a40 con 0x7f1a5006b760 2023-11-24T08:08:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a4ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1a500b35b0 0x7f1a50160f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50254/0 (socket says 172.21.15.64:50254) 2023-11-24T08:08:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a4ffff640 1 -- 172.21.15.64:0/1008433809 learned_addr learned my addr 172.21.15.64:0/1008433809 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a4f7fe640 1 --2- 172.21.15.64:0/1008433809 >> v2:172.21.15.91:3300/0 conn(0x7f1a501460b0 0x7f1a500b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a47fff640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a5006b760 0x7f1a500b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871665466 0 0) 0x7f1a5006cd50 con 0x7f1a500b35b0 2023-11-24T08:08:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.151+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a3c003520 con 0x7f1a500b35b0 2023-11-24T08:08:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2405505053 0 0) 0x7f1a3c003520 con 0x7f1a500b35b0 2023-11-24T08:08:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a5006b760 msgr2=0x7f1a500b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a5006b760 0x7f1a500b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 >> v2:172.21.15.91:3300/0 conn(0x7f1a501460b0 msgr2=0x7f1a500b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 --2- 172.21.15.64:0/1008433809 >> v2:172.21.15.91:3300/0 conn(0x7f1a501460b0 0x7f1a500b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50146490 con 0x7f1a500b35b0 2023-11-24T08:08:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f1a480019f0 con 0x7f1a500b35b0 2023-11-24T08:08:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 >> v1:172.21.15.64:6789/0 conn(0x7f1a500b35b0 legacy=0x7f1a50160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c003970 0x7f1a3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0047c0 0x7f1a3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a5006b760 0x7f1a3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a50146a90 con 0x7f1a3c003970 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500b3a40 con 0x7f1a3c0047c0 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c00b9d0 con 0x7f1a5006b760 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4f7fe640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c003970 0x7f1a3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a47fff640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0047c0 0x7f1a3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.152+0000 7f1a4ffff640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a5006b760 0x7f1a3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.153+0000 7f1a4f7fe640 1 -- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a5006b760 msgr2=0x7f1a3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.153+0000 7f1a4f7fe640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a5006b760 0x7f1a3c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.153+0000 7f1a4f7fe640 1 -- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0047c0 msgr2=0x7f1a3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.153+0000 7f1a4f7fe640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a3c0047c0 0x7f1a3c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.153+0000 7f1a4f7fe640 1 -- 172.21.15.64:0/1008433809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c003520 con 0x7f1a3c003970 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.154+0000 7f1a4f7fe640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c003970 0x7f1a3c003d90 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009af0 tx=0x7f1a3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e81ce061df3f6dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.154+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4002c710 con 0x7f1a3c003970 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.154+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4002c870 con 0x7f1a3c003970 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.154+0000 7f1a4effd640 1 -- 172.21.15.64:0/1008433809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4002cc00 con 0x7f1a3c003970 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.155+0000 7f1a56002640 1 -- 172.21.15.64:0/1008433809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c003970 msgr2=0x7f1a3c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.155+0000 7f1a56002640 1 --2- 172.21.15.64:0/1008433809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a3c003970 0x7f1a3c003d90 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009af0 tx=0x7f1a3c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.155+0000 7f1a4ffff640 1 -- 172.21.15.64:0/1008433809 reap_dead start 2023-11-24T08:08:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.155+0000 7f1a56002640 1 -- 172.21.15.64:0/1008433809 shutdown_connections 2023-11-24T08:08:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.155+0000 7f1a56002640 1 -- 172.21.15.64:0/1008433809 >> 172.21.15.64:0/1008433809 conn(0x7f1a500663d0 msgr2=0x7f1a500667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.155+0000 7f1a56002640 1 -- 172.21.15.64:0/1008433809 shutdown_connections 2023-11-24T08:08:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.155+0000 7f1a56002640 1 -- 172.21.15.64:0/1008433809 wait complete. 2023-11-24T08:08:09.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.157+0000 7f1a56002640 1 Processor -- start 2023-11-24T08:08:09.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.157+0000 7f1a56002640 1 -- start start 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a56002640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c003970 0x7f1a50136d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a56002640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a501435e0 0x7f1a50143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a56002640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a5013b020 0x7f1a5013b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a56002640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a50143fa0 con 0x7f1a501435e0 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a56002640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5013bd50 con 0x7f1a5013b020 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a56002640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5013c030 con 0x7f1a3c003970 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a501435e0 0x7f1a50143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a501435e0 0x7f1a50143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35438/0 (socket says 172.21.15.64:35438) 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.158+0000 7f1a47fff640 1 -- 172.21.15.64:0/1300851914 learned_addr learned my addr 172.21.15.64:0/1300851914 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a4f7fe640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c003970 0x7f1a50136d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a4ffff640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a5013b020 0x7f1a5013b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a47fff640 1 -- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c003970 msgr2=0x7f1a50136d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a47fff640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c003970 0x7f1a50136d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a47fff640 1 -- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a5013b020 msgr2=0x7f1a5013b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a47fff640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a5013b020 0x7f1a5013b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a47fff640 1 -- 172.21.15.64:0/1300851914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5013c310 con 0x7f1a501435e0 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a4ffff640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a5013b020 0x7f1a5013b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a47fff640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a501435e0 0x7f1a50143a60 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1a38011600 tx=0x7f1a38011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a4cff9640 1 -- 172.21.15.64:0/1300851914 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800f920 con 0x7f1a501435e0 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a4cff9640 1 -- 172.21.15.64:0/1300851914 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3800fa80 con 0x7f1a501435e0 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a4cff9640 1 -- 172.21.15.64:0/1300851914 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800fdf0 con 0x7f1a501435e0 2023-11-24T08:08:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.159+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5013c5f0 con 0x7f1a501435e0 2023-11-24T08:08:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.160+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a501372b0 con 0x7f1a501435e0 2023-11-24T08:08:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.161+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a50074770 con 0x7f1a501435e0 2023-11-24T08:08:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.162+0000 7f1a4cff9640 1 -- 172.21.15.64:0/1300851914 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1a38015460 con 0x7f1a501435e0 2023-11-24T08:08:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.162+0000 7f1a4cff9640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a3006b410 0x7f1a3006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.163+0000 7f1a4cff9640 1 -- 172.21.15.64:0/1300851914 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1a3808a600 con 0x7f1a501435e0 2023-11-24T08:08:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.163+0000 7f1a4f7fe640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a3006b410 0x7f1a3006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.164+0000 7f1a4f7fe640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a3006b410 0x7f1a3006d8d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1a40036000 tx=0x7f1a40030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:09.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.167+0000 7f1a4cff9640 1 -- 172.21.15.64:0/1300851914 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a38054de0 con 0x7f1a501435e0 2023-11-24T08:08:09.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.404+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1a50070e00 con 0x7f1a501435e0 2023-11-24T08:08:09.407 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-11-24T08:08:09.405+0000 7fb13dee4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:08:09.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.409+0000 7f1a4cff9640 1 -- 172.21.15.64:0/1300851914 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1a38058400 con 0x7f1a501435e0 2023-11-24T08:08:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:08:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a3006b410 msgr2=0x7f1a3006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a3006b410 0x7f1a3006d8d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1a40036000 tx=0x7f1a40030000 comp rx=0 tx=0).stop 2023-11-24T08:08:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a501435e0 msgr2=0x7f1a50143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a501435e0 0x7f1a50143a60 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1a38011600 tx=0x7f1a38011ac0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 shutdown_connections 2023-11-24T08:08:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1a3006b410 0x7f1a3006d8d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1a5013b020 0x7f1a5013b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a501435e0 0x7f1a50143a60 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.411+0000 7f1a56002640 1 --2- 172.21.15.64:0/1300851914 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1a3c003970 0x7f1a50136d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.412+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 >> 172.21.15.64:0/1300851914 conn(0x7f1a500663d0 msgr2=0x7f1a500b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.412+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 shutdown_connections 2023-11-24T08:08:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.412+0000 7f1a56002640 1 -- 172.21.15.64:0/1300851914 wait complete. 2023-11-24T08:08:09.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:08:09.528 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:08:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.642+0000 7f9ca0a9b640 1 Processor -- start 2023-11-24T08:08:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.642+0000 7f9ca0a9b640 1 -- start start 2023-11-24T08:08:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9ca0a9b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 0x7f9c9c06e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9ca0a9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c9c072280 0x7f9c9c06c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9ca0a9b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f9c9c06c900 0x7f9c9c160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9ca0a9b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c0b2320 con 0x7f9c9c072280 2023-11-24T08:08:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9ca0a9b640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c06cd90 con 0x7f9c9c06c900 2023-11-24T08:08:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9ca0a9b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c06df30 con 0x7f9c9c160b10 2023-11-24T08:08:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c99d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c9c072280 0x7f9c9c06c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c99d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c9c072280 0x7f9c9c06c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35476/0 (socket says 172.21.15.64:35476) 2023-11-24T08:08:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c99d9b640 1 -- 172.21.15.64:0/4203782421 learned_addr learned my addr 172.21.15.64:0/4203782421 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c9a59c640 1 --2- 172.21.15.64:0/4203782421 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 0x7f9c9c06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c9ad9d640 1 --2- 172.21.15.64:0/4203782421 >> v2:172.21.15.91:3300/0 conn(0x7f9c9c06c900 0x7f9c9c160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c99d9b640 1 -- 172.21.15.64:0/4203782421 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 msgr2=0x7f9c9c06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c99d9b640 1 --2- 172.21.15.64:0/4203782421 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 0x7f9c9c06e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c99d9b640 1 -- 172.21.15.64:0/4203782421 >> v2:172.21.15.91:3300/0 conn(0x7f9c9c06c900 msgr2=0x7f9c9c160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c99d9b640 1 --2- 172.21.15.64:0/4203782421 >> v2:172.21.15.91:3300/0 conn(0x7f9c9c06c900 0x7f9c9c160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.645+0000 7f9c99d9b640 1 -- 172.21.15.64:0/4203782421 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c071780 con 0x7f9c9c072280 2023-11-24T08:08:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.646+0000 7f9c99d9b640 1 --2- 172.21.15.64:0/4203782421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c072280 0x7f9c9c06c3c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c06bec0 tx=0x7f9c8402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dfc4ed8a75d2cf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.646+0000 7f9c9959a640 1 -- 172.21.15.64:0/4203782421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c840317c0 con 0x7f9c9c072280 2023-11-24T08:08:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.646+0000 7f9c9959a640 1 -- 172.21.15.64:0/4203782421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c84031920 con 0x7f9c9c072280 2023-11-24T08:08:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.646+0000 7f9c9959a640 1 -- 172.21.15.64:0/4203782421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c84031cb0 con 0x7f9c9c072280 2023-11-24T08:08:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/4203782421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c072280 msgr2=0x7f9c9c06c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/4203782421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c072280 0x7f9c9c06c3c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c06bec0 tx=0x7f9c8402b860 comp rx=0 tx=0).stop 2023-11-24T08:08:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/4203782421 shutdown_connections 2023-11-24T08:08:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/4203782421 >> v2:172.21.15.91:3300/0 conn(0x7f9c9c06c900 0x7f9c9c160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/4203782421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c072280 0x7f9c9c06c3c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/4203782421 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 0x7f9c9c06e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/4203782421 >> 172.21.15.64:0/4203782421 conn(0x7f9c9c0b0ba0 msgr2=0x7f9c9c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/4203782421 shutdown_connections 2023-11-24T08:08:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.648+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/4203782421 wait complete. 2023-11-24T08:08:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.649+0000 7f9ca0a9b640 1 Processor -- start 2023-11-24T08:08:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.649+0000 7f9ca0a9b640 1 -- start start 2023-11-24T08:08:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.650+0000 7f9ca0a9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c06c900 0x7f9c9c0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.650+0000 7f9ca0a9b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9c072280 0x7f9c9c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.650+0000 7f9ca0a9b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 0x7f9c9c0ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.650+0000 7f9ca0a9b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0cbed0 con 0x7f9c9c06c900 2023-11-24T08:08:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.650+0000 7f9ca0a9b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0cc1b0 con 0x7f9c9c072280 2023-11-24T08:08:09.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.650+0000 7f9ca0a9b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0cc490 con 0x7f9c9c160b10 2023-11-24T08:08:09.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c06c900 0x7f9c9c0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c06c900 0x7f9c9c0cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35482/0 (socket says 172.21.15.64:35482) 2023-11-24T08:08:09.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 -- 172.21.15.64:0/3463202294 learned_addr learned my addr 172.21.15.64:0/3463202294 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c99d9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9c072280 0x7f9c9c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9ad9d640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 0x7f9c9c0ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 -- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 msgr2=0x7f9c9c0ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 0x7f9c9c0ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 -- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9c072280 msgr2=0x7f9c9c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9c072280 0x7f9c9c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 -- 172.21.15.64:0/3463202294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c840094d0 con 0x7f9c9c06c900 2023-11-24T08:08:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.651+0000 7f9c9a59c640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c06c900 0x7f9c9c0cd790 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c007e40 tx=0x7f9c8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.652+0000 7f9c92ffd640 1 -- 172.21.15.64:0/3463202294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c014620 con 0x7f9c9c06c900 2023-11-24T08:08:09.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.652+0000 7f9c92ffd640 1 -- 172.21.15.64:0/3463202294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c00ee90 con 0x7f9c9c06c900 2023-11-24T08:08:09.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.652+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c0cc770 con 0x7f9c9c06c900 2023-11-24T08:08:09.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.652+0000 7f9c92ffd640 1 -- 172.21.15.64:0/3463202294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0149c0 con 0x7f9c9c06c900 2023-11-24T08:08:09.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.652+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c0cdcd0 con 0x7f9c9c06c900 2023-11-24T08:08:09.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.653+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9c09d200 con 0x7f9c9c06c900 2023-11-24T08:08:09.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.654+0000 7f9c92ffd640 1 -- 172.21.15.64:0/3463202294 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f9c8c014b20 con 0x7f9c9c06c900 2023-11-24T08:08:09.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.655+0000 7f9c92ffd640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9c7c06b3e0 0x7f9c7c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:09.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.655+0000 7f9c92ffd640 1 -- 172.21.15.64:0/3463202294 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9c8c08aa10 con 0x7f9c9c06c900 2023-11-24T08:08:09.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.656+0000 7f9c99d9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9c7c06b3e0 0x7f9c7c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:09.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.657+0000 7f9c99d9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9c7c06b3e0 0x7f9c7c06d8a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9c84036000 tx=0x7f9c8403d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:09.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.659+0000 7f9c92ffd640 1 -- 172.21.15.64:0/3463202294 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c8c055240 con 0x7f9c9c06c900 2023-11-24T08:08:09.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:09.892+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9c9c071d20 con 0x7f9c9c06c900 2023-11-24T08:08:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.519+0000 7f9c92ffd640 1 -- 172.21.15.64:0/3463202294 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9c8c058860 con 0x7f9c9c06c900 2023-11-24T08:08:10.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9c7c06b3e0 msgr2=0x7f9c7c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:10.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9c7c06b3e0 0x7f9c7c06d8a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9c84036000 tx=0x7f9c8403d040 comp rx=0 tx=0).stop 2023-11-24T08:08:10.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c06c900 msgr2=0x7f9c9c0cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:10.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c06c900 0x7f9c9c0cd790 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c007e40 tx=0x7f9c8c00e4b0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 shutdown_connections 2023-11-24T08:08:10.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9c7c06b3e0 0x7f9c7c06d8a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9c9c160b10 0x7f9c9c0ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9c9c072280 0x7f9c9c0abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 --2- 172.21.15.64:0/3463202294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c06c900 0x7f9c9c0cd790 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 >> 172.21.15.64:0/3463202294 conn(0x7f9c9c0b0ba0 msgr2=0x7f9c9c0acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:10.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.523+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 shutdown_connections 2023-11-24T08:08:10.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.524+0000 7f9ca0a9b640 1 -- 172.21.15.64:0/3463202294 wait complete. 2023-11-24T08:08:10.546 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:08:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.770+0000 7f13f5bd0640 1 Processor -- start 2023-11-24T08:08:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.770+0000 7f13f5bd0640 1 -- start start 2023-11-24T08:08:10.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.772+0000 7f13f5bd0640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f13f01448c0 0x7f13f006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.772+0000 7f13f5bd0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f015bda0 0x7f13f0070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.772+0000 7f13f5bd0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f015c210 con 0x7f13f006d910 2023-11-24T08:08:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.772+0000 7f13f5bd0640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f006dd50 con 0x7f13f01448c0 2023-11-24T08:08:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.772+0000 7f13f5bd0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0144ca0 con 0x7f13f015bda0 2023-11-24T08:08:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.772+0000 7f13effff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f13f006d910 0x7f13f015b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50274/0 (socket says 172.21.15.64:50274) 2023-11-24T08:08:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.772+0000 7f13effff640 1 -- 172.21.15.64:0/3975591089 learned_addr learned my addr 172.21.15.64:0/3975591089 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13f53cf640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f015bda0 0x7f13f0070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13f4bce640 1 --2- 172.21.15.64:0/3975591089 >> v2:172.21.15.91:3300/0 conn(0x7f13f01448c0 0x7f13f006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359152618 0 0) 0x7f13dc001290 con 0x7f13f006d910 2023-11-24T08:08:10.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e0003520 con 0x7f13f006d910 2023-11-24T08:08:10.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1351672487 0 0) 0x7f13e0003520 con 0x7f13f006d910 2023-11-24T08:08:10.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f015bda0 msgr2=0x7f13f0070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f015bda0 0x7f13f0070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 >> v2:172.21.15.91:3300/0 conn(0x7f13f01448c0 msgr2=0x7f13f006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/3975591089 >> v2:172.21.15.91:3300/0 conn(0x7f13f01448c0 0x7f13f006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f015c450 con 0x7f13f006d910 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.773+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f13dc001980 con 0x7f13f006d910 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 >> v1:172.21.15.64:6789/0 conn(0x7f13f006d910 legacy=0x7f13f015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f015bda0 0x7f13e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13ef7fe640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13e0004d00 0x7f13e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f006dd50 con 0x7f13e00039e0 2023-11-24T08:08:10.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0144ca0 con 0x7f13e0004d00 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e000fd90 con 0x7f13f015bda0 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13f4bce640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13effff640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f015bda0 0x7f13e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13f53cf640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13e0004d00 0x7f13e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13f4bce640 1 -- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f015bda0 msgr2=0x7f13e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13f4bce640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f015bda0 0x7f13e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13f4bce640 1 -- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13e0004d00 msgr2=0x7f13e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13f4bce640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13e0004d00 0x7f13e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13f4bce640 1 -- 172.21.15.64:0/3975591089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0003520 con 0x7f13e00039e0 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.774+0000 7f13f53cf640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13e0004d00 0x7f13e0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:10.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.775+0000 7f13f4bce640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13e0003e00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009ac0 tx=0x7f13e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a27f191e03e5a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e402c710 con 0x7f13e00039e0 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e402c870 con 0x7f13e00039e0 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13ef7fe640 1 -- 172.21.15.64:0/3975591089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e402cc00 con 0x7f13e00039e0 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3975591089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 msgr2=0x7f13e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13f5bd0640 1 --2- 172.21.15.64:0/3975591089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13e0003e00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009ac0 tx=0x7f13e00057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13f53cf640 1 -- 172.21.15.64:0/3975591089 reap_dead start 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3975591089 shutdown_connections 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3975591089 >> 172.21.15.64:0/3975591089 conn(0x7f13f00b0aa0 msgr2=0x7f13f00b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3975591089 shutdown_connections 2023-11-24T08:08:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.776+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3975591089 wait complete. 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.778+0000 7f13f5bd0640 1 Processor -- start 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.778+0000 7f13f5bd0640 1 -- start start 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.779+0000 7f13f5bd0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13f013fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.779+0000 7f13f5bd0640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13f013c6c0 0x7f13f013d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.779+0000 7f13f5bd0640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f0158800 0x7f13f0158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.779+0000 7f13f5bd0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01591c0 con 0x7f13e00039e0 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.779+0000 7f13f5bd0640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0141670 con 0x7f13f013c6c0 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.779+0000 7f13f5bd0640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f013e130 con 0x7f13f0158800 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13f013fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13f013fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35500/0 (socket says 172.21.15.64:35500) 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 -- 172.21.15.64:0/3119718326 learned_addr learned my addr 172.21.15.64:0/3119718326 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:10.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f53cf640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f0158800 0x7f13f0158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13effff640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13f013c6c0 0x7f13f013d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 -- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f0158800 msgr2=0x7f13f0158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f0158800 0x7f13f0158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 -- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13f013c6c0 msgr2=0x7f13f013d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13f013c6c0 0x7f13f013d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 -- 172.21.15.64:0/3119718326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f013e410 con 0x7f13e00039e0 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f53cf640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f0158800 0x7f13f0158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.780+0000 7f13f4bce640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13f013fa80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f13e402c2b0 tx=0x7f13e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.781+0000 7f13ed7fa640 1 -- 172.21.15.64:0/3119718326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4003810 con 0x7f13e00039e0 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.781+0000 7f13ed7fa640 1 -- 172.21.15.64:0/3119718326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e402de90 con 0x7f13e00039e0 2023-11-24T08:08:10.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.781+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f013e6f0 con 0x7f13e00039e0 2023-11-24T08:08:10.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.781+0000 7f13ed7fa640 1 -- 172.21.15.64:0/3119718326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4003b70 con 0x7f13e00039e0 2023-11-24T08:08:10.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.781+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f013de20 con 0x7f13e00039e0 2023-11-24T08:08:10.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.782+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f0067960 con 0x7f13e00039e0 2023-11-24T08:08:10.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.783+0000 7f13ed7fa640 1 -- 172.21.15.64:0/3119718326 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f13e4003cd0 con 0x7f13e00039e0 2023-11-24T08:08:10.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.784+0000 7f13ed7fa640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f13d406b410 0x7f13d406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:10.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.784+0000 7f13ed7fa640 1 -- 172.21.15.64:0/3119718326 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f13e40abf80 con 0x7f13e00039e0 2023-11-24T08:08:10.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.784+0000 7f13effff640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f13d406b410 0x7f13d406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:10.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.785+0000 7f13effff640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f13d406b410 0x7f13d406d8d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f13dc001430 tx=0x7f13dc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:10.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.788+0000 7f13ed7fa640 1 -- 172.21.15.64:0/3119718326 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13e40768d0 con 0x7f13e00039e0 2023-11-24T08:08:10.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:10.981+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f13f004e840 con 0x7f13e00039e0 2023-11-24T08:08:11.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.540+0000 7f13ed7fa640 1 -- 172.21.15.64:0/3119718326 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) v1 ==== 181+0+0 (secure 0 0 0) 0x7f13e4079ef0 con 0x7f13e00039e0 2023-11-24T08:08:11.541 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f13d406b410 msgr2=0x7f13d406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f13d406b410 0x7f13d406d8d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f13dc001430 tx=0x7f13dc009040 comp rx=0 tx=0).stop 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 msgr2=0x7f13f013fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13f013fa80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f13e402c2b0 tx=0x7f13e4004730 comp rx=0 tx=0).stop 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 shutdown_connections 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f13d406b410 0x7f13d406d8d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f13f0158800 0x7f13f0158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f13f013c6c0 0x7f13f013d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 --2- 172.21.15.64:0/3119718326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13e00039e0 0x7f13f013fa80 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.544+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 >> 172.21.15.64:0/3119718326 conn(0x7f13f00b0aa0 msgr2=0x7f13f00b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.545+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 shutdown_connections 2023-11-24T08:08:11.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.545+0000 7f13f5bd0640 1 -- 172.21.15.64:0/3119718326 wait complete. 2023-11-24T08:08:11.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:08:11.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.798+0000 7f5d1534e640 1 Processor -- start 2023-11-24T08:08:11.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.798+0000 7f5d1534e640 1 -- start start 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.799+0000 7f5d1534e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5d10144770 0x7f5d1015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d1534e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d1015f0a0 0x7f5d100ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d1534e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1015f930 con 0x7f5d100b35a0 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d1534e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d100b43b0 con 0x7f5d10144770 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d1534e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1015f530 con 0x7f5d1015f0a0 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d0ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5d10144770 0x7f5d1015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d0ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5d10144770 0x7f5d1015eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35508/0 (socket says 172.21.15.64:35508) 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d0f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5d100b35a0 0x7f5d1006c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50282/0 (socket says 172.21.15.64:50282) 2023-11-24T08:08:11.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d0ffff640 1 -- 172.21.15.64:0/1722565091 learned_addr learned my addr 172.21.15.64:0/1722565091 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d14b4d640 1 --2- 172.21.15.64:0/1722565091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d1015f0a0 0x7f5d100ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d0ffff640 1 -- 172.21.15.64:0/1722565091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d1015f0a0 msgr2=0x7f5d100ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.800+0000 7f5d0ffff640 1 --2- 172.21.15.64:0/1722565091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d1015f0a0 0x7f5d100ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.801+0000 7f5d0ffff640 1 -- 172.21.15.64:0/1722565091 >> v1:172.21.15.64:6789/0 conn(0x7f5d100b35a0 legacy=0x7f5d1006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.801+0000 7f5d0ffff640 1 -- 172.21.15.64:0/1722565091 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d100ce9f0 con 0x7f5d10144770 2023-11-24T08:08:11.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.801+0000 7f5d0effd640 1 -- 172.21.15.64:0/1722565091 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663248231 0 0) 0x7f5d1015f930 con 0x7f5d100b35a0 2023-11-24T08:08:11.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.802+0000 7f5d0ffff640 1 --2- 172.21.15.64:0/1722565091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d1015eb60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009870 tx=0x7f5d0002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c243cd54d9c4bbd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:11.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.802+0000 7f5d0effd640 1 -- 172.21.15.64:0/1722565091 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00030840 con 0x7f5d10144770 2023-11-24T08:08:11.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.802+0000 7f5d0effd640 1 -- 172.21.15.64:0/1722565091 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d000309a0 con 0x7f5d10144770 2023-11-24T08:08:11.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.802+0000 7f5d0effd640 1 -- 172.21.15.64:0/1722565091 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00030d30 con 0x7f5d10144770 2023-11-24T08:08:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.803+0000 7f5d1534e640 1 -- 172.21.15.64:0/1722565091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 msgr2=0x7f5d1015eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.803+0000 7f5d1534e640 1 --2- 172.21.15.64:0/1722565091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d1015eb60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009870 tx=0x7f5d0002b980 comp rx=0 tx=0).stop 2023-11-24T08:08:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.803+0000 7f5d1534e640 1 -- 172.21.15.64:0/1722565091 shutdown_connections 2023-11-24T08:08:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.803+0000 7f5d1534e640 1 --2- 172.21.15.64:0/1722565091 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d1015f0a0 0x7f5d100ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.803+0000 7f5d1534e640 1 --2- 172.21.15.64:0/1722565091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d1015eb60 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.803+0000 7f5d1534e640 1 -- 172.21.15.64:0/1722565091 >> 172.21.15.64:0/1722565091 conn(0x7f5d100cec10 msgr2=0x7f5d100cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:11.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.803+0000 7f5d1534e640 1 -- 172.21.15.64:0/1722565091 shutdown_connections 2023-11-24T08:08:11.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.803+0000 7f5d1534e640 1 -- 172.21.15.64:0/1722565091 wait complete. 2023-11-24T08:08:11.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.805+0000 7f5d1534e640 1 Processor -- start 2023-11-24T08:08:11.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.805+0000 7f5d1534e640 1 -- start start 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.806+0000 7f5d1534e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d100b35a0 0x7f5d1014d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.806+0000 7f5d1534e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d101f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.806+0000 7f5d1534e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d1015f0a0 0x7f5d10139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.806+0000 7f5d1534e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1014da10 con 0x7f5d10144770 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.806+0000 7f5d1534e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1014dcf0 con 0x7f5d1015f0a0 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.806+0000 7f5d1534e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1014dfd0 con 0x7f5d100b35a0 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.806+0000 7f5d0f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d101f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d101f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35514/0 (socket says 172.21.15.64:35514) 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0f7fe640 1 -- 172.21.15.64:0/162695695 learned_addr learned my addr 172.21.15.64:0/162695695 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0ffff640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d100b35a0 0x7f5d1014d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d14b4d640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d1015f0a0 0x7f5d10139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0f7fe640 1 -- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d100b35a0 msgr2=0x7f5d1014d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0f7fe640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d100b35a0 0x7f5d1014d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0f7fe640 1 -- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d1015f0a0 msgr2=0x7f5d10139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0f7fe640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d1015f0a0 0x7f5d10139d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0f7fe640 1 -- 172.21.15.64:0/162695695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d000094d0 con 0x7f5d10144770 2023-11-24T08:08:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d14b4d640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d1015f0a0 0x7f5d10139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:11.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.807+0000 7f5d0f7fe640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d101f6040 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c4d0 tx=0x7f5d0400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:11.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.808+0000 7f5d0cff9640 1 -- 172.21.15.64:0/162695695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d040074e0 con 0x7f5d10144770 2023-11-24T08:08:11.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.808+0000 7f5d0cff9640 1 -- 172.21.15.64:0/162695695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d04007660 con 0x7f5d10144770 2023-11-24T08:08:11.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.808+0000 7f5d0cff9640 1 -- 172.21.15.64:0/162695695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04004a20 con 0x7f5d10144770 2023-11-24T08:08:11.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.808+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1014e2b0 con 0x7f5d10144770 2023-11-24T08:08:11.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.808+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1014d6b0 con 0x7f5d10144770 2023-11-24T08:08:11.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.809+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1014e7d0 con 0x7f5d10144770 2023-11-24T08:08:11.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.810+0000 7f5d0cff9640 1 -- 172.21.15.64:0/162695695 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5d04004b80 con 0x7f5d10144770 2023-11-24T08:08:11.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.811+0000 7f5d0cff9640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5cfc06b3e0 0x7f5cfc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:11.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.811+0000 7f5d0cff9640 1 -- 172.21.15.64:0/162695695 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f5d04089100 con 0x7f5d10144770 2023-11-24T08:08:11.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.812+0000 7f5d0ffff640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5cfc06b3e0 0x7f5cfc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:11.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.813+0000 7f5d0ffff640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5cfc06b3e0 0x7f5cfc06d8a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5d00036000 tx=0x7f5d00008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:11.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:11.816+0000 7f5d0cff9640 1 -- 172.21.15.64:0/162695695 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d040bbe90 con 0x7f5d10144770 2023-11-24T08:08:12.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.021+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d1004e840 con 0x7f5d10144770 2023-11-24T08:08:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.551+0000 7f5d0cff9640 1 -- 172.21.15.64:0/162695695 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5d040180d0 con 0x7f5d10144770 2023-11-24T08:08:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-11-24T08:08:12.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.554+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5cfc06b3e0 msgr2=0x7f5cfc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:12.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.554+0000 7f5d1534e640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5cfc06b3e0 0x7f5cfc06d8a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5d00036000 tx=0x7f5d00008030 comp rx=0 tx=0).stop 2023-11-24T08:08:12.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.554+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 msgr2=0x7f5d101f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:12.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.554+0000 7f5d1534e640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d101f6040 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c4d0 tx=0x7f5d0400c990 comp rx=0 tx=0).stop 2023-11-24T08:08:12.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.555+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 shutdown_connections 2023-11-24T08:08:12.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.555+0000 7f5d1534e640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f5cfc06b3e0 0x7f5cfc06d8a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.555+0000 7f5d1534e640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5d1015f0a0 0x7f5d10139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.555+0000 7f5d1534e640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d10144770 0x7f5d101f6040 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.555+0000 7f5d1534e640 1 --2- 172.21.15.64:0/162695695 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5d100b35a0 0x7f5d1014d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.555+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 >> 172.21.15.64:0/162695695 conn(0x7f5d100cec10 msgr2=0x7f5d100b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:12.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.558+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 shutdown_connections 2023-11-24T08:08:12.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.558+0000 7f5d1534e640 1 -- 172.21.15.64:0/162695695 wait complete. 2023-11-24T08:08:12.582 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:08:12.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.803+0000 7fb32f59e640 1 Processor -- start 2023-11-24T08:08:12.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.804+0000 7fb32f59e640 1 -- start start 2023-11-24T08:08:12.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.805+0000 7fb32f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb33006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:12.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32f59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb3300c43a0 0x7fb33006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:12.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32f59e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300ce0e0 con 0x7fb330070fe0 2023-11-24T08:08:12.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32f59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33015d670 con 0x7fb3300c43a0 2023-11-24T08:08:12.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300c6300 con 0x7fb330070c00 2023-11-24T08:08:12.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32e59c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb33006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:12.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32dd9b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb3300c43a0 0x7fb33006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:12.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32dd9b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb3300c43a0 0x7fb33006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:34298/0 (socket says 172.21.15.64:34298) 2023-11-24T08:08:12.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32e59c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb33006e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:57742/0 (socket says 172.21.15.64:57742) 2023-11-24T08:08:12.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.806+0000 7fb32dd9b640 1 -- 172.21.15.64:0/669008775 learned_addr learned my addr 172.21.15.64:0/669008775 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:12.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.807+0000 7fb32e59c640 1 -- 172.21.15.64:0/669008775 >> v2:172.21.15.91:3300/0 conn(0x7fb3300c43a0 msgr2=0x7fb33006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:12.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.807+0000 7fb32e59c640 1 --2- 172.21.15.64:0/669008775 >> v2:172.21.15.91:3300/0 conn(0x7fb3300c43a0 0x7fb33006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.807+0000 7fb32e59c640 1 -- 172.21.15.64:0/669008775 >> v1:172.21.15.91:6789/0 conn(0x7fb330070fe0 legacy=0x7fb33006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:12.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.807+0000 7fb32e59c640 1 -- 172.21.15.64:0/669008775 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300c48f0 con 0x7fb330070c00 2023-11-24T08:08:12.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.809+0000 7fb32e59c640 1 --2- 172.21.15.64:0/669008775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb33006e400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb320009d90 tx=0x7fb32002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbec4e36de582290 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.809+0000 7fb32d59a640 1 -- 172.21.15.64:0/669008775 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32002cc50 con 0x7fb330070c00 2023-11-24T08:08:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32d59a640 1 -- 172.21.15.64:0/669008775 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32002cdb0 con 0x7fb330070c00 2023-11-24T08:08:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32d59a640 1 -- 172.21.15.64:0/669008775 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320034660 con 0x7fb330070c00 2023-11-24T08:08:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32f59e640 1 -- 172.21.15.64:0/669008775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 msgr2=0x7fb33006e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32f59e640 1 --2- 172.21.15.64:0/669008775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb33006e400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb320009d90 tx=0x7fb32002bea0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32f59e640 1 -- 172.21.15.64:0/669008775 shutdown_connections 2023-11-24T08:08:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32f59e640 1 --2- 172.21.15.64:0/669008775 >> v2:172.21.15.91:3300/0 conn(0x7fb3300c43a0 0x7fb33006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32f59e640 1 --2- 172.21.15.64:0/669008775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb33006e400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32f59e640 1 -- 172.21.15.64:0/669008775 >> 172.21.15.64:0/669008775 conn(0x7fb3300cf500 msgr2=0x7fb3300cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32f59e640 1 -- 172.21.15.64:0/669008775 shutdown_connections 2023-11-24T08:08:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.810+0000 7fb32f59e640 1 -- 172.21.15.64:0/669008775 wait complete. 2023-11-24T08:08:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.812+0000 7fb32f59e640 1 Processor -- start 2023-11-24T08:08:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.812+0000 7fb32f59e640 1 -- start start 2023-11-24T08:08:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.813+0000 7fb32f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb3300c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.813+0000 7fb32f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb330070fe0 0x7fb3300c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.813+0000 7fb32f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300c43a0 0x7fb3300c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.813+0000 7fb32f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300c8080 con 0x7fb3300c43a0 2023-11-24T08:08:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.813+0000 7fb32f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300c8360 con 0x7fb330070fe0 2023-11-24T08:08:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.813+0000 7fb32f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300c8640 con 0x7fb330070c00 2023-11-24T08:08:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300c43a0 0x7fb3300c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300c43a0 0x7fb3300c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35520/0 (socket says 172.21.15.64:35520) 2023-11-24T08:08:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 -- 172.21.15.64:0/1542673929 learned_addr learned my addr 172.21.15.64:0/1542673929 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32e59c640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb3300c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32dd9b640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb330070fe0 0x7fb3300c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 -- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 msgr2=0x7fb3300c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb3300c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 -- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb330070fe0 msgr2=0x7fb3300c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb330070fe0 0x7fb3300c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 -- 172.21.15.64:0/1542673929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb320009a70 con 0x7fb3300c43a0 2023-11-24T08:08:12.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.814+0000 7fb32ed9d640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300c43a0 0x7fb3300c7390 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb328001eb0 tx=0x7fb32800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:12.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.815+0000 7fb326ffd640 1 -- 172.21.15.64:0/1542673929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280052d0 con 0x7fb3300c43a0 2023-11-24T08:08:12.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.815+0000 7fb326ffd640 1 -- 172.21.15.64:0/1542673929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328005430 con 0x7fb3300c43a0 2023-11-24T08:08:12.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.815+0000 7fb326ffd640 1 -- 172.21.15.64:0/1542673929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328003810 con 0x7fb3300c43a0 2023-11-24T08:08:12.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.815+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3300c8920 con 0x7fb3300c43a0 2023-11-24T08:08:12.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.815+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33014bf40 con 0x7fb3300c43a0 2023-11-24T08:08:12.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.816+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3300c8e40 con 0x7fb3300c43a0 2023-11-24T08:08:12.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.817+0000 7fb326ffd640 1 -- 172.21.15.64:0/1542673929 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb328003970 con 0x7fb3300c43a0 2023-11-24T08:08:12.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.818+0000 7fb326ffd640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb31006b410 0x7fb31006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:12.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.818+0000 7fb326ffd640 1 -- 172.21.15.64:0/1542673929 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3280892b0 con 0x7fb3300c43a0 2023-11-24T08:08:12.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.819+0000 7fb32e59c640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb31006b410 0x7fb31006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:12.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.820+0000 7fb32e59c640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb31006b410 0x7fb31006d8d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb320036000 tx=0x7fb320030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:12.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:12.822+0000 7fb326ffd640 1 -- 172.21.15.64:0/1542673929 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3300c8e40 con 0x7fb3300c43a0 2023-11-24T08:08:13.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.016+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3300c8e40 con 0x7fb3300c43a0 2023-11-24T08:08:13.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.023+0000 7fb326ffd640 1 -- 172.21.15.64:0/1542673929 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3300c8e40 con 0x7fb3300c43a0 2023-11-24T08:08:13.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb31006b410 msgr2=0x7fb31006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:13.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb31006b410 0x7fb31006d8d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb320036000 tx=0x7fb320030000 comp rx=0 tx=0).stop 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300c43a0 msgr2=0x7fb3300c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300c43a0 0x7fb3300c7390 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb328001eb0 tx=0x7fb32800c890 comp rx=0 tx=0).stop 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 shutdown_connections 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb31006b410 0x7fb31006d8d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300c43a0 0x7fb3300c7390 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb330070fe0 0x7fb3300c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 --2- 172.21.15.64:0/1542673929 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb330070c00 0x7fb3300c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 >> 172.21.15.64:0/1542673929 conn(0x7fb3300cf500 msgr2=0x7fb3300b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 shutdown_connections 2023-11-24T08:08:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.025+0000 7fb32f59e640 1 -- 172.21.15.64:0/1542673929 wait complete. 2023-11-24T08:08:13.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T08:08:13.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.280+0000 7ffa9a71d640 1 Processor -- start 2023-11-24T08:08:13.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.280+0000 7ffa9a71d640 1 -- start start 2023-11-24T08:08:13.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.281+0000 7ffa9a71d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffa9406d520 0x7ffa9406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:13.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.282+0000 7ffa9a71d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9415c360 con 0x7ffa940ad720 2023-11-24T08:08:13.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.282+0000 7ffa9a71d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9415faf0 con 0x7ffa940abff0 2023-11-24T08:08:13.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.282+0000 7ffa9a71d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa940b41a0 con 0x7ffa9406d520 2023-11-24T08:08:13.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.282+0000 7ffa8bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffa940ad720 0x7ffa9415ecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50288/0 (socket says 172.21.15.64:50288) 2023-11-24T08:08:13.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.282+0000 7ffa8bfff640 1 -- 172.21.15.64:0/509328939 learned_addr learned my addr 172.21.15.64:0/509328939 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:13.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.282+0000 7ffa93fff640 1 --2- 172.21.15.64:0/509328939 >> v2:172.21.15.64:3300/0 conn(0x7ffa9406d520 0x7ffa9406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:13.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.282+0000 7ffa937fe640 1 -- 172.21.15.64:0/509328939 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360633016 0 0) 0x7ffa9415c360 con 0x7ffa940ad720 2023-11-24T08:08:13.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.282+0000 7ffa937fe640 1 -- 172.21.15.64:0/509328939 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa80003520 con 0x7ffa940ad720 2023-11-24T08:08:13.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.283+0000 7ffa93fff640 1 -- 172.21.15.64:0/509328939 >> v1:172.21.15.91:6789/0 conn(0x7ffa940abff0 legacy=0x7ffa9415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:13.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.283+0000 7ffa93fff640 1 -- 172.21.15.64:0/509328939 >> v1:172.21.15.64:6789/0 conn(0x7ffa940ad720 legacy=0x7ffa9415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:13.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.283+0000 7ffa93fff640 1 -- 172.21.15.64:0/509328939 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9406de40 con 0x7ffa9406d520 2023-11-24T08:08:13.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.284+0000 7ffa93fff640 1 --2- 172.21.15.64:0/509328939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa9406d520 0x7ffa9406d900 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ffa840097a0 tx=0x7ffa8402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd697f2cfcba247 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:13.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.284+0000 7ffa937fe640 1 -- 172.21.15.64:0/509328939 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84030610 con 0x7ffa9406d520 2023-11-24T08:08:13.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.284+0000 7ffa937fe640 1 -- 172.21.15.64:0/509328939 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa84030770 con 0x7ffa9406d520 2023-11-24T08:08:13.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.284+0000 7ffa937fe640 1 -- 172.21.15.64:0/509328939 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84030b00 con 0x7ffa9406d520 2023-11-24T08:08:13.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.285+0000 7ffa9a71d640 1 -- 172.21.15.64:0/509328939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa9406d520 msgr2=0x7ffa9406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:13.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.285+0000 7ffa9a71d640 1 --2- 172.21.15.64:0/509328939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa9406d520 0x7ffa9406d900 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ffa840097a0 tx=0x7ffa8402b960 comp rx=0 tx=0).stop 2023-11-24T08:08:13.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.285+0000 7ffa9a71d640 1 -- 172.21.15.64:0/509328939 shutdown_connections 2023-11-24T08:08:13.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.285+0000 7ffa9a71d640 1 --2- 172.21.15.64:0/509328939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa9406d520 0x7ffa9406d900 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:13.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.285+0000 7ffa9a71d640 1 -- 172.21.15.64:0/509328939 >> 172.21.15.64:0/509328939 conn(0x7ffa940b0d80 msgr2=0x7ffa940b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:13.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.285+0000 7ffa9a71d640 1 -- 172.21.15.64:0/509328939 shutdown_connections 2023-11-24T08:08:13.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.285+0000 7ffa9a71d640 1 -- 172.21.15.64:0/509328939 wait complete. 2023-11-24T08:08:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.287+0000 7ffa9a71d640 1 Processor -- start 2023-11-24T08:08:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.287+0000 7ffa9a71d640 1 -- start start 2023-11-24T08:08:13.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa9a71d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffa9406d520 0x7ffa94146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:13.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa9a71d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa940abff0 0x7ffa940ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:13.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa9a71d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa940ad720 0x7ffa94159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa9a71d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940c91a0 con 0x7ffa940ad720 2023-11-24T08:08:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa9a71d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940c9480 con 0x7ffa940abff0 2023-11-24T08:08:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa9a71d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940c9760 con 0x7ffa9406d520 2023-11-24T08:08:13.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa98cba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa940ad720 0x7ffa94159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:13.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa98cba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa940ad720 0x7ffa94159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35544/0 (socket says 172.21.15.64:35544) 2023-11-24T08:08:13.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa98cba640 1 -- 172.21.15.64:0/4230963952 learned_addr learned my addr 172.21.15.64:0/4230963952 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:13.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa98cba640 1 -- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffa9406d520 msgr2=0x7ffa94146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:13.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa98cba640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffa9406d520 0x7ffa94146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:13.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa98cba640 1 -- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa940abff0 msgr2=0x7ffa940ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:13.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa98cba640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa940abff0 0x7ffa940ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:13.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.288+0000 7ffa98cba640 1 -- 172.21.15.64:0/4230963952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa84009400 con 0x7ffa940ad720 2023-11-24T08:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.289+0000 7ffa98cba640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa940ad720 0x7ffa94159fd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00c730 tx=0x7ffa8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.289+0000 7ffa917fa640 1 -- 172.21.15.64:0/4230963952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c003b70 con 0x7ffa940ad720 2023-11-24T08:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.289+0000 7ffa917fa640 1 -- 172.21.15.64:0/4230963952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c003cd0 con 0x7ffa940ad720 2023-11-24T08:08:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.289+0000 7ffa917fa640 1 -- 172.21.15.64:0/4230963952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00aa90 con 0x7ffa940ad720 2023-11-24T08:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.289+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa940c9a40 con 0x7ffa940ad720 2023-11-24T08:08:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.289+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9415a510 con 0x7ffa940ad720 2023-11-24T08:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.290+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa940ca780 con 0x7ffa940ad720 2023-11-24T08:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.291+0000 7ffa917fa640 1 -- 172.21.15.64:0/4230963952 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ffa8c00abf0 con 0x7ffa940ad720 2023-11-24T08:08:13.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.292+0000 7ffa917fa640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa7406b3e0 0x7ffa7406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.292+0000 7ffa917fa640 1 -- 172.21.15.64:0/4230963952 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa8c089bf0 con 0x7ffa940ad720 2023-11-24T08:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.293+0000 7ffa93fff640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa7406b3e0 0x7ffa7406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.294+0000 7ffa93fff640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa7406b3e0 0x7ffa7406d8a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffa84036000 tx=0x7ffa84030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.297+0000 7ffa917fa640 1 -- 172.21.15.64:0/4230963952 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa8c0546f0 con 0x7ffa940ad720 2023-11-24T08:08:13.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:13.561+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7ffa940c9f60 con 0x7ffa940ad720 2023-11-24T08:08:14.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.563+0000 7ffa917fa640 1 -- 172.21.15.64:0/4230963952 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v258) v1 ==== 166+0+0 (secure 0 0 0) 0x7ffa940c9f60 con 0x7ffa940ad720 2023-11-24T08:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.567+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa7406b3e0 msgr2=0x7ffa7406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.567+0000 7ffa9a71d640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa7406b3e0 0x7ffa7406d8a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffa84036000 tx=0x7ffa84030000 comp rx=0 tx=0).stop 2023-11-24T08:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.567+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa940ad720 msgr2=0x7ffa94159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:14.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.567+0000 7ffa9a71d640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa940ad720 0x7ffa94159fd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00c730 tx=0x7ffa8c00cbf0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.568+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 shutdown_connections 2023-11-24T08:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.568+0000 7ffa9a71d640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ffa7406b3e0 0x7ffa7406d8a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.568+0000 7ffa9a71d640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa940ad720 0x7ffa94159fd0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.568+0000 7ffa9a71d640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ffa940abff0 0x7ffa940ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.568+0000 7ffa9a71d640 1 --2- 172.21.15.64:0/4230963952 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ffa9406d520 0x7ffa94146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.568+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 >> 172.21.15.64:0/4230963952 conn(0x7ffa940b0d80 msgr2=0x7ffa940b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:14.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.568+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 shutdown_connections 2023-11-24T08:08:14.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.568+0000 7ffa9a71d640 1 -- 172.21.15.64:0/4230963952 wait complete. 2023-11-24T08:08:14.588 INFO:tasks.cephfs_test_runner:tests the scenario where 'fs subvolume snapshot info' command ... ok 2023-11-24T08:08:14.589 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:08:14.589 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.812+0000 7f784759e640 1 Processor -- start 2023-11-24T08:08:14.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.812+0000 7f784759e640 1 -- start start 2023-11-24T08:08:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.814+0000 7f784759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7848143570 0x7f78480cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.814+0000 7f784759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f784815e6b0 0x7f78480cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.814+0000 7f784759e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7848071700 con 0x7f7848143950 2023-11-24T08:08:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.814+0000 7f784759e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784815b410 con 0x7f784815e6b0 2023-11-24T08:08:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.814+0000 7f784759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784815b130 con 0x7f7848143570 2023-11-24T08:08:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.814+0000 7f7846d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f784815e6b0 0x7f78480cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.814+0000 7f7846d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f784815e6b0 0x7f78480cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35548/0 (socket says 172.21.15.64:35548) 2023-11-24T08:08:14.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.814+0000 7f7846d9d640 1 -- 172.21.15.64:0/3850606338 learned_addr learned my addr 172.21.15.64:0/3850606338 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.815+0000 7f784659c640 1 --2- 172.21.15.64:0/3850606338 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7848143570 0x7f78480cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.815+0000 7f7846d9d640 1 -- 172.21.15.64:0/3850606338 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7848143570 msgr2=0x7f78480cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.815+0000 7f7846d9d640 1 --2- 172.21.15.64:0/3850606338 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7848143570 0x7f78480cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.815+0000 7f7846d9d640 1 -- 172.21.15.64:0/3850606338 >> v1:172.21.15.91:6789/0 conn(0x7f7848143950 legacy=0x7f784815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.815+0000 7f7846d9d640 1 -- 172.21.15.64:0/3850606338 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784815dbd0 con 0x7f784815e6b0 2023-11-24T08:08:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.816+0000 7f7846d9d640 1 --2- 172.21.15.64:0/3850606338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784815e6b0 0x7f78480cf590 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f7840009e50 tx=0x7f784002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ea734ec4c88202 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:14.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.817+0000 7f784559a640 1 -- 172.21.15.64:0/3850606338 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784002ca70 con 0x7f784815e6b0 2023-11-24T08:08:14.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.817+0000 7f784559a640 1 -- 172.21.15.64:0/3850606338 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784002cbd0 con 0x7f784815e6b0 2023-11-24T08:08:14.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.817+0000 7f784559a640 1 -- 172.21.15.64:0/3850606338 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400343f0 con 0x7f784815e6b0 2023-11-24T08:08:14.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.818+0000 7f784759e640 1 -- 172.21.15.64:0/3850606338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784815e6b0 msgr2=0x7f78480cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:14.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.818+0000 7f784759e640 1 --2- 172.21.15.64:0/3850606338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784815e6b0 0x7f78480cf590 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f7840009e50 tx=0x7f784002bed0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.818+0000 7f784759e640 1 -- 172.21.15.64:0/3850606338 shutdown_connections 2023-11-24T08:08:14.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.818+0000 7f784759e640 1 --2- 172.21.15.64:0/3850606338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784815e6b0 0x7f78480cf590 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.818+0000 7f784759e640 1 --2- 172.21.15.64:0/3850606338 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7848143570 0x7f78480cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.818+0000 7f784759e640 1 -- 172.21.15.64:0/3850606338 >> 172.21.15.64:0/3850606338 conn(0x7f784806a860 msgr2=0x7f784806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:14.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.818+0000 7f784759e640 1 -- 172.21.15.64:0/3850606338 shutdown_connections 2023-11-24T08:08:14.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.818+0000 7f784759e640 1 -- 172.21.15.64:0/3850606338 wait complete. 2023-11-24T08:08:14.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.820+0000 7f784759e640 1 Processor -- start 2023-11-24T08:08:14.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.820+0000 7f784759e640 1 -- start start 2023-11-24T08:08:14.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7848143570 0x7f78481388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:14.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784759e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7848143950 0x7f78480a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:14.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f784815e6b0 0x7f78480aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:14.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7848138e30 con 0x7f7848143570 2023-11-24T08:08:14.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784759e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78480a8820 con 0x7f7848143950 2023-11-24T08:08:14.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78480a8b00 con 0x7f784815e6b0 2023-11-24T08:08:14.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7848143570 0x7f78481388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:14.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7848143570 0x7f78481388f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35564/0 (socket says 172.21.15.64:35564) 2023-11-24T08:08:14.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.821+0000 7f784659c640 1 -- 172.21.15.64:0/75087906 learned_addr learned my addr 172.21.15.64:0/75087906 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f784659c640 1 -- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f784815e6b0 msgr2=0x7f78480aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f784659c640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f784815e6b0 0x7f78480aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f7845d9b640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7848143950 0x7f78480a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f784659c640 1 -- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7848143950 msgr2=0x7f78480a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f784659c640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7848143950 0x7f78480a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f784659c640 1 -- 172.21.15.64:0/75087906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009b30 con 0x7f7848143570 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f7845d9b640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7848143950 0x7f78480a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f784659c640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7848143570 0x7f78481388f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f783800d5d0 tx=0x7f783800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f7836ffd640 1 -- 172.21.15.64:0/75087906 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78380125f0 con 0x7f7848143570 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f7836ffd640 1 -- 172.21.15.64:0/75087906 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7838012770 con 0x7f7848143570 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78480a8de0 con 0x7f7848143570 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.822+0000 7f7836ffd640 1 -- 172.21.15.64:0/75087906 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838012b20 con 0x7f7848143570 2023-11-24T08:08:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.823+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78480a93a0 con 0x7f7848143570 2023-11-24T08:08:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.824+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784804e840 con 0x7f7848143570 2023-11-24T08:08:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.825+0000 7f7836ffd640 1 -- 172.21.15.64:0/75087906 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f7838012c80 con 0x7f7848143570 2023-11-24T08:08:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.825+0000 7f7836ffd640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f782806b3e0 0x7f782806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.825+0000 7f7836ffd640 1 -- 172.21.15.64:0/75087906 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f783808a210 con 0x7f7848143570 2023-11-24T08:08:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.826+0000 7f7845d9b640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f782806b3e0 0x7f782806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.827+0000 7f7845d9b640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f782806b3e0 0x7f782806d8a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f784806aef0 tx=0x7f783c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:14.830+0000 7f7836ffd640 1 -- 172.21.15.64:0/75087906 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7838054de0 con 0x7f7848143570 2023-11-24T08:08:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.028+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78480b3910 con 0x7f7848143570 2023-11-24T08:08:15.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.030+0000 7f7836ffd640 1 -- 172.21.15.64:0/75087906 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+12935 (secure 0 0 0) 0x7f7838088e60 con 0x7f7848143570 2023-11-24T08:08:15.034 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:15.035 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:12.540754+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6833/45948764":"2023-11-25T08:08:09.406219+0000","172.21.15.64:6832/45948764":"2023-11-25T08:08:09.406219+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:15.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.033+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f782806b3e0 msgr2=0x7f782806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.033+0000 7f784759e640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f782806b3e0 0x7f782806d8a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f784806aef0 tx=0x7f783c009040 comp rx=0 tx=0).stop 2023-11-24T08:08:15.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.033+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7848143570 msgr2=0x7f78481388f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.033+0000 7f784759e640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7848143570 0x7f78481388f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f783800d5d0 tx=0x7f783800da90 comp rx=0 tx=0).stop 2023-11-24T08:08:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.034+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 shutdown_connections 2023-11-24T08:08:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.034+0000 7f784759e640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f782806b3e0 0x7f782806d8a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.034+0000 7f784759e640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f784815e6b0 0x7f78480aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.034+0000 7f784759e640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f7848143950 0x7f78480a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.034+0000 7f784759e640 1 --2- 172.21.15.64:0/75087906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7848143570 0x7f78481388f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.034+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 >> 172.21.15.64:0/75087906 conn(0x7f784806a860 msgr2=0x7f78480b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.034+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 shutdown_connections 2023-11-24T08:08:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.034+0000 7f784759e640 1 -- 172.21.15.64:0/75087906 wait complete. 2023-11-24T08:08:15.056 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:15.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.280+0000 7fd99b121640 1 Processor -- start 2023-11-24T08:08:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.281+0000 7fd99b121640 1 -- start start 2023-11-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.282+0000 7fd99b121640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd99415d3b0 0x7fd9940ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.282+0000 7fd99b121640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940c6bf0 con 0x7fd9940d0fa0 2023-11-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.282+0000 7fd99b121640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940d1430 con 0x7fd9940d1d90 2023-11-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.282+0000 7fd99b121640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9940cedc0 con 0x7fd99415d3b0 2023-11-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99991e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd9940d0fa0 0x7fd9940d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50296/0 (socket says 172.21.15.64:50296) 2023-11-24T08:08:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99991e640 1 -- 172.21.15.64:0/1241537221 learned_addr learned my addr 172.21.15.64:0/1241537221 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99a11f640 1 --2- 172.21.15.64:0/1241537221 >> v2:172.21.15.91:3300/0 conn(0x7fd99415d3b0 0x7fd9940ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99911d640 1 -- 172.21.15.64:0/1241537221 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005614409 0 0) 0x7fd9940c6bf0 con 0x7fd9940d0fa0 2023-11-24T08:08:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99911d640 1 -- 172.21.15.64:0/1241537221 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd97c003520 con 0x7fd9940d0fa0 2023-11-24T08:08:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99a11f640 1 -- 172.21.15.64:0/1241537221 >> v1:172.21.15.91:6789/0 conn(0x7fd9940d1d90 legacy=0x7fd9940b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99911d640 1 -- 172.21.15.64:0/1241537221 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270934835 0 0) 0x7fd97c003520 con 0x7fd9940d0fa0 2023-11-24T08:08:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99a11f640 1 -- 172.21.15.64:0/1241537221 >> v1:172.21.15.64:6789/0 conn(0x7fd9940d0fa0 legacy=0x7fd9940d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.283+0000 7fd99a11f640 1 -- 172.21.15.64:0/1241537221 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940c6e40 con 0x7fd99415d3b0 2023-11-24T08:08:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.285+0000 7fd99a11f640 1 --2- 172.21.15.64:0/1241537221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99415d3b0 0x7fd9940ce6c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd984009d90 tx=0x7fd98402bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86bdc6410f50e78b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:15.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.285+0000 7fd99911d640 1 -- 172.21.15.64:0/1241537221 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988001670 con 0x7fd99415d3b0 2023-11-24T08:08:15.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.285+0000 7fd99911d640 1 -- 172.21.15.64:0/1241537221 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98402c8b0 con 0x7fd99415d3b0 2023-11-24T08:08:15.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.285+0000 7fd99911d640 1 -- 172.21.15.64:0/1241537221 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98402cc40 con 0x7fd99415d3b0 2023-11-24T08:08:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.286+0000 7fd99b121640 1 -- 172.21.15.64:0/1241537221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99415d3b0 msgr2=0x7fd9940ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.286+0000 7fd99b121640 1 --2- 172.21.15.64:0/1241537221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99415d3b0 0x7fd9940ce6c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd984009d90 tx=0x7fd98402bd80 comp rx=0 tx=0).stop 2023-11-24T08:08:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.286+0000 7fd99b121640 1 -- 172.21.15.64:0/1241537221 shutdown_connections 2023-11-24T08:08:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.286+0000 7fd99b121640 1 --2- 172.21.15.64:0/1241537221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd99415d3b0 0x7fd9940ce6c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.286+0000 7fd99b121640 1 -- 172.21.15.64:0/1241537221 >> 172.21.15.64:0/1241537221 conn(0x7fd9940d30a0 msgr2=0x7fd9940d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.286+0000 7fd99b121640 1 -- 172.21.15.64:0/1241537221 shutdown_connections 2023-11-24T08:08:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.286+0000 7fd99b121640 1 -- 172.21.15.64:0/1241537221 wait complete. 2023-11-24T08:08:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.288+0000 7fd99b121640 1 Processor -- start 2023-11-24T08:08:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.288+0000 7fd99b121640 1 -- start start 2023-11-24T08:08:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99b121640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9940d0fa0 0x7fd9940c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99b121640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9940d1d90 0x7fd9940cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99b121640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99415d3b0 0x7fd99415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99b121640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940b65d0 con 0x7fd9940d1d90 2023-11-24T08:08:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99b121640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940b5cc0 con 0x7fd9940d0fa0 2023-11-24T08:08:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99b121640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99415af70 con 0x7fd99415d3b0 2023-11-24T08:08:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99991e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9940d1d90 0x7fd9940cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99991e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9940d1d90 0x7fd9940cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35574/0 (socket says 172.21.15.64:35574) 2023-11-24T08:08:15.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.289+0000 7fd99991e640 1 -- 172.21.15.64:0/194096606 learned_addr learned my addr 172.21.15.64:0/194096606 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.290+0000 7fd99a11f640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9940d0fa0 0x7fd9940c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.290+0000 7fd99a920640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99415d3b0 0x7fd99415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.290+0000 7fd99991e640 1 -- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99415d3b0 msgr2=0x7fd99415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.290+0000 7fd99991e640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99415d3b0 0x7fd99415a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.290+0000 7fd99991e640 1 -- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9940d0fa0 msgr2=0x7fd9940c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.290+0000 7fd99991e640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9940d0fa0 0x7fd9940c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.290+0000 7fd99991e640 1 -- 172.21.15.64:0/194096606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984009a70 con 0x7fd9940d1d90 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.290+0000 7fd99991e640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9940d1d90 0x7fd9940cb9d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c760 tx=0x7fd98800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.291+0000 7fd992ffd640 1 -- 172.21.15.64:0/194096606 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988005ad0 con 0x7fd9940d1d90 2023-11-24T08:08:15.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.291+0000 7fd992ffd640 1 -- 172.21.15.64:0/194096606 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd988005c30 con 0x7fd9940d1d90 2023-11-24T08:08:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.291+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99409daf0 con 0x7fd9940d1d90 2023-11-24T08:08:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.291+0000 7fd992ffd640 1 -- 172.21.15.64:0/194096606 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988007070 con 0x7fd9940d1d90 2023-11-24T08:08:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.291+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9940b61f0 con 0x7fd9940d1d90 2023-11-24T08:08:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.292+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99404e840 con 0x7fd9940d1d90 2023-11-24T08:08:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.293+0000 7fd992ffd640 1 -- 172.21.15.64:0/194096606 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd9880071d0 con 0x7fd9940d1d90 2023-11-24T08:08:15.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.294+0000 7fd992ffd640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd98006b3e0 0x7fd98006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.295+0000 7fd992ffd640 1 -- 172.21.15.64:0/194096606 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd988089b50 con 0x7fd9940d1d90 2023-11-24T08:08:15.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.295+0000 7fd99a11f640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd98006b3e0 0x7fd98006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.296+0000 7fd99a11f640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd98006b3e0 0x7fd98006d8a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd984036000 tx=0x7fd984030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:15.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.299+0000 7fd992ffd640 1 -- 172.21.15.64:0/194096606 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd988054610 con 0x7fd9940d1d90 2023-11-24T08:08:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.491+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9940b5530 con 0x7fd9940d1d90 2023-11-24T08:08:15.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.492+0000 7fd992ffd640 1 -- 172.21.15.64:0/194096606 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd988057c30 con 0x7fd9940d1d90 2023-11-24T08:08:15.494 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd98006b3e0 msgr2=0x7fd98006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd98006b3e0 0x7fd98006d8a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd984036000 tx=0x7fd984030000 comp rx=0 tx=0).stop 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9940d1d90 msgr2=0x7fd9940cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9940d1d90 0x7fd9940cb9d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c760 tx=0x7fd98800cc20 comp rx=0 tx=0).stop 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 shutdown_connections 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd98006b3e0 0x7fd98006d8a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd99415d3b0 0x7fd99415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9940d1d90 0x7fd9940cb9d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 --2- 172.21.15.64:0/194096606 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd9940d0fa0 0x7fd9940c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 >> 172.21.15.64:0/194096606 conn(0x7fd9940d30a0 msgr2=0x7fd9941601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:15.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 shutdown_connections 2023-11-24T08:08:15.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.495+0000 7fd99b121640 1 -- 172.21.15.64:0/194096606 wait complete. 2023-11-24T08:08:15.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T08:08:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.739+0000 7f90b089b640 1 Processor -- start 2023-11-24T08:08:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.739+0000 7f90b089b640 1 -- start start 2023-11-24T08:08:15.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90b089b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90b089b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f90ac0ad720 0x7f90ac15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90b089b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ac15c360 con 0x7f90ac0abff0 2023-11-24T08:08:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90b089b640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ac0b41a0 con 0x7f90ac0ad720 2023-11-24T08:08:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90b089b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac06de40 con 0x7f90ac06d520 2023-11-24T08:08:15.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90aad9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f90ac0abff0 0x7f90ac15f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50304/0 (socket says 172.21.15.64:50304) 2023-11-24T08:08:15.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90aad9d640 1 -- 172.21.15.64:0/2590577457 learned_addr learned my addr 172.21.15.64:0/2590577457 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:15.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90a9d9b640 1 --2- 172.21.15.64:0/2590577457 >> v2:172.21.15.91:3300/0 conn(0x7f90ac0ad720 0x7f90ac15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.741+0000 7f90aa59c640 1 --2- 172.21.15.64:0/2590577457 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.742+0000 7f90a959a640 1 -- 172.21.15.64:0/2590577457 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419314768 0 0) 0x7f90ac15c360 con 0x7f90ac0abff0 2023-11-24T08:08:15.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.742+0000 7f90a959a640 1 -- 172.21.15.64:0/2590577457 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9094003520 con 0x7f90ac0abff0 2023-11-24T08:08:15.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.742+0000 7f90a959a640 1 -- 172.21.15.64:0/2590577457 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 336436680 0 0) 0x7f9094003520 con 0x7f90ac0abff0 2023-11-24T08:08:15.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.742+0000 7f90a9d9b640 1 -- 172.21.15.64:0/2590577457 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 msgr2=0x7f90ac06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:15.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.742+0000 7f90a9d9b640 1 --2- 172.21.15.64:0/2590577457 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.742+0000 7f90a9d9b640 1 -- 172.21.15.64:0/2590577457 >> v1:172.21.15.64:6789/0 conn(0x7f90ac0abff0 legacy=0x7f90ac15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.742+0000 7f90a9d9b640 1 -- 172.21.15.64:0/2590577457 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac15ba30 con 0x7f90ac0ad720 2023-11-24T08:08:15.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.744+0000 7f90a9d9b640 1 --2- 172.21.15.64:0/2590577457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0ad720 0x7f90ac15ecd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f90a00097a0 tx=0x7f90a002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9850e3106b957c0d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:15.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.744+0000 7f90a959a640 1 -- 172.21.15.64:0/2590577457 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40019f0 con 0x7f90ac0ad720 2023-11-24T08:08:15.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.744+0000 7f90a959a640 1 -- 172.21.15.64:0/2590577457 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a002e050 con 0x7f90ac0ad720 2023-11-24T08:08:15.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.744+0000 7f90a959a640 1 -- 172.21.15.64:0/2590577457 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0030730 con 0x7f90ac0ad720 2023-11-24T08:08:15.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.744+0000 7f90b089b640 1 -- 172.21.15.64:0/2590577457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0ad720 msgr2=0x7f90ac15ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.744+0000 7f90b089b640 1 --2- 172.21.15.64:0/2590577457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0ad720 0x7f90ac15ecd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f90a00097a0 tx=0x7f90a002b790 comp rx=0 tx=0).stop 2023-11-24T08:08:15.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.745+0000 7f90b089b640 1 -- 172.21.15.64:0/2590577457 shutdown_connections 2023-11-24T08:08:15.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.745+0000 7f90b089b640 1 --2- 172.21.15.64:0/2590577457 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0ad720 0x7f90ac15ecd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.745+0000 7f90b089b640 1 --2- 172.21.15.64:0/2590577457 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.745+0000 7f90b089b640 1 -- 172.21.15.64:0/2590577457 >> 172.21.15.64:0/2590577457 conn(0x7f90ac0b0d80 msgr2=0x7f90ac0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:15.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.745+0000 7f90b089b640 1 -- 172.21.15.64:0/2590577457 shutdown_connections 2023-11-24T08:08:15.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.745+0000 7f90b089b640 1 -- 172.21.15.64:0/2590577457 wait complete. 2023-11-24T08:08:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.746+0000 7f90b089b640 1 Processor -- start 2023-11-24T08:08:15.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.747+0000 7f90b089b640 1 -- start start 2023-11-24T08:08:15.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90b089b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90b089b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0abff0 0x7f90ac15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90b089b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90ac0ad720 0x7f90ac0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90b089b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac14f7a0 con 0x7f90ac0ad720 2023-11-24T08:08:15.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90b089b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac14fa80 con 0x7f90ac0abff0 2023-11-24T08:08:15.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90b089b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac14fd60 con 0x7f90ac06d520 2023-11-24T08:08:15.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90ac0ad720 0x7f90ac0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90ac0ad720 0x7f90ac0cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35588/0 (socket says 172.21.15.64:35588) 2023-11-24T08:08:15.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aad9d640 1 -- 172.21.15.64:0/3867314983 learned_addr learned my addr 172.21.15.64:0/3867314983 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:15.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90a9d9b640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0abff0 0x7f90ac15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aa59c640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aad9d640 1 -- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 msgr2=0x7f90ac14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aad9d640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac14ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aad9d640 1 -- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0abff0 msgr2=0x7f90ac15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:15.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aad9d640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0abff0 0x7f90ac15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:15.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aad9d640 1 -- 172.21.15.64:0/3867314983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0009400 con 0x7f90ac0ad720 2023-11-24T08:08:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.748+0000 7f90aa59c640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.749+0000 7f90aad9d640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90ac0ad720 0x7f90ac0cc470 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f90a4001560 tx=0x7f90a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.749+0000 7f909affd640 1 -- 172.21.15.64:0/3867314983 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4012400 con 0x7f90ac0ad720 2023-11-24T08:08:15.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.749+0000 7f909affd640 1 -- 172.21.15.64:0/3867314983 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a4012560 con 0x7f90ac0ad720 2023-11-24T08:08:15.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.749+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac0cbc70 con 0x7f90ac0ad720 2023-11-24T08:08:15.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.749+0000 7f909affd640 1 -- 172.21.15.64:0/3867314983 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4012910 con 0x7f90ac0ad720 2023-11-24T08:08:15.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.749+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac14f4b0 con 0x7f90ac0ad720 2023-11-24T08:08:15.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.750+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac0cc190 con 0x7f90ac0ad720 2023-11-24T08:08:15.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.751+0000 7f909affd640 1 -- 172.21.15.64:0/3867314983 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f90a4012a70 con 0x7f90ac0ad720 2023-11-24T08:08:15.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.752+0000 7f909affd640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f908c06b3e0 0x7f908c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:15.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.752+0000 7f909affd640 1 -- 172.21.15.64:0/3867314983 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f90a408a5d0 con 0x7f90ac0ad720 2023-11-24T08:08:15.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.753+0000 7f90aa59c640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f908c06b3e0 0x7f908c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:15.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.754+0000 7f90aa59c640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f908c06b3e0 0x7f908c06d8a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f909c009ac0 tx=0x7f909c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:15.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:15.756+0000 7f909affd640 1 -- 172.21.15.64:0/3867314983 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90ac0cc190 con 0x7f90ac0ad720 2023-11-24T08:08:15.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:08:15.915 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:08:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:16.025+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f90ac0cc190 con 0x7f90ac0ad720 2023-11-24T08:08:17.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.035+0000 7f909affd640 1 -- 172.21.15.64:0/3867314983 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v260) v1 ==== 167+0+0 (secure 0 0 0) 0x7f90ac0cc190 con 0x7f90ac0ad720 2023-11-24T08:08:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.039+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f908c06b3e0 msgr2=0x7f908c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.039+0000 7f90b089b640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f908c06b3e0 0x7f908c06d8a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f909c009ac0 tx=0x7f909c008040 comp rx=0 tx=0).stop 2023-11-24T08:08:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.039+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90ac0ad720 msgr2=0x7f90ac0cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.039+0000 7f90b089b640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90ac0ad720 0x7f90ac0cc470 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f90a4001560 tx=0x7f90a400a430 comp rx=0 tx=0).stop 2023-11-24T08:08:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.039+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 shutdown_connections 2023-11-24T08:08:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.039+0000 7f90b089b640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f908c06b3e0 0x7f908c06d8a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.040+0000 7f90b089b640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90ac0ad720 0x7f90ac0cc470 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.040+0000 7f90b089b640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f90ac0abff0 0x7f90ac15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.040+0000 7f90b089b640 1 --2- 172.21.15.64:0/3867314983 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f90ac06d520 0x7f90ac14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.040+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 >> 172.21.15.64:0/3867314983 conn(0x7f90ac0b0d80 msgr2=0x7f90ac0acc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.040+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 shutdown_connections 2023-11-24T08:08:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.040+0000 7f90b089b640 1 -- 172.21.15.64:0/3867314983 wait complete. 2023-11-24T08:08:17.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:08:17.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.284+0000 7fe5dffff640 1 Processor -- start 2023-11-24T08:08:17.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.284+0000 7fe5dffff640 1 -- start start 2023-11-24T08:08:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.286+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e015bda0 0x7fe5e0165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.286+0000 7fe5dffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e0071080 con 0x7fe5e01448c0 2023-11-24T08:08:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.286+0000 7fe5dffff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e0144ca0 con 0x7fe5e006dae0 2023-11-24T08:08:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.286+0000 7fe5dffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015c230 con 0x7fe5e015bda0 2023-11-24T08:08:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.286+0000 7fe5deffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe5e01448c0 0x7fe5e006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55646/0 (socket says 172.21.15.64:55646) 2023-11-24T08:08:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.286+0000 7fe5deffd640 1 -- 172.21.15.64:0/2919501449 learned_addr learned my addr 172.21.15.64:0/2919501449 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5df7fe640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e015bda0 0x7fe5e0165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297639146 0 0) 0x7fe5e0071080 con 0x7fe5e01448c0 2023-11-24T08:08:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5c8003520 con 0x7fe5e01448c0 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 620669114 0 0) 0x7fe5c8003520 con 0x7fe5e01448c0 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e015bda0 msgr2=0x7fe5e0165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5ddffb640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e015bda0 0x7fe5e0165ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 >> v1:172.21.15.91:6789/0 conn(0x7fe5e006dae0 legacy=0x7fe5e015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e015c450 con 0x7fe5e01448c0 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.287+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fe5d0001670 con 0x7fe5e01448c0 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 >> v1:172.21.15.64:6789/0 conn(0x7fe5e01448c0 legacy=0x7fe5e006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5ddffb640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5c8003a10 0x7fe5c8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5ddffb640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5c80047c0 0x7fe5c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5ddffb640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e015bda0 0x7fe5c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015c230 con 0x7fe5c8003a10 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c800ba80 con 0x7fe5c80047c0 2023-11-24T08:08:17.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c800bc40 con 0x7fe5e015bda0 2023-11-24T08:08:17.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5deffd640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5c8003a10 0x7fe5c8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:17.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5deffd640 1 -- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e015bda0 msgr2=0x7fe5c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:17.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5deffd640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e015bda0 0x7fe5c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5deffd640 1 -- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5c80047c0 msgr2=0x7fe5c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:17.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5deffd640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5c80047c0 0x7fe5c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.288+0000 7fe5deffd640 1 -- 172.21.15.64:0/2919501449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8003520 con 0x7fe5c8003a10 2023-11-24T08:08:17.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.289+0000 7fe5deffd640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5c8003a10 0x7fe5c8003e30 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c190 tx=0x7fe5c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2f3e8c3caa6a1d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:17.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0005ad0 con 0x7fe5c8003a10 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d0005c30 con 0x7fe5c8003a10 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5ddffb640 1 -- 172.21.15.64:0/2919501449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d002c8a0 con 0x7fe5c8003a10 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5dffff640 1 -- 172.21.15.64:0/2919501449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5c8003a10 msgr2=0x7fe5c8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5dffff640 1 --2- 172.21.15.64:0/2919501449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5c8003a10 0x7fe5c8003e30 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c190 tx=0x7fe5c80057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5df7fe640 1 -- 172.21.15.64:0/2919501449 reap_dead start 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5dffff640 1 -- 172.21.15.64:0/2919501449 shutdown_connections 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5dffff640 1 -- 172.21.15.64:0/2919501449 >> 172.21.15.64:0/2919501449 conn(0x7fe5e00b0aa0 msgr2=0x7fe5e00b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5dffff640 1 -- 172.21.15.64:0/2919501449 shutdown_connections 2023-11-24T08:08:17.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.290+0000 7fe5dffff640 1 -- 172.21.15.64:0/2919501449 wait complete. 2023-11-24T08:08:17.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.292+0000 7fe5dffff640 1 Processor -- start 2023-11-24T08:08:17.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.292+0000 7fe5dffff640 1 -- start start 2023-11-24T08:08:17.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.293+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5c8003a10 0x7fe5e01f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:17.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.293+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e013a240 0x7fe5e013a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:17.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.293+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e00b3400 0x7fe5e00b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:17.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.293+0000 7fe5dffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e01537f0 con 0x7fe5e00b3400 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.293+0000 7fe5dffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e00b3dc0 con 0x7fe5c8003a10 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.293+0000 7fe5dffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e0153a00 con 0x7fe5e013a240 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e00b3400 0x7fe5e00b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e00b3400 0x7fe5e00b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43622/0 (socket says 172.21.15.64:43622) 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5df7fe640 1 -- 172.21.15.64:0/3717882855 learned_addr learned my addr 172.21.15.64:0/3717882855 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5deffd640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5c8003a10 0x7fe5e01f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5de7fc640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e013a240 0x7fe5e013a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5df7fe640 1 -- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e013a240 msgr2=0x7fe5e013a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5df7fe640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e013a240 0x7fe5e013a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5df7fe640 1 -- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5c8003a10 msgr2=0x7fe5e01f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5df7fe640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5c8003a10 0x7fe5e01f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5df7fe640 1 -- 172.21.15.64:0/3717882855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0153ce0 con 0x7fe5e00b3400 2023-11-24T08:08:17.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.294+0000 7fe5de7fc640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e013a240 0x7fe5e013a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:17.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.295+0000 7fe5df7fe640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e00b3400 0x7fe5e00b3880 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00cd830 tx=0x7fe5d800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:17.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.295+0000 7fe5c7fff640 1 -- 172.21.15.64:0/3717882855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8010040 con 0x7fe5e00b3400 2023-11-24T08:08:17.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.295+0000 7fe5c7fff640 1 -- 172.21.15.64:0/3717882855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d800bae0 con 0x7fe5e00b3400 2023-11-24T08:08:17.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.295+0000 7fe5c7fff640 1 -- 172.21.15.64:0/3717882855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800be70 con 0x7fe5e00b3400 2023-11-24T08:08:17.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.295+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e0153fc0 con 0x7fe5e00b3400 2023-11-24T08:08:17.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.295+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e0155950 con 0x7fe5e00b3400 2023-11-24T08:08:17.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.296+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e004e840 con 0x7fe5e00b3400 2023-11-24T08:08:17.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.297+0000 7fe5c7fff640 1 -- 172.21.15.64:0/3717882855 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe5d801d400 con 0x7fe5e00b3400 2023-11-24T08:08:17.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.298+0000 7fe5c7fff640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5cc06b3e0 0x7fe5cc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:17.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.298+0000 7fe5c7fff640 1 -- 172.21.15.64:0/3717882855 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe5d808a360 con 0x7fe5e00b3400 2023-11-24T08:08:17.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.299+0000 7fe5deffd640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5cc06b3e0 0x7fe5cc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:17.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.300+0000 7fe5deffd640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5cc06b3e0 0x7fe5cc06d8a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0034000 tx=0x7fe5d0009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:17.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.303+0000 7fe5c7fff640 1 -- 172.21.15.64:0/3717882855 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5d8058930 con 0x7fe5e00b3400 2023-11-24T08:08:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.497+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe5e00afbb0 con 0x7fe5e00b3400 2023-11-24T08:08:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.503+0000 7fe5c7fff640 1 -- 172.21.15.64:0/3717882855 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe5d80584c0 con 0x7fe5e00b3400 2023-11-24T08:08:17.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5cc06b3e0 msgr2=0x7fe5cc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5cc06b3e0 0x7fe5cc06d8a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0034000 tx=0x7fe5d0009030 comp rx=0 tx=0).stop 2023-11-24T08:08:17.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e00b3400 msgr2=0x7fe5e00b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e00b3400 0x7fe5e00b3880 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00cd830 tx=0x7fe5d800b480 comp rx=0 tx=0).stop 2023-11-24T08:08:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 shutdown_connections 2023-11-24T08:08:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5cc06b3e0 0x7fe5cc06d8a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e00b3400 0x7fe5e00b3880 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe5e013a240 0x7fe5e013a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 --2- 172.21.15.64:0/3717882855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe5c8003a10 0x7fe5e01f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:17.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 >> 172.21.15.64:0/3717882855 conn(0x7fe5e00b0aa0 msgr2=0x7fe5e01409c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:17.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.506+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 shutdown_connections 2023-11-24T08:08:17.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:17.507+0000 7fe5dffff640 1 -- 172.21.15.64:0/3717882855 wait complete. 2023-11-24T08:08:17.527 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:08:17.527 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:08:17.652 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:08:17.778 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:08:17.778 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:08:17.778 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:17.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:08:17.839 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:08:17.839 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:17.900 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:08:17.900 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:08:17.900 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:17.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:08:17.962 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:08:17.962 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:18.023 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:08:18.023 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:08:18.023 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:18.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.255+0000 7f105231d640 1 Processor -- start 2023-11-24T08:08:18.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.255+0000 7f105231d640 1 -- start start 2023-11-24T08:08:18.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.257+0000 7f105231d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f104c0ad720 0x7f104c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.257+0000 7f105231d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0abff0 0x7f104c0b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.257+0000 7f105231d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c15c360 con 0x7f104c06d520 2023-11-24T08:08:18.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.257+0000 7f105231d640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104c0b41a0 con 0x7f104c0ad720 2023-11-24T08:08:18.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.257+0000 7f105231d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c15ba30 con 0x7f104c0abff0 2023-11-24T08:08:18.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.257+0000 7f104bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f104c06d520 0x7f104c06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55658/0 (socket says 172.21.15.64:55658) 2023-11-24T08:08:18.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.257+0000 7f104bfff640 1 -- 172.21.15.64:0/2319021878 learned_addr learned my addr 172.21.15.64:0/2319021878 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:18.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f1043fff640 1 --2- 172.21.15.64:0/2319021878 >> v2:172.21.15.91:3300/0 conn(0x7f104c0ad720 0x7f104c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f10508ba640 1 --2- 172.21.15.64:0/2319021878 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0abff0 0x7f104c0b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f104b7fe640 1 -- 172.21.15.64:0/2319021878 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480338528 0 0) 0x7f104c15c360 con 0x7f104c06d520 2023-11-24T08:08:18.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f104b7fe640 1 -- 172.21.15.64:0/2319021878 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1038003520 con 0x7f104c06d520 2023-11-24T08:08:18.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f1043fff640 1 -- 172.21.15.64:0/2319021878 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0abff0 msgr2=0x7f104c0b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:18.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f104b7fe640 1 -- 172.21.15.64:0/2319021878 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3530438245 0 0) 0x7f1038003520 con 0x7f104c06d520 2023-11-24T08:08:18.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f1043fff640 1 --2- 172.21.15.64:0/2319021878 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0abff0 0x7f104c0b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f1043fff640 1 -- 172.21.15.64:0/2319021878 >> v1:172.21.15.64:6789/0 conn(0x7f104c06d520 legacy=0x7f104c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.258+0000 7f1043fff640 1 -- 172.21.15.64:0/2319021878 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c15bd10 con 0x7f104c0ad720 2023-11-24T08:08:18.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.260+0000 7f1043fff640 1 --2- 172.21.15.64:0/2319021878 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c0ad720 0x7f104c0b2080 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f10340097a0 tx=0x7f103402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1806017b847efa7e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:18.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.260+0000 7f104b7fe640 1 -- 172.21.15.64:0/2319021878 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c001670 con 0x7f104c0ad720 2023-11-24T08:08:18.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.260+0000 7f104b7fe640 1 -- 172.21.15.64:0/2319021878 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103402e050 con 0x7f104c0ad720 2023-11-24T08:08:18.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.260+0000 7f104b7fe640 1 -- 172.21.15.64:0/2319021878 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034030730 con 0x7f104c0ad720 2023-11-24T08:08:18.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.261+0000 7f105231d640 1 -- 172.21.15.64:0/2319021878 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c0ad720 msgr2=0x7f104c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.261+0000 7f105231d640 1 --2- 172.21.15.64:0/2319021878 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c0ad720 0x7f104c0b2080 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f10340097a0 tx=0x7f103402b790 comp rx=0 tx=0).stop 2023-11-24T08:08:18.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.261+0000 7f105231d640 1 -- 172.21.15.64:0/2319021878 shutdown_connections 2023-11-24T08:08:18.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.261+0000 7f105231d640 1 --2- 172.21.15.64:0/2319021878 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0abff0 0x7f104c0b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.261+0000 7f105231d640 1 --2- 172.21.15.64:0/2319021878 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c0ad720 0x7f104c0b2080 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.261+0000 7f105231d640 1 -- 172.21.15.64:0/2319021878 >> 172.21.15.64:0/2319021878 conn(0x7f104c0b0d80 msgr2=0x7f104c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:18.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.261+0000 7f105231d640 1 -- 172.21.15.64:0/2319021878 shutdown_connections 2023-11-24T08:08:18.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.261+0000 7f105231d640 1 -- 172.21.15.64:0/2319021878 wait complete. 2023-11-24T08:08:18.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.263+0000 7f105231d640 1 Processor -- start 2023-11-24T08:08:18.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.263+0000 7f105231d640 1 -- start start 2023-11-24T08:08:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.264+0000 7f105231d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c06d520 0x7f104c1f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.264+0000 7f105231d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104c0abff0 0x7f104c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.264+0000 7f105231d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0ad720 0x7f104c15ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.264+0000 7f105231d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c159ff0 con 0x7f104c0abff0 2023-11-24T08:08:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.264+0000 7f105231d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c1f0950 con 0x7f104c06d520 2023-11-24T08:08:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.264+0000 7f105231d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c159370 con 0x7f104c0ad720 2023-11-24T08:08:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104c0abff0 0x7f104c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104c0abff0 0x7f104c0cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43632/0 (socket says 172.21.15.64:43632) 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 -- 172.21.15.64:0/1605174916 learned_addr learned my addr 172.21.15.64:0/1605174916 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f104bfff640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c06d520 0x7f104c1f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f10508ba640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0ad720 0x7f104c15ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 -- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0ad720 msgr2=0x7f104c15ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0ad720 0x7f104c15ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 -- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c06d520 msgr2=0x7f104c1f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c06d520 0x7f104c1f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 -- 172.21.15.64:0/1605174916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1034009400 con 0x7f104c0abff0 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f104bfff640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c06d520 0x7f104c1f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.265+0000 7f1043fff640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104c0abff0 0x7f104c0cdc40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1034000c00 tx=0x7f10340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:18.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.266+0000 7f10497fa640 1 -- 172.21.15.64:0/1605174916 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103403b810 con 0x7f104c0abff0 2023-11-24T08:08:18.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.266+0000 7f10497fa640 1 -- 172.21.15.64:0/1605174916 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103403b970 con 0x7f104c0abff0 2023-11-24T08:08:18.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.266+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c159650 con 0x7f104c0abff0 2023-11-24T08:08:18.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.266+0000 7f10497fa640 1 -- 172.21.15.64:0/1605174916 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103403bd20 con 0x7f104c0abff0 2023-11-24T08:08:18.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.266+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c159c10 con 0x7f104c0abff0 2023-11-24T08:08:18.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.267+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104c15a130 con 0x7f104c0abff0 2023-11-24T08:08:18.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.268+0000 7f10497fa640 1 -- 172.21.15.64:0/1605174916 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1034039040 con 0x7f104c0abff0 2023-11-24T08:08:18.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.269+0000 7f10497fa640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102c06b3e0 0x7f102c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.269+0000 7f10497fa640 1 -- 172.21.15.64:0/1605174916 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f10340ab570 con 0x7f104c0abff0 2023-11-24T08:08:18.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.270+0000 7f104bfff640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102c06b3e0 0x7f102c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.271+0000 7f104bfff640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102c06b3e0 0x7f102c06d8a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f103c0011e0 tx=0x7f103c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:18.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.273+0000 7f10497fa640 1 -- 172.21.15.64:0/1605174916 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f104c15a130 con 0x7f104c0abff0 2023-11-24T08:08:18.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.512+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f104c15a130 con 0x7f104c0abff0 2023-11-24T08:08:18.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.512+0000 7f10497fa640 1 -- 172.21.15.64:0/1605174916 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f1034037e80 con 0x7f104c0abff0 2023-11-24T08:08:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:18.515 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/3149637441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":3149637441},{"type":"v1","addr":"172.21.15.91:6836","nonce":3149637441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5761,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2159744284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2159744284},{"type":"v1","addr":"172.21.15.91:6837","nonce":2159744284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5764,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1010090226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1010090226},{"type":"v1","addr":"172.21.15.64:6835","nonce":1010090226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7077,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/339494575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":339494575},{"type":"v1","addr":"172.21.15.64:6833","nonce":339494575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2023-11-24T08:08:18.515 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2023-11-24T08:08:18.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102c06b3e0 msgr2=0x7f102c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102c06b3e0 0x7f102c06d8a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f103c0011e0 tx=0x7f103c009040 comp rx=0 tx=0).stop 2023-11-24T08:08:18.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104c0abff0 msgr2=0x7f104c0cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104c0abff0 0x7f104c0cdc40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1034000c00 tx=0x7f10340046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 shutdown_connections 2023-11-24T08:08:18.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f102c06b3e0 0x7f102c06d8a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f104c0ad720 0x7f104c15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104c0abff0 0x7f104c0cdc40 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 --2- 172.21.15.64:0/1605174916 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f104c06d520 0x7f104c1f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.515+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 >> 172.21.15.64:0/1605174916 conn(0x7f104c0b0d80 msgr2=0x7f104c0aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:18.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.516+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 shutdown_connections 2023-11-24T08:08:18.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.516+0000 7f105231d640 1 -- 172.21.15.64:0/1605174916 wait complete. 2023-11-24T08:08:18.536 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:08:18.536 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:08:18.536 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:08:18.536 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:08:18.536 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:08:18.536 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:08:18.537 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:08:18.537 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:08:18.537 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:08:18.537 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:08:18.537 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:08:18.537 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:08:18.538 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:08:18.538 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:08:18.538 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:08:18.538 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:08:18.553 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:08:18.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:08:18.554 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:08:18.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:08:18.560 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:08:18.560 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:08:18.561 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:08:18.561 DEBUG:teuthology.orchestra.run.smithi091:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:08:18.602 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:08:18.603 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:18.609 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:08:18.610 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:18.610 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:08:18.610 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:18.616 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:08:18.617 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:18.617 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:08:18.765 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:08:18.767 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-11-24T08:08:18.777 INFO:tasks.ceph.mds.d.smithi091.stdout:starting mds.d at 2023-11-24T08:08:18.779 INFO:tasks.ceph.mds.b.smithi091.stdout:starting mds.b at 2023-11-24T08:08:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.806+0000 7f157fdc9640 1 Processor -- start 2023-11-24T08:08:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.806+0000 7f157fdc9640 1 -- start start 2023-11-24T08:08:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.807+0000 7f157fdc9640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f157815b340 0x7f15780abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.808+0000 7f157fdc9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f157806dc30 0x7f15780b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.808+0000 7f157fdc9640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780ac530 con 0x7f15780cbcb0 2023-11-24T08:08:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.808+0000 7f157fdc9640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15780b2080 con 0x7f157815b340 2023-11-24T08:08:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.808+0000 7f157fdc9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15780b2360 con 0x7f157806dc30 2023-11-24T08:08:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.808+0000 7f157db65640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f157815b340 0x7f15780abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.808+0000 7f157e366640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f157806dc30 0x7f15780b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.808+0000 7f157db65640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f157815b340 0x7f15780abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:38768/0 (socket says 172.21.15.64:38768) 2023-11-24T08:08:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.808+0000 7f157db65640 1 -- 172.21.15.64:0/1429115707 learned_addr learned my addr 172.21.15.64:0/1429115707 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.809+0000 7f157e366640 1 -- 172.21.15.64:0/1429115707 >> v2:172.21.15.91:3300/0 conn(0x7f157815b340 msgr2=0x7f15780abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.809+0000 7f157e366640 1 --2- 172.21.15.64:0/1429115707 >> v2:172.21.15.91:3300/0 conn(0x7f157815b340 0x7f15780abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.809+0000 7f157e366640 1 -- 172.21.15.64:0/1429115707 >> v1:172.21.15.91:6789/0 conn(0x7f15780cbcb0 legacy=0x7f157806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.809+0000 7f157e366640 1 -- 172.21.15.64:0/1429115707 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780b2640 con 0x7f157806dc30 2023-11-24T08:08:18.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.809+0000 7f157cb63640 1 -- 172.21.15.64:0/1429115707 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800916889 0 0) 0x7f15780ac530 con 0x7f15780cbcb0 2023-11-24T08:08:18.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.809+0000 7f157db65640 1 --2- 172.21.15.64:0/1429115707 >> v2:172.21.15.91:3300/0 conn(0x7f157815b340 0x7f15780abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:18.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.810+0000 7f157e366640 1 --2- 172.21.15.64:0/1429115707 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f157806dc30 0x7f15780b3120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1570009d90 tx=0x7f157002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=663b35a6d99366bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:18.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.810+0000 7f157cb63640 1 -- 172.21.15.64:0/1429115707 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157002cc60 con 0x7f157806dc30 2023-11-24T08:08:18.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.810+0000 7f157cb63640 1 -- 172.21.15.64:0/1429115707 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157002cdc0 con 0x7f157806dc30 2023-11-24T08:08:18.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.810+0000 7f157cb63640 1 -- 172.21.15.64:0/1429115707 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570034660 con 0x7f157806dc30 2023-11-24T08:08:18.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.811+0000 7f157fdc9640 1 -- 172.21.15.64:0/1429115707 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f157806dc30 msgr2=0x7f15780b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.811+0000 7f157fdc9640 1 --2- 172.21.15.64:0/1429115707 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f157806dc30 0x7f15780b3120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1570009d90 tx=0x7f157002beb0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.811+0000 7f157fdc9640 1 -- 172.21.15.64:0/1429115707 shutdown_connections 2023-11-24T08:08:18.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.811+0000 7f157fdc9640 1 --2- 172.21.15.64:0/1429115707 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f157806dc30 0x7f15780b3120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.811+0000 7f157fdc9640 1 --2- 172.21.15.64:0/1429115707 >> v2:172.21.15.91:3300/0 conn(0x7f157815b340 0x7f15780abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.811+0000 7f157fdc9640 1 -- 172.21.15.64:0/1429115707 >> 172.21.15.64:0/1429115707 conn(0x7f15780b0d80 msgr2=0x7f15780b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:18.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.811+0000 7f157fdc9640 1 -- 172.21.15.64:0/1429115707 shutdown_connections 2023-11-24T08:08:18.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.811+0000 7f157fdc9640 1 -- 172.21.15.64:0/1429115707 wait complete. 2023-11-24T08:08:18.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.813+0000 7f157fdc9640 1 Processor -- start 2023-11-24T08:08:18.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.813+0000 7f157fdc9640 1 -- start start 2023-11-24T08:08:18.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.814+0000 7f157fdc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157806dc30 0x7f1578071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.814+0000 7f157fdc9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15780cbcb0 0x7f15780ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.814+0000 7f157fdc9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f157815b340 0x7f15781f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.814+0000 7f157fdc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781f6820 con 0x7f157806dc30 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.814+0000 7f157fdc9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780724e0 con 0x7f157815b340 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.814+0000 7f157fdc9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1578158ce0 con 0x7f15780cbcb0 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157d364640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15780cbcb0 0x7f15780ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157e366640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f157815b340 0x7f15781f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157db65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157806dc30 0x7f1578071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157e366640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f157815b340 0x7f15781f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:38778/0 (socket says 172.21.15.64:38778) 2023-11-24T08:08:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157e366640 1 -- 172.21.15.64:0/4192919842 learned_addr learned my addr 172.21.15.64:0/4192919842 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:18.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157d364640 1 -- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f157815b340 msgr2=0x7f15781f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157d364640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f157815b340 0x7f15781f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157d364640 1 -- 172.21.15.64:0/4192919842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157806dc30 msgr2=0x7f1578071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:18.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157d364640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157806dc30 0x7f1578071fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:18.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.815+0000 7f157d364640 1 -- 172.21.15.64:0/4192919842 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740094d0 con 0x7f15780cbcb0 2023-11-24T08:08:18.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.816+0000 7f157d364640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15780cbcb0 0x7f15780ca830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f156800c410 tx=0x7f156800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:18.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.816+0000 7f156e7fc640 1 -- 172.21.15.64:0/4192919842 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15680035f0 con 0x7f15780cbcb0 2023-11-24T08:08:18.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.816+0000 7f156e7fc640 1 -- 172.21.15.64:0/4192919842 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1568003750 con 0x7f15780cbcb0 2023-11-24T08:08:18.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.816+0000 7f156e7fc640 1 -- 172.21.15.64:0/4192919842 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568003b40 con 0x7f15780cbcb0 2023-11-24T08:08:18.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.816+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1570009a70 con 0x7f15780cbcb0 2023-11-24T08:08:18.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.816+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157806ccf0 con 0x7f15780cbcb0 2023-11-24T08:08:18.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.817+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1578074770 con 0x7f15780cbcb0 2023-11-24T08:08:18.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.819+0000 7f156e7fc640 1 -- 172.21.15.64:0/4192919842 <== mon.2 v2:172.21.15.91:3301/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1568003ca0 con 0x7f15780cbcb0 2023-11-24T08:08:18.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.819+0000 7f156e7fc640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f155c06b3e0 0x7f155c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:18.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.820+0000 7f156e7fc640 1 -- 172.21.15.64:0/4192919842 <== mon.2 v2:172.21.15.91:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1568089220 con 0x7f15780cbcb0 2023-11-24T08:08:18.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.820+0000 7f157db65640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f155c06b3e0 0x7f155c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:18.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.821+0000 7f157db65640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f155c06b3e0 0x7f155c06d8a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1574006ca0 tx=0x7f1574009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:18.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:18.825+0000 7f156e7fc640 1 -- 172.21.15.64:0/4192919842 <== mon.2 v2:172.21.15.91:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1568053ed0 con 0x7f15780cbcb0 2023-11-24T08:08:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.035+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1578071d80 con 0x7f15780cbcb0 2023-11-24T08:08:19.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.070+0000 7f156e7fc640 1 -- 172.21.15.64:0/4192919842 <== mon.2 v2:172.21.15.91:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) v1 ==== 97+0+0 (secure 0 0 0) 0x7f15680574f0 con 0x7f15780cbcb0 2023-11-24T08:08:19.072 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-11-24T08:08:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f155c06b3e0 msgr2=0x7f155c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f155c06b3e0 0x7f155c06d8a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1574006ca0 tx=0x7f1574009240 comp rx=0 tx=0).stop 2023-11-24T08:08:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15780cbcb0 msgr2=0x7f15780ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15780cbcb0 0x7f15780ca830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f156800c410 tx=0x7f156800c810 comp rx=0 tx=0).stop 2023-11-24T08:08:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 shutdown_connections 2023-11-24T08:08:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f155c06b3e0 0x7f155c06d8a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f157815b340 0x7f15781f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f15780cbcb0 0x7f15780ca830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.073+0000 7f157fdc9640 1 --2- 172.21.15.64:0/4192919842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157806dc30 0x7f1578071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.074+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 >> 172.21.15.64:0/4192919842 conn(0x7f15780b0d80 msgr2=0x7f15780cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:19.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.074+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 shutdown_connections 2023-11-24T08:08:19.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.074+0000 7f157fdc9640 1 -- 172.21.15.64:0/4192919842 wait complete. 2023-11-24T08:08:19.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:08:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.319+0000 7ff149133640 1 Processor -- start 2023-11-24T08:08:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.319+0000 7ff149133640 1 -- start start 2023-11-24T08:08:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.321+0000 7ff149133640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff14406d520 0x7ff14406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.321+0000 7ff149133640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1440ad720 0x7ff14415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.321+0000 7ff149133640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14415c360 con 0x7ff1440abff0 2023-11-24T08:08:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.321+0000 7ff149133640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440b41a0 con 0x7ff14406d520 2023-11-24T08:08:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.321+0000 7ff149133640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14406de40 con 0x7ff1440ad720 2023-11-24T08:08:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.321+0000 7ff142d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff14406d520 0x7ff14406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.321+0000 7ff14359e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff1440abff0 0x7ff14415f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55672/0 (socket says 172.21.15.64:55672) 2023-11-24T08:08:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.321+0000 7ff14359e640 1 -- 172.21.15.64:0/2421999808 learned_addr learned my addr 172.21.15.64:0/2421999808 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:19.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.322+0000 7ff142d9d640 1 -- 172.21.15.64:0/2421999808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1440ad720 msgr2=0x7ff14415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:19.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.322+0000 7ff14259c640 1 --2- 172.21.15.64:0/2421999808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1440ad720 0x7ff14415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.322+0000 7ff142d9d640 1 --2- 172.21.15.64:0/2421999808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1440ad720 0x7ff14415ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.322+0000 7ff142d9d640 1 -- 172.21.15.64:0/2421999808 >> v1:172.21.15.64:6789/0 conn(0x7ff1440abff0 legacy=0x7ff14415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.322+0000 7ff142d9d640 1 -- 172.21.15.64:0/2421999808 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14415ba30 con 0x7ff14406d520 2023-11-24T08:08:19.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.322+0000 7ff141d9b640 1 -- 172.21.15.64:0/2421999808 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944990737 0 0) 0x7ff14415c360 con 0x7ff1440abff0 2023-11-24T08:08:19.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.322+0000 7ff14259c640 1 --2- 172.21.15.64:0/2421999808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1440ad720 0x7ff14415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.323+0000 7ff142d9d640 1 --2- 172.21.15.64:0/2421999808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff14406d520 0x7ff14406d900 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff1340097a0 tx=0x7ff13402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e6a7d47a8efe97 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.323+0000 7ff141d9b640 1 -- 172.21.15.64:0/2421999808 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13402d040 con 0x7ff14406d520 2023-11-24T08:08:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.323+0000 7ff141d9b640 1 -- 172.21.15.64:0/2421999808 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff134031810 con 0x7ff14406d520 2023-11-24T08:08:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.323+0000 7ff141d9b640 1 -- 172.21.15.64:0/2421999808 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134031ba0 con 0x7ff14406d520 2023-11-24T08:08:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.324+0000 7ff149133640 1 -- 172.21.15.64:0/2421999808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff14406d520 msgr2=0x7ff14406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.324+0000 7ff149133640 1 --2- 172.21.15.64:0/2421999808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff14406d520 0x7ff14406d900 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff1340097a0 tx=0x7ff13402b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.324+0000 7ff149133640 1 -- 172.21.15.64:0/2421999808 shutdown_connections 2023-11-24T08:08:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.324+0000 7ff149133640 1 --2- 172.21.15.64:0/2421999808 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1440ad720 0x7ff14415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.324+0000 7ff149133640 1 --2- 172.21.15.64:0/2421999808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff14406d520 0x7ff14406d900 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.324+0000 7ff149133640 1 -- 172.21.15.64:0/2421999808 >> 172.21.15.64:0/2421999808 conn(0x7ff1440b0d80 msgr2=0x7ff1440b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.324+0000 7ff149133640 1 -- 172.21.15.64:0/2421999808 shutdown_connections 2023-11-24T08:08:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.324+0000 7ff149133640 1 -- 172.21.15.64:0/2421999808 wait complete. 2023-11-24T08:08:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.326+0000 7ff149133640 1 Processor -- start 2023-11-24T08:08:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.326+0000 7ff149133640 1 -- start start 2023-11-24T08:08:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.327+0000 7ff149133640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff14406d520 0x7ff144153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.327+0000 7ff149133640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1440abff0 0x7ff144140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.327+0000 7ff149133640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1440ad720 0x7ff14415a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.327+0000 7ff149133640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1440ce450 con 0x7ff1440ad720 2023-11-24T08:08:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.327+0000 7ff149133640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441543e0 con 0x7ff1440abff0 2023-11-24T08:08:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.327+0000 7ff149133640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff144144380 con 0x7ff14406d520 2023-11-24T08:08:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1440ad720 0x7ff14415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1440ad720 0x7ff14415a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43672/0 (socket says 172.21.15.64:43672) 2023-11-24T08:08:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 -- 172.21.15.64:0/1407524767 learned_addr learned my addr 172.21.15.64:0/1407524767 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14259c640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1440abff0 0x7ff144140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff142d9d640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff14406d520 0x7ff144153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 -- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff14406d520 msgr2=0x7ff144153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff14406d520 0x7ff144153ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 -- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1440abff0 msgr2=0x7ff144140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1440abff0 0x7ff144140fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 -- 172.21.15.64:0/1407524767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134009400 con 0x7ff1440ad720 2023-11-24T08:08:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14259c640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1440abff0 0x7ff144140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.328+0000 7ff14359e640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1440ad720 0x7ff14415a320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff13c001560 tx=0x7ff13c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.329+0000 7ff12b7fe640 1 -- 172.21.15.64:0/1407524767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00a640 con 0x7ff1440ad720 2023-11-24T08:08:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.329+0000 7ff12b7fe640 1 -- 172.21.15.64:0/1407524767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c00a7a0 con 0x7ff1440ad720 2023-11-24T08:08:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.329+0000 7ff12b7fe640 1 -- 172.21.15.64:0/1407524767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00ab30 con 0x7ff1440ad720 2023-11-24T08:08:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.329+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff144141510 con 0x7ff1440ad720 2023-11-24T08:08:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.329+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1440cdb20 con 0x7ff1440ad720 2023-11-24T08:08:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.330+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1440ce600 con 0x7ff1440ad720 2023-11-24T08:08:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.331+0000 7ff12b7fe640 1 -- 172.21.15.64:0/1407524767 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff13c00ac90 con 0x7ff1440ad720 2023-11-24T08:08:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.332+0000 7ff12b7fe640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff13006b3e0 0x7ff13006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.332+0000 7ff12b7fe640 1 -- 172.21.15.64:0/1407524767 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff13c0892f0 con 0x7ff1440ad720 2023-11-24T08:08:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.333+0000 7ff142d9d640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff13006b3e0 0x7ff13006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.334+0000 7ff142d9d640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff13006b3e0 0x7ff13006d8a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff134036000 tx=0x7ff13403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.337+0000 7ff12b7fe640 1 -- 172.21.15.64:0/1407524767 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1440ce600 con 0x7ff1440ad720 2023-11-24T08:08:19.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.597+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff144074770 con 0x7ff1440ad720 2023-11-24T08:08:19.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.598+0000 7ff12b7fe640 1 -- 172.21.15.64:0/1407524767 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff13c018090 con 0x7ff1440ad720 2023-11-24T08:08:19.600 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:19.600 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllV/IJxAAMEWwBM8H9uSLw5Bd93bScA==", 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBl35hoEBAADvW80b1cRhhgBRkYgNaftg==", 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBlZ1rGORAAjjOrPcL663guE4Q9MKA+6g==", 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.603 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlqV+1IRAAaLczF6+eDIuMM6X5Jgni/A==", 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCuWGBlsrGpIBAAVTJlhEtf7rPd1J4wUZRWTg==", 2023-11-24T08:08:19.604 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCxWGBl+u/pLBAAwh85K8evI69aY9Nij5dBRQ==", 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:08:19.605 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC1WGBl1kGECxAAtoQ9UI319UpAU7OwZehzew==", 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC4WGBlSlE2JRAAjPT1020nD4Dnrw1hrB2bKQ==", 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-11-24T08:08:19.607 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC/WGBlrRSGKRAAaiWZZvvd7O8PuipXF1oAUQ==", 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDDWGBls+mkBxAAJZXdXFq1+8QC5dBE14+YJA==", 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:08:19.608 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDGWGBl71+WIBAA/NNCMwpi0ycNJvXZFHKuIA==", 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:08:19.609 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJWGBl11BPKxAAnXhK4EqlIyvbHCpHzPCDUg==", 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA==", 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-11-24T08:08:19.611 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlsyr3DBAASDVvIWUkg06i7YARod5spw==", 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlhsnfABAA5sZEDm1Uyin3dIOEoMX+gg==", 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.612 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCqWGBlT9iOFxAAiupeG8QnYzYA8pJYftfxNA==", 2023-11-24T08:08:19.613 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-11-24T08:08:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-11-24T08:08:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-11-24T08:08:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-11-24T08:08:19.614 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-11-24T08:08:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCnWGBlKY7iFBAAmkZ+pYWFmZ7aPH06BiE0+w==", 2023-11-24T08:08:19.615 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:08:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-11-24T08:08:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-11-24T08:08:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.617 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-11-24T08:08:19.617 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBllqymGhAA+wKOSJeuBn60R4gZJ3edoA==", 2023-11-24T08:08:19.617 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.617 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:08:19.618 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:08:19.618 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:19.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:19.618 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-11-24T08:08:19.618 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCoWGBloKAEDhAAFSBzeCcgrTtmsfXrxKqCRQ==", 2023-11-24T08:08:19.619 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-11-24T08:08:19.619 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-11-24T08:08:19.619 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-11-24T08:08:19.619 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-11-24T08:08:19.619 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.619 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:19.619 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-11-24T08:08:19.619 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:08:19.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff13006b3e0 msgr2=0x7ff13006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff13006b3e0 0x7ff13006d8a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff134036000 tx=0x7ff13403c040 comp rx=0 tx=0).stop 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1440ad720 msgr2=0x7ff14415a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1440ad720 0x7ff14415a320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff13c001560 tx=0x7ff13c00cc70 comp rx=0 tx=0).stop 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 shutdown_connections 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff13006b3e0 0x7ff13006d8a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1440ad720 0x7ff14415a320 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1440abff0 0x7ff144140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 --2- 172.21.15.64:0/1407524767 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff14406d520 0x7ff144153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 >> 172.21.15.64:0/1407524767 conn(0x7ff1440b0d80 msgr2=0x7ff1440af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 shutdown_connections 2023-11-24T08:08:19.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.601+0000 7ff149133640 1 -- 172.21.15.64:0/1407524767 wait complete. 2023-11-24T08:08:19.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:19.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.851+0000 7f38b4863640 1 Processor -- start 2023-11-24T08:08:19.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.851+0000 7f38b4863640 1 -- start start 2023-11-24T08:08:19.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.852+0000 7f38b4863640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f38b015d370 0x7f38b00d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.852+0000 7f38b4863640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b00cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.852+0000 7f38b4863640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b00d0360 con 0x7f38b00cff00 2023-11-24T08:08:19.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.852+0000 7f38b4863640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b00afd30 con 0x7f38b015d370 2023-11-24T08:08:19.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.852+0000 7f38b4863640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00d0c30 con 0x7f38b00b1f00 2023-11-24T08:08:19.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.853+0000 7f38aed9d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f38b015d370 0x7f38b00d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.853+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b00cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.853+0000 7f38aed9d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f38b015d370 0x7f38b00d1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:38800/0 (socket says 172.21.15.64:38800) 2023-11-24T08:08:19.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.853+0000 7f38aed9d640 1 -- 172.21.15.64:0/3490806925 learned_addr learned my addr 172.21.15.64:0/3490806925 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.854+0000 7f38aed9d640 1 -- 172.21.15.64:0/3490806925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 msgr2=0x7f38b00cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.854+0000 7f38aed9d640 1 --2- 172.21.15.64:0/3490806925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b00cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.854+0000 7f38aed9d640 1 -- 172.21.15.64:0/3490806925 >> v1:172.21.15.91:6789/0 conn(0x7f38b00cff00 legacy=0x7f38b00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.854+0000 7f38aed9d640 1 -- 172.21.15.64:0/3490806925 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00cef60 con 0x7f38b015d370 2023-11-24T08:08:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.854+0000 7f38a7fff640 1 --2- 172.21.15.64:0/3490806925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b00cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:19.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.855+0000 7f38aed9d640 1 --2- 172.21.15.64:0/3490806925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 0x7f38b00d1c70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009870 tx=0x7f38a002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17ca7a907d3fb953 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:19.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38ae59c640 1 -- 172.21.15.64:0/3490806925 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0031640 con 0x7f38b015d370 2023-11-24T08:08:19.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38ae59c640 1 -- 172.21.15.64:0/3490806925 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a00317a0 con 0x7f38b015d370 2023-11-24T08:08:19.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38ae59c640 1 -- 172.21.15.64:0/3490806925 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0031b30 con 0x7f38b015d370 2023-11-24T08:08:19.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38b4863640 1 -- 172.21.15.64:0/3490806925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 msgr2=0x7f38b00d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38b4863640 1 --2- 172.21.15.64:0/3490806925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 0x7f38b00d1c70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009870 tx=0x7f38a002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38b4863640 1 -- 172.21.15.64:0/3490806925 shutdown_connections 2023-11-24T08:08:19.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38b4863640 1 --2- 172.21.15.64:0/3490806925 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b00cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38b4863640 1 --2- 172.21.15.64:0/3490806925 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 0x7f38b00d1c70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.856+0000 7f38b4863640 1 -- 172.21.15.64:0/3490806925 >> 172.21.15.64:0/3490806925 conn(0x7f38b009c3d0 msgr2=0x7f38b00d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:19.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.857+0000 7f38b4863640 1 -- 172.21.15.64:0/3490806925 shutdown_connections 2023-11-24T08:08:19.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.857+0000 7f38b4863640 1 -- 172.21.15.64:0/3490806925 wait complete. 2023-11-24T08:08:19.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.858+0000 7f38b4863640 1 Processor -- start 2023-11-24T08:08:19.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.858+0000 7f38b4863640 1 -- start start 2023-11-24T08:08:19.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.859+0000 7f38b4863640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.859+0000 7f38b4863640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b00cff00 0x7f38b00c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.859+0000 7f38b4863640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 0x7f38b00b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.859+0000 7f38b4863640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00c8fe0 con 0x7f38b00cff00 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.859+0000 7f38b4863640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b014ea60 con 0x7f38b015d370 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.859+0000 7f38b4863640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00c8360 con 0x7f38b00b1f00 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b00cff00 0x7f38b00c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b00cff00 0x7f38b00c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43682/0 (socket says 172.21.15.64:43682) 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38a7fff640 1 -- 172.21.15.64:0/2090832283 learned_addr learned my addr 172.21.15.64:0/2090832283 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38aed9d640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38a7fff640 1 -- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 msgr2=0x7f38b014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38a7fff640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b014e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38af59e640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 0x7f38b00b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38a7fff640 1 -- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 msgr2=0x7f38b00b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38a7fff640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 0x7f38b00b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38a7fff640 1 -- 172.21.15.64:0/2090832283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898009400 con 0x7f38b00cff00 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.860+0000 7f38af59e640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 0x7f38b00b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.861+0000 7f38a7fff640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b00cff00 0x7f38b00c9400 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f389800ebe0 tx=0x7f389800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.861+0000 7f38a77fe640 1 -- 172.21.15.64:0/2090832283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898009760 con 0x7f38b00cff00 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.861+0000 7f38a77fe640 1 -- 172.21.15.64:0/2090832283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389800cc30 con 0x7f38b00cff00 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.861+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a00094d0 con 0x7f38b00cff00 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.861+0000 7f38a77fe640 1 -- 172.21.15.64:0/2090832283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389800eee0 con 0x7f38b00cff00 2023-11-24T08:08:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.861+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b00c8920 con 0x7f38b00cff00 2023-11-24T08:08:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.862+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b00c9250 con 0x7f38b00cff00 2023-11-24T08:08:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.863+0000 7f38a77fe640 1 -- 172.21.15.64:0/2090832283 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f3898013470 con 0x7f38b00cff00 2023-11-24T08:08:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.864+0000 7f38a77fe640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f389006b3e0 0x7f389006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.864+0000 7f38a77fe640 1 -- 172.21.15.64:0/2090832283 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f389808a040 con 0x7f38b00cff00 2023-11-24T08:08:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.865+0000 7f38aed9d640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f389006b3e0 0x7f389006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:19.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.866+0000 7f38aed9d640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f389006b3e0 0x7f389006d8a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f38a0036000 tx=0x7f38a003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:19.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:19.868+0000 7f38a77fe640 1 -- 172.21.15.64:0/2090832283 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3898054ba0 con 0x7f38b00cff00 2023-11-24T08:08:20.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.058+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38b004e840 con 0x7f38b00cff00 2023-11-24T08:08:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.065+0000 7f38a77fe640 1 -- 172.21.15.64:0/2090832283 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+12810 (secure 0 0 0) 0x7f38980581c0 con 0x7f38b00cff00 2023-11-24T08:08:20.067 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:20.067 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":83,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:19.079527+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.067+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f389006b3e0 msgr2=0x7f389006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f389006b3e0 0x7f389006d8a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f38a0036000 tx=0x7f38a003c040 comp rx=0 tx=0).stop 2023-11-24T08:08:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b00cff00 msgr2=0x7f38b00c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b00cff00 0x7f38b00c9400 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f389800ebe0 tx=0x7f389800c520 comp rx=0 tx=0).stop 2023-11-24T08:08:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 shutdown_connections 2023-11-24T08:08:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f389006b3e0 0x7f389006d8a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f38b015d370 0x7f38b00b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38b00cff00 0x7f38b00c9400 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 --2- 172.21.15.64:0/2090832283 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f38b00b1f00 0x7f38b014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 >> 172.21.15.64:0/2090832283 conn(0x7f38b009c3d0 msgr2=0x7f38b00d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:20.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 shutdown_connections 2023-11-24T08:08:20.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.068+0000 7f38b4863640 1 -- 172.21.15.64:0/2090832283 wait complete. 2023-11-24T08:08:20.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:20.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:20.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.306+0000 7fb3297fb640 1 Processor -- start 2023-11-24T08:08:20.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.306+0000 7fb3297fb640 1 -- start start 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3297fb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb32415b350 0x7fb3240abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3297fb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32409cb30 0x7fb3240b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3297fb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3240adb60 0x7fb3240adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3297fb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32415b730 con 0x7fb3240adb60 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3297fb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3240ac740 con 0x7fb32415b350 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3297fb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240ac460 con 0x7fb32409cb30 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3237fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3240adb60 0x7fb3240adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3237fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3240adb60 0x7fb3240adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43688/0 (socket says 172.21.15.64:43688) 2023-11-24T08:08:20.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3237fe640 1 -- 172.21.15.64:0/870528022 learned_addr learned my addr 172.21.15.64:0/870528022 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:20.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3237fe640 1 -- 172.21.15.64:0/870528022 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32409cb30 msgr2=0x7fb3240b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:20.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb3227fc640 1 --2- 172.21.15.64:0/870528022 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32409cb30 0x7fb3240b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.308+0000 7fb322ffd640 1 --2- 172.21.15.64:0/870528022 >> v2:172.21.15.91:3300/0 conn(0x7fb32415b350 0x7fb3240abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.309+0000 7fb3237fe640 1 --2- 172.21.15.64:0/870528022 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32409cb30 0x7fb3240b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.309+0000 7fb3237fe640 1 -- 172.21.15.64:0/870528022 >> v2:172.21.15.91:3300/0 conn(0x7fb32415b350 msgr2=0x7fb3240abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:20.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.309+0000 7fb3237fe640 1 --2- 172.21.15.64:0/870528022 >> v2:172.21.15.91:3300/0 conn(0x7fb32415b350 0x7fb3240abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.309+0000 7fb3237fe640 1 -- 172.21.15.64:0/870528022 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240b2a50 con 0x7fb3240adb60 2023-11-24T08:08:20.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.309+0000 7fb322ffd640 1 --2- 172.21.15.64:0/870528022 >> v2:172.21.15.91:3300/0 conn(0x7fb32415b350 0x7fb3240abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:20.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.310+0000 7fb3237fe640 1 --2- 172.21.15.64:0/870528022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3240adb60 0x7fb3240adfb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb3240b5100 tx=0x7fb31c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bce3c1e12615a7e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:20.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.310+0000 7fb321ffb640 1 -- 172.21.15.64:0/870528022 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c031870 con 0x7fb3240adb60 2023-11-24T08:08:20.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.310+0000 7fb321ffb640 1 -- 172.21.15.64:0/870528022 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c0319d0 con 0x7fb3240adb60 2023-11-24T08:08:20.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.310+0000 7fb321ffb640 1 -- 172.21.15.64:0/870528022 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c031d60 con 0x7fb3240adb60 2023-11-24T08:08:20.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 -- 172.21.15.64:0/870528022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3240adb60 msgr2=0x7fb3240adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 --2- 172.21.15.64:0/870528022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3240adb60 0x7fb3240adfb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb3240b5100 tx=0x7fb31c02b910 comp rx=0 tx=0).stop 2023-11-24T08:08:20.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 -- 172.21.15.64:0/870528022 shutdown_connections 2023-11-24T08:08:20.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 --2- 172.21.15.64:0/870528022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3240adb60 0x7fb3240adfb0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 --2- 172.21.15.64:0/870528022 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32409cb30 0x7fb3240b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 --2- 172.21.15.64:0/870528022 >> v2:172.21.15.91:3300/0 conn(0x7fb32415b350 0x7fb3240abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 -- 172.21.15.64:0/870528022 >> 172.21.15.64:0/870528022 conn(0x7fb3240b0ba0 msgr2=0x7fb3240b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:20.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 -- 172.21.15.64:0/870528022 shutdown_connections 2023-11-24T08:08:20.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.311+0000 7fb3297fb640 1 -- 172.21.15.64:0/870528022 wait complete. 2023-11-24T08:08:20.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.313+0000 7fb3297fb640 1 Processor -- start 2023-11-24T08:08:20.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.313+0000 7fb3297fb640 1 -- start start 2023-11-24T08:08:20.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.314+0000 7fb3297fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb32409cb30 0x7fb324153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.314+0000 7fb3297fb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3240adb60 0x7fb324158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.314+0000 7fb3297fb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32415b350 0x7fb324152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.314+0000 7fb3297fb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb324154e30 con 0x7fb32409cb30 2023-11-24T08:08:20.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.314+0000 7fb3297fb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32414c0d0 con 0x7fb3240adb60 2023-11-24T08:08:20.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.314+0000 7fb3297fb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb324159250 con 0x7fb32415b350 2023-11-24T08:08:20.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.314+0000 7fb322ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb32409cb30 0x7fb324153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb322ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb32409cb30 0x7fb324153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43694/0 (socket says 172.21.15.64:43694) 2023-11-24T08:08:20.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb322ffd640 1 -- 172.21.15.64:0/4219033280 learned_addr learned my addr 172.21.15.64:0/4219033280 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:20.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb3227fc640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3240adb60 0x7fb324158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb3237fe640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32415b350 0x7fb324152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb322ffd640 1 -- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32415b350 msgr2=0x7fb324152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb322ffd640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32415b350 0x7fb324152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb322ffd640 1 -- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3240adb60 msgr2=0x7fb324158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb322ffd640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3240adb60 0x7fb324158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb322ffd640 1 -- 172.21.15.64:0/4219033280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c0094d0 con 0x7fb32409cb30 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.315+0000 7fb322ffd640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb32409cb30 0x7fb324153b40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb31400cf90 tx=0x7fb31400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.316+0000 7fb31b7fe640 1 -- 172.21.15.64:0/4219033280 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31400d840 con 0x7fb32409cb30 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.316+0000 7fb31b7fe640 1 -- 172.21.15.64:0/4219033280 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31400d9a0 con 0x7fb32409cb30 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.316+0000 7fb31b7fe640 1 -- 172.21.15.64:0/4219033280 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31400dd30 con 0x7fb32409cb30 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.316+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3241541b0 con 0x7fb32409cb30 2023-11-24T08:08:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.316+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb324154770 con 0x7fb32409cb30 2023-11-24T08:08:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.317+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32409d200 con 0x7fb32409cb30 2023-11-24T08:08:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.318+0000 7fb31b7fe640 1 -- 172.21.15.64:0/4219033280 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb314015440 con 0x7fb32409cb30 2023-11-24T08:08:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.319+0000 7fb31b7fe640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb30406b3e0 0x7fb30406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.319+0000 7fb31b7fe640 1 -- 172.21.15.64:0/4219033280 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb31408a620 con 0x7fb32409cb30 2023-11-24T08:08:20.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.320+0000 7fb3227fc640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb30406b3e0 0x7fb30406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.321+0000 7fb3227fc640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb30406b3e0 0x7fb30406d8a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb30c006fd0 tx=0x7fb30c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:20.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.323+0000 7fb31b7fe640 1 -- 172.21.15.64:0/4219033280 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3140552b0 con 0x7fb32409cb30 2023-11-24T08:08:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.517+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb32404e920 con 0x7fb32409cb30 2023-11-24T08:08:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.518+0000 7fb31b7fe640 1 -- 172.21.15.64:0/4219033280 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb31401a020 con 0x7fb32409cb30 2023-11-24T08:08:20.520 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:08:20.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb30406b3e0 msgr2=0x7fb30406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb30406b3e0 0x7fb30406d8a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb30c006fd0 tx=0x7fb30c008040 comp rx=0 tx=0).stop 2023-11-24T08:08:20.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb32409cb30 msgr2=0x7fb324153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb32409cb30 0x7fb324153b40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb31400cf90 tx=0x7fb31400a7d0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 shutdown_connections 2023-11-24T08:08:20.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb30406b3e0 0x7fb30406d8a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb32415b350 0x7fb324152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb3240adb60 0x7fb324158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 --2- 172.21.15.64:0/4219033280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb32409cb30 0x7fb324153b40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 >> 172.21.15.64:0/4219033280 conn(0x7fb3240b0ba0 msgr2=0x7fb3240aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:20.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 shutdown_connections 2023-11-24T08:08:20.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.521+0000 7fb3297fb640 1 -- 172.21.15.64:0/4219033280 wait complete. 2023-11-24T08:08:20.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:08:20.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.750+0000 7fcbd359e640 1 Processor -- start 2023-11-24T08:08:20.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.751+0000 7fcbd359e640 1 -- start start 2023-11-24T08:08:20.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.752+0000 7fcbd359e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcbd413dfd0 0x7fcbd415be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.752+0000 7fcbd359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcbd40cf500 0x7fcbd406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.752+0000 7fcbd359e640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd40ce7c0 con 0x7fcbd40cf050 2023-11-24T08:08:20.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.752+0000 7fcbd359e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd4140900 con 0x7fcbd40cf500 2023-11-24T08:08:20.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.752+0000 7fcbd359e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd415fa70 con 0x7fcbd413dfd0 2023-11-24T08:08:20.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.753+0000 7fcbd2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcbd40cf500 0x7fcbd406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.753+0000 7fcbd2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcbd40cf500 0x7fcbd406c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43702/0 (socket says 172.21.15.64:43702) 2023-11-24T08:08:20.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.753+0000 7fcbd2d9d640 1 -- 172.21.15.64:0/2735640061 learned_addr learned my addr 172.21.15.64:0/2735640061 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:20.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.753+0000 7fcbd259c640 1 --2- 172.21.15.64:0/2735640061 >> v2:172.21.15.91:3300/0 conn(0x7fcbd413dfd0 0x7fcbd415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.753+0000 7fcbd2d9d640 1 -- 172.21.15.64:0/2735640061 >> v2:172.21.15.91:3300/0 conn(0x7fcbd413dfd0 msgr2=0x7fcbd415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:20.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.753+0000 7fcbd2d9d640 1 --2- 172.21.15.64:0/2735640061 >> v2:172.21.15.91:3300/0 conn(0x7fcbd413dfd0 0x7fcbd415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.753+0000 7fcbd2d9d640 1 -- 172.21.15.64:0/2735640061 >> v1:172.21.15.91:6789/0 conn(0x7fcbd40cf050 legacy=0x7fcbd406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:20.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.753+0000 7fcbd2d9d640 1 -- 172.21.15.64:0/2735640061 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd415ee60 con 0x7fcbd40cf500 2023-11-24T08:08:20.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.754+0000 7fcbd2d9d640 1 --2- 172.21.15.64:0/2735640061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf500 0x7fcbd406c840 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40cfc10 tx=0x7fcbcc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df19e786bf02ac2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:20.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd159a640 1 -- 172.21.15.64:0/2735640061 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc031870 con 0x7fcbd40cf500 2023-11-24T08:08:20.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd159a640 1 -- 172.21.15.64:0/2735640061 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc0319d0 con 0x7fcbd40cf500 2023-11-24T08:08:20.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd159a640 1 -- 172.21.15.64:0/2735640061 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc031d60 con 0x7fcbd40cf500 2023-11-24T08:08:20.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd359e640 1 -- 172.21.15.64:0/2735640061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf500 msgr2=0x7fcbd406c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd359e640 1 --2- 172.21.15.64:0/2735640061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf500 0x7fcbd406c840 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40cfc10 tx=0x7fcbcc02b910 comp rx=0 tx=0).stop 2023-11-24T08:08:20.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd359e640 1 -- 172.21.15.64:0/2735640061 shutdown_connections 2023-11-24T08:08:20.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd359e640 1 --2- 172.21.15.64:0/2735640061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf500 0x7fcbd406c840 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd359e640 1 --2- 172.21.15.64:0/2735640061 >> v2:172.21.15.91:3300/0 conn(0x7fcbd413dfd0 0x7fcbd415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.755+0000 7fcbd359e640 1 -- 172.21.15.64:0/2735640061 >> 172.21.15.64:0/2735640061 conn(0x7fcbd406a860 msgr2=0x7fcbd406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:20.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.756+0000 7fcbd359e640 1 -- 172.21.15.64:0/2735640061 shutdown_connections 2023-11-24T08:08:20.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.756+0000 7fcbd359e640 1 -- 172.21.15.64:0/2735640061 wait complete. 2023-11-24T08:08:20.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.757+0000 7fcbd359e640 1 Processor -- start 2023-11-24T08:08:20.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.757+0000 7fcbd359e640 1 -- start start 2023-11-24T08:08:20.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.758+0000 7fcbd359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf050 0x7fcbd41396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.758+0000 7fcbd359e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcbd40cf500 0x7fcbd415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.758+0000 7fcbd359e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd413dfd0 0x7fcbd415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.758+0000 7fcbd359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4139c20 con 0x7fcbd40cf050 2023-11-24T08:08:20.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.758+0000 7fcbd359e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4157750 con 0x7fcbd413dfd0 2023-11-24T08:08:20.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.758+0000 7fcbd359e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4156b40 con 0x7fcbd40cf500 2023-11-24T08:08:20.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf050 0x7fcbd41396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf050 0x7fcbd41396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43708/0 (socket says 172.21.15.64:43708) 2023-11-24T08:08:20.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 -- 172.21.15.64:0/837275557 learned_addr learned my addr 172.21.15.64:0/837275557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:20.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd2d9d640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd413dfd0 0x7fcbd415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 -- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcbd40cf500 msgr2=0x7fcbd415cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcbd40cf500 0x7fcbd415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 -- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd413dfd0 msgr2=0x7fcbd415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd413dfd0 0x7fcbd415d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 -- 172.21.15.64:0/837275557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0094d0 con 0x7fcbd40cf050 2023-11-24T08:08:20.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.759+0000 7fcbd259c640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf050 0x7fcbd41396e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4007e40 tx=0x7fcbc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:20.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.760+0000 7fcbcaffd640 1 -- 172.21.15.64:0/837275557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4014620 con 0x7fcbd40cf050 2023-11-24T08:08:20.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.760+0000 7fcbcaffd640 1 -- 172.21.15.64:0/837275557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc400ee90 con 0x7fcbd40cf050 2023-11-24T08:08:20.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.760+0000 7fcbcaffd640 1 -- 172.21.15.64:0/837275557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc40149c0 con 0x7fcbd40cf050 2023-11-24T08:08:20.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.760+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd4156e20 con 0x7fcbd40cf050 2023-11-24T08:08:20.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.760+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd41573e0 con 0x7fcbd40cf050 2023-11-24T08:08:20.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.761+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd4157570 con 0x7fcbd40cf050 2023-11-24T08:08:20.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.762+0000 7fcbcaffd640 1 -- 172.21.15.64:0/837275557 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fcbc4014b20 con 0x7fcbd40cf050 2023-11-24T08:08:20.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.763+0000 7fcbcaffd640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcbb406b3e0 0x7fcbb406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:20.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.763+0000 7fcbcaffd640 1 -- 172.21.15.64:0/837275557 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcbc408a740 con 0x7fcbd40cf050 2023-11-24T08:08:20.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.764+0000 7fcbd1d9b640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcbb406b3e0 0x7fcbb406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:20.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.765+0000 7fcbd1d9b640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcbb406b3e0 0x7fcbb406d8a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcbd414ae90 tx=0x7fcbbc0079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:20.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.767+0000 7fcbcaffd640 1 -- 172.21.15.64:0/837275557 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcbc4055240 con 0x7fcbd40cf050 2023-11-24T08:08:20.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.963+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcbd404e840 con 0x7fcbd40cf050 2023-11-24T08:08:20.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.964+0000 7fcbcaffd640 1 -- 172.21.15.64:0/837275557 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcbc4058860 con 0x7fcbd40cf050 2023-11-24T08:08:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:20.971 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:08:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:20.971 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-11-24T08:08:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-11-24T08:08:20.972 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:20.972 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:08:20.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcbb406b3e0 msgr2=0x7fcbb406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcbb406b3e0 0x7fcbb406d8a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcbd414ae90 tx=0x7fcbbc0079b0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf050 msgr2=0x7fcbd41396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:20.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf050 0x7fcbd41396e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4007e40 tx=0x7fcbc400e4b0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 shutdown_connections 2023-11-24T08:08:20.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcbb406b3e0 0x7fcbb406d8a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcbd413dfd0 0x7fcbd415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcbd40cf500 0x7fcbd415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 --2- 172.21.15.64:0/837275557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd40cf050 0x7fcbd41396e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:20.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 >> 172.21.15.64:0/837275557 conn(0x7fcbd406a860 msgr2=0x7fcbd413c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:20.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 shutdown_connections 2023-11-24T08:08:20.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:20.967+0000 7fcbd359e640 1 -- 172.21.15.64:0/837275557 wait complete. 2023-11-24T08:08:20.987 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:08:20.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:08:21.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.205+0000 7f0d4f59e640 1 Processor -- start 2023-11-24T08:08:21.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.206+0000 7f0d4f59e640 1 -- start start 2023-11-24T08:08:21.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.207+0000 7f0d4f59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0d50153ea0 0x7f0d500ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.207+0000 7f0d4f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d50070850 0x7f0d50071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.207+0000 7f0d4f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d500ce650 con 0x7f0d50154280 2023-11-24T08:08:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.207+0000 7f0d4f59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d50154710 con 0x7f0d50153ea0 2023-11-24T08:08:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.207+0000 7f0d4f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5015e0a0 con 0x7f0d50070850 2023-11-24T08:08:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.208+0000 7f0d47fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0d50154280 0x7f0d5015d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55674/0 (socket says 172.21.15.64:55674) 2023-11-24T08:08:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.208+0000 7f0d47fff640 1 -- 172.21.15.64:0/1524850244 learned_addr learned my addr 172.21.15.64:0/1524850244 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:21.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.208+0000 7f0d4e59c640 1 --2- 172.21.15.64:0/1524850244 >> v2:172.21.15.91:3300/0 conn(0x7f0d50153ea0 0x7f0d500ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.208+0000 7f0d4ed9d640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d50070850 0x7f0d50071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.208+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936653734 0 0) 0x7f0d500ce650 con 0x7f0d50154280 2023-11-24T08:08:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.208+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d3c003520 con 0x7f0d50154280 2023-11-24T08:08:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438251003 0 0) 0x7f0d3c003520 con 0x7f0d50154280 2023-11-24T08:08:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d50070850 msgr2=0x7f0d50071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d50070850 0x7f0d50071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 >> v2:172.21.15.91:3300/0 conn(0x7f0d50153ea0 msgr2=0x7f0d500ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 --2- 172.21.15.64:0/1524850244 >> v2:172.21.15.91:3300/0 conn(0x7f0d50153ea0 0x7f0d500ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5015e380 con 0x7f0d50154280 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0d38001670 con 0x7f0d50154280 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 >> v1:172.21.15.64:6789/0 conn(0x7f0d50154280 legacy=0x7f0d5015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d50070850 0x7f0d3c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c003fb0 0x7f0d3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d3c004d00 0x7f0d3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50154710 con 0x7f0d3c003fb0 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5015e0a0 con 0x7f0d3c004d00 2023-11-24T08:08:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c00ba00 con 0x7f0d50070850 2023-11-24T08:08:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.209+0000 7f0d47fff640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c003fb0 0x7f0d3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.210+0000 7f0d4e59c640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d50070850 0x7f0d3c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.210+0000 7f0d47fff640 1 -- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d50070850 msgr2=0x7f0d3c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.210+0000 7f0d47fff640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d50070850 0x7f0d3c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.210+0000 7f0d47fff640 1 -- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d3c004d00 msgr2=0x7f0d3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.210+0000 7f0d4ed9d640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d3c004d00 0x7f0d3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.210+0000 7f0d47fff640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d3c004d00 0x7f0d3c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.210+0000 7f0d47fff640 1 -- 172.21.15.64:0/1524850244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c003520 con 0x7f0d3c003fb0 2023-11-24T08:08:21.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.211+0000 7f0d47fff640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c003fb0 0x7f0d3c0047c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c110 tx=0x7f0d3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eed59bf458a476a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:21.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.211+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38005ad0 con 0x7f0d3c003fb0 2023-11-24T08:08:21.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.211+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d38005c50 con 0x7f0d3c003fb0 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.211+0000 7f0d4dd9b640 1 -- 172.21.15.64:0/1524850244 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3802c830 con 0x7f0d3c003fb0 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.212+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1524850244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c003fb0 msgr2=0x7f0d3c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.212+0000 7f0d4f59e640 1 --2- 172.21.15.64:0/1524850244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c003fb0 0x7f0d3c0047c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c110 tx=0x7f0d3c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.212+0000 7f0d4ed9d640 1 -- 172.21.15.64:0/1524850244 reap_dead start 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.212+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1524850244 shutdown_connections 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.212+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1524850244 >> 172.21.15.64:0/1524850244 conn(0x7f0d500cec10 msgr2=0x7f0d500cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.212+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1524850244 shutdown_connections 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.212+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1524850244 wait complete. 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.214+0000 7f0d4f59e640 1 Processor -- start 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.214+0000 7f0d4f59e640 1 -- start start 2023-11-24T08:08:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d4f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d3c003fb0 0x7f0d501508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d4f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d50156a20 0x7f0d50156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d4f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d50158c70 0x7f0d501590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d4f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d501573e0 con 0x7f0d50156a20 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d4f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50150de0 con 0x7f0d50158c70 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d4f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500ae830 con 0x7f0d3c003fb0 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d50156a20 0x7f0d50156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d50156a20 0x7f0d50156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43720/0 (socket says 172.21.15.64:43720) 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d47fff640 1 -- 172.21.15.64:0/1835928659 learned_addr learned my addr 172.21.15.64:0/1835928659 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.215+0000 7f0d4e59c640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d3c003fb0 0x7f0d501508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d4ed9d640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d50158c70 0x7f0d501590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d47fff640 1 -- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d3c003fb0 msgr2=0x7f0d501508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d47fff640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d3c003fb0 0x7f0d501508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d47fff640 1 -- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d50158c70 msgr2=0x7f0d501590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d47fff640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d50158c70 0x7f0d501590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d47fff640 1 -- 172.21.15.64:0/1835928659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500aeb10 con 0x7f0d50156a20 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d4ed9d640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d50158c70 0x7f0d501590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d47fff640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d50156a20 0x7f0d50156ea0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c230 tx=0x7f0d38001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d46ffd640 1 -- 172.21.15.64:0/1835928659 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38039070 con 0x7f0d50156a20 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d46ffd640 1 -- 172.21.15.64:0/1835928659 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3802cc40 con 0x7f0d50156a20 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d46ffd640 1 -- 172.21.15.64:0/1835928659 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38007070 con 0x7f0d50156a20 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.216+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d500aedf0 con 0x7f0d50156a20 2023-11-24T08:08:21.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.217+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d500af3b0 con 0x7f0d50156a20 2023-11-24T08:08:21.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.217+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d500af540 con 0x7f0d50156a20 2023-11-24T08:08:21.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.218+0000 7f0d46ffd640 1 -- 172.21.15.64:0/1835928659 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0d38032070 con 0x7f0d50156a20 2023-11-24T08:08:21.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.219+0000 7f0d46ffd640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d3006b410 0x7f0d3006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:21.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.219+0000 7f0d46ffd640 1 -- 172.21.15.64:0/1835928659 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0d3803c040 con 0x7f0d50156a20 2023-11-24T08:08:21.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.220+0000 7f0d4e59c640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d3006b410 0x7f0d3006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:21.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.221+0000 7f0d4e59c640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d3006b410 0x7f0d3006d8d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009a90 tx=0x7f0d40008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.224+0000 7f0d46ffd640 1 -- 172.21.15.64:0/1835928659 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d38075bb0 con 0x7f0d50156a20 2023-11-24T08:08:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:21.409+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0d5004e840 con 0x7f0d50156a20 2023-11-24T08:08:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.087+0000 7f0d46ffd640 1 -- 172.21.15.64:0/1835928659 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0d380791d0 con 0x7f0d50156a20 2023-11-24T08:08:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-11-24T08:08:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.091+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d3006b410 msgr2=0x7f0d3006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:22.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.091+0000 7f0d4f59e640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d3006b410 0x7f0d3006d8d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009a90 tx=0x7f0d40008040 comp rx=0 tx=0).stop 2023-11-24T08:08:22.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.091+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d50156a20 msgr2=0x7f0d50156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:22.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.091+0000 7f0d4f59e640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d50156a20 0x7f0d50156ea0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c230 tx=0x7f0d38001930 comp rx=0 tx=0).stop 2023-11-24T08:08:22.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.092+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 shutdown_connections 2023-11-24T08:08:22.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.092+0000 7f0d4f59e640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0d3006b410 0x7f0d3006d8d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:22.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.092+0000 7f0d4f59e640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0d50158c70 0x7f0d501590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:22.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.092+0000 7f0d4f59e640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d50156a20 0x7f0d50156ea0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:22.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.092+0000 7f0d4f59e640 1 --2- 172.21.15.64:0/1835928659 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0d3c003fb0 0x7f0d501508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:22.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.092+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 >> 172.21.15.64:0/1835928659 conn(0x7f0d500cec10 msgr2=0x7f0d500b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:22.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.093+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 shutdown_connections 2023-11-24T08:08:22.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.093+0000 7f0d4f59e640 1 -- 172.21.15.64:0/1835928659 wait complete. 2023-11-24T08:08:22.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:08:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.334+0000 7f0ac32ad640 1 Processor -- start 2023-11-24T08:08:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.334+0000 7f0ac32ad640 1 -- start start 2023-11-24T08:08:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.336+0000 7f0ac32ad640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0abc06dc30 0x7f0abc0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.336+0000 7f0ac32ad640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc06d7a0 con 0x7f0abc0acd70 2023-11-24T08:08:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.336+0000 7f0ac32ad640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc06ce70 con 0x7f0abc070490 2023-11-24T08:08:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.336+0000 7f0ac32ad640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc15a5b0 con 0x7f0abc06dc30 2023-11-24T08:08:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.336+0000 7f0ac1049640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0abc0acd70 0x7f0abc0ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55680/0 (socket says 172.21.15.64:55680) 2023-11-24T08:08:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.336+0000 7f0ac1049640 1 -- 172.21.15.64:0/537585315 learned_addr learned my addr 172.21.15.64:0/537585315 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:22.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.337+0000 7f0ac0848640 1 --2- 172.21.15.64:0/537585315 >> v2:172.21.15.91:3300/0 conn(0x7f0abc06dc30 0x7f0abc0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:22.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.337+0000 7f0ab3fff640 1 -- 172.21.15.64:0/537585315 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628088860 0 0) 0x7f0abc06d7a0 con 0x7f0abc0acd70 2023-11-24T08:08:22.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.337+0000 7f0ab3fff640 1 -- 172.21.15.64:0/537585315 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa8003520 con 0x7f0abc0acd70 2023-11-24T08:08:22.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.338+0000 7f0ac0848640 1 -- 172.21.15.64:0/537585315 >> v1:172.21.15.91:6789/0 conn(0x7f0abc070490 legacy=0x7f0abc070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:22.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.338+0000 7f0ac0848640 1 -- 172.21.15.64:0/537585315 >> v1:172.21.15.64:6789/0 conn(0x7f0abc0acd70 legacy=0x7f0abc0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:22.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.338+0000 7f0ab3fff640 1 -- 172.21.15.64:0/537585315 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 46653557 0 0) 0x7f0aa8003520 con 0x7f0abc0acd70 2023-11-24T08:08:22.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.338+0000 7f0ac0848640 1 -- 172.21.15.64:0/537585315 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc06d150 con 0x7f0abc06dc30 2023-11-24T08:08:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.340+0000 7f0ac0848640 1 --2- 172.21.15.64:0/537585315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 0x7f0abc0ae490 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0097a0 tx=0x7f0aac02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a98beab99b9e120 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.340+0000 7f0ab3fff640 1 -- 172.21.15.64:0/537585315 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac030610 con 0x7f0abc06dc30 2023-11-24T08:08:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.340+0000 7f0ab3fff640 1 -- 172.21.15.64:0/537585315 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac030770 con 0x7f0abc06dc30 2023-11-24T08:08:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.340+0000 7f0ab3fff640 1 -- 172.21.15.64:0/537585315 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac030b00 con 0x7f0abc06dc30 2023-11-24T08:08:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.341+0000 7f0ac32ad640 1 -- 172.21.15.64:0/537585315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 msgr2=0x7f0abc0ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.341+0000 7f0ac32ad640 1 --2- 172.21.15.64:0/537585315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 0x7f0abc0ae490 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0097a0 tx=0x7f0aac02b960 comp rx=0 tx=0).stop 2023-11-24T08:08:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.341+0000 7f0ac32ad640 1 -- 172.21.15.64:0/537585315 shutdown_connections 2023-11-24T08:08:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.341+0000 7f0ac32ad640 1 --2- 172.21.15.64:0/537585315 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 0x7f0abc0ae490 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.341+0000 7f0ac32ad640 1 -- 172.21.15.64:0/537585315 >> 172.21.15.64:0/537585315 conn(0x7f0abc0b0ba0 msgr2=0x7f0abc0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.341+0000 7f0ac32ad640 1 -- 172.21.15.64:0/537585315 shutdown_connections 2023-11-24T08:08:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.341+0000 7f0ac32ad640 1 -- 172.21.15.64:0/537585315 wait complete. 2023-11-24T08:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.343+0000 7f0ac32ad640 1 Processor -- start 2023-11-24T08:08:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.343+0000 7f0ac32ad640 1 -- start start 2023-11-24T08:08:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac32ad640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 0x7f0abc1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac32ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0abc070490 0x7f0abc155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac32ad640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0abc0acd70 0x7f0abc1eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac32ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0ae9d0 con 0x7f0abc070490 2023-11-24T08:08:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac32ad640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc1efae0 con 0x7f0abc06dc30 2023-11-24T08:08:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac32ad640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc1f2ff0 con 0x7f0abc0acd70 2023-11-24T08:08:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac0848640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0abc070490 0x7f0abc155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac0848640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0abc070490 0x7f0abc155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43734/0 (socket says 172.21.15.64:43734) 2023-11-24T08:08:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.344+0000 7f0ac0848640 1 -- 172.21.15.64:0/794282958 learned_addr learned my addr 172.21.15.64:0/794282958 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ac1049640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 0x7f0abc1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ac184a640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0abc0acd70 0x7f0abc1eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ac0848640 1 -- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0abc0acd70 msgr2=0x7f0abc1eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ac0848640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0abc0acd70 0x7f0abc1eeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ac0848640 1 -- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 msgr2=0x7f0abc1f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ac0848640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 0x7f0abc1f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ac0848640 1 -- 172.21.15.64:0/794282958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac009400 con 0x7f0abc070490 2023-11-24T08:08:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ac0848640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0abc070490 0x7f0abc155c20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0aac02bf70 tx=0x7f0aac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.345+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/794282958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac03e070 con 0x7f0abc070490 2023-11-24T08:08:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.346+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/794282958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac030e30 con 0x7f0abc070490 2023-11-24T08:08:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.346+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/794282958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0399a0 con 0x7f0abc070490 2023-11-24T08:08:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.346+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc156160 con 0x7f0abc070490 2023-11-24T08:08:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.346+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc1ef800 con 0x7f0abc070490 2023-11-24T08:08:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.347+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc09d200 con 0x7f0abc070490 2023-11-24T08:08:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.348+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/794282958 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0aac042070 con 0x7f0abc070490 2023-11-24T08:08:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.349+0000 7f0ab1ffb640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0aa006b3e0 0x7f0aa006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.349+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/794282958 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0aac0abd70 con 0x7f0abc070490 2023-11-24T08:08:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.350+0000 7f0ac1049640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0aa006b3e0 0x7f0aa006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.351+0000 7f0ac1049640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0aa006b3e0 0x7f0aa006d8a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005c00 tx=0x7f0ab8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.354+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/794282958 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0aac076640 con 0x7f0abc070490 2023-11-24T08:08:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:22.549+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0abc04e920 con 0x7f0abc070490 2023-11-24T08:08:23.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.093+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/794282958 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0aac079c60 con 0x7f0abc070490 2023-11-24T08:08:23.095 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-11-24T08:08:23.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.099+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0aa006b3e0 msgr2=0x7f0aa006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:23.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.099+0000 7f0ac32ad640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0aa006b3e0 0x7f0aa006d8a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005c00 tx=0x7f0ab8009040 comp rx=0 tx=0).stop 2023-11-24T08:08:23.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.099+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0abc070490 msgr2=0x7f0abc155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:23.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.099+0000 7f0ac32ad640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0abc070490 0x7f0abc155c20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0aac02bf70 tx=0x7f0aac0046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.100+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 shutdown_connections 2023-11-24T08:08:23.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.100+0000 7f0ac32ad640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0aa006b3e0 0x7f0aa006d8a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.100+0000 7f0ac32ad640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0abc0acd70 0x7f0abc1eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.100+0000 7f0ac32ad640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0abc070490 0x7f0abc155c20 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.100+0000 7f0ac32ad640 1 --2- 172.21.15.64:0/794282958 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0abc06dc30 0x7f0abc1f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.100+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 >> 172.21.15.64:0/794282958 conn(0x7f0abc0b0ba0 msgr2=0x7f0abc0aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:23.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.101+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 shutdown_connections 2023-11-24T08:08:23.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.101+0000 7f0ac32ad640 1 -- 172.21.15.64:0/794282958 wait complete. 2023-11-24T08:08:23.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:08:23.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.327+0000 7fdd1170f640 1 Processor -- start 2023-11-24T08:08:23.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.327+0000 7fdd1170f640 1 -- start start 2023-11-24T08:08:23.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.329+0000 7fdd1170f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd0c15b350 0x7fdd0c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:23.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.329+0000 7fdd1170f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c09cb30 0x7fdd0c0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.329+0000 7fdd1170f640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c15b730 con 0x7fdd0c0adb60 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.329+0000 7fdd1170f640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c0ac740 con 0x7fdd0c15b350 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.329+0000 7fdd1170f640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0ac460 con 0x7fdd0c09cb30 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.329+0000 7fdd0affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd0c15b350 0x7fdd0c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.329+0000 7fdd0affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd0c15b350 0x7fdd0c0abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43736/0 (socket says 172.21.15.64:43736) 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.329+0000 7fdd0affd640 1 -- 172.21.15.64:0/1363674994 learned_addr learned my addr 172.21.15.64:0/1363674994 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.330+0000 7fdd03fff640 1 --2- 172.21.15.64:0/1363674994 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c09cb30 0x7fdd0c0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.330+0000 7fdd0affd640 1 -- 172.21.15.64:0/1363674994 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c09cb30 msgr2=0x7fdd0c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.330+0000 7fdd0affd640 1 --2- 172.21.15.64:0/1363674994 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c09cb30 0x7fdd0c0b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.330+0000 7fdd0affd640 1 -- 172.21.15.64:0/1363674994 >> v1:172.21.15.91:6789/0 conn(0x7fdd0c0adb60 legacy=0x7fdd0c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:23.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.330+0000 7fdd0affd640 1 -- 172.21.15.64:0/1363674994 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0b2a50 con 0x7fdd0c15b350 2023-11-24T08:08:23.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.331+0000 7fdd0affd640 1 --2- 172.21.15.64:0/1363674994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c15b350 0x7fdd0c0abe80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009870 tx=0x7fdcfc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe0ad87c7c18f38 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:23.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.331+0000 7fdd0a7fc640 1 -- 172.21.15.64:0/1363674994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc031640 con 0x7fdd0c15b350 2023-11-24T08:08:23.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.331+0000 7fdd0a7fc640 1 -- 172.21.15.64:0/1363674994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc0317a0 con 0x7fdd0c15b350 2023-11-24T08:08:23.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.331+0000 7fdd0a7fc640 1 -- 172.21.15.64:0/1363674994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc031b30 con 0x7fdd0c15b350 2023-11-24T08:08:23.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.332+0000 7fdd1170f640 1 -- 172.21.15.64:0/1363674994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c15b350 msgr2=0x7fdd0c0abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:23.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.332+0000 7fdd1170f640 1 --2- 172.21.15.64:0/1363674994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c15b350 0x7fdd0c0abe80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009870 tx=0x7fdcfc02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.332+0000 7fdd1170f640 1 -- 172.21.15.64:0/1363674994 shutdown_connections 2023-11-24T08:08:23.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.332+0000 7fdd1170f640 1 --2- 172.21.15.64:0/1363674994 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c09cb30 0x7fdd0c0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.332+0000 7fdd1170f640 1 --2- 172.21.15.64:0/1363674994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c15b350 0x7fdd0c0abe80 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.332+0000 7fdd1170f640 1 -- 172.21.15.64:0/1363674994 >> 172.21.15.64:0/1363674994 conn(0x7fdd0c0b0ba0 msgr2=0x7fdd0c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:23.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.332+0000 7fdd1170f640 1 -- 172.21.15.64:0/1363674994 shutdown_connections 2023-11-24T08:08:23.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.332+0000 7fdd1170f640 1 -- 172.21.15.64:0/1363674994 wait complete. 2023-11-24T08:08:23.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.334+0000 7fdd1170f640 1 Processor -- start 2023-11-24T08:08:23.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.334+0000 7fdd1170f640 1 -- start start 2023-11-24T08:08:23.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.335+0000 7fdd1170f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c09cb30 0x7fdd0c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:23.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.335+0000 7fdd1170f640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd0c0adb60 0x7fdd0c146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:23.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.335+0000 7fdd1170f640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c15b350 0x7fdd0c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:23.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.335+0000 7fdd1170f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c14f060 con 0x7fdd0c09cb30 2023-11-24T08:08:23.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.335+0000 7fdd1170f640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c14adf0 con 0x7fdd0c0adb60 2023-11-24T08:08:23.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.335+0000 7fdd1170f640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c14b0d0 con 0x7fdd0c15b350 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c09cb30 0x7fdd0c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c09cb30 0x7fdd0c14f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43752/0 (socket says 172.21.15.64:43752) 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 -- 172.21.15.64:0/4126266883 learned_addr learned my addr 172.21.15.64:0/4126266883 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0b7fe640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c15b350 0x7fdd0c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd03fff640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd0c0adb60 0x7fdd0c146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 -- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c15b350 msgr2=0x7fdd0c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c15b350 0x7fdd0c14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 -- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd0c0adb60 msgr2=0x7fdd0c146a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd0c0adb60 0x7fdd0c146a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 -- 172.21.15.64:0/4126266883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0094d0 con 0x7fdd0c09cb30 2023-11-24T08:08:23.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0b7fe640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c15b350 0x7fdd0c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:23.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd03fff640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd0c0adb60 0x7fdd0c146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:23.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.336+0000 7fdd0affd640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c09cb30 0x7fdd0c14f4f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0090b0 tx=0x7fdcfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:23.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.337+0000 7fdd037fe640 1 -- 172.21.15.64:0/4126266883 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc03ac60 con 0x7fdd0c09cb30 2023-11-24T08:08:23.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.337+0000 7fdd037fe640 1 -- 172.21.15.64:0/4126266883 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc03adc0 con 0x7fdd0c09cb30 2023-11-24T08:08:23.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.337+0000 7fdd037fe640 1 -- 172.21.15.64:0/4126266883 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc044610 con 0x7fdd0c09cb30 2023-11-24T08:08:23.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.337+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c14ecf0 con 0x7fdd0c09cb30 2023-11-24T08:08:23.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.337+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c14e9e0 con 0x7fdd0c09cb30 2023-11-24T08:08:23.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.338+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0c14ee80 con 0x7fdd0c09cb30 2023-11-24T08:08:23.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.340+0000 7fdd037fe640 1 -- 172.21.15.64:0/4126266883 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fdcfc02f050 con 0x7fdd0c09cb30 2023-11-24T08:08:23.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.341+0000 7fdd037fe640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcec06b3e0 0x7fdcec06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:23.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.341+0000 7fdd037fe640 1 -- 172.21.15.64:0/4126266883 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdcfc0ac170 con 0x7fdd0c09cb30 2023-11-24T08:08:23.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.341+0000 7fdd03fff640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcec06b3e0 0x7fdcec06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:23.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.343+0000 7fdd03fff640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcec06b3e0 0x7fdcec06d8a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c15ad60 tx=0x7fdcf4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:23.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.347+0000 7fdd037fe640 1 -- 172.21.15.64:0/4126266883 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdcfc076980 con 0x7fdd0c09cb30 2023-11-24T08:08:23.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:23.581+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdd0c15b730 con 0x7fdd0c09cb30 2023-11-24T08:08:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.139+0000 7fdd037fe640 1 -- 172.21.15.64:0/4126266883 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdd0c15b730 con 0x7fdd0c09cb30 2023-11-24T08:08:24.142 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 12 and data pool 13 2023-11-24T08:08:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.148+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcec06b3e0 msgr2=0x7fdcec06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.148+0000 7fdd1170f640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcec06b3e0 0x7fdcec06d8a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c15ad60 tx=0x7fdcf4008040 comp rx=0 tx=0).stop 2023-11-24T08:08:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.148+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c09cb30 msgr2=0x7fdd0c14f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.148+0000 7fdd1170f640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c09cb30 0x7fdd0c14f4f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0090b0 tx=0x7fdcfc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.153+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 shutdown_connections 2023-11-24T08:08:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.153+0000 7fdd1170f640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fdcec06b3e0 0x7fdcec06d8a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.153+0000 7fdd1170f640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fdd0c15b350 0x7fdd0c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.153+0000 7fdd1170f640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fdd0c0adb60 0x7fdd0c146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.153+0000 7fdd1170f640 1 --2- 172.21.15.64:0/4126266883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c09cb30 0x7fdd0c14f4f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.153+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 >> 172.21.15.64:0/4126266883 conn(0x7fdd0c0b0ba0 msgr2=0x7fdd0c0b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.154+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 shutdown_connections 2023-11-24T08:08:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.155+0000 7fdd1170f640 1 -- 172.21.15.64:0/4126266883 wait complete. 2023-11-24T08:08:24.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:24.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.375+0000 7f67d94ff640 1 Processor -- start 2023-11-24T08:08:24.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.375+0000 7f67d94ff640 1 -- start start 2023-11-24T08:08:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67d94ff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f67d40cf050 0x7f67d40cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67d94ff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d40cd7b0 0x7f67d40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67d94ff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d415fa70 con 0x7f67d413dfd0 2023-11-24T08:08:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67d94ff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d40cfa40 con 0x7f67d40cf050 2023-11-24T08:08:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67d94ff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d415c580 con 0x7f67d40cd7b0 2023-11-24T08:08:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67cbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f67d40cf050 0x7f67d40cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67cbfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f67d40cf050 0x7f67d40cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43764/0 (socket says 172.21.15.64:43764) 2023-11-24T08:08:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67cbfff640 1 -- 172.21.15.64:0/2157400793 learned_addr learned my addr 172.21.15.64:0/2157400793 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:24.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67cbfff640 1 -- 172.21.15.64:0/2157400793 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d40cd7b0 msgr2=0x7f67d40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:24.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67cbfff640 1 --2- 172.21.15.64:0/2157400793 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d40cd7b0 0x7f67d40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67cbfff640 1 -- 172.21.15.64:0/2157400793 >> v1:172.21.15.91:6789/0 conn(0x7f67d413dfd0 legacy=0x7f67d415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:24.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.377+0000 7f67cbfff640 1 -- 172.21.15.64:0/2157400793 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4140900 con 0x7f67d40cf050 2023-11-24T08:08:24.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.379+0000 7f67cbfff640 1 --2- 172.21.15.64:0/2157400793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cf050 0x7f67d40cf500 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f67d406e070 tx=0x7f67bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf116f5347054d87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:24.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.379+0000 7f67d37fe640 1 -- 172.21.15.64:0/2157400793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc031830 con 0x7f67d40cf050 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.379+0000 7f67d37fe640 1 -- 172.21.15.64:0/2157400793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc031990 con 0x7f67d40cf050 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.379+0000 7f67d37fe640 1 -- 172.21.15.64:0/2157400793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc031d20 con 0x7f67d40cf050 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.379+0000 7f67d94ff640 1 -- 172.21.15.64:0/2157400793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cf050 msgr2=0x7f67d40cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.379+0000 7f67d94ff640 1 --2- 172.21.15.64:0/2157400793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cf050 0x7f67d40cf500 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f67d406e070 tx=0x7f67bc02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.380+0000 7f67d94ff640 1 -- 172.21.15.64:0/2157400793 shutdown_connections 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.380+0000 7f67d94ff640 1 --2- 172.21.15.64:0/2157400793 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d40cd7b0 0x7f67d40cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.380+0000 7f67d94ff640 1 --2- 172.21.15.64:0/2157400793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cf050 0x7f67d40cf500 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.380+0000 7f67d94ff640 1 -- 172.21.15.64:0/2157400793 >> 172.21.15.64:0/2157400793 conn(0x7f67d406a860 msgr2=0x7f67d406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.380+0000 7f67d94ff640 1 -- 172.21.15.64:0/2157400793 shutdown_connections 2023-11-24T08:08:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.380+0000 7f67d94ff640 1 -- 172.21.15.64:0/2157400793 wait complete. 2023-11-24T08:08:24.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.382+0000 7f67d94ff640 1 Processor -- start 2023-11-24T08:08:24.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.382+0000 7f67d94ff640 1 -- start start 2023-11-24T08:08:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d94ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cd7b0 0x7f67d4159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d94ff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67d40cf050 0x7f67d41552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d94ff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d413dfd0 0x7f67d415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d94ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4159e30 con 0x7f67d40cd7b0 2023-11-24T08:08:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d94ff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4155c00 con 0x7f67d40cf050 2023-11-24T08:08:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d94ff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4155ee0 con 0x7f67d413dfd0 2023-11-24T08:08:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cd7b0 0x7f67d4159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cd7b0 0x7f67d4159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43776/0 (socket says 172.21.15.64:43776) 2023-11-24T08:08:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.383+0000 7f67d3fff640 1 -- 172.21.15.64:0/3742058910 learned_addr learned my addr 172.21.15.64:0/3742058910 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67cbfff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67d40cf050 0x7f67d41552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d3fff640 1 -- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d413dfd0 msgr2=0x7f67d415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d8cfe640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d413dfd0 0x7f67d415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d3fff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d413dfd0 0x7f67d415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d3fff640 1 -- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67d40cf050 msgr2=0x7f67d41552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d3fff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67d40cf050 0x7f67d41552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d3fff640 1 -- 172.21.15.64:0/3742058910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0094d0 con 0x7f67d40cd7b0 2023-11-24T08:08:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67cbfff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67d40cf050 0x7f67d41552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d3fff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cd7b0 0x7f67d4159840 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f67c400b3f0 tx=0x7f67c400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d17fa640 1 -- 172.21.15.64:0/3742058910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4011830 con 0x7f67d40cd7b0 2023-11-24T08:08:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d17fa640 1 -- 172.21.15.64:0/3742058910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c4011990 con 0x7f67d40cd7b0 2023-11-24T08:08:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d17fa640 1 -- 172.21.15.64:0/3742058910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4011d40 con 0x7f67d40cd7b0 2023-11-24T08:08:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.384+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d41561c0 con 0x7f67d40cd7b0 2023-11-24T08:08:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.385+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d415b300 con 0x7f67d40cd7b0 2023-11-24T08:08:24.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.386+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d404e840 con 0x7f67d40cd7b0 2023-11-24T08:08:24.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.387+0000 7f67d17fa640 1 -- 172.21.15.64:0/3742058910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f67c401a400 con 0x7f67d40cd7b0 2023-11-24T08:08:24.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.388+0000 7f67d17fa640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f67b406b3e0 0x7f67b406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.388+0000 7f67d17fa640 1 -- 172.21.15.64:0/3742058910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67c4089ee0 con 0x7f67d40cd7b0 2023-11-24T08:08:24.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.388+0000 7f67cbfff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f67b406b3e0 0x7f67b406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.389+0000 7f67cbfff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f67b406b3e0 0x7f67b406d8a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f67bc036000 tx=0x7f67bc03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:24.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.393+0000 7f67d17fa640 1 -- 172.21.15.64:0/3742058910 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67c408d030 con 0x7f67d40cd7b0 2023-11-24T08:08:24.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.591+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d413bb90 con 0x7f67d40cd7b0 2023-11-24T08:08:24.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.593+0000 7f67d17fa640 1 -- 172.21.15.64:0/3742058910 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16525 (secure 0 0 0) 0x7f67c4015090 con 0x7f67d40cd7b0 2023-11-24T08:08:24.596 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:24.596 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":86,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:23.582878+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:21.467760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:22.606730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:24.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f67b406b3e0 msgr2=0x7f67b406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f67b406b3e0 0x7f67b406d8a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f67bc036000 tx=0x7f67bc03c040 comp rx=0 tx=0).stop 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cd7b0 msgr2=0x7f67d4159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cd7b0 0x7f67d4159840 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f67c400b3f0 tx=0x7f67c400b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 shutdown_connections 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f67b406b3e0 0x7f67b406d8a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f67d413dfd0 0x7f67d415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f67d40cf050 0x7f67d41552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 --2- 172.21.15.64:0/3742058910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d40cd7b0 0x7f67d4159840 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.596+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 >> 172.21.15.64:0/3742058910 conn(0x7f67d406a860 msgr2=0x7f67d413cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.597+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 shutdown_connections 2023-11-24T08:08:24.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.597+0000 7f67d94ff640 1 -- 172.21.15.64:0/3742058910 wait complete. 2023-11-24T08:08:24.617 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:24.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.839+0000 7f16cca71640 1 Processor -- start 2023-11-24T08:08:24.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.839+0000 7f16cca71640 1 -- start start 2023-11-24T08:08:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.841+0000 7f16cca71640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16c813dfd0 0x7f16c815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.841+0000 7f16cca71640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16c80cf4d0 0x7f16c806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.841+0000 7f16cca71640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c80ce7c0 con 0x7f16c80cf050 2023-11-24T08:08:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.841+0000 7f16cca71640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c8140900 con 0x7f16c80cf4d0 2023-11-24T08:08:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.841+0000 7f16cca71640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c815fa70 con 0x7f16c813dfd0 2023-11-24T08:08:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.841+0000 7f16c7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16c80cf4d0 0x7f16c806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.841+0000 7f16c7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16c80cf4d0 0x7f16c806c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43788/0 (socket says 172.21.15.64:43788) 2023-11-24T08:08:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.841+0000 7f16c7fff640 1 -- 172.21.15.64:0/3383476526 learned_addr learned my addr 172.21.15.64:0/3383476526 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.842+0000 7f16c7fff640 1 -- 172.21.15.64:0/3383476526 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16c813dfd0 msgr2=0x7f16c815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.842+0000 7f16c7fff640 1 --2- 172.21.15.64:0/3383476526 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16c813dfd0 0x7f16c815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.842+0000 7f16c7fff640 1 -- 172.21.15.64:0/3383476526 >> v1:172.21.15.91:6789/0 conn(0x7f16c80cf050 legacy=0x7f16c806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.842+0000 7f16c7fff640 1 -- 172.21.15.64:0/3383476526 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c815ee60 con 0x7f16c80cf4d0 2023-11-24T08:08:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.843+0000 7f16c7fff640 1 --2- 172.21.15.64:0/3383476526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf4d0 0x7f16c806c840 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009870 tx=0x7f16c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7080dd0a77182d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.843+0000 7f16c6ffd640 1 -- 172.21.15.64:0/3383476526 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c0031640 con 0x7f16c80cf4d0 2023-11-24T08:08:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.843+0000 7f16c6ffd640 1 -- 172.21.15.64:0/3383476526 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c00317a0 con 0x7f16c80cf4d0 2023-11-24T08:08:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.843+0000 7f16c6ffd640 1 -- 172.21.15.64:0/3383476526 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c0031b30 con 0x7f16c80cf4d0 2023-11-24T08:08:24.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.844+0000 7f16cca71640 1 -- 172.21.15.64:0/3383476526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf4d0 msgr2=0x7f16c806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.844+0000 7f16cca71640 1 --2- 172.21.15.64:0/3383476526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf4d0 0x7f16c806c840 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009870 tx=0x7f16c002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.844+0000 7f16cca71640 1 -- 172.21.15.64:0/3383476526 shutdown_connections 2023-11-24T08:08:24.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.844+0000 7f16cca71640 1 --2- 172.21.15.64:0/3383476526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf4d0 0x7f16c806c840 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.844+0000 7f16cca71640 1 --2- 172.21.15.64:0/3383476526 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16c813dfd0 0x7f16c815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.844+0000 7f16cca71640 1 -- 172.21.15.64:0/3383476526 >> 172.21.15.64:0/3383476526 conn(0x7f16c806a860 msgr2=0x7f16c806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:24.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.844+0000 7f16cca71640 1 -- 172.21.15.64:0/3383476526 shutdown_connections 2023-11-24T08:08:24.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.844+0000 7f16cca71640 1 -- 172.21.15.64:0/3383476526 wait complete. 2023-11-24T08:08:24.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.846+0000 7f16cca71640 1 Processor -- start 2023-11-24T08:08:24.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.846+0000 7f16cca71640 1 -- start start 2023-11-24T08:08:24.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16cca71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf050 0x7f16c81550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16cca71640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16c80cf4d0 0x7f16c813a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16cca71640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16c813dfd0 0x7f16c8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16cca71640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c813ab40 con 0x7f16c80cf050 2023-11-24T08:08:24.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16cca71640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c813ae20 con 0x7f16c813dfd0 2023-11-24T08:08:24.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16cca71640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c813b100 con 0x7f16c80cf4d0 2023-11-24T08:08:24.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf050 0x7f16c81550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf050 0x7f16c81550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43804/0 (socket says 172.21.15.64:43804) 2023-11-24T08:08:24.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.847+0000 7f16c77fe640 1 -- 172.21.15.64:0/501398540 learned_addr learned my addr 172.21.15.64:0/501398540 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:24.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c7fff640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16c813dfd0 0x7f16c8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c77fe640 1 -- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16c80cf4d0 msgr2=0x7f16c813a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c77fe640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16c80cf4d0 0x7f16c813a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c77fe640 1 -- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16c813dfd0 msgr2=0x7f16c8071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:24.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c77fe640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16c813dfd0 0x7f16c8071180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:24.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c77fe640 1 -- 172.21.15.64:0/501398540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00094d0 con 0x7f16c80cf050 2023-11-24T08:08:24.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c7fff640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16c813dfd0 0x7f16c8071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:24.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c77fe640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf050 0x7f16c81550e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f16b800ba50 tx=0x7f16b800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:24.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c4ff9640 1 -- 172.21.15.64:0/501398540 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800cbe0 con 0x7f16c80cf050 2023-11-24T08:08:24.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16c4ff9640 1 -- 172.21.15.64:0/501398540 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800cd60 con 0x7f16c80cf050 2023-11-24T08:08:24.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.848+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c813b3e0 con 0x7f16c80cf050 2023-11-24T08:08:24.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.849+0000 7f16c4ff9640 1 -- 172.21.15.64:0/501398540 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8014680 con 0x7f16c80cf050 2023-11-24T08:08:24.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.849+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c8155620 con 0x7f16c80cf050 2023-11-24T08:08:24.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.850+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c813b900 con 0x7f16c80cf050 2023-11-24T08:08:24.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.851+0000 7f16c4ff9640 1 -- 172.21.15.64:0/501398540 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f16b80147e0 con 0x7f16c80cf050 2023-11-24T08:08:24.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.852+0000 7f16c4ff9640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f16a806b410 0x7f16a806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:24.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.852+0000 7f16c4ff9640 1 -- 172.21.15.64:0/501398540 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16b8089fe0 con 0x7f16c80cf050 2023-11-24T08:08:24.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.852+0000 7f16beffd640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f16a806b410 0x7f16a806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:24.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.853+0000 7f16beffd640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f16a806b410 0x7f16a806d8d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f16c80cc400 tx=0x7f16b0006ba0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:24.856+0000 7f16c4ff9640 1 -- 172.21.15.64:0/501398540 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16c813b900 con 0x7f16c80cf050 2023-11-24T08:08:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.057+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16c813b900 con 0x7f16c80cf050 2023-11-24T08:08:25.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.060+0000 7f16c4ff9640 1 -- 172.21.15.64:0/501398540 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16525 (secure 0 0 0) 0x7f16c813b900 con 0x7f16c80cf050 2023-11-24T08:08:25.062 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:25.062 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":86,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:23.582878+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:21.467760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:22.606730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:25.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f16a806b410 msgr2=0x7f16a806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:25.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f16a806b410 0x7f16a806d8d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f16c80cc400 tx=0x7f16b0006ba0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf050 msgr2=0x7f16c81550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf050 0x7f16c81550e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f16b800ba50 tx=0x7f16b800bf10 comp rx=0 tx=0).stop 2023-11-24T08:08:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 shutdown_connections 2023-11-24T08:08:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f16a806b410 0x7f16a806d8d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16c813dfd0 0x7f16c8071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16c80cf4d0 0x7f16c813a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 --2- 172.21.15.64:0/501398540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c80cf050 0x7f16c81550e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 >> 172.21.15.64:0/501398540 conn(0x7f16c806a860 msgr2=0x7f16c815de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:25.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 shutdown_connections 2023-11-24T08:08:25.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.063+0000 7f16cca71640 1 -- 172.21.15.64:0/501398540 wait complete. 2023-11-24T08:08:25.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:08:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.302+0000 7f176b59e640 1 Processor -- start 2023-11-24T08:08:25.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.302+0000 7f176b59e640 1 -- start start 2023-11-24T08:08:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.303+0000 7f176b59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f176c0d1d90 0x7f176c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.303+0000 7f176b59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c0d0fa0 0x7f176c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.303+0000 7f176b59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0c6bf0 con 0x7f176c15d3b0 2023-11-24T08:08:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.303+0000 7f176b59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c0d13f0 con 0x7f176c0d1d90 2023-11-24T08:08:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.303+0000 7f176b59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cedc0 con 0x7f176c0d0fa0 2023-11-24T08:08:25.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.303+0000 7f176a59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f176c15d3b0 0x7f176c0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55694/0 (socket says 172.21.15.64:55694) 2023-11-24T08:08:25.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176a59c640 1 -- 172.21.15.64:0/2171126511 learned_addr learned my addr 172.21.15.64:0/2171126511 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:25.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911734783 0 0) 0x7f176c0c6bf0 con 0x7f176c15d3b0 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1758003520 con 0x7f176c15d3b0 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f1769d9b640 1 --2- 172.21.15.64:0/2171126511 >> v2:172.21.15.91:3300/0 conn(0x7f176c0d1d90 0x7f176c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176ad9d640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c0d0fa0 0x7f176c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 998888759 0 0) 0x7f1758003520 con 0x7f176c15d3b0 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c0d0fa0 msgr2=0x7f176c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c0d0fa0 0x7f176c0b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 >> v2:172.21.15.91:3300/0 conn(0x7f176c0d1d90 msgr2=0x7f176c0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 --2- 172.21.15.64:0/2171126511 >> v2:172.21.15.91:3300/0 conn(0x7f176c0d1d90 0x7f176c0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0c6e40 con 0x7f176c15d3b0 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.304+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f175c0019f0 con 0x7f176c15d3b0 2023-11-24T08:08:25.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 >> v1:172.21.15.64:6789/0 conn(0x7f176c15d3b0 legacy=0x7f176c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176959a640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c0d0fa0 0x7f17580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176959a640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1758004060 0x7f17580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176959a640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1758004d00 0x7f1758005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0d13f0 con 0x7f1758004d00 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cedc0 con 0x7f1758004060 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175800bbf0 con 0x7f176c0d0fa0 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176ad9d640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1758004d00 0x7f1758005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f176a59c640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c0d0fa0 0x7f17580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.305+0000 7f1769d9b640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1758004060 0x7f17580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.306+0000 7f176ad9d640 1 -- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c0d0fa0 msgr2=0x7f17580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.306+0000 7f176ad9d640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c0d0fa0 0x7f17580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.306+0000 7f176ad9d640 1 -- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1758004060 msgr2=0x7f17580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:25.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.306+0000 7f176ad9d640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1758004060 0x7f17580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.306+0000 7f176ad9d640 1 -- 172.21.15.64:0/2171126511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1758003520 con 0x7f1758004d00 2023-11-24T08:08:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.307+0000 7f176ad9d640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1758004d00 0x7f1758005180 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f176400ba30 tx=0x7f17580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2daa1d03416888 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.307+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764032640 con 0x7f1758004d00 2023-11-24T08:08:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.307+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17640327a0 con 0x7f1758004d00 2023-11-24T08:08:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.307+0000 7f176959a640 1 -- 172.21.15.64:0/2171126511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764032b30 con 0x7f1758004d00 2023-11-24T08:08:25.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.308+0000 7f176b59e640 1 -- 172.21.15.64:0/2171126511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1758004d00 msgr2=0x7f1758005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.308+0000 7f176b59e640 1 --2- 172.21.15.64:0/2171126511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1758004d00 0x7f1758005180 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f176400ba30 tx=0x7f17580057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.308+0000 7f176ad9d640 1 -- 172.21.15.64:0/2171126511 reap_dead start 2023-11-24T08:08:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.308+0000 7f176b59e640 1 -- 172.21.15.64:0/2171126511 shutdown_connections 2023-11-24T08:08:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.308+0000 7f176b59e640 1 -- 172.21.15.64:0/2171126511 >> 172.21.15.64:0/2171126511 conn(0x7f176c0d30a0 msgr2=0x7f176c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.308+0000 7f176b59e640 1 -- 172.21.15.64:0/2171126511 shutdown_connections 2023-11-24T08:08:25.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.308+0000 7f176b59e640 1 -- 172.21.15.64:0/2171126511 wait complete. 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.310+0000 7f176b59e640 1 Processor -- start 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.310+0000 7f176b59e640 1 -- start start 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.311+0000 7f176b59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c1524c0 0x7f176c0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.311+0000 7f176b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c0afc60 0x7f176c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.311+0000 7f176b59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f176c1528a0 0x7f176c0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.311+0000 7f176b59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b2490 con 0x7f176c0afc60 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.311+0000 7f176b59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b2110 con 0x7f176c1528a0 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.311+0000 7f176b59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b8b10 con 0x7f176c1524c0 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c0afc60 0x7f176c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c0afc60 0x7f176c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43826/0 (socket says 172.21.15.64:43826) 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 -- 172.21.15.64:0/1247894374 learned_addr learned my addr 172.21.15.64:0/1247894374 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f176a59c640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c1524c0 0x7f176c0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f176ad9d640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f176c1528a0 0x7f176c0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 -- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c1524c0 msgr2=0x7f176c0cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c1524c0 0x7f176c0cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 -- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f176c1528a0 msgr2=0x7f176c0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f176c1528a0 0x7f176c0cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 -- 172.21.15.64:0/1247894374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0b8df0 con 0x7f176c0afc60 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f176ad9d640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f176c1528a0 0x7f176c0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.312+0000 7f1769d9b640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c0afc60 0x7f176c0cc240 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f175400f5a0 tx=0x7f175400fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:25.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.313+0000 7f1762ffd640 1 -- 172.21.15.64:0/1247894374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754014870 con 0x7f176c0afc60 2023-11-24T08:08:25.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.313+0000 7f1762ffd640 1 -- 172.21.15.64:0/1247894374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17540149d0 con 0x7f176c0afc60 2023-11-24T08:08:25.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.313+0000 7f1762ffd640 1 -- 172.21.15.64:0/1247894374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754014d40 con 0x7f176c0afc60 2023-11-24T08:08:25.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.313+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c0ccf80 con 0x7f176c0afc60 2023-11-24T08:08:25.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.313+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c1f6650 con 0x7f176c0afc60 2023-11-24T08:08:25.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.314+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c0b25d0 con 0x7f176c0afc60 2023-11-24T08:08:25.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.315+0000 7f1762ffd640 1 -- 172.21.15.64:0/1247894374 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1754015410 con 0x7f176c0afc60 2023-11-24T08:08:25.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.316+0000 7f1762ffd640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f174c06b410 0x7f174c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:25.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.316+0000 7f1762ffd640 1 -- 172.21.15.64:0/1247894374 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f175408a9a0 con 0x7f176c0afc60 2023-11-24T08:08:25.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.317+0000 7f176a59c640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f174c06b410 0x7f174c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:25.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.318+0000 7f176a59c640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f174c06b410 0x7f174c06d8d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f175c004bc0 tx=0x7f175c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:25.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.321+0000 7f1762ffd640 1 -- 172.21.15.64:0/1247894374 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f176c0b25d0 con 0x7f176c0afc60 2023-11-24T08:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:25.552+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f176c0b25d0 con 0x7f176c0afc60 2023-11-24T08:08:26.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.186+0000 7f1762ffd640 1 -- 172.21.15.64:0/1247894374 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f176c0b25d0 con 0x7f176c0afc60 2023-11-24T08:08:26.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f174c06b410 msgr2=0x7f174c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:26.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f174c06b410 0x7f174c06d8d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f175c004bc0 tx=0x7f175c009040 comp rx=0 tx=0).stop 2023-11-24T08:08:26.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c0afc60 msgr2=0x7f176c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:26.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c0afc60 0x7f176c0cc240 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f175400f5a0 tx=0x7f175400fa60 comp rx=0 tx=0).stop 2023-11-24T08:08:26.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 shutdown_connections 2023-11-24T08:08:26.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f174c06b410 0x7f174c06d8d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:26.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f176c1528a0 0x7f176c0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:26.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c0afc60 0x7f176c0cc240 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:26.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 --2- 172.21.15.64:0/1247894374 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f176c1524c0 0x7f176c0cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:26.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 >> 172.21.15.64:0/1247894374 conn(0x7f176c0d30a0 msgr2=0x7f176c0c7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:26.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.190+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 shutdown_connections 2023-11-24T08:08:26.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.191+0000 7f176b59e640 1 -- 172.21.15.64:0/1247894374 wait complete. 2023-11-24T08:08:26.212 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:08:26.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:08:26.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.445+0000 7f9c02fcb640 1 Processor -- start 2023-11-24T08:08:26.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.445+0000 7f9c02fcb640 1 -- start start 2023-11-24T08:08:26.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.447+0000 7f9c02fcb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9bfc15f270 0x7f9bfc0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:26.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.447+0000 7f9c02fcb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc15f930 con 0x7f9bfc06d9e0 2023-11-24T08:08:26.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.447+0000 7f9c02fcb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0b3b10 con 0x7f9bfc144770 2023-11-24T08:08:26.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.447+0000 7f9c02fcb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc0b43b0 con 0x7f9bfc15f270 2023-11-24T08:08:26.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.447+0000 7f9c017c8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9bfc15f270 0x7f9bfc0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:26.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.447+0000 7f9c017c8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9bfc15f270 0x7f9bfc0cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43832/0 (socket says 172.21.15.64:43832) 2023-11-24T08:08:26.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.447+0000 7f9c017c8640 1 -- 172.21.15.64:0/2616417158 learned_addr learned my addr 172.21.15.64:0/2616417158 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:26.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.447+0000 7f9c027ca640 1 --1- 172.21.15.64:0/2616417158 >> v1:172.21.15.64:6789/0 conn(0x7f9bfc06d9e0 0x7f9bfc0cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55708/0 (socket says 172.21.15.64:55708) 2023-11-24T08:08:26.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.448+0000 7f9c017c8640 1 -- 172.21.15.64:0/2616417158 >> v1:172.21.15.91:6789/0 conn(0x7f9bfc144770 legacy=0x7f9bfc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:26.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.448+0000 7f9c017c8640 1 -- 172.21.15.64:0/2616417158 >> v1:172.21.15.64:6789/0 conn(0x7f9bfc06d9e0 legacy=0x7f9bfc0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:26.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.448+0000 7f9c017c8640 1 -- 172.21.15.64:0/2616417158 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0b3df0 con 0x7f9bfc15f270 2023-11-24T08:08:26.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.449+0000 7f9c017c8640 1 --2- 172.21.15.64:0/2616417158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0cd4a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009870 tx=0x7f9bec02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5f51222e139349 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:26.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.449+0000 7f9c00fc7640 1 -- 172.21.15.64:0/2616417158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec02f850 con 0x7f9bfc15f270 2023-11-24T08:08:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.449+0000 7f9c00fc7640 1 -- 172.21.15.64:0/2616417158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec02f9b0 con 0x7f9bfc15f270 2023-11-24T08:08:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.449+0000 7f9c00fc7640 1 -- 172.21.15.64:0/2616417158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec02fd40 con 0x7f9bfc15f270 2023-11-24T08:08:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.450+0000 7f9c02fcb640 1 -- 172.21.15.64:0/2616417158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 msgr2=0x7f9bfc0cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.450+0000 7f9c02fcb640 1 --2- 172.21.15.64:0/2616417158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0cd4a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009870 tx=0x7f9bec02ba30 comp rx=0 tx=0).stop 2023-11-24T08:08:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.450+0000 7f9c02fcb640 1 -- 172.21.15.64:0/2616417158 shutdown_connections 2023-11-24T08:08:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.450+0000 7f9c02fcb640 1 --2- 172.21.15.64:0/2616417158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0cd4a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.450+0000 7f9c02fcb640 1 -- 172.21.15.64:0/2616417158 >> 172.21.15.64:0/2616417158 conn(0x7f9bfc0cec10 msgr2=0x7f9bfc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.450+0000 7f9c02fcb640 1 -- 172.21.15.64:0/2616417158 shutdown_connections 2023-11-24T08:08:26.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.450+0000 7f9c02fcb640 1 -- 172.21.15.64:0/2616417158 wait complete. 2023-11-24T08:08:26.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.452+0000 7f9c02fcb640 1 Processor -- start 2023-11-24T08:08:26.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.452+0000 7f9c02fcb640 1 -- start start 2023-11-24T08:08:26.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.453+0000 7f9c02fcb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bfc06d9e0 0x7f9bfc06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:26.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.453+0000 7f9c02fcb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bfc144770 0x7f9bfc13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:26.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.453+0000 7f9c02fcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:26.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.453+0000 7f9c02fcb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc13e910 con 0x7f9bfc15f270 2023-11-24T08:08:26.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.453+0000 7f9c02fcb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc13ebf0 con 0x7f9bfc06d9e0 2023-11-24T08:08:26.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.453+0000 7f9c02fcb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc13eed0 con 0x7f9bfc144770 2023-11-24T08:08:26.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.453+0000 7f9c027ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:26.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c027ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43848/0 (socket says 172.21.15.64:43848) 2023-11-24T08:08:26.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c027ca640 1 -- 172.21.15.64:0/286842022 learned_addr learned my addr 172.21.15.64:0/286842022 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:26.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c017c8640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bfc144770 0x7f9bfc13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:26.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c01fc9640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bfc06d9e0 0x7f9bfc06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:26.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c027ca640 1 -- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bfc144770 msgr2=0x7f9bfc13f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:26.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c027ca640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bfc144770 0x7f9bfc13f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:26.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c027ca640 1 -- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bfc06d9e0 msgr2=0x7f9bfc06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:26.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c027ca640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bfc06d9e0 0x7f9bfc06cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:26.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c027ca640 1 -- 172.21.15.64:0/286842022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0094d0 con 0x7f9bfc15f270 2023-11-24T08:08:26.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.454+0000 7f9c027ca640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0ad990 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c780 tx=0x7f9bf400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:26.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.455+0000 7f9bf27fc640 1 -- 172.21.15.64:0/286842022 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400ce90 con 0x7f9bfc15f270 2023-11-24T08:08:26.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.455+0000 7f9bf27fc640 1 -- 172.21.15.64:0/286842022 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf400a640 con 0x7f9bfc15f270 2023-11-24T08:08:26.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.455+0000 7f9bf27fc640 1 -- 172.21.15.64:0/286842022 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400a9f0 con 0x7f9bfc15f270 2023-11-24T08:08:26.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.455+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc13f1b0 con 0x7f9bfc15f270 2023-11-24T08:08:26.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.455+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc06d080 con 0x7f9bfc15f270 2023-11-24T08:08:26.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.456+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc04e840 con 0x7f9bfc15f270 2023-11-24T08:08:26.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.457+0000 7f9bf27fc640 1 -- 172.21.15.64:0/286842022 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f9bf400ab50 con 0x7f9bfc15f270 2023-11-24T08:08:26.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.458+0000 7f9bf27fc640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9be006b3e0 0x7f9be006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:26.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.458+0000 7f9bf27fc640 1 -- 172.21.15.64:0/286842022 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bf4089690 con 0x7f9bfc15f270 2023-11-24T08:08:26.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.458+0000 7f9c01fc9640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9be006b3e0 0x7f9be006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:26.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.460+0000 7f9c01fc9640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9be006b3e0 0x7f9be006d8a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8000f60 tx=0x7f9bf8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:26.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.463+0000 7f9bf27fc640 1 -- 172.21.15.64:0/286842022 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bf4057b20 con 0x7f9bfc15f270 2023-11-24T08:08:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:26.694+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9bfc13ea50 con 0x7f9bfc15f270 2023-11-24T08:08:27.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.189+0000 7f9bf27fc640 1 -- 172.21.15.64:0/286842022 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9bfc13ea50 con 0x7f9bfc15f270 2023-11-24T08:08:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.192+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9be006b3e0 msgr2=0x7f9be006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.192+0000 7f9c02fcb640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9be006b3e0 0x7f9be006d8a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8000f60 tx=0x7f9bf8009040 comp rx=0 tx=0).stop 2023-11-24T08:08:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.192+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 msgr2=0x7f9bfc0ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.192+0000 7f9c02fcb640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0ad990 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c780 tx=0x7f9bf400cc40 comp rx=0 tx=0).stop 2023-11-24T08:08:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.193+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 shutdown_connections 2023-11-24T08:08:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.193+0000 7f9c02fcb640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f9be006b3e0 0x7f9be006d8a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.193+0000 7f9c02fcb640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bfc15f270 0x7f9bfc0ad990 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.193+0000 7f9c02fcb640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9bfc144770 0x7f9bfc13f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.193+0000 7f9c02fcb640 1 --2- 172.21.15.64:0/286842022 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9bfc06d9e0 0x7f9bfc06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.193+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 >> 172.21.15.64:0/286842022 conn(0x7f9bfc0cec10 msgr2=0x7f9bfc0b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.193+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 shutdown_connections 2023-11-24T08:08:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.193+0000 7f9c02fcb640 1 -- 172.21.15.64:0/286842022 wait complete. 2023-11-24T08:08:27.218 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:08:27.218 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:27.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.439+0000 7fd2623ba640 1 Processor -- start 2023-11-24T08:08:27.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.439+0000 7fd2623ba640 1 -- start start 2023-11-24T08:08:27.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.440+0000 7fd2623ba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd25c0b1f00 0x7fd25c0d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.440+0000 7fd2623ba640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0afd30 con 0x7fd25c15d370 2023-11-24T08:08:27.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.440+0000 7fd2623ba640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0d0c30 con 0x7fd25c0d00d0 2023-11-24T08:08:27.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.440+0000 7fd2623ba640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c0cef60 con 0x7fd25c0b1f00 2023-11-24T08:08:27.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.440+0000 7fd2613b8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd25c15d370 0x7fd25c0cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46058/0 (socket says 172.21.15.64:46058) 2023-11-24T08:08:27.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.440+0000 7fd2613b8640 1 -- 172.21.15.64:0/3584917545 learned_addr learned my addr 172.21.15.64:0/3584917545 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:27.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.440+0000 7fd261bb9640 1 --2- 172.21.15.64:0/3584917545 >> v2:172.21.15.64:3300/0 conn(0x7fd25c0b1f00 0x7fd25c0d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.441+0000 7fd253fff640 1 -- 172.21.15.64:0/3584917545 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008823790 0 0) 0x7fd25c0afd30 con 0x7fd25c15d370 2023-11-24T08:08:27.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.441+0000 7fd253fff640 1 -- 172.21.15.64:0/3584917545 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd248003520 con 0x7fd25c15d370 2023-11-24T08:08:27.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.441+0000 7fd261bb9640 1 -- 172.21.15.64:0/3584917545 >> v1:172.21.15.91:6789/0 conn(0x7fd25c0d00d0 legacy=0x7fd25c0b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:27.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.441+0000 7fd261bb9640 1 -- 172.21.15.64:0/3584917545 >> v1:172.21.15.64:6789/0 conn(0x7fd25c15d370 legacy=0x7fd25c0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.441+0000 7fd261bb9640 1 -- 172.21.15.64:0/3584917545 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0d0ef0 con 0x7fd25c0b1f00 2023-11-24T08:08:27.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.443+0000 7fd261bb9640 1 --2- 172.21.15.64:0/3584917545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b1f00 0x7fd25c0d1c70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd254009d90 tx=0x7fd25402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1554c8920d2507bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:27.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.443+0000 7fd253fff640 1 -- 172.21.15.64:0/3584917545 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25402cc60 con 0x7fd25c0b1f00 2023-11-24T08:08:27.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.443+0000 7fd253fff640 1 -- 172.21.15.64:0/3584917545 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25402cdc0 con 0x7fd25c0b1f00 2023-11-24T08:08:27.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.443+0000 7fd253fff640 1 -- 172.21.15.64:0/3584917545 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254034660 con 0x7fd25c0b1f00 2023-11-24T08:08:27.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.444+0000 7fd2623ba640 1 -- 172.21.15.64:0/3584917545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b1f00 msgr2=0x7fd25c0d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.444+0000 7fd2623ba640 1 --2- 172.21.15.64:0/3584917545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b1f00 0x7fd25c0d1c70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd254009d90 tx=0x7fd25402beb0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.444+0000 7fd2623ba640 1 -- 172.21.15.64:0/3584917545 shutdown_connections 2023-11-24T08:08:27.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.444+0000 7fd2623ba640 1 --2- 172.21.15.64:0/3584917545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b1f00 0x7fd25c0d1c70 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.444+0000 7fd2623ba640 1 -- 172.21.15.64:0/3584917545 >> 172.21.15.64:0/3584917545 conn(0x7fd25c09c3d0 msgr2=0x7fd25c0d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:27.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.444+0000 7fd2623ba640 1 -- 172.21.15.64:0/3584917545 shutdown_connections 2023-11-24T08:08:27.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.444+0000 7fd2623ba640 1 -- 172.21.15.64:0/3584917545 wait complete. 2023-11-24T08:08:27.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.446+0000 7fd2623ba640 1 Processor -- start 2023-11-24T08:08:27.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.446+0000 7fd2623ba640 1 -- start start 2023-11-24T08:08:27.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.447+0000 7fd2623ba640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd25c0b1f00 0x7fd25c15b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.447+0000 7fd2623ba640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd25c0d00d0 0x7fd25c160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.447+0000 7fd2623ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c15d370 0x7fd25c0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.447+0000 7fd2623ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0b81c0 con 0x7fd25c15d370 2023-11-24T08:08:27.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.447+0000 7fd2623ba640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0b7140 con 0x7fd25c0d00d0 2023-11-24T08:08:27.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.447+0000 7fd2623ba640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c15b5a0 con 0x7fd25c0b1f00 2023-11-24T08:08:27.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.447+0000 7fd261bb9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c15d370 0x7fd25c0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd261bb9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c15d370 0x7fd25c0b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60410/0 (socket says 172.21.15.64:60410) 2023-11-24T08:08:27.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd261bb9640 1 -- 172.21.15.64:0/3361002309 learned_addr learned my addr 172.21.15.64:0/3361002309 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:27.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd261bb9640 1 -- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd25c0b1f00 msgr2=0x7fd25c15b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd261bb9640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd25c0b1f00 0x7fd25c15b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd261bb9640 1 -- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd25c0d00d0 msgr2=0x7fd25c160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd260bb7640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd25c0d00d0 0x7fd25c160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd261bb9640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd25c0d00d0 0x7fd25c160370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd261bb9640 1 -- 172.21.15.64:0/3361002309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254009a70 con 0x7fd25c15d370 2023-11-24T08:08:27.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.448+0000 7fd261bb9640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c15d370 0x7fd25c0b11a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd254009d90 tx=0x7fd254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:27.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.449+0000 7fd251ffb640 1 -- 172.21.15.64:0/3361002309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254040650 con 0x7fd25c15d370 2023-11-24T08:08:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.449+0000 7fd251ffb640 1 -- 172.21.15.64:0/3361002309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2540407b0 con 0x7fd25c15d370 2023-11-24T08:08:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.449+0000 7fd251ffb640 1 -- 172.21.15.64:0/3361002309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254040b60 con 0x7fd25c15d370 2023-11-24T08:08:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.449+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c0b7400 con 0x7fd25c15d370 2023-11-24T08:08:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.449+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0b79c0 con 0x7fd25c15d370 2023-11-24T08:08:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.450+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c04e840 con 0x7fd25c15d370 2023-11-24T08:08:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.451+0000 7fd251ffb640 1 -- 172.21.15.64:0/3361002309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd254040cc0 con 0x7fd25c15d370 2023-11-24T08:08:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.452+0000 7fd251ffb640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd24006b3e0 0x7fd24006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.452+0000 7fd251ffb640 1 -- 172.21.15.64:0/3361002309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2540ace00 con 0x7fd25c15d370 2023-11-24T08:08:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.453+0000 7fd2613b8640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd24006b3e0 0x7fd24006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.454+0000 7fd2613b8640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd24006b3e0 0x7fd24006d8a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd258001570 tx=0x7fd258007930 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.456+0000 7fd251ffb640 1 -- 172.21.15.64:0/3361002309 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd254077640 con 0x7fd25c15d370 2023-11-24T08:08:27.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.689+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd25c066650 con 0x7fd25c15d370 2023-11-24T08:08:27.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.691+0000 7fd251ffb640 1 -- 172.21.15.64:0/3361002309 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4912 (secure 0 0 0) 0x7fd254009450 con 0x7fd25c15d370 2023-11-24T08:08:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:27.693 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/970726691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":970726691},{"type":"v1","addr":"172.21.15.64:6835","nonce":970726691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2338720158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2338720158},{"type":"v1","addr":"172.21.15.91:6836","nonce":2338720158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2942920208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2942920208},{"type":"v1","addr":"172.21.15.91:6837","nonce":2942920208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:23.583381+0000","modified":"2023-11-24T08:08:26.696160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/3674316997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3674316997},{"type":"v1","addr":"172.21.15.64:6834","nonce":3674316997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:08:27.693 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 53 2023-11-24T08:08:27.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd24006b3e0 msgr2=0x7fd24006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd24006b3e0 0x7fd24006d8a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd258001570 tx=0x7fd258007930 comp rx=0 tx=0).stop 2023-11-24T08:08:27.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c15d370 msgr2=0x7fd25c0b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c15d370 0x7fd25c0b11a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd254009d90 tx=0x7fd254004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:27.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 shutdown_connections 2023-11-24T08:08:27.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd24006b3e0 0x7fd24006d8a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c15d370 0x7fd25c0b11a0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd25c0d00d0 0x7fd25c160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 --2- 172.21.15.64:0/3361002309 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd25c0b1f00 0x7fd25c15b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 >> 172.21.15.64:0/3361002309 conn(0x7fd25c09c3d0 msgr2=0x7fd25c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:27.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 shutdown_connections 2023-11-24T08:08:27.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.693+0000 7fd2623ba640 1 -- 172.21.15.64:0/3361002309 wait complete. 2023-11-24T08:08:27.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:27.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.934+0000 7f18c5439640 1 Processor -- start 2023-11-24T08:08:27.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.934+0000 7f18c5439640 1 -- start start 2023-11-24T08:08:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.936+0000 7f18c5439640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f18c013dfd0 0x7f18c015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.936+0000 7f18c5439640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c00cf050 0x7f18c00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.936+0000 7f18c5439640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c00ce7c0 con 0x7f18c006c130 2023-11-24T08:08:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.936+0000 7f18c5439640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c00cfa10 con 0x7f18c013dfd0 2023-11-24T08:08:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.936+0000 7f18c5439640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0140900 con 0x7f18c00cf050 2023-11-24T08:08:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.936+0000 7f18bffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f18c013dfd0 0x7f18c015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.936+0000 7f18bffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f18c013dfd0 0x7f18c015be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60416/0 (socket says 172.21.15.64:60416) 2023-11-24T08:08:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.936+0000 7f18bffff640 1 -- 172.21.15.64:0/1660670175 learned_addr learned my addr 172.21.15.64:0/1660670175 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.937+0000 7f18b7fff640 1 --2- 172.21.15.64:0/1660670175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c00cf050 0x7f18c00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.937+0000 7f18bffff640 1 -- 172.21.15.64:0/1660670175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c00cf050 msgr2=0x7f18c00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.937+0000 7f18bffff640 1 --2- 172.21.15.64:0/1660670175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c00cf050 0x7f18c00cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.937+0000 7f18bffff640 1 -- 172.21.15.64:0/1660670175 >> v1:172.21.15.64:6789/0 conn(0x7f18c006c130 legacy=0x7f18c006c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.937+0000 7f18bffff640 1 -- 172.21.15.64:0/1660670175 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c015fa70 con 0x7f18c013dfd0 2023-11-24T08:08:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.937+0000 7f18b7fff640 1 --2- 172.21.15.64:0/1660670175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c00cf050 0x7f18c00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.938+0000 7f18bffff640 1 --2- 172.21.15.64:0/1660670175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c013dfd0 0x7f18c015be70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009870 tx=0x7f18b002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d462e1447dde95a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.938+0000 7f18bf7fe640 1 -- 172.21.15.64:0/1660670175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0030840 con 0x7f18c013dfd0 2023-11-24T08:08:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.938+0000 7f18bf7fe640 1 -- 172.21.15.64:0/1660670175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b00309a0 con 0x7f18c013dfd0 2023-11-24T08:08:27.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.938+0000 7f18bf7fe640 1 -- 172.21.15.64:0/1660670175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0030d30 con 0x7f18c013dfd0 2023-11-24T08:08:27.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.939+0000 7f18c5439640 1 -- 172.21.15.64:0/1660670175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c013dfd0 msgr2=0x7f18c015be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.939+0000 7f18c5439640 1 --2- 172.21.15.64:0/1660670175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c013dfd0 0x7f18c015be70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009870 tx=0x7f18b002b980 comp rx=0 tx=0).stop 2023-11-24T08:08:27.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.939+0000 7f18c5439640 1 -- 172.21.15.64:0/1660670175 shutdown_connections 2023-11-24T08:08:27.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.939+0000 7f18c5439640 1 --2- 172.21.15.64:0/1660670175 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c00cf050 0x7f18c00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.939+0000 7f18c5439640 1 --2- 172.21.15.64:0/1660670175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c013dfd0 0x7f18c015be70 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.939+0000 7f18c5439640 1 -- 172.21.15.64:0/1660670175 >> 172.21.15.64:0/1660670175 conn(0x7f18c006a860 msgr2=0x7f18c006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.939+0000 7f18c5439640 1 -- 172.21.15.64:0/1660670175 shutdown_connections 2023-11-24T08:08:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.939+0000 7f18c5439640 1 -- 172.21.15.64:0/1660670175 wait complete. 2023-11-24T08:08:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.941+0000 7f18c5439640 1 Processor -- start 2023-11-24T08:08:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.941+0000 7f18c5439640 1 -- start start 2023-11-24T08:08:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.942+0000 7f18c5439640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c006c130 0x7f18c0071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.942+0000 7f18c5439640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c00cf050 0x7f18c00b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.942+0000 7f18c5439640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c013dfd0 0x7f18c00b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.942+0000 7f18c5439640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c015bcd0 con 0x7f18c00cf050 2023-11-24T08:08:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.942+0000 7f18c5439640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0070250 con 0x7f18c006c130 2023-11-24T08:08:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.942+0000 7f18c5439640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0070530 con 0x7f18c013dfd0 2023-11-24T08:08:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.942+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c00cf050 0x7f18c00b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c00cf050 0x7f18c00b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60420/0 (socket says 172.21.15.64:60420) 2023-11-24T08:08:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18b7fff640 1 -- 172.21.15.64:0/706304516 learned_addr learned my addr 172.21.15.64:0/706304516 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18bffff640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c006c130 0x7f18c0071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18b7fff640 1 -- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c013dfd0 msgr2=0x7f18c00b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18b7fff640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c013dfd0 0x7f18c00b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18b7fff640 1 -- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c006c130 msgr2=0x7f18c0071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18b7fff640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c006c130 0x7f18c0071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18b7fff640 1 -- 172.21.15.64:0/706304516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00094d0 con 0x7f18c00cf050 2023-11-24T08:08:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.943+0000 7f18b7fff640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c00cf050 0x7f18c00b2a10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c7f0 tx=0x7f18a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.944+0000 7f18bd7fa640 1 -- 172.21.15.64:0/706304516 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8007a30 con 0x7f18c00cf050 2023-11-24T08:08:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.944+0000 7f18bd7fa640 1 -- 172.21.15.64:0/706304516 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a8007b90 con 0x7f18c00cf050 2023-11-24T08:08:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.944+0000 7f18bd7fa640 1 -- 172.21.15.64:0/706304516 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800ce40 con 0x7f18c00cf050 2023-11-24T08:08:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.944+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c0070810 con 0x7f18c00cf050 2023-11-24T08:08:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.944+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c0072030 con 0x7f18c00cf050 2023-11-24T08:08:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.945+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c0071010 con 0x7f18c00cf050 2023-11-24T08:08:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.946+0000 7f18bd7fa640 1 -- 172.21.15.64:0/706304516 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f18a80113f0 con 0x7f18c00cf050 2023-11-24T08:08:27.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.947+0000 7f18bd7fa640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18a006b410 0x7f18a006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:27.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.947+0000 7f18bd7fa640 1 -- 172.21.15.64:0/706304516 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18a808a360 con 0x7f18c00cf050 2023-11-24T08:08:27.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.948+0000 7f18bffff640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18a006b410 0x7f18a006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:27.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.949+0000 7f18bffff640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18a006b410 0x7f18a006d8d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f18b0036000 tx=0x7f18b003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:27.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:27.951+0000 7f18bd7fa640 1 -- 172.21.15.64:0/706304516 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18c0071010 con 0x7f18c00cf050 2023-11-24T08:08:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.147+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18c0071010 con 0x7f18c00cf050 2023-11-24T08:08:28.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.149+0000 7f18bd7fa640 1 -- 172.21.15.64:0/706304516 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16512 (secure 0 0 0) 0x7f18c0071010 con 0x7f18c00cf050 2023-11-24T08:08:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:28.152 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":87,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:25.137033+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:21.467760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:22.606730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18a006b410 msgr2=0x7f18a006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18a006b410 0x7f18a006d8d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f18b0036000 tx=0x7f18b003b040 comp rx=0 tx=0).stop 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c00cf050 msgr2=0x7f18c00b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c00cf050 0x7f18c00b2a10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c7f0 tx=0x7f18a800cbf0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 shutdown_connections 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f18a006b410 0x7f18a006d8d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f18c013dfd0 0x7f18c00b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c00cf050 0x7f18c00b2a10 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 --2- 172.21.15.64:0/706304516 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f18c006c130 0x7f18c0071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.152+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 >> 172.21.15.64:0/706304516 conn(0x7f18c006a860 msgr2=0x7f18c015ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:28.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.153+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 shutdown_connections 2023-11-24T08:08:28.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.153+0000 7f18c5439640 1 -- 172.21.15.64:0/706304516 wait complete. 2023-11-24T08:08:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:08:28.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.390+0000 7f0117fff640 1 Processor -- start 2023-11-24T08:08:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.390+0000 7f0117fff640 1 -- start start 2023-11-24T08:08:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.392+0000 7f0117fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01180cf500 0x7f01180cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.392+0000 7f0117fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0118140900 con 0x7f011813dfd0 2023-11-24T08:08:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.392+0000 7f0117fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011815fa70 con 0x7f01180cf050 2023-11-24T08:08:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.392+0000 7f0117fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011815c580 con 0x7f01180cf500 2023-11-24T08:08:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.392+0000 7f0116ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f011813dfd0 0x7f011815be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46080/0 (socket says 172.21.15.64:46080) 2023-11-24T08:08:28.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.392+0000 7f0116ffd640 1 -- 172.21.15.64:0/1771203542 learned_addr learned my addr 172.21.15.64:0/1771203542 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:28.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.392+0000 7f01177fe640 1 --2- 172.21.15.64:0/1771203542 >> v2:172.21.15.64:3300/0 conn(0x7f01180cf500 0x7f01180cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.393+0000 7f01177fe640 1 -- 172.21.15.64:0/1771203542 >> v1:172.21.15.91:6789/0 conn(0x7f01180cf050 legacy=0x7f01180cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:28.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.393+0000 7f01177fe640 1 -- 172.21.15.64:0/1771203542 >> v1:172.21.15.64:6789/0 conn(0x7f011813dfd0 legacy=0x7f011815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.393+0000 7f01177fe640 1 -- 172.21.15.64:0/1771203542 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011815ee60 con 0x7f01180cf500 2023-11-24T08:08:28.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.394+0000 7f01177fe640 1 --2- 172.21.15.64:0/1771203542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f01180cdec0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0110009d90 tx=0x7f011002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef6887e5b1dfa8b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:28.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.394+0000 7f01167fc640 1 -- 172.21.15.64:0/1771203542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011002cc60 con 0x7f01180cf500 2023-11-24T08:08:28.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.394+0000 7f01167fc640 1 -- 172.21.15.64:0/1771203542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011002cdc0 con 0x7f01180cf500 2023-11-24T08:08:28.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.394+0000 7f01167fc640 1 -- 172.21.15.64:0/1771203542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110034660 con 0x7f01180cf500 2023-11-24T08:08:28.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.395+0000 7f0117fff640 1 -- 172.21.15.64:0/1771203542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 msgr2=0x7f01180cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.395+0000 7f0117fff640 1 --2- 172.21.15.64:0/1771203542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f01180cdec0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0110009d90 tx=0x7f011002beb0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.395+0000 7f0117fff640 1 -- 172.21.15.64:0/1771203542 shutdown_connections 2023-11-24T08:08:28.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.395+0000 7f0117fff640 1 --2- 172.21.15.64:0/1771203542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f01180cdec0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.395+0000 7f0117fff640 1 -- 172.21.15.64:0/1771203542 >> 172.21.15.64:0/1771203542 conn(0x7f011806a860 msgr2=0x7f011806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.395+0000 7f0117fff640 1 -- 172.21.15.64:0/1771203542 shutdown_connections 2023-11-24T08:08:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.395+0000 7f0117fff640 1 -- 172.21.15.64:0/1771203542 wait complete. 2023-11-24T08:08:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.397+0000 7f0117fff640 1 Processor -- start 2023-11-24T08:08:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.397+0000 7f0117fff640 1 -- start start 2023-11-24T08:08:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f0117fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01180cf050 0x7f01181f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f0117fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f0118154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f0117fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f011813dfd0 0x7f011813b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f0117fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01181f43b0 con 0x7f01180cf500 2023-11-24T08:08:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f0117fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118159600 con 0x7f01180cf050 2023-11-24T08:08:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f0117fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01181f6b80 con 0x7f011813dfd0 2023-11-24T08:08:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f010ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f0118154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f010ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f0118154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60432/0 (socket says 172.21.15.64:60432) 2023-11-24T08:08:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.398+0000 7f010ffff640 1 -- 172.21.15.64:0/1057124895 learned_addr learned my addr 172.21.15.64:0/1057124895 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f0116ffd640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01180cf050 0x7f01181f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f010ffff640 1 -- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f011813dfd0 msgr2=0x7f011813b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f010ffff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f011813dfd0 0x7f011813b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f010ffff640 1 -- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01180cf050 msgr2=0x7f01181f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f010ffff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01180cf050 0x7f01181f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f010ffff640 1 -- 172.21.15.64:0/1057124895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0110009a70 con 0x7f01180cf500 2023-11-24T08:08:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f0116ffd640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01180cf050 0x7f01181f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f010ffff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f0118154f50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f010000c430 tx=0x7f010000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f0114ff9640 1 -- 172.21.15.64:0/1057124895 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100011790 con 0x7f01180cf500 2023-11-24T08:08:28.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f0114ff9640 1 -- 172.21.15.64:0/1057124895 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01000118f0 con 0x7f01180cf500 2023-11-24T08:08:28.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.399+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01181f6e60 con 0x7f01180cf500 2023-11-24T08:08:28.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.400+0000 7f0114ff9640 1 -- 172.21.15.64:0/1057124895 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100011ce0 con 0x7f01180cf500 2023-11-24T08:08:28.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.400+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01181f7420 con 0x7f01180cf500 2023-11-24T08:08:28.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.401+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011806dc50 con 0x7f01180cf500 2023-11-24T08:08:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.402+0000 7f0114ff9640 1 -- 172.21.15.64:0/1057124895 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f010001c460 con 0x7f01180cf500 2023-11-24T08:08:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.403+0000 7f0114ff9640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f806b3e0 0x7f00f806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.403+0000 7f0114ff9640 1 -- 172.21.15.64:0/1057124895 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f010008a050 con 0x7f01180cf500 2023-11-24T08:08:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.403+0000 7f0116ffd640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f806b3e0 0x7f00f806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.405+0000 7f0116ffd640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f806b3e0 0x7f00f806d8a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0108005880 tx=0x7f0108009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:28.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.408+0000 7f0114ff9640 1 -- 172.21.15.64:0/1057124895 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0100054850 con 0x7f01180cf500 2023-11-24T08:08:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.598+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f011815f700 con 0x7f00f806b3e0 2023-11-24T08:08:28.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.622+0000 7f0114ff9640 1 -- 172.21.15.64:0/1057124895 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281726 (secure 0 0 0) 0x7f0108001c40 con 0x7f00f806b3e0 2023-11-24T08:08:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:28.630 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:08:28.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f806b3e0 msgr2=0x7f00f806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f806b3e0 0x7f00f806d8a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0108005880 tx=0x7f0108009040 comp rx=0 tx=0).stop 2023-11-24T08:08:28.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 msgr2=0x7f0118154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f0118154f50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f010000c430 tx=0x7f010000c8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 shutdown_connections 2023-11-24T08:08:28.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f806b3e0 0x7f00f806d8a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f011813dfd0 0x7f011813b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01180cf500 0x7f0118154f50 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 --2- 172.21.15.64:0/1057124895 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f01180cf050 0x7f01181f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.629+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 >> 172.21.15.64:0/1057124895 conn(0x7f011806a860 msgr2=0x7f011815d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:28.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.630+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 shutdown_connections 2023-11-24T08:08:28.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.630+0000 7f0117fff640 1 -- 172.21.15.64:0/1057124895 wait complete. 2023-11-24T08:08:28.652 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2023-11-24T08:08:28.113534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258780,"kb_used_data":9316,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715940,"statfs":{"total":773094113280,"available":772829122560,"internally_reserved":0,"allocated":9539584,"data_stored":4748709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002610"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.125419+0000","last_change":"2023-11-24T08:08:23.125419+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.125419+0000","last_undegraded":"2023-11-24T08:08:23.125419+0000","last_fullsized":"2023-11-24T08:08:23.125419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111910+0000","last_change":"2023-11-24T08:08:23.111910+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111910+0000","last_undegraded":"2023-11-24T08:08:23.111910+0000","last_fullsized":"2023-11-24T08:08:23.111910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111691+0000","last_change":"2023-11-24T08:08:23.111691+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111691+0000","last_undegraded":"2023-11-24T08:08:23.111691+0000","last_fullsized":"2023-11-24T08:08:23.111691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119453+0000","last_change":"2023-11-24T08:08:23.119453+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119453+0000","last_undegraded":"2023-11-24T08:08:23.119453+0000","last_fullsized":"2023-11-24T08:08:23.119453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.104404+0000","last_change":"2023-11-24T08:08:23.104404+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.104404+0000","last_undegraded":"2023-11-24T08:08:23.104404+0000","last_fullsized":"2023-11-24T08:08:23.104404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114326+0000","last_change":"2023-11-24T08:08:23.114326+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114326+0000","last_undegraded":"2023-11-24T08:08:23.114326+0000","last_fullsized":"2023-11-24T08:08:23.114326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114692+0000","last_change":"2023-11-24T08:08:23.114692+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114692+0000","last_undegraded":"2023-11-24T08:08:23.114692+0000","last_fullsized":"2023-11-24T08:08:23.114692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114757+0000","last_change":"2023-11-24T08:08:23.114757+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114757+0000","last_undegraded":"2023-11-24T08:08:23.114757+0000","last_fullsized":"2023-11-24T08:08:23.114757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116155+0000","last_change":"2023-11-24T08:08:23.116155+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116155+0000","last_undegraded":"2023-11-24T08:08:23.116155+0000","last_fullsized":"2023-11-24T08:08:23.116155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118098+0000","last_change":"2023-11-24T08:08:23.118098+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118098+0000","last_undegraded":"2023-11-24T08:08:23.118098+0000","last_fullsized":"2023-11-24T08:08:23.118098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112412+0000","last_change":"2023-11-24T08:08:23.112412+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112412+0000","last_undegraded":"2023-11-24T08:08:23.112412+0000","last_fullsized":"2023-11-24T08:08:23.112412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.125090+0000","last_change":"2023-11-24T08:08:23.125090+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.125090+0000","last_undegraded":"2023-11-24T08:08:23.125090+0000","last_fullsized":"2023-11-24T08:08:23.125090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116279+0000","last_change":"2023-11-24T08:08:23.116279+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116279+0000","last_undegraded":"2023-11-24T08:08:23.116279+0000","last_fullsized":"2023-11-24T08:08:23.116279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115991+0000","last_change":"2023-11-24T08:08:23.115991+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115991+0000","last_undegraded":"2023-11-24T08:08:23.115991+0000","last_fullsized":"2023-11-24T08:08:23.115991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103193+0000","last_change":"2023-11-24T08:08:23.103193+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103193+0000","last_undegraded":"2023-11-24T08:08:23.103193+0000","last_fullsized":"2023-11-24T08:08:23.103193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.105524+0000","last_change":"2023-11-24T08:08:23.105524+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.105524+0000","last_undegraded":"2023-11-24T08:08:23.105524+0000","last_fullsized":"2023-11-24T08:08:23.105524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103155+0000","last_change":"2023-11-24T08:08:23.103155+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103155+0000","last_undegraded":"2023-11-24T08:08:23.103155+0000","last_fullsized":"2023-11-24T08:08:23.103155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.120117+0000","last_change":"2023-11-24T08:08:23.120117+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.120117+0000","last_undegraded":"2023-11-24T08:08:23.120117+0000","last_fullsized":"2023-11-24T08:08:23.120117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118749+0000","last_change":"2023-11-24T08:08:23.118749+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118749+0000","last_undegraded":"2023-11-24T08:08:23.118749+0000","last_fullsized":"2023-11-24T08:08:23.118749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118836+0000","last_change":"2023-11-24T08:08:23.118836+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118836+0000","last_undegraded":"2023-11-24T08:08:23.118836+0000","last_fullsized":"2023-11-24T08:08:23.118836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115676+0000","last_change":"2023-11-24T08:08:23.115676+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115676+0000","last_undegraded":"2023-11-24T08:08:23.115676+0000","last_fullsized":"2023-11-24T08:08:23.115676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111546+0000","last_change":"2023-11-24T08:08:23.111546+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111546+0000","last_undegraded":"2023-11-24T08:08:23.111546+0000","last_fullsized":"2023-11-24T08:08:23.111546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117271+0000","last_change":"2023-11-24T08:08:23.117271+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117271+0000","last_undegraded":"2023-11-24T08:08:23.117271+0000","last_fullsized":"2023-11-24T08:08:23.117271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117195+0000","last_change":"2023-11-24T08:08:23.117195+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117195+0000","last_undegraded":"2023-11-24T08:08:23.117195+0000","last_fullsized":"2023-11-24T08:08:23.117195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110410+0000","last_change":"2023-11-24T08:08:23.110410+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110410+0000","last_undegraded":"2023-11-24T08:08:23.110410+0000","last_fullsized":"2023-11-24T08:08:23.110410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.124187+0000","last_change":"2023-11-24T08:08:23.124187+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.124187+0000","last_undegraded":"2023-11-24T08:08:23.124187+0000","last_fullsized":"2023-11-24T08:08:23.124187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110905+0000","last_change":"2023-11-24T08:08:23.110905+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110905+0000","last_undegraded":"2023-11-24T08:08:23.110905+0000","last_fullsized":"2023-11-24T08:08:23.110905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.123814+0000","last_change":"2023-11-24T08:08:23.123814+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.123814+0000","last_undegraded":"2023-11-24T08:08:23.123814+0000","last_fullsized":"2023-11-24T08:08:23.123814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110123+0000","last_change":"2023-11-24T08:08:23.110123+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110123+0000","last_undegraded":"2023-11-24T08:08:23.110123+0000","last_fullsized":"2023-11-24T08:08:23.110123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110824+0000","last_change":"2023-11-24T08:08:23.110824+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110824+0000","last_undegraded":"2023-11-24T08:08:23.110824+0000","last_fullsized":"2023-11-24T08:08:23.110824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.102988+0000","last_change":"2023-11-24T08:08:23.102988+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.102988+0000","last_undegraded":"2023-11-24T08:08:23.102988+0000","last_fullsized":"2023-11-24T08:08:23.102988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119610+0000","last_change":"2023-11-24T08:08:23.119610+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119610+0000","last_undegraded":"2023-11-24T08:08:23.119610+0000","last_fullsized":"2023-11-24T08:08:23.119610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.105615+0000","last_change":"2023-11-24T08:08:23.105615+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.105615+0000","last_undegraded":"2023-11-24T08:08:23.105615+0000","last_fullsized":"2023-11-24T08:08:23.105615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136824+0000","last_change":"2023-11-24T08:08:23.134349+0000","last_active":"2023-11-24T08:08:23.136824+0000","last_peered":"2023-11-24T08:08:23.136824+0000","last_clean":"2023-11-24T08:08:23.136824+0000","last_became_active":"2023-11-24T08:08:23.134026+0000","last_became_peered":"2023-11-24T08:08:23.134026+0000","last_unstale":"2023-11-24T08:08:23.136824+0000","last_undegraded":"2023-11-24T08:08:23.136824+0000","last_fullsized":"2023-11-24T08:08:23.136824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:30.483247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.124855+0000","last_change":"2023-11-24T08:08:23.124855+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.124855+0000","last_undegraded":"2023-11-24T08:08:23.124855+0000","last_fullsized":"2023-11-24T08:08:23.124855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129876+0000","last_change":"2023-11-24T08:08:23.125215+0000","last_active":"2023-11-24T08:08:23.129876+0000","last_peered":"2023-11-24T08:08:23.129876+0000","last_clean":"2023-11-24T08:08:23.129876+0000","last_became_active":"2023-11-24T08:08:23.124899+0000","last_became_peered":"2023-11-24T08:08:23.124899+0000","last_unstale":"2023-11-24T08:08:23.129876+0000","last_undegraded":"2023-11-24T08:08:23.129876+0000","last_fullsized":"2023-11-24T08:08:23.129876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:32.242969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.121284+0000","last_change":"2023-11-24T08:08:23.121284+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.121284+0000","last_undegraded":"2023-11-24T08:08:23.121284+0000","last_fullsized":"2023-11-24T08:08:23.121284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135774+0000","last_change":"2023-11-24T08:08:23.132484+0000","last_active":"2023-11-24T08:08:23.135774+0000","last_peered":"2023-11-24T08:08:23.135774+0000","last_clean":"2023-11-24T08:08:23.135774+0000","last_became_active":"2023-11-24T08:08:23.132220+0000","last_became_peered":"2023-11-24T08:08:23.132220+0000","last_unstale":"2023-11-24T08:08:23.135774+0000","last_undegraded":"2023-11-24T08:08:23.135774+0000","last_fullsized":"2023-11-24T08:08:23.135774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:57.911680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110525+0000","last_change":"2023-11-24T08:08:23.110525+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110525+0000","last_undegraded":"2023-11-24T08:08:23.110525+0000","last_fullsized":"2023-11-24T08:08:23.110525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135536+0000","last_change":"2023-11-24T08:08:23.132020+0000","last_active":"2023-11-24T08:08:23.135536+0000","last_peered":"2023-11-24T08:08:23.135536+0000","last_clean":"2023-11-24T08:08:23.135536+0000","last_became_active":"2023-11-24T08:08:23.131713+0000","last_became_peered":"2023-11-24T08:08:23.131713+0000","last_unstale":"2023-11-24T08:08:23.135536+0000","last_undegraded":"2023-11-24T08:08:23.135536+0000","last_fullsized":"2023-11-24T08:08:23.135536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.740441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110163+0000","last_change":"2023-11-24T08:08:23.110163+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110163+0000","last_undegraded":"2023-11-24T08:08:23.110163+0000","last_fullsized":"2023-11-24T08:08:23.110163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130037+0000","last_change":"2023-11-24T08:08:23.127933+0000","last_active":"2023-11-24T08:08:23.130037+0000","last_peered":"2023-11-24T08:08:23.130037+0000","last_clean":"2023-11-24T08:08:23.130037+0000","last_became_active":"2023-11-24T08:08:23.127651+0000","last_became_peered":"2023-11-24T08:08:23.127651+0000","last_unstale":"2023-11-24T08:08:23.130037+0000","last_undegraded":"2023-11-24T08:08:23.130037+0000","last_fullsized":"2023-11-24T08:08:23.130037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:35.016477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103184+0000","last_change":"2023-11-24T08:08:23.103184+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103184+0000","last_undegraded":"2023-11-24T08:08:23.103184+0000","last_fullsized":"2023-11-24T08:08:23.103184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.126956+0000","last_change":"2023-11-24T08:08:23.120911+0000","last_active":"2023-11-24T08:08:23.126956+0000","last_peered":"2023-11-24T08:08:23.126956+0000","last_clean":"2023-11-24T08:08:23.126956+0000","last_became_active":"2023-11-24T08:08:23.117203+0000","last_became_peered":"2023-11-24T08:08:23.117203+0000","last_unstale":"2023-11-24T08:08:23.126956+0000","last_undegraded":"2023-11-24T08:08:23.126956+0000","last_fullsized":"2023-11-24T08:08:23.126956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:36.336333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119024+0000","last_change":"2023-11-24T08:08:23.119024+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119024+0000","last_undegraded":"2023-11-24T08:08:23.119024+0000","last_fullsized":"2023-11-24T08:08:23.119024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129621+0000","last_change":"2023-11-24T08:08:23.124305+0000","last_active":"2023-11-24T08:08:23.129621+0000","last_peered":"2023-11-24T08:08:23.129621+0000","last_clean":"2023-11-24T08:08:23.129621+0000","last_became_active":"2023-11-24T08:08:23.123993+0000","last_became_peered":"2023-11-24T08:08:23.123993+0000","last_unstale":"2023-11-24T08:08:23.129621+0000","last_undegraded":"2023-11-24T08:08:23.129621+0000","last_fullsized":"2023-11-24T08:08:23.129621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:06.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.101930+0000","last_change":"2023-11-24T08:08:23.101930+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.101930+0000","last_undegraded":"2023-11-24T08:08:23.101930+0000","last_fullsized":"2023-11-24T08:08:23.101930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.127166+0000","last_change":"2023-11-24T08:08:23.118333+0000","last_active":"2023-11-24T08:08:23.127166+0000","last_peered":"2023-11-24T08:08:23.127166+0000","last_clean":"2023-11-24T08:08:23.127166+0000","last_became_active":"2023-11-24T08:08:23.116900+0000","last_became_peered":"2023-11-24T08:08:23.116900+0000","last_unstale":"2023-11-24T08:08:23.127166+0000","last_undegraded":"2023-11-24T08:08:23.127166+0000","last_fullsized":"2023-11-24T08:08:23.127166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:29.860915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.102421+0000","last_change":"2023-11-24T08:08:23.102421+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.102421+0000","last_undegraded":"2023-11-24T08:08:23.102421+0000","last_fullsized":"2023-11-24T08:08:23.102421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134469+0000","last_change":"2023-11-24T08:08:23.130844+0000","last_active":"2023-11-24T08:08:23.134469+0000","last_peered":"2023-11-24T08:08:23.134469+0000","last_clean":"2023-11-24T08:08:23.134469+0000","last_became_active":"2023-11-24T08:08:23.130493+0000","last_became_peered":"2023-11-24T08:08:23.130493+0000","last_unstale":"2023-11-24T08:08:23.134469+0000","last_undegraded":"2023-11-24T08:08:23.134469+0000","last_fullsized":"2023-11-24T08:08:23.134469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:35.016477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112793+0000","last_change":"2023-11-24T08:08:23.112793+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112793+0000","last_undegraded":"2023-11-24T08:08:23.112793+0000","last_fullsized":"2023-11-24T08:08:23.112793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134012+0000","last_change":"2023-11-24T08:08:23.129847+0000","last_active":"2023-11-24T08:08:23.134012+0000","last_peered":"2023-11-24T08:08:23.134012+0000","last_clean":"2023-11-24T08:08:23.134012+0000","last_became_active":"2023-11-24T08:08:23.128603+0000","last_became_peered":"2023-11-24T08:08:23.128603+0000","last_unstale":"2023-11-24T08:08:23.134012+0000","last_undegraded":"2023-11-24T08:08:23.134012+0000","last_fullsized":"2023-11-24T08:08:23.134012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:36.336333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.121568+0000","last_change":"2023-11-24T08:08:23.121568+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.121568+0000","last_undegraded":"2023-11-24T08:08:23.121568+0000","last_fullsized":"2023-11-24T08:08:23.121568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136085+0000","last_change":"2023-11-24T08:08:23.133517+0000","last_active":"2023-11-24T08:08:23.136085+0000","last_peered":"2023-11-24T08:08:23.136085+0000","last_clean":"2023-11-24T08:08:23.136085+0000","last_became_active":"2023-11-24T08:08:23.132959+0000","last_became_peered":"2023-11-24T08:08:23.132959+0000","last_unstale":"2023-11-24T08:08:23.136085+0000","last_undegraded":"2023-11-24T08:08:23.136085+0000","last_fullsized":"2023-11-24T08:08:23.136085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:12:25.699627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.108974+0000","last_change":"2023-11-24T08:08:23.108974+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.108974+0000","last_undegraded":"2023-11-24T08:08:23.108974+0000","last_fullsized":"2023-11-24T08:08:23.108974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130716+0000","last_change":"2023-11-24T08:08:23.126238+0000","last_active":"2023-11-24T08:08:23.130716+0000","last_peered":"2023-11-24T08:08:23.130716+0000","last_clean":"2023-11-24T08:08:23.130716+0000","last_became_active":"2023-11-24T08:08:23.125933+0000","last_became_peered":"2023-11-24T08:08:23.125933+0000","last_unstale":"2023-11-24T08:08:23.130716+0000","last_undegraded":"2023-11-24T08:08:23.130716+0000","last_fullsized":"2023-11-24T08:08:23.130716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:14.689949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.113332+0000","last_change":"2023-11-24T08:08:23.113332+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.113332+0000","last_undegraded":"2023-11-24T08:08:23.113332+0000","last_fullsized":"2023-11-24T08:08:23.113332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.138954+0000","last_change":"2023-11-24T08:08:23.132368+0000","last_active":"2023-11-24T08:08:23.138954+0000","last_peered":"2023-11-24T08:08:23.138954+0000","last_clean":"2023-11-24T08:08:23.138954+0000","last_became_active":"2023-11-24T08:08:23.132111+0000","last_became_peered":"2023-11-24T08:08:23.132111+0000","last_unstale":"2023-11-24T08:08:23.138954+0000","last_undegraded":"2023-11-24T08:08:23.138954+0000","last_fullsized":"2023-11-24T08:08:23.138954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:20.663330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111249+0000","last_change":"2023-11-24T08:08:23.111249+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111249+0000","last_undegraded":"2023-11-24T08:08:23.111249+0000","last_fullsized":"2023-11-24T08:08:23.111249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.121538+0000","last_change":"2023-11-24T08:08:23.116189+0000","last_active":"2023-11-24T08:08:23.121538+0000","last_peered":"2023-11-24T08:08:23.121538+0000","last_clean":"2023-11-24T08:08:23.121538+0000","last_became_active":"2023-11-24T08:08:23.115181+0000","last_became_peered":"2023-11-24T08:08:23.115181+0000","last_unstale":"2023-11-24T08:08:23.121538+0000","last_undegraded":"2023-11-24T08:08:23.121538+0000","last_fullsized":"2023-11-24T08:08:23.121538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:51.802097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109869+0000","last_change":"2023-11-24T08:08:23.109869+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109869+0000","last_undegraded":"2023-11-24T08:08:23.109869+0000","last_fullsized":"2023-11-24T08:08:23.109869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130705+0000","last_change":"2023-11-24T08:08:23.127485+0000","last_active":"2023-11-24T08:08:23.130705+0000","last_peered":"2023-11-24T08:08:23.130705+0000","last_clean":"2023-11-24T08:08:23.130705+0000","last_became_active":"2023-11-24T08:08:23.127216+0000","last_became_peered":"2023-11-24T08:08:23.127216+0000","last_unstale":"2023-11-24T08:08:23.130705+0000","last_undegraded":"2023-11-24T08:08:23.130705+0000","last_fullsized":"2023-11-24T08:08:23.130705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:30.483247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109666+0000","last_change":"2023-11-24T08:08:23.109666+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109666+0000","last_undegraded":"2023-11-24T08:08:23.109666+0000","last_fullsized":"2023-11-24T08:08:23.109666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.137030+0000","last_change":"2023-11-24T08:08:23.135139+0000","last_active":"2023-11-24T08:08:23.137030+0000","last_peered":"2023-11-24T08:08:23.137030+0000","last_clean":"2023-11-24T08:08:23.137030+0000","last_became_active":"2023-11-24T08:08:23.134748+0000","last_became_peered":"2023-11-24T08:08:23.134748+0000","last_unstale":"2023-11-24T08:08:23.137030+0000","last_undegraded":"2023-11-24T08:08:23.137030+0000","last_fullsized":"2023-11-24T08:08:23.137030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:45.681033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111275+0000","last_change":"2023-11-24T08:08:23.111275+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111275+0000","last_undegraded":"2023-11-24T08:08:23.111275+0000","last_fullsized":"2023-11-24T08:08:23.111275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135948+0000","last_change":"2023-11-24T08:08:23.132828+0000","last_active":"2023-11-24T08:08:23.135948+0000","last_peered":"2023-11-24T08:08:23.135948+0000","last_clean":"2023-11-24T08:08:23.135948+0000","last_became_active":"2023-11-24T08:08:23.132502+0000","last_became_peered":"2023-11-24T08:08:23.132502+0000","last_unstale":"2023-11-24T08:08:23.135948+0000","last_undegraded":"2023-11-24T08:08:23.135948+0000","last_fullsized":"2023-11-24T08:08:23.135948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:31:30.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.123529+0000","last_change":"2023-11-24T08:08:23.123529+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.123529+0000","last_undegraded":"2023-11-24T08:08:23.123529+0000","last_fullsized":"2023-11-24T08:08:23.123529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130927+0000","last_change":"2023-11-24T08:08:23.128247+0000","last_active":"2023-11-24T08:08:23.130927+0000","last_peered":"2023-11-24T08:08:23.130927+0000","last_clean":"2023-11-24T08:08:23.130927+0000","last_became_active":"2023-11-24T08:08:23.127889+0000","last_became_peered":"2023-11-24T08:08:23.127889+0000","last_unstale":"2023-11-24T08:08:23.130927+0000","last_undegraded":"2023-11-24T08:08:23.130927+0000","last_fullsized":"2023-11-24T08:08:23.130927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:52.156451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.104547+0000","last_change":"2023-11-24T08:08:23.104547+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.104547+0000","last_undegraded":"2023-11-24T08:08:23.104547+0000","last_fullsized":"2023-11-24T08:08:23.104547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135167+0000","last_change":"2023-11-24T08:08:23.130898+0000","last_active":"2023-11-24T08:08:23.135167+0000","last_peered":"2023-11-24T08:08:23.135167+0000","last_clean":"2023-11-24T08:08:23.135167+0000","last_became_active":"2023-11-24T08:08:23.130435+0000","last_became_peered":"2023-11-24T08:08:23.130435+0000","last_unstale":"2023-11-24T08:08:23.135167+0000","last_undegraded":"2023-11-24T08:08:23.135167+0000","last_fullsized":"2023-11-24T08:08:23.135167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:02.959505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115515+0000","last_change":"2023-11-24T08:08:23.115515+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115515+0000","last_undegraded":"2023-11-24T08:08:23.115515+0000","last_fullsized":"2023-11-24T08:08:23.115515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135146+0000","last_change":"2023-11-24T08:08:23.129858+0000","last_active":"2023-11-24T08:08:23.135146+0000","last_peered":"2023-11-24T08:08:23.135146+0000","last_clean":"2023-11-24T08:08:23.135146+0000","last_became_active":"2023-11-24T08:08:23.129534+0000","last_became_peered":"2023-11-24T08:08:23.129534+0000","last_unstale":"2023-11-24T08:08:23.135146+0000","last_undegraded":"2023-11-24T08:08:23.135146+0000","last_fullsized":"2023-11-24T08:08:23.135146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:43:41.791702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116698+0000","last_change":"2023-11-24T08:08:23.116698+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116698+0000","last_undegraded":"2023-11-24T08:08:23.116698+0000","last_fullsized":"2023-11-24T08:08:23.116698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.131723+0000","last_change":"2023-11-24T08:08:23.129448+0000","last_active":"2023-11-24T08:08:23.131723+0000","last_peered":"2023-11-24T08:08:23.131723+0000","last_clean":"2023-11-24T08:08:23.131723+0000","last_became_active":"2023-11-24T08:08:23.129016+0000","last_became_peered":"2023-11-24T08:08:23.129016+0000","last_unstale":"2023-11-24T08:08:23.131723+0000","last_undegraded":"2023-11-24T08:08:23.131723+0000","last_fullsized":"2023-11-24T08:08:23.131723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:57.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116746+0000","last_change":"2023-11-24T08:08:23.116746+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116746+0000","last_undegraded":"2023-11-24T08:08:23.116746+0000","last_fullsized":"2023-11-24T08:08:23.116746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.139416+0000","last_change":"2023-11-24T08:08:23.134387+0000","last_active":"2023-11-24T08:08:23.139416+0000","last_peered":"2023-11-24T08:08:23.139416+0000","last_clean":"2023-11-24T08:08:23.139416+0000","last_became_active":"2023-11-24T08:08:23.134082+0000","last_became_peered":"2023-11-24T08:08:23.134082+0000","last_unstale":"2023-11-24T08:08:23.139416+0000","last_undegraded":"2023-11-24T08:08:23.139416+0000","last_fullsized":"2023-11-24T08:08:23.139416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:29.860915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112898+0000","last_change":"2023-11-24T08:08:23.112898+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112898+0000","last_undegraded":"2023-11-24T08:08:23.112898+0000","last_fullsized":"2023-11-24T08:08:23.112898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135574+0000","last_change":"2023-11-24T08:08:23.127111+0000","last_active":"2023-11-24T08:08:23.135574+0000","last_peered":"2023-11-24T08:08:23.135574+0000","last_clean":"2023-11-24T08:08:23.135574+0000","last_became_active":"2023-11-24T08:08:23.126828+0000","last_became_peered":"2023-11-24T08:08:23.126828+0000","last_unstale":"2023-11-24T08:08:23.135574+0000","last_undegraded":"2023-11-24T08:08:23.135574+0000","last_fullsized":"2023-11-24T08:08:23.135574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:27.777382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103781+0000","last_change":"2023-11-24T08:08:23.103781+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103781+0000","last_undegraded":"2023-11-24T08:08:23.103781+0000","last_fullsized":"2023-11-24T08:08:23.103781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129854+0000","last_change":"2023-11-24T08:08:23.125635+0000","last_active":"2023-11-24T08:08:23.129854+0000","last_peered":"2023-11-24T08:08:23.129854+0000","last_clean":"2023-11-24T08:08:23.129854+0000","last_became_active":"2023-11-24T08:08:23.125335+0000","last_became_peered":"2023-11-24T08:08:23.125335+0000","last_unstale":"2023-11-24T08:08:23.129854+0000","last_undegraded":"2023-11-24T08:08:23.129854+0000","last_fullsized":"2023-11-24T08:08:23.129854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:20.751833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112141+0000","last_change":"2023-11-24T08:08:23.112141+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112141+0000","last_undegraded":"2023-11-24T08:08:23.112141+0000","last_fullsized":"2023-11-24T08:08:23.112141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129828+0000","last_change":"2023-11-24T08:08:23.125737+0000","last_active":"2023-11-24T08:08:23.129828+0000","last_peered":"2023-11-24T08:08:23.129828+0000","last_clean":"2023-11-24T08:08:23.129828+0000","last_became_active":"2023-11-24T08:08:23.125403+0000","last_became_peered":"2023-11-24T08:08:23.125403+0000","last_unstale":"2023-11-24T08:08:23.129828+0000","last_undegraded":"2023-11-24T08:08:23.129828+0000","last_fullsized":"2023-11-24T08:08:23.129828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:57.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117189+0000","last_change":"2023-11-24T08:08:23.117189+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117189+0000","last_undegraded":"2023-11-24T08:08:23.117189+0000","last_fullsized":"2023-11-24T08:08:23.117189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136817+0000","last_change":"2023-11-24T08:08:23.134364+0000","last_active":"2023-11-24T08:08:23.136817+0000","last_peered":"2023-11-24T08:08:23.136817+0000","last_clean":"2023-11-24T08:08:23.136817+0000","last_became_active":"2023-11-24T08:08:23.134008+0000","last_became_peered":"2023-11-24T08:08:23.134008+0000","last_unstale":"2023-11-24T08:08:23.136817+0000","last_undegraded":"2023-11-24T08:08:23.136817+0000","last_fullsized":"2023-11-24T08:08:23.136817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:57.287980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103133+0000","last_change":"2023-11-24T08:08:23.103133+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103133+0000","last_undegraded":"2023-11-24T08:08:23.103133+0000","last_fullsized":"2023-11-24T08:08:23.103133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134107+0000","last_change":"2023-11-24T08:08:23.130703+0000","last_active":"2023-11-24T08:08:23.134107+0000","last_peered":"2023-11-24T08:08:23.134107+0000","last_clean":"2023-11-24T08:08:23.134107+0000","last_became_active":"2023-11-24T08:08:23.130442+0000","last_became_peered":"2023-11-24T08:08:23.130442+0000","last_unstale":"2023-11-24T08:08:23.134107+0000","last_undegraded":"2023-11-24T08:08:23.134107+0000","last_fullsized":"2023-11-24T08:08:23.134107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:14.668163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.113310+0000","last_change":"2023-11-24T08:08:23.113310+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.113310+0000","last_undegraded":"2023-11-24T08:08:23.113310+0000","last_fullsized":"2023-11-24T08:08:23.113310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136704+0000","last_change":"2023-11-24T08:08:23.134436+0000","last_active":"2023-11-24T08:08:23.136704+0000","last_peered":"2023-11-24T08:08:23.136704+0000","last_clean":"2023-11-24T08:08:23.136704+0000","last_became_active":"2023-11-24T08:08:23.133914+0000","last_became_peered":"2023-11-24T08:08:23.133914+0000","last_unstale":"2023-11-24T08:08:23.136704+0000","last_undegraded":"2023-11-24T08:08:23.136704+0000","last_fullsized":"2023-11-24T08:08:23.136704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:45.936686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.120558+0000","last_change":"2023-11-24T08:08:23.120558+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.120558+0000","last_undegraded":"2023-11-24T08:08:23.120558+0000","last_fullsized":"2023-11-24T08:08:23.120558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129603+0000","last_change":"2023-11-24T08:08:23.124236+0000","last_active":"2023-11-24T08:08:23.129603+0000","last_peered":"2023-11-24T08:08:23.129603+0000","last_clean":"2023-11-24T08:08:23.129603+0000","last_became_active":"2023-11-24T08:08:23.123917+0000","last_became_peered":"2023-11-24T08:08:23.123917+0000","last_unstale":"2023-11-24T08:08:23.129603+0000","last_undegraded":"2023-11-24T08:08:23.129603+0000","last_fullsized":"2023-11-24T08:08:23.129603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:14.668163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115029+0000","last_change":"2023-11-24T08:08:23.115029+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115029+0000","last_undegraded":"2023-11-24T08:08:23.115029+0000","last_fullsized":"2023-11-24T08:08:23.115029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.131926+0000","last_change":"2023-11-24T08:08:23.128663+0000","last_active":"2023-11-24T08:08:23.131926+0000","last_peered":"2023-11-24T08:08:23.131926+0000","last_clean":"2023-11-24T08:08:23.131926+0000","last_became_active":"2023-11-24T08:08:23.128380+0000","last_became_peered":"2023-11-24T08:08:23.128380+0000","last_unstale":"2023-11-24T08:08:23.131926+0000","last_undegraded":"2023-11-24T08:08:23.131926+0000","last_fullsized":"2023-11-24T08:08:23.131926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:32.242969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116516+0000","last_change":"2023-11-24T08:08:23.116516+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116516+0000","last_undegraded":"2023-11-24T08:08:23.116516+0000","last_fullsized":"2023-11-24T08:08:23.116516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136078+0000","last_change":"2023-11-24T08:08:23.133278+0000","last_active":"2023-11-24T08:08:23.136078+0000","last_peered":"2023-11-24T08:08:23.136078+0000","last_clean":"2023-11-24T08:08:23.136078+0000","last_became_active":"2023-11-24T08:08:23.132849+0000","last_became_peered":"2023-11-24T08:08:23.132849+0000","last_unstale":"2023-11-24T08:08:23.136078+0000","last_undegraded":"2023-11-24T08:08:23.136078+0000","last_fullsized":"2023-11-24T08:08:23.136078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:31:30.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109344+0000","last_change":"2023-11-24T08:08:23.109344+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109344+0000","last_undegraded":"2023-11-24T08:08:23.109344+0000","last_fullsized":"2023-11-24T08:08:23.109344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.137736+0000","last_change":"2023-11-24T08:08:23.136137+0000","last_active":"2023-11-24T08:08:23.137736+0000","last_peered":"2023-11-24T08:08:23.137736+0000","last_clean":"2023-11-24T08:08:23.137736+0000","last_became_active":"2023-11-24T08:08:23.135553+0000","last_became_peered":"2023-11-24T08:08:23.135553+0000","last_unstale":"2023-11-24T08:08:23.137736+0000","last_undegraded":"2023-11-24T08:08:23.137736+0000","last_fullsized":"2023-11-24T08:08:23.137736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:55.555959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T08:08:22.094050+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:22.094050+0000","last_peered":"2023-11-24T08:08:22.094050+0000","last_clean":"2023-11-24T08:08:22.094050+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:22.094050+0000","last_undegraded":"2023-11-24T08:08:22.094050+0000","last_fullsized":"2023-11-24T08:08:22.094050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":1036492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.995,"15min":1.0009999999999999},"min":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.516,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.046,"15min":1.079},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.403,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.028}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.747,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6799999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.3069999999999999,"15min":1.3720000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.712,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.4379999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.268,"15min":1.3029999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.355}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.26,"15min":1.3129999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8859999999999999,"5min":2.524,"15min":2.524},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3959999999999999,"15min":1.498},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7889999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.472}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.391,"15min":1.5109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8520000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.268},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.323,"15min":1.365},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6779999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.95499999999999996}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.5189999999999999,"15min":1.599},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.23},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9750000000000001,"5min":2.944,"15min":2.944},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.631,"15min":1.7350000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.915,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.5389999999999999,"15min":1.643},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.962,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.3069999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.319},"min":{"1min":0.73099999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.6579999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.171},{"interface":"front","average":{"1min":1.173,"5min":1.262,"15min":1.335},"min":{"1min":0.81899999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.3819999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.63300000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.593,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2270000000000001,"15min":1.3029999999999999},"min":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7090000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.409}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.204,"15min":1.2869999999999999},"min":{"1min":0.86699999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.825,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3260000000000001,"15min":1.476},"min":{"1min":0.84999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5169999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.2170000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3680000000000001,"15min":1.4370000000000001},"min":{"1min":1.0980000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.8089999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.468},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.45,"15min":1.492},"min":{"1min":1.024,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.821,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.4359999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.526,"15min":1.724},"min":{"1min":0.995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.627,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.1279999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9299999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.4890000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.5760000000000001,"15min":1.778},"min":{"1min":0.84399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.784,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.714,"15min":1.9179999999999999},"min":{"1min":1.3640000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8660000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.599,"15min":1.6759999999999999},"min":{"1min":1.175,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0190000000000001,"5min":2.98,"15min":2.98},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.5800000000000001,"15min":1.6379999999999999},"min":{"1min":1.0549999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9810000000000001,"5min":2.29,"15min":2.29},"last":0.97399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0129999999999999,"15min":1.0489999999999999},"min":{"1min":0.68899999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6419999999999999,"5min":1.764,"15min":1.764},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.016},"min":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5369999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.80000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9530000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.149,"5min":1.1000000000000001,"15min":1.0580000000000001},"min":{"1min":0.78600000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.589,"5min":1.635,"15min":1.635},"last":1.1319999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.214,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.252,"15min":1.224},"min":{"1min":0.81100000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.2690000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.3959999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.381,"15min":1.4259999999999999},"min":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.7400000000000002,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.2250000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.302,"15min":1.304},"min":{"1min":0.67700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.885,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.448,"15min":1.4239999999999999},"min":{"1min":0.91700000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.3620000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.4790000000000001,"15min":1.48},"min":{"1min":0.621,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.042},{"interface":"front","average":{"1min":1.502,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":0.76500000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.292}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.55,"15min":1.538},"min":{"1min":0.78900000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.9370000000000003,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.5640000000000001,"15min":1.538},"min":{"1min":0.74299999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.323,"5min":2.859,"15min":2.859},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.034,"15min":1.04},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.534,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.393},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.68500000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.617,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0700000000000001,"15min":1.081},"min":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5660000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.319},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1970000000000001,"15min":1.2330000000000001},"min":{"1min":0.72499999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5760000000000001,"5min":1.903,"15min":1.903},"last":0.91600000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1910000000000001,"15min":1.2010000000000001},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7390000000000001,"5min":1.823,"15min":1.823},"last":1.24},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.248},"min":{"1min":0.72999999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.534,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.0489999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.278,"15min":1.296},"min":{"1min":0.76300000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7829999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.222},{"interface":"front","average":{"1min":1.196,"5min":1.286,"15min":1.306},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.478,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.306,"15min":1.304},"min":{"1min":1.095,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8120000000000001,"5min":1.944,"15min":1.944},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.365,"15min":1.367},"min":{"1min":1.0960000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0049999999999999,"5min":2.012,"15min":2.012},"last":1.018}]},{"osd":6,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.385,"15min":1.371},"min":{"1min":1.0229999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.736,"5min":2.105,"15min":2.105},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.615,"5min":1.508,"15min":1.514},"min":{"1min":1.306,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.143}]},{"osd":7,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.452,"15min":1.4370000000000001},"min":{"1min":1.1599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0379999999999998,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.524},{"interface":"front","average":{"1min":1.494,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":1.2430000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.827,"5min":2.722,"15min":2.722},"last":1.101}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.3560000000000001,"5min":1.585,"15min":1.585},"last":1.284},{"interface":"front","average":{"1min":0.87,"5min":0.90500000000000003,"15min":0.94199999999999995},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.488,"5min":1.51,"15min":1.51},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.01,"15min":1.129},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4299999999999999,"5min":3.246,"15min":3.246},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.995,"5min":0.97999999999999998,"15min":0.96899999999999997},"min":{"1min":0.51400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.6299999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1140000000000001,"15min":1.1739999999999999},"min":{"1min":0.63800000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.988,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.876},{"interface":"front","average":{"1min":1.155,"5min":1.1120000000000001,"15min":1.1539999999999999},"min":{"1min":0.68999999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8009999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.4299999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2390000000000001,"15min":1.262},"min":{"1min":0.94999999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6950000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.325,"15min":1.3799999999999999},"min":{"1min":0.83799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8919999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.222}]},{"osd":5,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3939999999999999,"15min":1.45},"min":{"1min":0.96799999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.839,"5min":2.395,"15min":2.395},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.89800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.7250000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4219999999999999,"15min":1.4319999999999999},"min":{"1min":0.95999999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0230000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.393,"15min":1.4159999999999999},"min":{"1min":1.0269999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.417,"15min":1.4419999999999999},"min":{"1min":0.98199999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.024},{"interface":"front","average":{"1min":1.486,"5min":1.5029999999999999,"15min":1.573},"min":{"1min":1.1120000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0510000000000002,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.343}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83699999999999997},"min":{"1min":0.44500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.369,"5min":1.524,"15min":1.524},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95799999999999996,"15min":1.01},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.694,"5min":2.133,"15min":2.133},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.087,"15min":1.2070000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4359999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0569999999999999},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.875,"5min":1.998,"15min":1.998},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.258,"15min":1.3160000000000001},"min":{"1min":0.63700000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.722,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.1220000000000001,"15min":1.1859999999999999},"min":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.9350000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.204}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.248,"15min":1.321},"min":{"1min":0.82799999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.843,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.774},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.411,"15min":1.4930000000000001},"min":{"1min":0.75900000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8280000000000001,"5min":3.161,"15min":3.161},"last":1.2569999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.325,"15min":1.387},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.1629999999999998,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3200000000000001,"15min":1.3999999999999999},"min":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6220000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.8919999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4750000000000001,"15min":1.579},"min":{"1min":0.73299999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8420000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.665},{"interface":"front","average":{"1min":1.405,"5min":1.4470000000000001,"15min":1.552},"min":{"1min":0.93100000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0510000000000002,"5min":2.702,"15min":2.702},"last":1.4790000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.504},"min":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.996,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.9970000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.53,"15min":1.6319999999999999},"min":{"1min":0.81999999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.907,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.605}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.88800000000000001},"min":{"1min":0.53900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.95599999999999996,"15min":1.042},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6459999999999999,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0680000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94599999999999995,"15min":1.018},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.724,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1399999999999999,"15min":1.216},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8959999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.3500000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1990000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.127,"15min":1.2150000000000001},"min":{"1min":0.60399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.0899999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.383}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.198,"15min":1.24},"min":{"1min":0.81799999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.645,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.379,"15min":1.4410000000000001},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7909999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.393,"15min":1.446},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.8740000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.415,"15min":1.466},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.855,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.3260000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.4059999999999999,"15min":1.45},"min":{"1min":0.92900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9630000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.613},{"interface":"front","average":{"1min":1.256,"5min":1.3939999999999999,"15min":1.466},"min":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.669,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.216}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.48,"15min":1.548},"min":{"1min":1.0940000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6739999999999999,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.284},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.536,"15min":1.5840000000000001},"min":{"1min":1.0209999999999999,"5min":0.872,"15min":0.872},"max":{"1min":2.335,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1634304,"data_stored":1035473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.876,"15min":0.88},"min":{"1min":0.504,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2110000000000001,"5min":3.423,"15min":3.423},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.997,"15min":0.95399999999999996},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.661,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.96099999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.081,"15min":1.0649999999999999},"min":{"1min":0.60899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5589999999999999,"5min":3.899,"15min":3.899},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.99399999999999999,"15min":0.95799999999999996},"min":{"1min":0.46899999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.3140000000000001,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.282,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.2250000000000001},"min":{"1min":0.65300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.621,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.1619999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3,"15min":1.2689999999999999},"min":{"1min":0.81999999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6459999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.387},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.482,"15min":1.476},"min":{"1min":0.99199999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8959999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4510000000000001,"15min":1.393},"min":{"1min":0.96599999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.4550000000000001,"15min":1.4379999999999999},"min":{"1min":0.80000000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.0110000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.0430000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.399,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.786,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.226},{"interface":"front","average":{"1min":1.49,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":0.84899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9359999999999999,"5min":3.879,"15min":3.879},"last":2.1070000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.54,"15min":1.54},"min":{"1min":0.83499999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":3.823,"15min":3.823},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.5049999999999999,"15min":1.492},"min":{"1min":0.88300000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8740000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.48}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:28.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:28.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.922+0000 7f6704a1a640 1 Processor -- start 2023-11-24T08:08:28.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.923+0000 7f6704a1a640 1 -- start start 2023-11-24T08:08:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.924+0000 7f6704a1a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6700157770 0x7f670006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.924+0000 7f6704a1a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f670006d080 0x7f670015d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.924+0000 7f6704a1a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6700161670 con 0x7f670015ddb0 2023-11-24T08:08:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.924+0000 7f6704a1a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670006d500 con 0x7f670006d080 2023-11-24T08:08:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.924+0000 7f6704a1a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6700071730 con 0x7f6700157770 2023-11-24T08:08:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.925+0000 7f66f7fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f670006d080 0x7f670015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.925+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6700157770 0x7f670006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.925+0000 7f66f7fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f670006d080 0x7f670015d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:52686/0 (socket says 172.21.15.64:52686) 2023-11-24T08:08:28.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.926+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6700157770 0x7f670006cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:41446/0 (socket says 172.21.15.64:41446) 2023-11-24T08:08:28.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.926+0000 7f66f7fff640 1 -- 172.21.15.64:0/1833379847 learned_addr learned my addr 172.21.15.64:0/1833379847 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:28.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.925+0000 7f66fffff640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f670015ddb0 0x7f67000ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:41726/0 (socket says 172.21.15.64:41726) 2023-11-24T08:08:28.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.926+0000 7f66ff7fe640 1 -- 172.21.15.64:0/1833379847 >> v2:172.21.15.91:3300/0 conn(0x7f670006d080 msgr2=0x7f670015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:28.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.926+0000 7f66ff7fe640 1 --2- 172.21.15.64:0/1833379847 >> v2:172.21.15.91:3300/0 conn(0x7f670006d080 0x7f670015d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.926+0000 7f66ff7fe640 1 -- 172.21.15.64:0/1833379847 >> v1:172.21.15.91:6789/0 conn(0x7f670015ddb0 legacy=0x7f67000ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.926+0000 7f66ff7fe640 1 -- 172.21.15.64:0/1833379847 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670015e610 con 0x7f6700157770 2023-11-24T08:08:28.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.926+0000 7f66f7fff640 1 --2- 172.21.15.64:0/1833379847 >> v2:172.21.15.91:3300/0 conn(0x7f670006d080 0x7f670015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:28.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.928+0000 7f66ff7fe640 1 --2- 172.21.15.64:0/1833379847 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6700157770 0x7f670006cb40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66f00097a0 tx=0x7f66f002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42467f723b997788 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:28.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.929+0000 7f66feffd640 1 -- 172.21.15.64:0/1833379847 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0031830 con 0x7f6700157770 2023-11-24T08:08:28.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.929+0000 7f66feffd640 1 -- 172.21.15.64:0/1833379847 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0031990 con 0x7f6700157770 2023-11-24T08:08:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.929+0000 7f66feffd640 1 -- 172.21.15.64:0/1833379847 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0031d20 con 0x7f6700157770 2023-11-24T08:08:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.929+0000 7f6704a1a640 1 -- 172.21.15.64:0/1833379847 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6700157770 msgr2=0x7f670006cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.929+0000 7f6704a1a640 1 --2- 172.21.15.64:0/1833379847 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6700157770 0x7f670006cb40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66f00097a0 tx=0x7f66f002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.930+0000 7f6704a1a640 1 -- 172.21.15.64:0/1833379847 shutdown_connections 2023-11-24T08:08:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.930+0000 7f6704a1a640 1 --2- 172.21.15.64:0/1833379847 >> v2:172.21.15.91:3300/0 conn(0x7f670006d080 0x7f670015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.930+0000 7f6704a1a640 1 --2- 172.21.15.64:0/1833379847 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6700157770 0x7f670006cb40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.930+0000 7f6704a1a640 1 -- 172.21.15.64:0/1833379847 >> 172.21.15.64:0/1833379847 conn(0x7f670006a860 msgr2=0x7f670006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.930+0000 7f6704a1a640 1 -- 172.21.15.64:0/1833379847 shutdown_connections 2023-11-24T08:08:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.930+0000 7f6704a1a640 1 -- 172.21.15.64:0/1833379847 wait complete. 2023-11-24T08:08:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.932+0000 7f6704a1a640 1 Processor -- start 2023-11-24T08:08:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.932+0000 7f6704a1a640 1 -- start start 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f6704a1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f670006d080 0x7f67000cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f6704a1a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6700157770 0x7f670015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f6704a1a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f670015ddb0 0x7f6700154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f6704a1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000ce500 con 0x7f670006d080 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f6704a1a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700158190 con 0x7f6700157770 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f6704a1a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67001427e0 con 0x7f670015ddb0 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f670006d080 0x7f67000cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f670006d080 0x7f67000cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60438/0 (socket says 172.21.15.64:60438) 2023-11-24T08:08:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f66ff7fe640 1 -- 172.21.15.64:0/3049085790 learned_addr learned my addr 172.21.15.64:0/3049085790 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.933+0000 7f66f7fff640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6700157770 0x7f670015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66fffff640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f670015ddb0 0x7f6700154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66ff7fe640 1 -- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f670015ddb0 msgr2=0x7f6700154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66ff7fe640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f670015ddb0 0x7f6700154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66ff7fe640 1 -- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6700157770 msgr2=0x7f670015cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66ff7fe640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6700157770 0x7f670015cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66ff7fe640 1 -- 172.21.15.64:0/3049085790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e80094d0 con 0x7f670006d080 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66f7fff640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6700157770 0x7f670015cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66ff7fe640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f670006d080 0x7f67000cde40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f66f002bee0 tx=0x7f66f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66fcff9640 1 -- 172.21.15.64:0/3049085790 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f003a6f0 con 0x7f670006d080 2023-11-24T08:08:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66fcff9640 1 -- 172.21.15.64:0/3049085790 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f003a870 con 0x7f670006d080 2023-11-24T08:08:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f66fcff9640 1 -- 172.21.15.64:0/3049085790 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f003ac20 con 0x7f670006d080 2023-11-24T08:08:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.934+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f0009400 con 0x7f670006d080 2023-11-24T08:08:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.935+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6700142da0 con 0x7f670006d080 2023-11-24T08:08:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.936+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670004e840 con 0x7f670006d080 2023-11-24T08:08:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.937+0000 7f66fcff9640 1 -- 172.21.15.64:0/3049085790 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f66f0030380 con 0x7f670006d080 2023-11-24T08:08:28.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.938+0000 7f66fcff9640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f66e006b3e0 0x7f66e006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.938+0000 7f66fcff9640 1 -- 172.21.15.64:0/3049085790 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66f00abb50 con 0x7f670006d080 2023-11-24T08:08:28.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.938+0000 7f66f7fff640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f66e006b3e0 0x7f66e006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.939+0000 7f66f7fff640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f66e006b3e0 0x7f66e006d8a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009c50 tx=0x7f66e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:28.942+0000 7f66fcff9640 1 -- 172.21.15.64:0/3049085790 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66f0076240 con 0x7f670006d080 2023-11-24T08:08:29.060 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-11-24T08:08:29.058+0000 7f5c57c47640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:08:29.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.131+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6700157020 con 0x7f66e006b3e0 2023-11-24T08:08:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.153+0000 7f66fcff9640 1 -- 172.21.15.64:0/3049085790 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281726 (secure 0 0 0) 0x7f6700157020 con 0x7f66e006b3e0 2023-11-24T08:08:29.159 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f66e006b3e0 msgr2=0x7f66e006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f66e006b3e0 0x7f66e006d8a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009c50 tx=0x7f66e8008040 comp rx=0 tx=0).stop 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f670006d080 msgr2=0x7f67000cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f670006d080 0x7f67000cde40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f66f002bee0 tx=0x7f66f00046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 shutdown_connections 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f66e006b3e0 0x7f66e006d8a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f670015ddb0 0x7f6700154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6700157770 0x7f670015cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 --2- 172.21.15.64:0/3049085790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f670006d080 0x7f67000cde40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 >> 172.21.15.64:0/3049085790 conn(0x7f670006a860 msgr2=0x7f67000b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:29.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 shutdown_connections 2023-11-24T08:08:29.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.158+0000 7f6704a1a640 1 -- 172.21.15.64:0/3049085790 wait complete. 2023-11-24T08:08:29.178 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2023-11-24T08:08:28.113534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258780,"kb_used_data":9316,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715940,"statfs":{"total":773094113280,"available":772829122560,"internally_reserved":0,"allocated":9539584,"data_stored":4748709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002610"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.125419+0000","last_change":"2023-11-24T08:08:23.125419+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.125419+0000","last_undegraded":"2023-11-24T08:08:23.125419+0000","last_fullsized":"2023-11-24T08:08:23.125419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111910+0000","last_change":"2023-11-24T08:08:23.111910+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111910+0000","last_undegraded":"2023-11-24T08:08:23.111910+0000","last_fullsized":"2023-11-24T08:08:23.111910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111691+0000","last_change":"2023-11-24T08:08:23.111691+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111691+0000","last_undegraded":"2023-11-24T08:08:23.111691+0000","last_fullsized":"2023-11-24T08:08:23.111691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119453+0000","last_change":"2023-11-24T08:08:23.119453+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119453+0000","last_undegraded":"2023-11-24T08:08:23.119453+0000","last_fullsized":"2023-11-24T08:08:23.119453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.104404+0000","last_change":"2023-11-24T08:08:23.104404+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.104404+0000","last_undegraded":"2023-11-24T08:08:23.104404+0000","last_fullsized":"2023-11-24T08:08:23.104404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114326+0000","last_change":"2023-11-24T08:08:23.114326+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114326+0000","last_undegraded":"2023-11-24T08:08:23.114326+0000","last_fullsized":"2023-11-24T08:08:23.114326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114692+0000","last_change":"2023-11-24T08:08:23.114692+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114692+0000","last_undegraded":"2023-11-24T08:08:23.114692+0000","last_fullsized":"2023-11-24T08:08:23.114692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114757+0000","last_change":"2023-11-24T08:08:23.114757+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114757+0000","last_undegraded":"2023-11-24T08:08:23.114757+0000","last_fullsized":"2023-11-24T08:08:23.114757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116155+0000","last_change":"2023-11-24T08:08:23.116155+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116155+0000","last_undegraded":"2023-11-24T08:08:23.116155+0000","last_fullsized":"2023-11-24T08:08:23.116155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118098+0000","last_change":"2023-11-24T08:08:23.118098+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118098+0000","last_undegraded":"2023-11-24T08:08:23.118098+0000","last_fullsized":"2023-11-24T08:08:23.118098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112412+0000","last_change":"2023-11-24T08:08:23.112412+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112412+0000","last_undegraded":"2023-11-24T08:08:23.112412+0000","last_fullsized":"2023-11-24T08:08:23.112412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.125090+0000","last_change":"2023-11-24T08:08:23.125090+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.125090+0000","last_undegraded":"2023-11-24T08:08:23.125090+0000","last_fullsized":"2023-11-24T08:08:23.125090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116279+0000","last_change":"2023-11-24T08:08:23.116279+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116279+0000","last_undegraded":"2023-11-24T08:08:23.116279+0000","last_fullsized":"2023-11-24T08:08:23.116279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115991+0000","last_change":"2023-11-24T08:08:23.115991+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115991+0000","last_undegraded":"2023-11-24T08:08:23.115991+0000","last_fullsized":"2023-11-24T08:08:23.115991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103193+0000","last_change":"2023-11-24T08:08:23.103193+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103193+0000","last_undegraded":"2023-11-24T08:08:23.103193+0000","last_fullsized":"2023-11-24T08:08:23.103193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.105524+0000","last_change":"2023-11-24T08:08:23.105524+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.105524+0000","last_undegraded":"2023-11-24T08:08:23.105524+0000","last_fullsized":"2023-11-24T08:08:23.105524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103155+0000","last_change":"2023-11-24T08:08:23.103155+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103155+0000","last_undegraded":"2023-11-24T08:08:23.103155+0000","last_fullsized":"2023-11-24T08:08:23.103155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.120117+0000","last_change":"2023-11-24T08:08:23.120117+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.120117+0000","last_undegraded":"2023-11-24T08:08:23.120117+0000","last_fullsized":"2023-11-24T08:08:23.120117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118749+0000","last_change":"2023-11-24T08:08:23.118749+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118749+0000","last_undegraded":"2023-11-24T08:08:23.118749+0000","last_fullsized":"2023-11-24T08:08:23.118749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118836+0000","last_change":"2023-11-24T08:08:23.118836+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118836+0000","last_undegraded":"2023-11-24T08:08:23.118836+0000","last_fullsized":"2023-11-24T08:08:23.118836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115676+0000","last_change":"2023-11-24T08:08:23.115676+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115676+0000","last_undegraded":"2023-11-24T08:08:23.115676+0000","last_fullsized":"2023-11-24T08:08:23.115676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111546+0000","last_change":"2023-11-24T08:08:23.111546+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111546+0000","last_undegraded":"2023-11-24T08:08:23.111546+0000","last_fullsized":"2023-11-24T08:08:23.111546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117271+0000","last_change":"2023-11-24T08:08:23.117271+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117271+0000","last_undegraded":"2023-11-24T08:08:23.117271+0000","last_fullsized":"2023-11-24T08:08:23.117271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117195+0000","last_change":"2023-11-24T08:08:23.117195+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117195+0000","last_undegraded":"2023-11-24T08:08:23.117195+0000","last_fullsized":"2023-11-24T08:08:23.117195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110410+0000","last_change":"2023-11-24T08:08:23.110410+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110410+0000","last_undegraded":"2023-11-24T08:08:23.110410+0000","last_fullsized":"2023-11-24T08:08:23.110410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.124187+0000","last_change":"2023-11-24T08:08:23.124187+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.124187+0000","last_undegraded":"2023-11-24T08:08:23.124187+0000","last_fullsized":"2023-11-24T08:08:23.124187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110905+0000","last_change":"2023-11-24T08:08:23.110905+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110905+0000","last_undegraded":"2023-11-24T08:08:23.110905+0000","last_fullsized":"2023-11-24T08:08:23.110905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.123814+0000","last_change":"2023-11-24T08:08:23.123814+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.123814+0000","last_undegraded":"2023-11-24T08:08:23.123814+0000","last_fullsized":"2023-11-24T08:08:23.123814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110123+0000","last_change":"2023-11-24T08:08:23.110123+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110123+0000","last_undegraded":"2023-11-24T08:08:23.110123+0000","last_fullsized":"2023-11-24T08:08:23.110123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110824+0000","last_change":"2023-11-24T08:08:23.110824+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110824+0000","last_undegraded":"2023-11-24T08:08:23.110824+0000","last_fullsized":"2023-11-24T08:08:23.110824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.102988+0000","last_change":"2023-11-24T08:08:23.102988+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.102988+0000","last_undegraded":"2023-11-24T08:08:23.102988+0000","last_fullsized":"2023-11-24T08:08:23.102988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119610+0000","last_change":"2023-11-24T08:08:23.119610+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119610+0000","last_undegraded":"2023-11-24T08:08:23.119610+0000","last_fullsized":"2023-11-24T08:08:23.119610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.105615+0000","last_change":"2023-11-24T08:08:23.105615+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.105615+0000","last_undegraded":"2023-11-24T08:08:23.105615+0000","last_fullsized":"2023-11-24T08:08:23.105615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136824+0000","last_change":"2023-11-24T08:08:23.134349+0000","last_active":"2023-11-24T08:08:23.136824+0000","last_peered":"2023-11-24T08:08:23.136824+0000","last_clean":"2023-11-24T08:08:23.136824+0000","last_became_active":"2023-11-24T08:08:23.134026+0000","last_became_peered":"2023-11-24T08:08:23.134026+0000","last_unstale":"2023-11-24T08:08:23.136824+0000","last_undegraded":"2023-11-24T08:08:23.136824+0000","last_fullsized":"2023-11-24T08:08:23.136824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:30.483247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.124855+0000","last_change":"2023-11-24T08:08:23.124855+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.124855+0000","last_undegraded":"2023-11-24T08:08:23.124855+0000","last_fullsized":"2023-11-24T08:08:23.124855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129876+0000","last_change":"2023-11-24T08:08:23.125215+0000","last_active":"2023-11-24T08:08:23.129876+0000","last_peered":"2023-11-24T08:08:23.129876+0000","last_clean":"2023-11-24T08:08:23.129876+0000","last_became_active":"2023-11-24T08:08:23.124899+0000","last_became_peered":"2023-11-24T08:08:23.124899+0000","last_unstale":"2023-11-24T08:08:23.129876+0000","last_undegraded":"2023-11-24T08:08:23.129876+0000","last_fullsized":"2023-11-24T08:08:23.129876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:32.242969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.121284+0000","last_change":"2023-11-24T08:08:23.121284+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.121284+0000","last_undegraded":"2023-11-24T08:08:23.121284+0000","last_fullsized":"2023-11-24T08:08:23.121284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135774+0000","last_change":"2023-11-24T08:08:23.132484+0000","last_active":"2023-11-24T08:08:23.135774+0000","last_peered":"2023-11-24T08:08:23.135774+0000","last_clean":"2023-11-24T08:08:23.135774+0000","last_became_active":"2023-11-24T08:08:23.132220+0000","last_became_peered":"2023-11-24T08:08:23.132220+0000","last_unstale":"2023-11-24T08:08:23.135774+0000","last_undegraded":"2023-11-24T08:08:23.135774+0000","last_fullsized":"2023-11-24T08:08:23.135774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:57.911680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110525+0000","last_change":"2023-11-24T08:08:23.110525+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110525+0000","last_undegraded":"2023-11-24T08:08:23.110525+0000","last_fullsized":"2023-11-24T08:08:23.110525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135536+0000","last_change":"2023-11-24T08:08:23.132020+0000","last_active":"2023-11-24T08:08:23.135536+0000","last_peered":"2023-11-24T08:08:23.135536+0000","last_clean":"2023-11-24T08:08:23.135536+0000","last_became_active":"2023-11-24T08:08:23.131713+0000","last_became_peered":"2023-11-24T08:08:23.131713+0000","last_unstale":"2023-11-24T08:08:23.135536+0000","last_undegraded":"2023-11-24T08:08:23.135536+0000","last_fullsized":"2023-11-24T08:08:23.135536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.740441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110163+0000","last_change":"2023-11-24T08:08:23.110163+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110163+0000","last_undegraded":"2023-11-24T08:08:23.110163+0000","last_fullsized":"2023-11-24T08:08:23.110163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130037+0000","last_change":"2023-11-24T08:08:23.127933+0000","last_active":"2023-11-24T08:08:23.130037+0000","last_peered":"2023-11-24T08:08:23.130037+0000","last_clean":"2023-11-24T08:08:23.130037+0000","last_became_active":"2023-11-24T08:08:23.127651+0000","last_became_peered":"2023-11-24T08:08:23.127651+0000","last_unstale":"2023-11-24T08:08:23.130037+0000","last_undegraded":"2023-11-24T08:08:23.130037+0000","last_fullsized":"2023-11-24T08:08:23.130037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:35.016477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103184+0000","last_change":"2023-11-24T08:08:23.103184+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103184+0000","last_undegraded":"2023-11-24T08:08:23.103184+0000","last_fullsized":"2023-11-24T08:08:23.103184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.126956+0000","last_change":"2023-11-24T08:08:23.120911+0000","last_active":"2023-11-24T08:08:23.126956+0000","last_peered":"2023-11-24T08:08:23.126956+0000","last_clean":"2023-11-24T08:08:23.126956+0000","last_became_active":"2023-11-24T08:08:23.117203+0000","last_became_peered":"2023-11-24T08:08:23.117203+0000","last_unstale":"2023-11-24T08:08:23.126956+0000","last_undegraded":"2023-11-24T08:08:23.126956+0000","last_fullsized":"2023-11-24T08:08:23.126956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:36.336333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119024+0000","last_change":"2023-11-24T08:08:23.119024+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119024+0000","last_undegraded":"2023-11-24T08:08:23.119024+0000","last_fullsized":"2023-11-24T08:08:23.119024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129621+0000","last_change":"2023-11-24T08:08:23.124305+0000","last_active":"2023-11-24T08:08:23.129621+0000","last_peered":"2023-11-24T08:08:23.129621+0000","last_clean":"2023-11-24T08:08:23.129621+0000","last_became_active":"2023-11-24T08:08:23.123993+0000","last_became_peered":"2023-11-24T08:08:23.123993+0000","last_unstale":"2023-11-24T08:08:23.129621+0000","last_undegraded":"2023-11-24T08:08:23.129621+0000","last_fullsized":"2023-11-24T08:08:23.129621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:06.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.101930+0000","last_change":"2023-11-24T08:08:23.101930+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.101930+0000","last_undegraded":"2023-11-24T08:08:23.101930+0000","last_fullsized":"2023-11-24T08:08:23.101930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.127166+0000","last_change":"2023-11-24T08:08:23.118333+0000","last_active":"2023-11-24T08:08:23.127166+0000","last_peered":"2023-11-24T08:08:23.127166+0000","last_clean":"2023-11-24T08:08:23.127166+0000","last_became_active":"2023-11-24T08:08:23.116900+0000","last_became_peered":"2023-11-24T08:08:23.116900+0000","last_unstale":"2023-11-24T08:08:23.127166+0000","last_undegraded":"2023-11-24T08:08:23.127166+0000","last_fullsized":"2023-11-24T08:08:23.127166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:29.860915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.102421+0000","last_change":"2023-11-24T08:08:23.102421+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.102421+0000","last_undegraded":"2023-11-24T08:08:23.102421+0000","last_fullsized":"2023-11-24T08:08:23.102421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134469+0000","last_change":"2023-11-24T08:08:23.130844+0000","last_active":"2023-11-24T08:08:23.134469+0000","last_peered":"2023-11-24T08:08:23.134469+0000","last_clean":"2023-11-24T08:08:23.134469+0000","last_became_active":"2023-11-24T08:08:23.130493+0000","last_became_peered":"2023-11-24T08:08:23.130493+0000","last_unstale":"2023-11-24T08:08:23.134469+0000","last_undegraded":"2023-11-24T08:08:23.134469+0000","last_fullsized":"2023-11-24T08:08:23.134469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:35.016477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112793+0000","last_change":"2023-11-24T08:08:23.112793+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112793+0000","last_undegraded":"2023-11-24T08:08:23.112793+0000","last_fullsized":"2023-11-24T08:08:23.112793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134012+0000","last_change":"2023-11-24T08:08:23.129847+0000","last_active":"2023-11-24T08:08:23.134012+0000","last_peered":"2023-11-24T08:08:23.134012+0000","last_clean":"2023-11-24T08:08:23.134012+0000","last_became_active":"2023-11-24T08:08:23.128603+0000","last_became_peered":"2023-11-24T08:08:23.128603+0000","last_unstale":"2023-11-24T08:08:23.134012+0000","last_undegraded":"2023-11-24T08:08:23.134012+0000","last_fullsized":"2023-11-24T08:08:23.134012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:36.336333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.121568+0000","last_change":"2023-11-24T08:08:23.121568+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.121568+0000","last_undegraded":"2023-11-24T08:08:23.121568+0000","last_fullsized":"2023-11-24T08:08:23.121568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136085+0000","last_change":"2023-11-24T08:08:23.133517+0000","last_active":"2023-11-24T08:08:23.136085+0000","last_peered":"2023-11-24T08:08:23.136085+0000","last_clean":"2023-11-24T08:08:23.136085+0000","last_became_active":"2023-11-24T08:08:23.132959+0000","last_became_peered":"2023-11-24T08:08:23.132959+0000","last_unstale":"2023-11-24T08:08:23.136085+0000","last_undegraded":"2023-11-24T08:08:23.136085+0000","last_fullsized":"2023-11-24T08:08:23.136085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:12:25.699627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.108974+0000","last_change":"2023-11-24T08:08:23.108974+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.108974+0000","last_undegraded":"2023-11-24T08:08:23.108974+0000","last_fullsized":"2023-11-24T08:08:23.108974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130716+0000","last_change":"2023-11-24T08:08:23.126238+0000","last_active":"2023-11-24T08:08:23.130716+0000","last_peered":"2023-11-24T08:08:23.130716+0000","last_clean":"2023-11-24T08:08:23.130716+0000","last_became_active":"2023-11-24T08:08:23.125933+0000","last_became_peered":"2023-11-24T08:08:23.125933+0000","last_unstale":"2023-11-24T08:08:23.130716+0000","last_undegraded":"2023-11-24T08:08:23.130716+0000","last_fullsized":"2023-11-24T08:08:23.130716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:14.689949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.113332+0000","last_change":"2023-11-24T08:08:23.113332+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.113332+0000","last_undegraded":"2023-11-24T08:08:23.113332+0000","last_fullsized":"2023-11-24T08:08:23.113332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.138954+0000","last_change":"2023-11-24T08:08:23.132368+0000","last_active":"2023-11-24T08:08:23.138954+0000","last_peered":"2023-11-24T08:08:23.138954+0000","last_clean":"2023-11-24T08:08:23.138954+0000","last_became_active":"2023-11-24T08:08:23.132111+0000","last_became_peered":"2023-11-24T08:08:23.132111+0000","last_unstale":"2023-11-24T08:08:23.138954+0000","last_undegraded":"2023-11-24T08:08:23.138954+0000","last_fullsized":"2023-11-24T08:08:23.138954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:20.663330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111249+0000","last_change":"2023-11-24T08:08:23.111249+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111249+0000","last_undegraded":"2023-11-24T08:08:23.111249+0000","last_fullsized":"2023-11-24T08:08:23.111249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.121538+0000","last_change":"2023-11-24T08:08:23.116189+0000","last_active":"2023-11-24T08:08:23.121538+0000","last_peered":"2023-11-24T08:08:23.121538+0000","last_clean":"2023-11-24T08:08:23.121538+0000","last_became_active":"2023-11-24T08:08:23.115181+0000","last_became_peered":"2023-11-24T08:08:23.115181+0000","last_unstale":"2023-11-24T08:08:23.121538+0000","last_undegraded":"2023-11-24T08:08:23.121538+0000","last_fullsized":"2023-11-24T08:08:23.121538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:51.802097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109869+0000","last_change":"2023-11-24T08:08:23.109869+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109869+0000","last_undegraded":"2023-11-24T08:08:23.109869+0000","last_fullsized":"2023-11-24T08:08:23.109869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130705+0000","last_change":"2023-11-24T08:08:23.127485+0000","last_active":"2023-11-24T08:08:23.130705+0000","last_peered":"2023-11-24T08:08:23.130705+0000","last_clean":"2023-11-24T08:08:23.130705+0000","last_became_active":"2023-11-24T08:08:23.127216+0000","last_became_peered":"2023-11-24T08:08:23.127216+0000","last_unstale":"2023-11-24T08:08:23.130705+0000","last_undegraded":"2023-11-24T08:08:23.130705+0000","last_fullsized":"2023-11-24T08:08:23.130705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:30.483247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109666+0000","last_change":"2023-11-24T08:08:23.109666+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109666+0000","last_undegraded":"2023-11-24T08:08:23.109666+0000","last_fullsized":"2023-11-24T08:08:23.109666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.137030+0000","last_change":"2023-11-24T08:08:23.135139+0000","last_active":"2023-11-24T08:08:23.137030+0000","last_peered":"2023-11-24T08:08:23.137030+0000","last_clean":"2023-11-24T08:08:23.137030+0000","last_became_active":"2023-11-24T08:08:23.134748+0000","last_became_peered":"2023-11-24T08:08:23.134748+0000","last_unstale":"2023-11-24T08:08:23.137030+0000","last_undegraded":"2023-11-24T08:08:23.137030+0000","last_fullsized":"2023-11-24T08:08:23.137030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:45.681033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111275+0000","last_change":"2023-11-24T08:08:23.111275+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111275+0000","last_undegraded":"2023-11-24T08:08:23.111275+0000","last_fullsized":"2023-11-24T08:08:23.111275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135948+0000","last_change":"2023-11-24T08:08:23.132828+0000","last_active":"2023-11-24T08:08:23.135948+0000","last_peered":"2023-11-24T08:08:23.135948+0000","last_clean":"2023-11-24T08:08:23.135948+0000","last_became_active":"2023-11-24T08:08:23.132502+0000","last_became_peered":"2023-11-24T08:08:23.132502+0000","last_unstale":"2023-11-24T08:08:23.135948+0000","last_undegraded":"2023-11-24T08:08:23.135948+0000","last_fullsized":"2023-11-24T08:08:23.135948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:31:30.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.123529+0000","last_change":"2023-11-24T08:08:23.123529+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.123529+0000","last_undegraded":"2023-11-24T08:08:23.123529+0000","last_fullsized":"2023-11-24T08:08:23.123529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130927+0000","last_change":"2023-11-24T08:08:23.128247+0000","last_active":"2023-11-24T08:08:23.130927+0000","last_peered":"2023-11-24T08:08:23.130927+0000","last_clean":"2023-11-24T08:08:23.130927+0000","last_became_active":"2023-11-24T08:08:23.127889+0000","last_became_peered":"2023-11-24T08:08:23.127889+0000","last_unstale":"2023-11-24T08:08:23.130927+0000","last_undegraded":"2023-11-24T08:08:23.130927+0000","last_fullsized":"2023-11-24T08:08:23.130927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:52.156451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.104547+0000","last_change":"2023-11-24T08:08:23.104547+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.104547+0000","last_undegraded":"2023-11-24T08:08:23.104547+0000","last_fullsized":"2023-11-24T08:08:23.104547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135167+0000","last_change":"2023-11-24T08:08:23.130898+0000","last_active":"2023-11-24T08:08:23.135167+0000","last_peered":"2023-11-24T08:08:23.135167+0000","last_clean":"2023-11-24T08:08:23.135167+0000","last_became_active":"2023-11-24T08:08:23.130435+0000","last_became_peered":"2023-11-24T08:08:23.130435+0000","last_unstale":"2023-11-24T08:08:23.135167+0000","last_undegraded":"2023-11-24T08:08:23.135167+0000","last_fullsized":"2023-11-24T08:08:23.135167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:02.959505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115515+0000","last_change":"2023-11-24T08:08:23.115515+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115515+0000","last_undegraded":"2023-11-24T08:08:23.115515+0000","last_fullsized":"2023-11-24T08:08:23.115515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135146+0000","last_change":"2023-11-24T08:08:23.129858+0000","last_active":"2023-11-24T08:08:23.135146+0000","last_peered":"2023-11-24T08:08:23.135146+0000","last_clean":"2023-11-24T08:08:23.135146+0000","last_became_active":"2023-11-24T08:08:23.129534+0000","last_became_peered":"2023-11-24T08:08:23.129534+0000","last_unstale":"2023-11-24T08:08:23.135146+0000","last_undegraded":"2023-11-24T08:08:23.135146+0000","last_fullsized":"2023-11-24T08:08:23.135146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:43:41.791702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116698+0000","last_change":"2023-11-24T08:08:23.116698+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116698+0000","last_undegraded":"2023-11-24T08:08:23.116698+0000","last_fullsized":"2023-11-24T08:08:23.116698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.131723+0000","last_change":"2023-11-24T08:08:23.129448+0000","last_active":"2023-11-24T08:08:23.131723+0000","last_peered":"2023-11-24T08:08:23.131723+0000","last_clean":"2023-11-24T08:08:23.131723+0000","last_became_active":"2023-11-24T08:08:23.129016+0000","last_became_peered":"2023-11-24T08:08:23.129016+0000","last_unstale":"2023-11-24T08:08:23.131723+0000","last_undegraded":"2023-11-24T08:08:23.131723+0000","last_fullsized":"2023-11-24T08:08:23.131723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:57.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116746+0000","last_change":"2023-11-24T08:08:23.116746+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116746+0000","last_undegraded":"2023-11-24T08:08:23.116746+0000","last_fullsized":"2023-11-24T08:08:23.116746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.139416+0000","last_change":"2023-11-24T08:08:23.134387+0000","last_active":"2023-11-24T08:08:23.139416+0000","last_peered":"2023-11-24T08:08:23.139416+0000","last_clean":"2023-11-24T08:08:23.139416+0000","last_became_active":"2023-11-24T08:08:23.134082+0000","last_became_peered":"2023-11-24T08:08:23.134082+0000","last_unstale":"2023-11-24T08:08:23.139416+0000","last_undegraded":"2023-11-24T08:08:23.139416+0000","last_fullsized":"2023-11-24T08:08:23.139416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:29.860915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112898+0000","last_change":"2023-11-24T08:08:23.112898+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112898+0000","last_undegraded":"2023-11-24T08:08:23.112898+0000","last_fullsized":"2023-11-24T08:08:23.112898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135574+0000","last_change":"2023-11-24T08:08:23.127111+0000","last_active":"2023-11-24T08:08:23.135574+0000","last_peered":"2023-11-24T08:08:23.135574+0000","last_clean":"2023-11-24T08:08:23.135574+0000","last_became_active":"2023-11-24T08:08:23.126828+0000","last_became_peered":"2023-11-24T08:08:23.126828+0000","last_unstale":"2023-11-24T08:08:23.135574+0000","last_undegraded":"2023-11-24T08:08:23.135574+0000","last_fullsized":"2023-11-24T08:08:23.135574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:27.777382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103781+0000","last_change":"2023-11-24T08:08:23.103781+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103781+0000","last_undegraded":"2023-11-24T08:08:23.103781+0000","last_fullsized":"2023-11-24T08:08:23.103781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129854+0000","last_change":"2023-11-24T08:08:23.125635+0000","last_active":"2023-11-24T08:08:23.129854+0000","last_peered":"2023-11-24T08:08:23.129854+0000","last_clean":"2023-11-24T08:08:23.129854+0000","last_became_active":"2023-11-24T08:08:23.125335+0000","last_became_peered":"2023-11-24T08:08:23.125335+0000","last_unstale":"2023-11-24T08:08:23.129854+0000","last_undegraded":"2023-11-24T08:08:23.129854+0000","last_fullsized":"2023-11-24T08:08:23.129854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:20.751833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112141+0000","last_change":"2023-11-24T08:08:23.112141+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112141+0000","last_undegraded":"2023-11-24T08:08:23.112141+0000","last_fullsized":"2023-11-24T08:08:23.112141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129828+0000","last_change":"2023-11-24T08:08:23.125737+0000","last_active":"2023-11-24T08:08:23.129828+0000","last_peered":"2023-11-24T08:08:23.129828+0000","last_clean":"2023-11-24T08:08:23.129828+0000","last_became_active":"2023-11-24T08:08:23.125403+0000","last_became_peered":"2023-11-24T08:08:23.125403+0000","last_unstale":"2023-11-24T08:08:23.129828+0000","last_undegraded":"2023-11-24T08:08:23.129828+0000","last_fullsized":"2023-11-24T08:08:23.129828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:57.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117189+0000","last_change":"2023-11-24T08:08:23.117189+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117189+0000","last_undegraded":"2023-11-24T08:08:23.117189+0000","last_fullsized":"2023-11-24T08:08:23.117189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136817+0000","last_change":"2023-11-24T08:08:23.134364+0000","last_active":"2023-11-24T08:08:23.136817+0000","last_peered":"2023-11-24T08:08:23.136817+0000","last_clean":"2023-11-24T08:08:23.136817+0000","last_became_active":"2023-11-24T08:08:23.134008+0000","last_became_peered":"2023-11-24T08:08:23.134008+0000","last_unstale":"2023-11-24T08:08:23.136817+0000","last_undegraded":"2023-11-24T08:08:23.136817+0000","last_fullsized":"2023-11-24T08:08:23.136817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:57.287980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103133+0000","last_change":"2023-11-24T08:08:23.103133+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103133+0000","last_undegraded":"2023-11-24T08:08:23.103133+0000","last_fullsized":"2023-11-24T08:08:23.103133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134107+0000","last_change":"2023-11-24T08:08:23.130703+0000","last_active":"2023-11-24T08:08:23.134107+0000","last_peered":"2023-11-24T08:08:23.134107+0000","last_clean":"2023-11-24T08:08:23.134107+0000","last_became_active":"2023-11-24T08:08:23.130442+0000","last_became_peered":"2023-11-24T08:08:23.130442+0000","last_unstale":"2023-11-24T08:08:23.134107+0000","last_undegraded":"2023-11-24T08:08:23.134107+0000","last_fullsized":"2023-11-24T08:08:23.134107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:14.668163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.113310+0000","last_change":"2023-11-24T08:08:23.113310+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.113310+0000","last_undegraded":"2023-11-24T08:08:23.113310+0000","last_fullsized":"2023-11-24T08:08:23.113310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136704+0000","last_change":"2023-11-24T08:08:23.134436+0000","last_active":"2023-11-24T08:08:23.136704+0000","last_peered":"2023-11-24T08:08:23.136704+0000","last_clean":"2023-11-24T08:08:23.136704+0000","last_became_active":"2023-11-24T08:08:23.133914+0000","last_became_peered":"2023-11-24T08:08:23.133914+0000","last_unstale":"2023-11-24T08:08:23.136704+0000","last_undegraded":"2023-11-24T08:08:23.136704+0000","last_fullsized":"2023-11-24T08:08:23.136704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:45.936686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.120558+0000","last_change":"2023-11-24T08:08:23.120558+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.120558+0000","last_undegraded":"2023-11-24T08:08:23.120558+0000","last_fullsized":"2023-11-24T08:08:23.120558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129603+0000","last_change":"2023-11-24T08:08:23.124236+0000","last_active":"2023-11-24T08:08:23.129603+0000","last_peered":"2023-11-24T08:08:23.129603+0000","last_clean":"2023-11-24T08:08:23.129603+0000","last_became_active":"2023-11-24T08:08:23.123917+0000","last_became_peered":"2023-11-24T08:08:23.123917+0000","last_unstale":"2023-11-24T08:08:23.129603+0000","last_undegraded":"2023-11-24T08:08:23.129603+0000","last_fullsized":"2023-11-24T08:08:23.129603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:14.668163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115029+0000","last_change":"2023-11-24T08:08:23.115029+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115029+0000","last_undegraded":"2023-11-24T08:08:23.115029+0000","last_fullsized":"2023-11-24T08:08:23.115029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.131926+0000","last_change":"2023-11-24T08:08:23.128663+0000","last_active":"2023-11-24T08:08:23.131926+0000","last_peered":"2023-11-24T08:08:23.131926+0000","last_clean":"2023-11-24T08:08:23.131926+0000","last_became_active":"2023-11-24T08:08:23.128380+0000","last_became_peered":"2023-11-24T08:08:23.128380+0000","last_unstale":"2023-11-24T08:08:23.131926+0000","last_undegraded":"2023-11-24T08:08:23.131926+0000","last_fullsized":"2023-11-24T08:08:23.131926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:32.242969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116516+0000","last_change":"2023-11-24T08:08:23.116516+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116516+0000","last_undegraded":"2023-11-24T08:08:23.116516+0000","last_fullsized":"2023-11-24T08:08:23.116516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136078+0000","last_change":"2023-11-24T08:08:23.133278+0000","last_active":"2023-11-24T08:08:23.136078+0000","last_peered":"2023-11-24T08:08:23.136078+0000","last_clean":"2023-11-24T08:08:23.136078+0000","last_became_active":"2023-11-24T08:08:23.132849+0000","last_became_peered":"2023-11-24T08:08:23.132849+0000","last_unstale":"2023-11-24T08:08:23.136078+0000","last_undegraded":"2023-11-24T08:08:23.136078+0000","last_fullsized":"2023-11-24T08:08:23.136078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:31:30.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109344+0000","last_change":"2023-11-24T08:08:23.109344+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109344+0000","last_undegraded":"2023-11-24T08:08:23.109344+0000","last_fullsized":"2023-11-24T08:08:23.109344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.137736+0000","last_change":"2023-11-24T08:08:23.136137+0000","last_active":"2023-11-24T08:08:23.137736+0000","last_peered":"2023-11-24T08:08:23.137736+0000","last_clean":"2023-11-24T08:08:23.137736+0000","last_became_active":"2023-11-24T08:08:23.135553+0000","last_became_peered":"2023-11-24T08:08:23.135553+0000","last_unstale":"2023-11-24T08:08:23.137736+0000","last_undegraded":"2023-11-24T08:08:23.137736+0000","last_fullsized":"2023-11-24T08:08:23.137736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:55.555959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T08:08:22.094050+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:22.094050+0000","last_peered":"2023-11-24T08:08:22.094050+0000","last_clean":"2023-11-24T08:08:22.094050+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:22.094050+0000","last_undegraded":"2023-11-24T08:08:22.094050+0000","last_fullsized":"2023-11-24T08:08:22.094050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":1036492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.995,"15min":1.0009999999999999},"min":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.516,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.046,"15min":1.079},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.403,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.028}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.747,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6799999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.3069999999999999,"15min":1.3720000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.712,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.4379999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.268,"15min":1.3029999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.355}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.26,"15min":1.3129999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8859999999999999,"5min":2.524,"15min":2.524},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3959999999999999,"15min":1.498},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7889999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.472}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.391,"15min":1.5109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8520000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.268},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.323,"15min":1.365},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6779999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.95499999999999996}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.5189999999999999,"15min":1.599},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.23},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9750000000000001,"5min":2.944,"15min":2.944},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.631,"15min":1.7350000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.915,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.5389999999999999,"15min":1.643},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.962,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.3069999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.319},"min":{"1min":0.73099999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.6579999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.171},{"interface":"front","average":{"1min":1.173,"5min":1.262,"15min":1.335},"min":{"1min":0.81899999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.3819999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.63300000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.593,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2270000000000001,"15min":1.3029999999999999},"min":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7090000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.409}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.204,"15min":1.2869999999999999},"min":{"1min":0.86699999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.825,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3260000000000001,"15min":1.476},"min":{"1min":0.84999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5169999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.2170000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3680000000000001,"15min":1.4370000000000001},"min":{"1min":1.0980000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.8089999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.468},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.45,"15min":1.492},"min":{"1min":1.024,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.821,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.4359999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.526,"15min":1.724},"min":{"1min":0.995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.627,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.1279999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9299999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.4890000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.5760000000000001,"15min":1.778},"min":{"1min":0.84399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.784,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.714,"15min":1.9179999999999999},"min":{"1min":1.3640000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8660000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.599,"15min":1.6759999999999999},"min":{"1min":1.175,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0190000000000001,"5min":2.98,"15min":2.98},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.5800000000000001,"15min":1.6379999999999999},"min":{"1min":1.0549999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9810000000000001,"5min":2.29,"15min":2.29},"last":0.97399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0129999999999999,"15min":1.0489999999999999},"min":{"1min":0.68899999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6419999999999999,"5min":1.764,"15min":1.764},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.016},"min":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5369999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.80000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9530000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.149,"5min":1.1000000000000001,"15min":1.0580000000000001},"min":{"1min":0.78600000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.589,"5min":1.635,"15min":1.635},"last":1.1319999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.214,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.252,"15min":1.224},"min":{"1min":0.81100000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.2690000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.3959999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.381,"15min":1.4259999999999999},"min":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.7400000000000002,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.2250000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.302,"15min":1.304},"min":{"1min":0.67700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.885,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.448,"15min":1.4239999999999999},"min":{"1min":0.91700000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.3620000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.4790000000000001,"15min":1.48},"min":{"1min":0.621,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.042},{"interface":"front","average":{"1min":1.502,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":0.76500000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.292}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.55,"15min":1.538},"min":{"1min":0.78900000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.9370000000000003,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.5640000000000001,"15min":1.538},"min":{"1min":0.74299999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.323,"5min":2.859,"15min":2.859},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.034,"15min":1.04},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.534,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.393},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.68500000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.617,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0700000000000001,"15min":1.081},"min":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5660000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.319},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1970000000000001,"15min":1.2330000000000001},"min":{"1min":0.72499999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5760000000000001,"5min":1.903,"15min":1.903},"last":0.91600000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1910000000000001,"15min":1.2010000000000001},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7390000000000001,"5min":1.823,"15min":1.823},"last":1.24},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.248},"min":{"1min":0.72999999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.534,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.0489999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.278,"15min":1.296},"min":{"1min":0.76300000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7829999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.222},{"interface":"front","average":{"1min":1.196,"5min":1.286,"15min":1.306},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.478,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.306,"15min":1.304},"min":{"1min":1.095,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8120000000000001,"5min":1.944,"15min":1.944},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.365,"15min":1.367},"min":{"1min":1.0960000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0049999999999999,"5min":2.012,"15min":2.012},"last":1.018}]},{"osd":6,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.385,"15min":1.371},"min":{"1min":1.0229999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.736,"5min":2.105,"15min":2.105},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.615,"5min":1.508,"15min":1.514},"min":{"1min":1.306,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.143}]},{"osd":7,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.452,"15min":1.4370000000000001},"min":{"1min":1.1599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0379999999999998,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.524},{"interface":"front","average":{"1min":1.494,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":1.2430000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.827,"5min":2.722,"15min":2.722},"last":1.101}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.3560000000000001,"5min":1.585,"15min":1.585},"last":1.284},{"interface":"front","average":{"1min":0.87,"5min":0.90500000000000003,"15min":0.94199999999999995},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.488,"5min":1.51,"15min":1.51},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.01,"15min":1.129},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4299999999999999,"5min":3.246,"15min":3.246},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.995,"5min":0.97999999999999998,"15min":0.96899999999999997},"min":{"1min":0.51400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.6299999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1140000000000001,"15min":1.1739999999999999},"min":{"1min":0.63800000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.988,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.876},{"interface":"front","average":{"1min":1.155,"5min":1.1120000000000001,"15min":1.1539999999999999},"min":{"1min":0.68999999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8009999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.4299999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2390000000000001,"15min":1.262},"min":{"1min":0.94999999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6950000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.325,"15min":1.3799999999999999},"min":{"1min":0.83799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8919999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.222}]},{"osd":5,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3939999999999999,"15min":1.45},"min":{"1min":0.96799999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.839,"5min":2.395,"15min":2.395},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.89800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.7250000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4219999999999999,"15min":1.4319999999999999},"min":{"1min":0.95999999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0230000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.393,"15min":1.4159999999999999},"min":{"1min":1.0269999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.417,"15min":1.4419999999999999},"min":{"1min":0.98199999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.024},{"interface":"front","average":{"1min":1.486,"5min":1.5029999999999999,"15min":1.573},"min":{"1min":1.1120000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0510000000000002,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.343}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83699999999999997},"min":{"1min":0.44500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.369,"5min":1.524,"15min":1.524},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95799999999999996,"15min":1.01},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.694,"5min":2.133,"15min":2.133},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.087,"15min":1.2070000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4359999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0569999999999999},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.875,"5min":1.998,"15min":1.998},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.258,"15min":1.3160000000000001},"min":{"1min":0.63700000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.722,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.1220000000000001,"15min":1.1859999999999999},"min":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.9350000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.204}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.248,"15min":1.321},"min":{"1min":0.82799999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.843,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.774},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.411,"15min":1.4930000000000001},"min":{"1min":0.75900000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8280000000000001,"5min":3.161,"15min":3.161},"last":1.2569999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.325,"15min":1.387},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.1629999999999998,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3200000000000001,"15min":1.3999999999999999},"min":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6220000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.8919999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4750000000000001,"15min":1.579},"min":{"1min":0.73299999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8420000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.665},{"interface":"front","average":{"1min":1.405,"5min":1.4470000000000001,"15min":1.552},"min":{"1min":0.93100000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0510000000000002,"5min":2.702,"15min":2.702},"last":1.4790000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.504},"min":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.996,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.9970000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.53,"15min":1.6319999999999999},"min":{"1min":0.81999999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.907,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.605}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.88800000000000001},"min":{"1min":0.53900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.95599999999999996,"15min":1.042},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6459999999999999,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0680000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94599999999999995,"15min":1.018},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.724,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1399999999999999,"15min":1.216},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8959999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.3500000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1990000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.127,"15min":1.2150000000000001},"min":{"1min":0.60399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.0899999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.383}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.198,"15min":1.24},"min":{"1min":0.81799999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.645,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.379,"15min":1.4410000000000001},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7909999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.393,"15min":1.446},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.8740000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.415,"15min":1.466},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.855,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.3260000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.4059999999999999,"15min":1.45},"min":{"1min":0.92900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9630000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.613},{"interface":"front","average":{"1min":1.256,"5min":1.3939999999999999,"15min":1.466},"min":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.669,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.216}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.48,"15min":1.548},"min":{"1min":1.0940000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6739999999999999,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.284},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.536,"15min":1.5840000000000001},"min":{"1min":1.0209999999999999,"5min":0.872,"15min":0.872},"max":{"1min":2.335,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1634304,"data_stored":1035473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.876,"15min":0.88},"min":{"1min":0.504,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2110000000000001,"5min":3.423,"15min":3.423},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.997,"15min":0.95399999999999996},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.661,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.96099999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.081,"15min":1.0649999999999999},"min":{"1min":0.60899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5589999999999999,"5min":3.899,"15min":3.899},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.99399999999999999,"15min":0.95799999999999996},"min":{"1min":0.46899999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.3140000000000001,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.282,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.2250000000000001},"min":{"1min":0.65300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.621,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.1619999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3,"15min":1.2689999999999999},"min":{"1min":0.81999999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6459999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.387},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.482,"15min":1.476},"min":{"1min":0.99199999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8959999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4510000000000001,"15min":1.393},"min":{"1min":0.96599999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.4550000000000001,"15min":1.4379999999999999},"min":{"1min":0.80000000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.0110000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.0430000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.399,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.786,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.226},{"interface":"front","average":{"1min":1.49,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":0.84899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9359999999999999,"5min":3.879,"15min":3.879},"last":2.1070000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.54,"15min":1.54},"min":{"1min":0.83499999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":3.823,"15min":3.823},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.5049999999999999,"15min":1.492},"min":{"1min":0.88300000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8740000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.48}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:29.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:29.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.447+0000 7f8abe0ff640 1 Processor -- start 2023-11-24T08:08:29.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.447+0000 7f8abe0ff640 1 -- start start 2023-11-24T08:08:29.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.449+0000 7f8abe0ff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8ab8143570 0x7f8ab80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.449+0000 7f8abe0ff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab8071700 con 0x7f8ab815e6b0 2023-11-24T08:08:29.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.449+0000 7f8abe0ff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab815b410 con 0x7f8ab8143950 2023-11-24T08:08:29.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.449+0000 7f8abe0ff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab815b130 con 0x7f8ab8143570 2023-11-24T08:08:29.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.449+0000 7f8abd8fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8ab815e6b0 0x7f8ab80b1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46094/0 (socket says 172.21.15.64:46094) 2023-11-24T08:08:29.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.449+0000 7f8abd8fe640 1 -- 172.21.15.64:0/3254849971 learned_addr learned my addr 172.21.15.64:0/3254849971 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8abd0fd640 1 --2- 172.21.15.64:0/3254849971 >> v2:172.21.15.91:3300/0 conn(0x7f8ab8143570 0x7f8ab80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018032983 0 0) 0x7f8ab0001490 con 0x7f8ab815e6b0 2023-11-24T08:08:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa4003520 con 0x7f8ab815e6b0 2023-11-24T08:08:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2011373077 0 0) 0x7f8aa4003520 con 0x7f8ab815e6b0 2023-11-24T08:08:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 >> v2:172.21.15.91:3300/0 conn(0x7f8ab8143570 msgr2=0x7f8ab80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 --2- 172.21.15.64:0/3254849971 >> v2:172.21.15.91:3300/0 conn(0x7f8ab8143570 0x7f8ab80cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 >> v1:172.21.15.91:6789/0 conn(0x7f8ab8143950 legacy=0x7f8ab815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab815dbd0 con 0x7f8ab815e6b0 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f8ab0001b80 con 0x7f8ab815e6b0 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 >> v1:172.21.15.64:6789/0 conn(0x7f8ab815e6b0 legacy=0x7f8ab80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.450+0000 7f8aaffff640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8aa4003a30 0x7f8aa4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8aaffff640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8aa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8aaffff640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8aa40052a0 0x7f8aa400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab815b130 con 0x7f8aa40047c0 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa400c170 con 0x7f8aa40052a0 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa400c330 con 0x7f8aa4003a30 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8abc8fc640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8aa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8abc8fc640 1 -- 172.21.15.64:0/3254849971 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8aa4003a30 msgr2=0x7f8aa4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:29.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8abc8fc640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8aa4003a30 0x7f8aa4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8abc8fc640 1 -- 172.21.15.64:0/3254849971 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8aa40052a0 msgr2=0x7f8aa400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:29.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8abc8fc640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8aa40052a0 0x7f8aa400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8abc8fc640 1 -- 172.21.15.64:0/3254849971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa4003520 con 0x7f8aa40047c0 2023-11-24T08:08:29.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.451+0000 7f8abd8fe640 1 -- 172.21.15.64:0/3254849971 reap_dead start 2023-11-24T08:08:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.452+0000 7f8abc8fc640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8aa4004c40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8001720 tx=0x7f8aa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48454def5aeb24d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80052d0 con 0x7f8aa40047c0 2023-11-24T08:08:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa802e050 con 0x7f8aa40047c0 2023-11-24T08:08:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8aaffff640 1 -- 172.21.15.64:0/3254849971 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8003830 con 0x7f8aa40047c0 2023-11-24T08:08:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8abe0ff640 1 -- 172.21.15.64:0/3254849971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 msgr2=0x7f8aa4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8abe0ff640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8aa4004c40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8001720 tx=0x7f8aa40057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8abe0ff640 1 -- 172.21.15.64:0/3254849971 shutdown_connections 2023-11-24T08:08:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8abe0ff640 1 --2- 172.21.15.64:0/3254849971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8aa4004c40 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8abe0ff640 1 -- 172.21.15.64:0/3254849971 >> 172.21.15.64:0/3254849971 conn(0x7f8ab806a860 msgr2=0x7f8ab806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8abe0ff640 1 -- 172.21.15.64:0/3254849971 shutdown_connections 2023-11-24T08:08:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.453+0000 7f8abe0ff640 1 -- 172.21.15.64:0/3254849971 wait complete. 2023-11-24T08:08:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.455+0000 7f8abe0ff640 1 Processor -- start 2023-11-24T08:08:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.455+0000 7f8abe0ff640 1 -- start start 2023-11-24T08:08:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.456+0000 7f8abe0ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8ab81403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.456+0000 7f8abe0ff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ab8159620 0x7f8ab8158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.456+0000 7f8abe0ff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ab80c7ec0 0x7f8ab80cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.456+0000 7f8abe0ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80cb630 con 0x7f8aa40047c0 2023-11-24T08:08:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.456+0000 7f8abe0ff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab81593f0 con 0x7f8ab80c7ec0 2023-11-24T08:08:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.456+0000 7f8abe0ff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80ca9b0 con 0x7f8ab8159620 2023-11-24T08:08:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8ab81403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8ab81403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60456/0 (socket says 172.21.15.64:60456) 2023-11-24T08:08:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 -- 172.21.15.64:0/1122573269 learned_addr learned my addr 172.21.15.64:0/1122573269 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 -- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ab8159620 msgr2=0x7f8ab8158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ab8159620 0x7f8ab8158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 -- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ab80c7ec0 msgr2=0x7f8ab80cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ab80c7ec0 0x7f8ab80cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 -- 172.21.15.64:0/1122573269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab80cac90 con 0x7f8aa40047c0 2023-11-24T08:08:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.457+0000 7f8abd0fd640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8ab81403c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400e770 tx=0x7f8ab400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.458+0000 7f8aadffb640 1 -- 172.21.15.64:0/1122573269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400c730 con 0x7f8aa40047c0 2023-11-24T08:08:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.458+0000 7f8aadffb640 1 -- 172.21.15.64:0/1122573269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab400c890 con 0x7f8aa40047c0 2023-11-24T08:08:29.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.458+0000 7f8aadffb640 1 -- 172.21.15.64:0/1122573269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400cc40 con 0x7f8aa40047c0 2023-11-24T08:08:29.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.458+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab80caf70 con 0x7f8aa40047c0 2023-11-24T08:08:29.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.458+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab80c8340 con 0x7f8aa40047c0 2023-11-24T08:08:29.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.459+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab804e840 con 0x7f8aa40047c0 2023-11-24T08:08:29.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.460+0000 7f8aadffb640 1 -- 172.21.15.64:0/1122573269 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8ab4013470 con 0x7f8aa40047c0 2023-11-24T08:08:29.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.461+0000 7f8aadffb640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a9c06b410 0x7f8a9c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.461+0000 7f8aadffb640 1 -- 172.21.15.64:0/1122573269 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ab408a2f0 con 0x7f8aa40047c0 2023-11-24T08:08:29.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.462+0000 7f8abc8fc640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a9c06b410 0x7f8a9c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.463+0000 7f8abc8fc640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a9c06b410 0x7f8a9c06d8d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8033000 tx=0x7f8aa802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.466+0000 7f8aadffb640 1 -- 172.21.15.64:0/1122573269 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ab4054af0 con 0x7f8aa40047c0 2023-11-24T08:08:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.663+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ab80c8560 con 0x7f8a9c06b410 2023-11-24T08:08:29.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.686+0000 7f8aadffb640 1 -- 172.21.15.64:0/1122573269 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281726 (secure 0 0 0) 0x7f8ab80c8560 con 0x7f8a9c06b410 2023-11-24T08:08:29.692 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.692+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a9c06b410 msgr2=0x7f8a9c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.692+0000 7f8abe0ff640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a9c06b410 0x7f8a9c06d8d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8033000 tx=0x7f8aa802f000 comp rx=0 tx=0).stop 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.692+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 msgr2=0x7f8ab81403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.692+0000 7f8abe0ff640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8ab81403c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400e770 tx=0x7f8ab400ec30 comp rx=0 tx=0).stop 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.693+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 shutdown_connections 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.693+0000 7f8abe0ff640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8a9c06b410 0x7f8a9c06d8d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.693+0000 7f8abe0ff640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8ab80c7ec0 0x7f8ab80cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.693+0000 7f8abe0ff640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8ab8159620 0x7f8ab8158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.693+0000 7f8abe0ff640 1 --2- 172.21.15.64:0/1122573269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aa40047c0 0x7f8ab81403c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.693+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 >> 172.21.15.64:0/1122573269 conn(0x7f8ab806a860 msgr2=0x7f8ab80ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.693+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 shutdown_connections 2023-11-24T08:08:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.693+0000 7f8abe0ff640 1 -- 172.21.15.64:0/1122573269 wait complete. 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2023-11-24T08:08:28.113534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258780,"kb_used_data":9316,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715940,"statfs":{"total":773094113280,"available":772829122560,"internally_reserved":0,"allocated":9539584,"data_stored":4748709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002610"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.125419+0000","last_change":"2023-11-24T08:08:23.125419+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.125419+0000","last_undegraded":"2023-11-24T08:08:23.125419+0000","last_fullsized":"2023-11-24T08:08:23.125419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111910+0000","last_change":"2023-11-24T08:08:23.111910+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111910+0000","last_undegraded":"2023-11-24T08:08:23.111910+0000","last_fullsized":"2023-11-24T08:08:23.111910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111691+0000","last_change":"2023-11-24T08:08:23.111691+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111691+0000","last_undegraded":"2023-11-24T08:08:23.111691+0000","last_fullsized":"2023-11-24T08:08:23.111691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119453+0000","last_change":"2023-11-24T08:08:23.119453+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119453+0000","last_undegraded":"2023-11-24T08:08:23.119453+0000","last_fullsized":"2023-11-24T08:08:23.119453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.104404+0000","last_change":"2023-11-24T08:08:23.104404+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.104404+0000","last_undegraded":"2023-11-24T08:08:23.104404+0000","last_fullsized":"2023-11-24T08:08:23.104404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114326+0000","last_change":"2023-11-24T08:08:23.114326+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114326+0000","last_undegraded":"2023-11-24T08:08:23.114326+0000","last_fullsized":"2023-11-24T08:08:23.114326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114692+0000","last_change":"2023-11-24T08:08:23.114692+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114692+0000","last_undegraded":"2023-11-24T08:08:23.114692+0000","last_fullsized":"2023-11-24T08:08:23.114692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.114757+0000","last_change":"2023-11-24T08:08:23.114757+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.114757+0000","last_undegraded":"2023-11-24T08:08:23.114757+0000","last_fullsized":"2023-11-24T08:08:23.114757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116155+0000","last_change":"2023-11-24T08:08:23.116155+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116155+0000","last_undegraded":"2023-11-24T08:08:23.116155+0000","last_fullsized":"2023-11-24T08:08:23.116155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118098+0000","last_change":"2023-11-24T08:08:23.118098+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118098+0000","last_undegraded":"2023-11-24T08:08:23.118098+0000","last_fullsized":"2023-11-24T08:08:23.118098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112412+0000","last_change":"2023-11-24T08:08:23.112412+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112412+0000","last_undegraded":"2023-11-24T08:08:23.112412+0000","last_fullsized":"2023-11-24T08:08:23.112412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.125090+0000","last_change":"2023-11-24T08:08:23.125090+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.125090+0000","last_undegraded":"2023-11-24T08:08:23.125090+0000","last_fullsized":"2023-11-24T08:08:23.125090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116279+0000","last_change":"2023-11-24T08:08:23.116279+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116279+0000","last_undegraded":"2023-11-24T08:08:23.116279+0000","last_fullsized":"2023-11-24T08:08:23.116279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115991+0000","last_change":"2023-11-24T08:08:23.115991+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115991+0000","last_undegraded":"2023-11-24T08:08:23.115991+0000","last_fullsized":"2023-11-24T08:08:23.115991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103193+0000","last_change":"2023-11-24T08:08:23.103193+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103193+0000","last_undegraded":"2023-11-24T08:08:23.103193+0000","last_fullsized":"2023-11-24T08:08:23.103193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.105524+0000","last_change":"2023-11-24T08:08:23.105524+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.105524+0000","last_undegraded":"2023-11-24T08:08:23.105524+0000","last_fullsized":"2023-11-24T08:08:23.105524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103155+0000","last_change":"2023-11-24T08:08:23.103155+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103155+0000","last_undegraded":"2023-11-24T08:08:23.103155+0000","last_fullsized":"2023-11-24T08:08:23.103155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.120117+0000","last_change":"2023-11-24T08:08:23.120117+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.120117+0000","last_undegraded":"2023-11-24T08:08:23.120117+0000","last_fullsized":"2023-11-24T08:08:23.120117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118749+0000","last_change":"2023-11-24T08:08:23.118749+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118749+0000","last_undegraded":"2023-11-24T08:08:23.118749+0000","last_fullsized":"2023-11-24T08:08:23.118749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.118836+0000","last_change":"2023-11-24T08:08:23.118836+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.118836+0000","last_undegraded":"2023-11-24T08:08:23.118836+0000","last_fullsized":"2023-11-24T08:08:23.118836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115676+0000","last_change":"2023-11-24T08:08:23.115676+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115676+0000","last_undegraded":"2023-11-24T08:08:23.115676+0000","last_fullsized":"2023-11-24T08:08:23.115676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111546+0000","last_change":"2023-11-24T08:08:23.111546+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111546+0000","last_undegraded":"2023-11-24T08:08:23.111546+0000","last_fullsized":"2023-11-24T08:08:23.111546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117271+0000","last_change":"2023-11-24T08:08:23.117271+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117271+0000","last_undegraded":"2023-11-24T08:08:23.117271+0000","last_fullsized":"2023-11-24T08:08:23.117271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117195+0000","last_change":"2023-11-24T08:08:23.117195+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117195+0000","last_undegraded":"2023-11-24T08:08:23.117195+0000","last_fullsized":"2023-11-24T08:08:23.117195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110410+0000","last_change":"2023-11-24T08:08:23.110410+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110410+0000","last_undegraded":"2023-11-24T08:08:23.110410+0000","last_fullsized":"2023-11-24T08:08:23.110410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.124187+0000","last_change":"2023-11-24T08:08:23.124187+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.124187+0000","last_undegraded":"2023-11-24T08:08:23.124187+0000","last_fullsized":"2023-11-24T08:08:23.124187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110905+0000","last_change":"2023-11-24T08:08:23.110905+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110905+0000","last_undegraded":"2023-11-24T08:08:23.110905+0000","last_fullsized":"2023-11-24T08:08:23.110905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.123814+0000","last_change":"2023-11-24T08:08:23.123814+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.123814+0000","last_undegraded":"2023-11-24T08:08:23.123814+0000","last_fullsized":"2023-11-24T08:08:23.123814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110123+0000","last_change":"2023-11-24T08:08:23.110123+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110123+0000","last_undegraded":"2023-11-24T08:08:23.110123+0000","last_fullsized":"2023-11-24T08:08:23.110123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110824+0000","last_change":"2023-11-24T08:08:23.110824+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110824+0000","last_undegraded":"2023-11-24T08:08:23.110824+0000","last_fullsized":"2023-11-24T08:08:23.110824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.102988+0000","last_change":"2023-11-24T08:08:23.102988+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.102988+0000","last_undegraded":"2023-11-24T08:08:23.102988+0000","last_fullsized":"2023-11-24T08:08:23.102988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119610+0000","last_change":"2023-11-24T08:08:23.119610+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119610+0000","last_undegraded":"2023-11-24T08:08:23.119610+0000","last_fullsized":"2023-11-24T08:08:23.119610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.105615+0000","last_change":"2023-11-24T08:08:23.105615+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.105615+0000","last_undegraded":"2023-11-24T08:08:23.105615+0000","last_fullsized":"2023-11-24T08:08:23.105615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136824+0000","last_change":"2023-11-24T08:08:23.134349+0000","last_active":"2023-11-24T08:08:23.136824+0000","last_peered":"2023-11-24T08:08:23.136824+0000","last_clean":"2023-11-24T08:08:23.136824+0000","last_became_active":"2023-11-24T08:08:23.134026+0000","last_became_peered":"2023-11-24T08:08:23.134026+0000","last_unstale":"2023-11-24T08:08:23.136824+0000","last_undegraded":"2023-11-24T08:08:23.136824+0000","last_fullsized":"2023-11-24T08:08:23.136824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:30.483247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.124855+0000","last_change":"2023-11-24T08:08:23.124855+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.124855+0000","last_undegraded":"2023-11-24T08:08:23.124855+0000","last_fullsized":"2023-11-24T08:08:23.124855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129876+0000","last_change":"2023-11-24T08:08:23.125215+0000","last_active":"2023-11-24T08:08:23.129876+0000","last_peered":"2023-11-24T08:08:23.129876+0000","last_clean":"2023-11-24T08:08:23.129876+0000","last_became_active":"2023-11-24T08:08:23.124899+0000","last_became_peered":"2023-11-24T08:08:23.124899+0000","last_unstale":"2023-11-24T08:08:23.129876+0000","last_undegraded":"2023-11-24T08:08:23.129876+0000","last_fullsized":"2023-11-24T08:08:23.129876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:32.242969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.121284+0000","last_change":"2023-11-24T08:08:23.121284+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.121284+0000","last_undegraded":"2023-11-24T08:08:23.121284+0000","last_fullsized":"2023-11-24T08:08:23.121284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135774+0000","last_change":"2023-11-24T08:08:23.132484+0000","last_active":"2023-11-24T08:08:23.135774+0000","last_peered":"2023-11-24T08:08:23.135774+0000","last_clean":"2023-11-24T08:08:23.135774+0000","last_became_active":"2023-11-24T08:08:23.132220+0000","last_became_peered":"2023-11-24T08:08:23.132220+0000","last_unstale":"2023-11-24T08:08:23.135774+0000","last_undegraded":"2023-11-24T08:08:23.135774+0000","last_fullsized":"2023-11-24T08:08:23.135774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:57.911680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110525+0000","last_change":"2023-11-24T08:08:23.110525+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110525+0000","last_undegraded":"2023-11-24T08:08:23.110525+0000","last_fullsized":"2023-11-24T08:08:23.110525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135536+0000","last_change":"2023-11-24T08:08:23.132020+0000","last_active":"2023-11-24T08:08:23.135536+0000","last_peered":"2023-11-24T08:08:23.135536+0000","last_clean":"2023-11-24T08:08:23.135536+0000","last_became_active":"2023-11-24T08:08:23.131713+0000","last_became_peered":"2023-11-24T08:08:23.131713+0000","last_unstale":"2023-11-24T08:08:23.135536+0000","last_undegraded":"2023-11-24T08:08:23.135536+0000","last_fullsized":"2023-11-24T08:08:23.135536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.740441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.110163+0000","last_change":"2023-11-24T08:08:23.110163+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.110163+0000","last_undegraded":"2023-11-24T08:08:23.110163+0000","last_fullsized":"2023-11-24T08:08:23.110163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130037+0000","last_change":"2023-11-24T08:08:23.127933+0000","last_active":"2023-11-24T08:08:23.130037+0000","last_peered":"2023-11-24T08:08:23.130037+0000","last_clean":"2023-11-24T08:08:23.130037+0000","last_became_active":"2023-11-24T08:08:23.127651+0000","last_became_peered":"2023-11-24T08:08:23.127651+0000","last_unstale":"2023-11-24T08:08:23.130037+0000","last_undegraded":"2023-11-24T08:08:23.130037+0000","last_fullsized":"2023-11-24T08:08:23.130037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:35.016477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103184+0000","last_change":"2023-11-24T08:08:23.103184+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103184+0000","last_undegraded":"2023-11-24T08:08:23.103184+0000","last_fullsized":"2023-11-24T08:08:23.103184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.126956+0000","last_change":"2023-11-24T08:08:23.120911+0000","last_active":"2023-11-24T08:08:23.126956+0000","last_peered":"2023-11-24T08:08:23.126956+0000","last_clean":"2023-11-24T08:08:23.126956+0000","last_became_active":"2023-11-24T08:08:23.117203+0000","last_became_peered":"2023-11-24T08:08:23.117203+0000","last_unstale":"2023-11-24T08:08:23.126956+0000","last_undegraded":"2023-11-24T08:08:23.126956+0000","last_fullsized":"2023-11-24T08:08:23.126956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:36.336333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.119024+0000","last_change":"2023-11-24T08:08:23.119024+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.119024+0000","last_undegraded":"2023-11-24T08:08:23.119024+0000","last_fullsized":"2023-11-24T08:08:23.119024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129621+0000","last_change":"2023-11-24T08:08:23.124305+0000","last_active":"2023-11-24T08:08:23.129621+0000","last_peered":"2023-11-24T08:08:23.129621+0000","last_clean":"2023-11-24T08:08:23.129621+0000","last_became_active":"2023-11-24T08:08:23.123993+0000","last_became_peered":"2023-11-24T08:08:23.123993+0000","last_unstale":"2023-11-24T08:08:23.129621+0000","last_undegraded":"2023-11-24T08:08:23.129621+0000","last_fullsized":"2023-11-24T08:08:23.129621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:06.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.101930+0000","last_change":"2023-11-24T08:08:23.101930+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.101930+0000","last_undegraded":"2023-11-24T08:08:23.101930+0000","last_fullsized":"2023-11-24T08:08:23.101930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.127166+0000","last_change":"2023-11-24T08:08:23.118333+0000","last_active":"2023-11-24T08:08:23.127166+0000","last_peered":"2023-11-24T08:08:23.127166+0000","last_clean":"2023-11-24T08:08:23.127166+0000","last_became_active":"2023-11-24T08:08:23.116900+0000","last_became_peered":"2023-11-24T08:08:23.116900+0000","last_unstale":"2023-11-24T08:08:23.127166+0000","last_undegraded":"2023-11-24T08:08:23.127166+0000","last_fullsized":"2023-11-24T08:08:23.127166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:29.860915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.102421+0000","last_change":"2023-11-24T08:08:23.102421+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.102421+0000","last_undegraded":"2023-11-24T08:08:23.102421+0000","last_fullsized":"2023-11-24T08:08:23.102421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134469+0000","last_change":"2023-11-24T08:08:23.130844+0000","last_active":"2023-11-24T08:08:23.134469+0000","last_peered":"2023-11-24T08:08:23.134469+0000","last_clean":"2023-11-24T08:08:23.134469+0000","last_became_active":"2023-11-24T08:08:23.130493+0000","last_became_peered":"2023-11-24T08:08:23.130493+0000","last_unstale":"2023-11-24T08:08:23.134469+0000","last_undegraded":"2023-11-24T08:08:23.134469+0000","last_fullsized":"2023-11-24T08:08:23.134469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:35.016477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112793+0000","last_change":"2023-11-24T08:08:23.112793+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112793+0000","last_undegraded":"2023-11-24T08:08:23.112793+0000","last_fullsized":"2023-11-24T08:08:23.112793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134012+0000","last_change":"2023-11-24T08:08:23.129847+0000","last_active":"2023-11-24T08:08:23.134012+0000","last_peered":"2023-11-24T08:08:23.134012+0000","last_clean":"2023-11-24T08:08:23.134012+0000","last_became_active":"2023-11-24T08:08:23.128603+0000","last_became_peered":"2023-11-24T08:08:23.128603+0000","last_unstale":"2023-11-24T08:08:23.134012+0000","last_undegraded":"2023-11-24T08:08:23.134012+0000","last_fullsized":"2023-11-24T08:08:23.134012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:36.336333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.121568+0000","last_change":"2023-11-24T08:08:23.121568+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.121568+0000","last_undegraded":"2023-11-24T08:08:23.121568+0000","last_fullsized":"2023-11-24T08:08:23.121568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136085+0000","last_change":"2023-11-24T08:08:23.133517+0000","last_active":"2023-11-24T08:08:23.136085+0000","last_peered":"2023-11-24T08:08:23.136085+0000","last_clean":"2023-11-24T08:08:23.136085+0000","last_became_active":"2023-11-24T08:08:23.132959+0000","last_became_peered":"2023-11-24T08:08:23.132959+0000","last_unstale":"2023-11-24T08:08:23.136085+0000","last_undegraded":"2023-11-24T08:08:23.136085+0000","last_fullsized":"2023-11-24T08:08:23.136085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:12:25.699627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.108974+0000","last_change":"2023-11-24T08:08:23.108974+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.108974+0000","last_undegraded":"2023-11-24T08:08:23.108974+0000","last_fullsized":"2023-11-24T08:08:23.108974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130716+0000","last_change":"2023-11-24T08:08:23.126238+0000","last_active":"2023-11-24T08:08:23.130716+0000","last_peered":"2023-11-24T08:08:23.130716+0000","last_clean":"2023-11-24T08:08:23.130716+0000","last_became_active":"2023-11-24T08:08:23.125933+0000","last_became_peered":"2023-11-24T08:08:23.125933+0000","last_unstale":"2023-11-24T08:08:23.130716+0000","last_undegraded":"2023-11-24T08:08:23.130716+0000","last_fullsized":"2023-11-24T08:08:23.130716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:14.689949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.113332+0000","last_change":"2023-11-24T08:08:23.113332+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.113332+0000","last_undegraded":"2023-11-24T08:08:23.113332+0000","last_fullsized":"2023-11-24T08:08:23.113332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.138954+0000","last_change":"2023-11-24T08:08:23.132368+0000","last_active":"2023-11-24T08:08:23.138954+0000","last_peered":"2023-11-24T08:08:23.138954+0000","last_clean":"2023-11-24T08:08:23.138954+0000","last_became_active":"2023-11-24T08:08:23.132111+0000","last_became_peered":"2023-11-24T08:08:23.132111+0000","last_unstale":"2023-11-24T08:08:23.138954+0000","last_undegraded":"2023-11-24T08:08:23.138954+0000","last_fullsized":"2023-11-24T08:08:23.138954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:20.663330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111249+0000","last_change":"2023-11-24T08:08:23.111249+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111249+0000","last_undegraded":"2023-11-24T08:08:23.111249+0000","last_fullsized":"2023-11-24T08:08:23.111249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.121538+0000","last_change":"2023-11-24T08:08:23.116189+0000","last_active":"2023-11-24T08:08:23.121538+0000","last_peered":"2023-11-24T08:08:23.121538+0000","last_clean":"2023-11-24T08:08:23.121538+0000","last_became_active":"2023-11-24T08:08:23.115181+0000","last_became_peered":"2023-11-24T08:08:23.115181+0000","last_unstale":"2023-11-24T08:08:23.121538+0000","last_undegraded":"2023-11-24T08:08:23.121538+0000","last_fullsized":"2023-11-24T08:08:23.121538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:51.802097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109869+0000","last_change":"2023-11-24T08:08:23.109869+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109869+0000","last_undegraded":"2023-11-24T08:08:23.109869+0000","last_fullsized":"2023-11-24T08:08:23.109869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130705+0000","last_change":"2023-11-24T08:08:23.127485+0000","last_active":"2023-11-24T08:08:23.130705+0000","last_peered":"2023-11-24T08:08:23.130705+0000","last_clean":"2023-11-24T08:08:23.130705+0000","last_became_active":"2023-11-24T08:08:23.127216+0000","last_became_peered":"2023-11-24T08:08:23.127216+0000","last_unstale":"2023-11-24T08:08:23.130705+0000","last_undegraded":"2023-11-24T08:08:23.130705+0000","last_fullsized":"2023-11-24T08:08:23.130705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:30.483247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109666+0000","last_change":"2023-11-24T08:08:23.109666+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109666+0000","last_undegraded":"2023-11-24T08:08:23.109666+0000","last_fullsized":"2023-11-24T08:08:23.109666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.137030+0000","last_change":"2023-11-24T08:08:23.135139+0000","last_active":"2023-11-24T08:08:23.137030+0000","last_peered":"2023-11-24T08:08:23.137030+0000","last_clean":"2023-11-24T08:08:23.137030+0000","last_became_active":"2023-11-24T08:08:23.134748+0000","last_became_peered":"2023-11-24T08:08:23.134748+0000","last_unstale":"2023-11-24T08:08:23.137030+0000","last_undegraded":"2023-11-24T08:08:23.137030+0000","last_fullsized":"2023-11-24T08:08:23.137030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:45.681033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.111275+0000","last_change":"2023-11-24T08:08:23.111275+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.111275+0000","last_undegraded":"2023-11-24T08:08:23.111275+0000","last_fullsized":"2023-11-24T08:08:23.111275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135948+0000","last_change":"2023-11-24T08:08:23.132828+0000","last_active":"2023-11-24T08:08:23.135948+0000","last_peered":"2023-11-24T08:08:23.135948+0000","last_clean":"2023-11-24T08:08:23.135948+0000","last_became_active":"2023-11-24T08:08:23.132502+0000","last_became_peered":"2023-11-24T08:08:23.132502+0000","last_unstale":"2023-11-24T08:08:23.135948+0000","last_undegraded":"2023-11-24T08:08:23.135948+0000","last_fullsized":"2023-11-24T08:08:23.135948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:31:30.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.123529+0000","last_change":"2023-11-24T08:08:23.123529+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.123529+0000","last_undegraded":"2023-11-24T08:08:23.123529+0000","last_fullsized":"2023-11-24T08:08:23.123529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.130927+0000","last_change":"2023-11-24T08:08:23.128247+0000","last_active":"2023-11-24T08:08:23.130927+0000","last_peered":"2023-11-24T08:08:23.130927+0000","last_clean":"2023-11-24T08:08:23.130927+0000","last_became_active":"2023-11-24T08:08:23.127889+0000","last_became_peered":"2023-11-24T08:08:23.127889+0000","last_unstale":"2023-11-24T08:08:23.130927+0000","last_undegraded":"2023-11-24T08:08:23.130927+0000","last_fullsized":"2023-11-24T08:08:23.130927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:52.156451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.104547+0000","last_change":"2023-11-24T08:08:23.104547+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.104547+0000","last_undegraded":"2023-11-24T08:08:23.104547+0000","last_fullsized":"2023-11-24T08:08:23.104547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135167+0000","last_change":"2023-11-24T08:08:23.130898+0000","last_active":"2023-11-24T08:08:23.135167+0000","last_peered":"2023-11-24T08:08:23.135167+0000","last_clean":"2023-11-24T08:08:23.135167+0000","last_became_active":"2023-11-24T08:08:23.130435+0000","last_became_peered":"2023-11-24T08:08:23.130435+0000","last_unstale":"2023-11-24T08:08:23.135167+0000","last_undegraded":"2023-11-24T08:08:23.135167+0000","last_fullsized":"2023-11-24T08:08:23.135167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:02.959505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115515+0000","last_change":"2023-11-24T08:08:23.115515+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115515+0000","last_undegraded":"2023-11-24T08:08:23.115515+0000","last_fullsized":"2023-11-24T08:08:23.115515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135146+0000","last_change":"2023-11-24T08:08:23.129858+0000","last_active":"2023-11-24T08:08:23.135146+0000","last_peered":"2023-11-24T08:08:23.135146+0000","last_clean":"2023-11-24T08:08:23.135146+0000","last_became_active":"2023-11-24T08:08:23.129534+0000","last_became_peered":"2023-11-24T08:08:23.129534+0000","last_unstale":"2023-11-24T08:08:23.135146+0000","last_undegraded":"2023-11-24T08:08:23.135146+0000","last_fullsized":"2023-11-24T08:08:23.135146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:43:41.791702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116698+0000","last_change":"2023-11-24T08:08:23.116698+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116698+0000","last_undegraded":"2023-11-24T08:08:23.116698+0000","last_fullsized":"2023-11-24T08:08:23.116698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.131723+0000","last_change":"2023-11-24T08:08:23.129448+0000","last_active":"2023-11-24T08:08:23.131723+0000","last_peered":"2023-11-24T08:08:23.131723+0000","last_clean":"2023-11-24T08:08:23.131723+0000","last_became_active":"2023-11-24T08:08:23.129016+0000","last_became_peered":"2023-11-24T08:08:23.129016+0000","last_unstale":"2023-11-24T08:08:23.131723+0000","last_undegraded":"2023-11-24T08:08:23.131723+0000","last_fullsized":"2023-11-24T08:08:23.131723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:57.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116746+0000","last_change":"2023-11-24T08:08:23.116746+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116746+0000","last_undegraded":"2023-11-24T08:08:23.116746+0000","last_fullsized":"2023-11-24T08:08:23.116746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.139416+0000","last_change":"2023-11-24T08:08:23.134387+0000","last_active":"2023-11-24T08:08:23.139416+0000","last_peered":"2023-11-24T08:08:23.139416+0000","last_clean":"2023-11-24T08:08:23.139416+0000","last_became_active":"2023-11-24T08:08:23.134082+0000","last_became_peered":"2023-11-24T08:08:23.134082+0000","last_unstale":"2023-11-24T08:08:23.139416+0000","last_undegraded":"2023-11-24T08:08:23.139416+0000","last_fullsized":"2023-11-24T08:08:23.139416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:29.860915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112898+0000","last_change":"2023-11-24T08:08:23.112898+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112898+0000","last_undegraded":"2023-11-24T08:08:23.112898+0000","last_fullsized":"2023-11-24T08:08:23.112898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.135574+0000","last_change":"2023-11-24T08:08:23.127111+0000","last_active":"2023-11-24T08:08:23.135574+0000","last_peered":"2023-11-24T08:08:23.135574+0000","last_clean":"2023-11-24T08:08:23.135574+0000","last_became_active":"2023-11-24T08:08:23.126828+0000","last_became_peered":"2023-11-24T08:08:23.126828+0000","last_unstale":"2023-11-24T08:08:23.135574+0000","last_undegraded":"2023-11-24T08:08:23.135574+0000","last_fullsized":"2023-11-24T08:08:23.135574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:27.777382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103781+0000","last_change":"2023-11-24T08:08:23.103781+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103781+0000","last_undegraded":"2023-11-24T08:08:23.103781+0000","last_fullsized":"2023-11-24T08:08:23.103781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129854+0000","last_change":"2023-11-24T08:08:23.125635+0000","last_active":"2023-11-24T08:08:23.129854+0000","last_peered":"2023-11-24T08:08:23.129854+0000","last_clean":"2023-11-24T08:08:23.129854+0000","last_became_active":"2023-11-24T08:08:23.125335+0000","last_became_peered":"2023-11-24T08:08:23.125335+0000","last_unstale":"2023-11-24T08:08:23.129854+0000","last_undegraded":"2023-11-24T08:08:23.129854+0000","last_fullsized":"2023-11-24T08:08:23.129854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:20.751833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.112141+0000","last_change":"2023-11-24T08:08:23.112141+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.112141+0000","last_undegraded":"2023-11-24T08:08:23.112141+0000","last_fullsized":"2023-11-24T08:08:23.112141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129828+0000","last_change":"2023-11-24T08:08:23.125737+0000","last_active":"2023-11-24T08:08:23.129828+0000","last_peered":"2023-11-24T08:08:23.129828+0000","last_clean":"2023-11-24T08:08:23.129828+0000","last_became_active":"2023-11-24T08:08:23.125403+0000","last_became_peered":"2023-11-24T08:08:23.125403+0000","last_unstale":"2023-11-24T08:08:23.129828+0000","last_undegraded":"2023-11-24T08:08:23.129828+0000","last_fullsized":"2023-11-24T08:08:23.129828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:57.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.117189+0000","last_change":"2023-11-24T08:08:23.117189+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.117189+0000","last_undegraded":"2023-11-24T08:08:23.117189+0000","last_fullsized":"2023-11-24T08:08:23.117189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136817+0000","last_change":"2023-11-24T08:08:23.134364+0000","last_active":"2023-11-24T08:08:23.136817+0000","last_peered":"2023-11-24T08:08:23.136817+0000","last_clean":"2023-11-24T08:08:23.136817+0000","last_became_active":"2023-11-24T08:08:23.134008+0000","last_became_peered":"2023-11-24T08:08:23.134008+0000","last_unstale":"2023-11-24T08:08:23.136817+0000","last_undegraded":"2023-11-24T08:08:23.136817+0000","last_fullsized":"2023-11-24T08:08:23.136817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:57.287980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.103133+0000","last_change":"2023-11-24T08:08:23.103133+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.103133+0000","last_undegraded":"2023-11-24T08:08:23.103133+0000","last_fullsized":"2023-11-24T08:08:23.103133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.134107+0000","last_change":"2023-11-24T08:08:23.130703+0000","last_active":"2023-11-24T08:08:23.134107+0000","last_peered":"2023-11-24T08:08:23.134107+0000","last_clean":"2023-11-24T08:08:23.134107+0000","last_became_active":"2023-11-24T08:08:23.130442+0000","last_became_peered":"2023-11-24T08:08:23.130442+0000","last_unstale":"2023-11-24T08:08:23.134107+0000","last_undegraded":"2023-11-24T08:08:23.134107+0000","last_fullsized":"2023-11-24T08:08:23.134107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:14.668163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.113310+0000","last_change":"2023-11-24T08:08:23.113310+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.113310+0000","last_undegraded":"2023-11-24T08:08:23.113310+0000","last_fullsized":"2023-11-24T08:08:23.113310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136704+0000","last_change":"2023-11-24T08:08:23.134436+0000","last_active":"2023-11-24T08:08:23.136704+0000","last_peered":"2023-11-24T08:08:23.136704+0000","last_clean":"2023-11-24T08:08:23.136704+0000","last_became_active":"2023-11-24T08:08:23.133914+0000","last_became_peered":"2023-11-24T08:08:23.133914+0000","last_unstale":"2023-11-24T08:08:23.136704+0000","last_undegraded":"2023-11-24T08:08:23.136704+0000","last_fullsized":"2023-11-24T08:08:23.136704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:45.936686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.120558+0000","last_change":"2023-11-24T08:08:23.120558+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.120558+0000","last_undegraded":"2023-11-24T08:08:23.120558+0000","last_fullsized":"2023-11-24T08:08:23.120558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.129603+0000","last_change":"2023-11-24T08:08:23.124236+0000","last_active":"2023-11-24T08:08:23.129603+0000","last_peered":"2023-11-24T08:08:23.129603+0000","last_clean":"2023-11-24T08:08:23.129603+0000","last_became_active":"2023-11-24T08:08:23.123917+0000","last_became_peered":"2023-11-24T08:08:23.123917+0000","last_unstale":"2023-11-24T08:08:23.129603+0000","last_undegraded":"2023-11-24T08:08:23.129603+0000","last_fullsized":"2023-11-24T08:08:23.129603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:14.668163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.115029+0000","last_change":"2023-11-24T08:08:23.115029+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.115029+0000","last_undegraded":"2023-11-24T08:08:23.115029+0000","last_fullsized":"2023-11-24T08:08:23.115029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.131926+0000","last_change":"2023-11-24T08:08:23.128663+0000","last_active":"2023-11-24T08:08:23.131926+0000","last_peered":"2023-11-24T08:08:23.131926+0000","last_clean":"2023-11-24T08:08:23.131926+0000","last_became_active":"2023-11-24T08:08:23.128380+0000","last_became_peered":"2023-11-24T08:08:23.128380+0000","last_unstale":"2023-11-24T08:08:23.131926+0000","last_undegraded":"2023-11-24T08:08:23.131926+0000","last_fullsized":"2023-11-24T08:08:23.131926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:32.242969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.116516+0000","last_change":"2023-11-24T08:08:23.116516+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.116516+0000","last_undegraded":"2023-11-24T08:08:23.116516+0000","last_fullsized":"2023-11-24T08:08:23.116516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.136078+0000","last_change":"2023-11-24T08:08:23.133278+0000","last_active":"2023-11-24T08:08:23.136078+0000","last_peered":"2023-11-24T08:08:23.136078+0000","last_clean":"2023-11-24T08:08:23.136078+0000","last_became_active":"2023-11-24T08:08:23.132849+0000","last_became_peered":"2023-11-24T08:08:23.132849+0000","last_unstale":"2023-11-24T08:08:23.136078+0000","last_undegraded":"2023-11-24T08:08:23.136078+0000","last_fullsized":"2023-11-24T08:08:23.136078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:31:30.834610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-11-24T08:08:23.109344+0000","last_change":"2023-11-24T08:08:23.109344+0000","last_active":"2023-11-24T08:08:23.087219+0000","last_peered":"2023-11-24T08:08:23.087219+0000","last_clean":"2023-11-24T08:08:23.087219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:08:23.109344+0000","last_undegraded":"2023-11-24T08:08:23.109344+0000","last_fullsized":"2023-11-24T08:08:23.109344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:08:23.137736+0000","last_change":"2023-11-24T08:08:23.136137+0000","last_active":"2023-11-24T08:08:23.137736+0000","last_peered":"2023-11-24T08:08:23.137736+0000","last_clean":"2023-11-24T08:08:23.137736+0000","last_became_active":"2023-11-24T08:08:23.135553+0000","last_became_peered":"2023-11-24T08:08:23.135553+0000","last_unstale":"2023-11-24T08:08:23.137736+0000","last_undegraded":"2023-11-24T08:08:23.137736+0000","last_fullsized":"2023-11-24T08:08:23.137736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:55.555959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T08:08:22.094050+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:22.094050+0000","last_peered":"2023-11-24T08:08:22.094050+0000","last_clean":"2023-11-24T08:08:22.094050+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:22.094050+0000","last_undegraded":"2023-11-24T08:08:22.094050+0000","last_fullsized":"2023-11-24T08:08:22.094050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":1036492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.995,"15min":1.0009999999999999},"min":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.516,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.046,"15min":1.079},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.403,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.028}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.747,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6799999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.3069999999999999,"15min":1.3720000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.712,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.4379999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.268,"15min":1.3029999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.355}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.26,"15min":1.3129999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8859999999999999,"5min":2.524,"15min":2.524},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3959999999999999,"15min":1.498},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7889999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.472}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.391,"15min":1.5109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8520000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.268},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.323,"15min":1.365},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6779999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.95499999999999996}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.5189999999999999,"15min":1.599},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.23},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9750000000000001,"5min":2.944,"15min":2.944},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.631,"15min":1.7350000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.915,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.5389999999999999,"15min":1.643},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.962,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.3069999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.319},"min":{"1min":0.73099999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.6579999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.171},{"interface":"front","average":{"1min":1.173,"5min":1.262,"15min":1.335},"min":{"1min":0.81899999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.3819999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.63300000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.593,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2270000000000001,"15min":1.3029999999999999},"min":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7090000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.409}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.204,"15min":1.2869999999999999},"min":{"1min":0.86699999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.825,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3260000000000001,"15min":1.476},"min":{"1min":0.84999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5169999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.2170000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3680000000000001,"15min":1.4370000000000001},"min":{"1min":1.0980000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.8089999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.468},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.45,"15min":1.492},"min":{"1min":1.024,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.821,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.4359999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.526,"15min":1.724},"min":{"1min":0.995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.627,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.1279999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9299999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.4890000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.5760000000000001,"15min":1.778},"min":{"1min":0.84399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.784,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.714,"15min":1.9179999999999999},"min":{"1min":1.3640000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8660000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.599,"15min":1.6759999999999999},"min":{"1min":1.175,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0190000000000001,"5min":2.98,"15min":2.98},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.5800000000000001,"15min":1.6379999999999999},"min":{"1min":1.0549999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9810000000000001,"5min":2.29,"15min":2.29},"last":0.97399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0129999999999999,"15min":1.0489999999999999},"min":{"1min":0.68899999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6419999999999999,"5min":1.764,"15min":1.764},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.016},"min":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5369999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.80000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9530000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.149,"5min":1.1000000000000001,"15min":1.0580000000000001},"min":{"1min":0.78600000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.589,"5min":1.635,"15min":1.635},"last":1.1319999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.214,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.252,"15min":1.224},"min":{"1min":0.81100000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.2690000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.3959999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.381,"15min":1.4259999999999999},"min":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.7400000000000002,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.2250000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.302,"15min":1.304},"min":{"1min":0.67700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.885,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.448,"15min":1.4239999999999999},"min":{"1min":0.91700000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.3620000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.4790000000000001,"15min":1.48},"min":{"1min":0.621,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.042},{"interface":"front","average":{"1min":1.502,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":0.76500000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.292}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.55,"15min":1.538},"min":{"1min":0.78900000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.9370000000000003,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.5640000000000001,"15min":1.538},"min":{"1min":0.74299999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.323,"5min":2.859,"15min":2.859},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.034,"15min":1.04},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.534,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.393},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.68500000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.617,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0700000000000001,"15min":1.081},"min":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5660000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.319},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1970000000000001,"15min":1.2330000000000001},"min":{"1min":0.72499999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5760000000000001,"5min":1.903,"15min":1.903},"last":0.91600000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1910000000000001,"15min":1.2010000000000001},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7390000000000001,"5min":1.823,"15min":1.823},"last":1.24},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.248},"min":{"1min":0.72999999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.534,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.0489999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.278,"15min":1.296},"min":{"1min":0.76300000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7829999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.222},{"interface":"front","average":{"1min":1.196,"5min":1.286,"15min":1.306},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.478,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.306,"15min":1.304},"min":{"1min":1.095,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8120000000000001,"5min":1.944,"15min":1.944},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.365,"15min":1.367},"min":{"1min":1.0960000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0049999999999999,"5min":2.012,"15min":2.012},"last":1.018}]},{"osd":6,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.385,"15min":1.371},"min":{"1min":1.0229999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.736,"5min":2.105,"15min":2.105},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.615,"5min":1.508,"15min":1.514},"min":{"1min":1.306,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.143}]},{"osd":7,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.452,"15min":1.4370000000000001},"min":{"1min":1.1599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0379999999999998,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.524},{"interface":"front","average":{"1min":1.494,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":1.2430000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.827,"5min":2.722,"15min":2.722},"last":1.101}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.3560000000000001,"5min":1.585,"15min":1.585},"last":1.284},{"interface":"front","average":{"1min":0.87,"5min":0.90500000000000003,"15min":0.94199999999999995},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.488,"5min":1.51,"15min":1.51},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.01,"15min":1.129},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4299999999999999,"5min":3.246,"15min":3.246},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.995,"5min":0.97999999999999998,"15min":0.96899999999999997},"min":{"1min":0.51400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.6299999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1140000000000001,"15min":1.1739999999999999},"min":{"1min":0.63800000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.988,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.876},{"interface":"front","average":{"1min":1.155,"5min":1.1120000000000001,"15min":1.1539999999999999},"min":{"1min":0.68999999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8009999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.4299999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2390000000000001,"15min":1.262},"min":{"1min":0.94999999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6950000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.325,"15min":1.3799999999999999},"min":{"1min":0.83799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8919999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.222}]},{"osd":5,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3939999999999999,"15min":1.45},"min":{"1min":0.96799999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.839,"5min":2.395,"15min":2.395},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.89800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.7250000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4219999999999999,"15min":1.4319999999999999},"min":{"1min":0.95999999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0230000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.393,"15min":1.4159999999999999},"min":{"1min":1.0269999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.417,"15min":1.4419999999999999},"min":{"1min":0.98199999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.024},{"interface":"front","average":{"1min":1.486,"5min":1.5029999999999999,"15min":1.573},"min":{"1min":1.1120000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0510000000000002,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.343}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83699999999999997},"min":{"1min":0.44500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.369,"5min":1.524,"15min":1.524},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95799999999999996,"15min":1.01},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.694,"5min":2.133,"15min":2.133},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.087,"15min":1.2070000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4359999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0569999999999999},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.875,"5min":1.998,"15min":1.998},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.258,"15min":1.3160000000000001},"min":{"1min":0.63700000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.722,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.1220000000000001,"15min":1.1859999999999999},"min":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.9350000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.204}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.248,"15min":1.321},"min":{"1min":0.82799999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.843,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.774},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.411,"15min":1.4930000000000001},"min":{"1min":0.75900000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8280000000000001,"5min":3.161,"15min":3.161},"last":1.2569999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.325,"15min":1.387},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.1629999999999998,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3200000000000001,"15min":1.3999999999999999},"min":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6220000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.8919999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4750000000000001,"15min":1.579},"min":{"1min":0.73299999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8420000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.665},{"interface":"front","average":{"1min":1.405,"5min":1.4470000000000001,"15min":1.552},"min":{"1min":0.93100000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0510000000000002,"5min":2.702,"15min":2.702},"last":1.4790000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.504},"min":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.996,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.9970000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.53,"15min":1.6319999999999999},"min":{"1min":0.81999999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.907,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.605}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":446124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.88800000000000001},"min":{"1min":0.53900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.95599999999999996,"15min":1.042},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6459999999999999,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0680000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94599999999999995,"15min":1.018},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.724,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1399999999999999,"15min":1.216},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8959999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.3500000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1990000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.127,"15min":1.2150000000000001},"min":{"1min":0.60399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.0899999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.383}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.198,"15min":1.24},"min":{"1min":0.81799999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.645,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.379,"15min":1.4410000000000001},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7909999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.393,"15min":1.446},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.8740000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.415,"15min":1.466},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.855,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.3260000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.4059999999999999,"15min":1.45},"min":{"1min":0.92900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9630000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.613},{"interface":"front","average":{"1min":1.256,"5min":1.3939999999999999,"15min":1.466},"min":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.669,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.216}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.48,"15min":1.548},"min":{"1min":1.0940000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6739999999999999,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.284},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.536,"15min":1.5840000000000001},"min":{"1min":1.0209999999999999,"5min":0.872,"15min":0.872},"max":{"1min":2.335,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1634304,"data_stored":1035473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.876,"15min":0.88},"min":{"1min":0.504,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2110000000000001,"5min":3.423,"15min":3.423},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.997,"15min":0.95399999999999996},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.661,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.96099999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.081,"15min":1.0649999999999999},"min":{"1min":0.60899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5589999999999999,"5min":3.899,"15min":3.899},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.99399999999999999,"15min":0.95799999999999996},"min":{"1min":0.46899999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.3140000000000001,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.282,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.2250000000000001},"min":{"1min":0.65300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.621,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.1619999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3,"15min":1.2689999999999999},"min":{"1min":0.81999999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6459999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.387},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.482,"15min":1.476},"min":{"1min":0.99199999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8959999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4510000000000001,"15min":1.393},"min":{"1min":0.96599999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.4550000000000001,"15min":1.4379999999999999},"min":{"1min":0.80000000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.0110000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.0430000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.399,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.786,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.226},{"interface":"front","average":{"1min":1.49,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":0.84899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9359999999999999,"5min":3.879,"15min":3.879},"last":2.1070000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.54,"15min":1.54},"min":{"1min":0.83499999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":3.823,"15min":3.823},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.5049999999999999,"15min":1.492},"min":{"1min":0.88300000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8740000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.48}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:29.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2023-11-24T08:08:29.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.125419+0000', 'last_change': '2023-11-24T08:08:23.125419+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.125419+0000', 'last_undegraded': '2023-11-24T08:08:23.125419+0000', 'last_fullsized': '2023-11-24T08:08:23.125419+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2023-11-24T08:08:29.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.111910+0000', 'last_change': '2023-11-24T08:08:23.111910+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.111910+0000', 'last_undegraded': '2023-11-24T08:08:23.111910+0000', 'last_fullsized': '2023-11-24T08:08:23.111910+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2023-11-24T08:08:29.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.111691+0000', 'last_change': '2023-11-24T08:08:23.111691+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.111691+0000', 'last_undegraded': '2023-11-24T08:08:23.111691+0000', 'last_fullsized': '2023-11-24T08:08:23.111691+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.119453+0000', 'last_change': '2023-11-24T08:08:23.119453+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.119453+0000', 'last_undegraded': '2023-11-24T08:08:23.119453+0000', 'last_fullsized': '2023-11-24T08:08:23.119453+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.104404+0000', 'last_change': '2023-11-24T08:08:23.104404+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.104404+0000', 'last_undegraded': '2023-11-24T08:08:23.104404+0000', 'last_fullsized': '2023-11-24T08:08:23.104404+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.114326+0000', 'last_change': '2023-11-24T08:08:23.114326+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.114326+0000', 'last_undegraded': '2023-11-24T08:08:23.114326+0000', 'last_fullsized': '2023-11-24T08:08:23.114326+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.114692+0000', 'last_change': '2023-11-24T08:08:23.114692+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.114692+0000', 'last_undegraded': '2023-11-24T08:08:23.114692+0000', 'last_fullsized': '2023-11-24T08:08:23.114692+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2023-11-24T08:08:29.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.114757+0000', 'last_change': '2023-11-24T08:08:23.114757+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.114757+0000', 'last_undegraded': '2023-11-24T08:08:23.114757+0000', 'last_fullsized': '2023-11-24T08:08:23.114757+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.116155+0000', 'last_change': '2023-11-24T08:08:23.116155+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.116155+0000', 'last_undegraded': '2023-11-24T08:08:23.116155+0000', 'last_fullsized': '2023-11-24T08:08:23.116155+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.118098+0000', 'last_change': '2023-11-24T08:08:23.118098+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.118098+0000', 'last_undegraded': '2023-11-24T08:08:23.118098+0000', 'last_fullsized': '2023-11-24T08:08:23.118098+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.112412+0000', 'last_change': '2023-11-24T08:08:23.112412+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.112412+0000', 'last_undegraded': '2023-11-24T08:08:23.112412+0000', 'last_fullsized': '2023-11-24T08:08:23.112412+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.125090+0000', 'last_change': '2023-11-24T08:08:23.125090+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.125090+0000', 'last_undegraded': '2023-11-24T08:08:23.125090+0000', 'last_fullsized': '2023-11-24T08:08:23.125090+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.116279+0000', 'last_change': '2023-11-24T08:08:23.116279+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.116279+0000', 'last_undegraded': '2023-11-24T08:08:23.116279+0000', 'last_fullsized': '2023-11-24T08:08:23.116279+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.115991+0000', 'last_change': '2023-11-24T08:08:23.115991+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.115991+0000', 'last_undegraded': '2023-11-24T08:08:23.115991+0000', 'last_fullsized': '2023-11-24T08:08:23.115991+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.103193+0000', 'last_change': '2023-11-24T08:08:23.103193+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.103193+0000', 'last_undegraded': '2023-11-24T08:08:23.103193+0000', 'last_fullsized': '2023-11-24T08:08:23.103193+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.105524+0000', 'last_change': '2023-11-24T08:08:23.105524+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.105524+0000', 'last_undegraded': '2023-11-24T08:08:23.105524+0000', 'last_fullsized': '2023-11-24T08:08:23.105524+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.103155+0000', 'last_change': '2023-11-24T08:08:23.103155+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.103155+0000', 'last_undegraded': '2023-11-24T08:08:23.103155+0000', 'last_fullsized': '2023-11-24T08:08:23.103155+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.120117+0000', 'last_change': '2023-11-24T08:08:23.120117+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.120117+0000', 'last_undegraded': '2023-11-24T08:08:23.120117+0000', 'last_fullsized': '2023-11-24T08:08:23.120117+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.118749+0000', 'last_change': '2023-11-24T08:08:23.118749+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.118749+0000', 'last_undegraded': '2023-11-24T08:08:23.118749+0000', 'last_fullsized': '2023-11-24T08:08:23.118749+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.118836+0000', 'last_change': '2023-11-24T08:08:23.118836+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.118836+0000', 'last_undegraded': '2023-11-24T08:08:23.118836+0000', 'last_fullsized': '2023-11-24T08:08:23.118836+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.115676+0000', 'last_change': '2023-11-24T08:08:23.115676+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.115676+0000', 'last_undegraded': '2023-11-24T08:08:23.115676+0000', 'last_fullsized': '2023-11-24T08:08:23.115676+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2023-11-24T08:08:29.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.111546+0000', 'last_change': '2023-11-24T08:08:23.111546+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.111546+0000', 'last_undegraded': '2023-11-24T08:08:23.111546+0000', 'last_fullsized': '2023-11-24T08:08:23.111546+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.117271+0000', 'last_change': '2023-11-24T08:08:23.117271+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.117271+0000', 'last_undegraded': '2023-11-24T08:08:23.117271+0000', 'last_fullsized': '2023-11-24T08:08:23.117271+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.117195+0000', 'last_change': '2023-11-24T08:08:23.117195+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.117195+0000', 'last_undegraded': '2023-11-24T08:08:23.117195+0000', 'last_fullsized': '2023-11-24T08:08:23.117195+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.110410+0000', 'last_change': '2023-11-24T08:08:23.110410+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.110410+0000', 'last_undegraded': '2023-11-24T08:08:23.110410+0000', 'last_fullsized': '2023-11-24T08:08:23.110410+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.124187+0000', 'last_change': '2023-11-24T08:08:23.124187+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.124187+0000', 'last_undegraded': '2023-11-24T08:08:23.124187+0000', 'last_fullsized': '2023-11-24T08:08:23.124187+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2023-11-24T08:08:29.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.110905+0000', 'last_change': '2023-11-24T08:08:23.110905+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.110905+0000', 'last_undegraded': '2023-11-24T08:08:23.110905+0000', 'last_fullsized': '2023-11-24T08:08:23.110905+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.123814+0000', 'last_change': '2023-11-24T08:08:23.123814+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.123814+0000', 'last_undegraded': '2023-11-24T08:08:23.123814+0000', 'last_fullsized': '2023-11-24T08:08:23.123814+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.110123+0000', 'last_change': '2023-11-24T08:08:23.110123+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.110123+0000', 'last_undegraded': '2023-11-24T08:08:23.110123+0000', 'last_fullsized': '2023-11-24T08:08:23.110123+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.110824+0000', 'last_change': '2023-11-24T08:08:23.110824+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.110824+0000', 'last_undegraded': '2023-11-24T08:08:23.110824+0000', 'last_fullsized': '2023-11-24T08:08:23.110824+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.102988+0000', 'last_change': '2023-11-24T08:08:23.102988+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.102988+0000', 'last_undegraded': '2023-11-24T08:08:23.102988+0000', 'last_fullsized': '2023-11-24T08:08:23.102988+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2023-11-24T08:08:29.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.119610+0000', 'last_change': '2023-11-24T08:08:23.119610+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.119610+0000', 'last_undegraded': '2023-11-24T08:08:23.119610+0000', 'last_fullsized': '2023-11-24T08:08:23.119610+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.105615+0000', 'last_change': '2023-11-24T08:08:23.105615+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.105615+0000', 'last_undegraded': '2023-11-24T08:08:23.105615+0000', 'last_fullsized': '2023-11-24T08:08:23.105615+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.124855+0000', 'last_change': '2023-11-24T08:08:23.124855+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.124855+0000', 'last_undegraded': '2023-11-24T08:08:23.124855+0000', 'last_fullsized': '2023-11-24T08:08:23.124855+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.121284+0000', 'last_change': '2023-11-24T08:08:23.121284+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.121284+0000', 'last_undegraded': '2023-11-24T08:08:23.121284+0000', 'last_fullsized': '2023-11-24T08:08:23.121284+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.110525+0000', 'last_change': '2023-11-24T08:08:23.110525+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.110525+0000', 'last_undegraded': '2023-11-24T08:08:23.110525+0000', 'last_fullsized': '2023-11-24T08:08:23.110525+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2023-11-24T08:08:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.110163+0000', 'last_change': '2023-11-24T08:08:23.110163+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.110163+0000', 'last_undegraded': '2023-11-24T08:08:23.110163+0000', 'last_fullsized': '2023-11-24T08:08:23.110163+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.103184+0000', 'last_change': '2023-11-24T08:08:23.103184+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.103184+0000', 'last_undegraded': '2023-11-24T08:08:23.103184+0000', 'last_fullsized': '2023-11-24T08:08:23.103184+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.119024+0000', 'last_change': '2023-11-24T08:08:23.119024+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.119024+0000', 'last_undegraded': '2023-11-24T08:08:23.119024+0000', 'last_fullsized': '2023-11-24T08:08:23.119024+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.101930+0000', 'last_change': '2023-11-24T08:08:23.101930+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.101930+0000', 'last_undegraded': '2023-11-24T08:08:23.101930+0000', 'last_fullsized': '2023-11-24T08:08:23.101930+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.102421+0000', 'last_change': '2023-11-24T08:08:23.102421+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.102421+0000', 'last_undegraded': '2023-11-24T08:08:23.102421+0000', 'last_fullsized': '2023-11-24T08:08:23.102421+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2023-11-24T08:08:29.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.112793+0000', 'last_change': '2023-11-24T08:08:23.112793+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.112793+0000', 'last_undegraded': '2023-11-24T08:08:23.112793+0000', 'last_fullsized': '2023-11-24T08:08:23.112793+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.121568+0000', 'last_change': '2023-11-24T08:08:23.121568+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.121568+0000', 'last_undegraded': '2023-11-24T08:08:23.121568+0000', 'last_fullsized': '2023-11-24T08:08:23.121568+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.108974+0000', 'last_change': '2023-11-24T08:08:23.108974+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.108974+0000', 'last_undegraded': '2023-11-24T08:08:23.108974+0000', 'last_fullsized': '2023-11-24T08:08:23.108974+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.113332+0000', 'last_change': '2023-11-24T08:08:23.113332+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.113332+0000', 'last_undegraded': '2023-11-24T08:08:23.113332+0000', 'last_fullsized': '2023-11-24T08:08:23.113332+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.111249+0000', 'last_change': '2023-11-24T08:08:23.111249+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.111249+0000', 'last_undegraded': '2023-11-24T08:08:23.111249+0000', 'last_fullsized': '2023-11-24T08:08:23.111249+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2023-11-24T08:08:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.109869+0000', 'last_change': '2023-11-24T08:08:23.109869+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.109869+0000', 'last_undegraded': '2023-11-24T08:08:23.109869+0000', 'last_fullsized': '2023-11-24T08:08:23.109869+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.109666+0000', 'last_change': '2023-11-24T08:08:23.109666+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.109666+0000', 'last_undegraded': '2023-11-24T08:08:23.109666+0000', 'last_fullsized': '2023-11-24T08:08:23.109666+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.111275+0000', 'last_change': '2023-11-24T08:08:23.111275+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.111275+0000', 'last_undegraded': '2023-11-24T08:08:23.111275+0000', 'last_fullsized': '2023-11-24T08:08:23.111275+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.123529+0000', 'last_change': '2023-11-24T08:08:23.123529+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.123529+0000', 'last_undegraded': '2023-11-24T08:08:23.123529+0000', 'last_fullsized': '2023-11-24T08:08:23.123529+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.104547+0000', 'last_change': '2023-11-24T08:08:23.104547+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.104547+0000', 'last_undegraded': '2023-11-24T08:08:23.104547+0000', 'last_fullsized': '2023-11-24T08:08:23.104547+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2023-11-24T08:08:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.115515+0000', 'last_change': '2023-11-24T08:08:23.115515+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.115515+0000', 'last_undegraded': '2023-11-24T08:08:23.115515+0000', 'last_fullsized': '2023-11-24T08:08:23.115515+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.116698+0000', 'last_change': '2023-11-24T08:08:23.116698+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.116698+0000', 'last_undegraded': '2023-11-24T08:08:23.116698+0000', 'last_fullsized': '2023-11-24T08:08:23.116698+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.116746+0000', 'last_change': '2023-11-24T08:08:23.116746+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.116746+0000', 'last_undegraded': '2023-11-24T08:08:23.116746+0000', 'last_fullsized': '2023-11-24T08:08:23.116746+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.112898+0000', 'last_change': '2023-11-24T08:08:23.112898+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.112898+0000', 'last_undegraded': '2023-11-24T08:08:23.112898+0000', 'last_fullsized': '2023-11-24T08:08:23.112898+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.103781+0000', 'last_change': '2023-11-24T08:08:23.103781+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.103781+0000', 'last_undegraded': '2023-11-24T08:08:23.103781+0000', 'last_fullsized': '2023-11-24T08:08:23.103781+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.112141+0000', 'last_change': '2023-11-24T08:08:23.112141+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.112141+0000', 'last_undegraded': '2023-11-24T08:08:23.112141+0000', 'last_fullsized': '2023-11-24T08:08:23.112141+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.117189+0000', 'last_change': '2023-11-24T08:08:23.117189+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.117189+0000', 'last_undegraded': '2023-11-24T08:08:23.117189+0000', 'last_fullsized': '2023-11-24T08:08:23.117189+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.103133+0000', 'last_change': '2023-11-24T08:08:23.103133+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.103133+0000', 'last_undegraded': '2023-11-24T08:08:23.103133+0000', 'last_fullsized': '2023-11-24T08:08:23.103133+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.113310+0000', 'last_change': '2023-11-24T08:08:23.113310+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.113310+0000', 'last_undegraded': '2023-11-24T08:08:23.113310+0000', 'last_fullsized': '2023-11-24T08:08:23.113310+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.120558+0000', 'last_change': '2023-11-24T08:08:23.120558+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.120558+0000', 'last_undegraded': '2023-11-24T08:08:23.120558+0000', 'last_fullsized': '2023-11-24T08:08:23.120558+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:08:29.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2023-11-24T08:08:29.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.115029+0000', 'last_change': '2023-11-24T08:08:23.115029+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.115029+0000', 'last_undegraded': '2023-11-24T08:08:23.115029+0000', 'last_fullsized': '2023-11-24T08:08:23.115029+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:08:29.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2023-11-24T08:08:29.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.116516+0000', 'last_change': '2023-11-24T08:08:23.116516+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.116516+0000', 'last_undegraded': '2023-11-24T08:08:23.116516+0000', 'last_fullsized': '2023-11-24T08:08:23.116516+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:08:29.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2023-11-24T08:08:29.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:08:23.109344+0000', 'last_change': '2023-11-24T08:08:23.109344+0000', 'last_active': '2023-11-24T08:08:23.087219+0000', 'last_peered': '2023-11-24T08:08:23.087219+0000', 'last_clean': '2023-11-24T08:08:23.087219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:08:23.109344+0000', 'last_undegraded': '2023-11-24T08:08:23.109344+0000', 'last_fullsized': '2023-11-24T08:08:23.109344+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'last_clean_scrub_stamp': '2023-11-24T08:08:23.087219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:08:29.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T08:08:29.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.984+0000 7f677e121640 1 Processor -- start 2023-11-24T08:08:29.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.984+0000 7f677e121640 1 -- start start 2023-11-24T08:08:29.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.986+0000 7f677e121640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f67780b4830 0x7f67780cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.986+0000 7f677e121640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677815c460 con 0x7f6778158af0 2023-11-24T08:08:29.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.986+0000 7f677e121640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6778158ed0 con 0x7f677815bfd0 2023-11-24T08:08:29.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.986+0000 7f677e121640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67780cc7e0 con 0x7f67780b4830 2023-11-24T08:08:29.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.986+0000 7f677d11f640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6778158af0 0x7f677815c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46096/0 (socket says 172.21.15.64:46096) 2023-11-24T08:08:29.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.986+0000 7f677d11f640 1 -- 172.21.15.64:0/2556464520 learned_addr learned my addr 172.21.15.64:0/2556464520 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:29.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.986+0000 7f677d920640 1 --2- 172.21.15.64:0/2556464520 >> v2:172.21.15.64:3300/0 conn(0x7f67780b4830 0x7f67780cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.986+0000 7f676ffff640 1 -- 172.21.15.64:0/2556464520 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353920877 0 0) 0x7f677815c460 con 0x7f6778158af0 2023-11-24T08:08:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.987+0000 7f676ffff640 1 -- 172.21.15.64:0/2556464520 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6764003520 con 0x7f6778158af0 2023-11-24T08:08:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.987+0000 7f677d920640 1 -- 172.21.15.64:0/2556464520 >> v1:172.21.15.91:6789/0 conn(0x7f677815bfd0 legacy=0x7f67780b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.987+0000 7f677d920640 1 -- 172.21.15.64:0/2556464520 >> v1:172.21.15.64:6789/0 conn(0x7f6778158af0 legacy=0x7f677815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.987+0000 7f677d920640 1 -- 172.21.15.64:0/2556464520 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67780ccac0 con 0x7f67780b4830 2023-11-24T08:08:29.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.988+0000 7f677d920640 1 --2- 172.21.15.64:0/2556464520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f67780cc0e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6770009d90 tx=0x7f677002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bad124d10671c7c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.988+0000 7f676ffff640 1 -- 172.21.15.64:0/2556464520 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677002cc60 con 0x7f67780b4830 2023-11-24T08:08:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.988+0000 7f676ffff640 1 -- 172.21.15.64:0/2556464520 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677002cdc0 con 0x7f67780b4830 2023-11-24T08:08:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.988+0000 7f676ffff640 1 -- 172.21.15.64:0/2556464520 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770034660 con 0x7f67780b4830 2023-11-24T08:08:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.989+0000 7f677e121640 1 -- 172.21.15.64:0/2556464520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 msgr2=0x7f67780cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.989+0000 7f677e121640 1 --2- 172.21.15.64:0/2556464520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f67780cc0e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6770009d90 tx=0x7f677002beb0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.989+0000 7f677e121640 1 -- 172.21.15.64:0/2556464520 shutdown_connections 2023-11-24T08:08:29.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.989+0000 7f677e121640 1 --2- 172.21.15.64:0/2556464520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f67780cc0e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.989+0000 7f677e121640 1 -- 172.21.15.64:0/2556464520 >> 172.21.15.64:0/2556464520 conn(0x7f67780b0b10 msgr2=0x7f67780b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:29.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.989+0000 7f677e121640 1 -- 172.21.15.64:0/2556464520 shutdown_connections 2023-11-24T08:08:29.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.989+0000 7f677e121640 1 -- 172.21.15.64:0/2556464520 wait complete. 2023-11-24T08:08:29.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.991+0000 7f677e121640 1 Processor -- start 2023-11-24T08:08:29.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.991+0000 7f677e121640 1 -- start start 2023-11-24T08:08:29.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677e121640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f6778152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677e121640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6778158af0 0x7f67780ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677e121640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f677815bfd0 0x7f67780cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677e121640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67780cb260 con 0x7f67780b4830 2023-11-24T08:08:29.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677e121640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67781f3730 con 0x7f677815bfd0 2023-11-24T08:08:29.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677e121640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67781f2b20 con 0x7f6778158af0 2023-11-24T08:08:29.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677d11f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f6778152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677d11f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f6778152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60482/0 (socket says 172.21.15.64:60482) 2023-11-24T08:08:29.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.992+0000 7f677d11f640 1 -- 172.21.15.64:0/483368099 learned_addr learned my addr 172.21.15.64:0/483368099 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f677d11f640 1 -- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6778158af0 msgr2=0x7f67780ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f677d11f640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6778158af0 0x7f67780ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f677d11f640 1 -- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f677815bfd0 msgr2=0x7f67780cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f677d11f640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f677815bfd0 0x7f67780cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f677d11f640 1 -- 172.21.15.64:0/483368099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6770009a70 con 0x7f67780b4830 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f677d11f640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f6778152e70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f677400c500 tx=0x7f677400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f676dffb640 1 -- 172.21.15.64:0/483368099 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774005ce0 con 0x7f67780b4830 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f676dffb640 1 -- 172.21.15.64:0/483368099 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774005e40 con 0x7f67780b4830 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67781f2e00 con 0x7f67780b4830 2023-11-24T08:08:29.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.993+0000 7f676dffb640 1 -- 172.21.15.64:0/483368099 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774007330 con 0x7f67780b4830 2023-11-24T08:08:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.994+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67781f33c0 con 0x7f67780b4830 2023-11-24T08:08:29.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.995+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677804e840 con 0x7f67780b4830 2023-11-24T08:08:29.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.996+0000 7f676dffb640 1 -- 172.21.15.64:0/483368099 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6774007490 con 0x7f67780b4830 2023-11-24T08:08:29.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.997+0000 7f676dffb640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f675c06b3e0 0x7f675c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.997+0000 7f676dffb640 1 -- 172.21.15.64:0/483368099 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6774089680 con 0x7f67780b4830 2023-11-24T08:08:29.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.998+0000 7f677c91e640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f675c06b3e0 0x7f675c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:29.999+0000 7f677c91e640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f675c06b3e0 0x7f675c06d8a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6768006fd0 tx=0x7f6768006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.001+0000 7f676dffb640 1 -- 172.21.15.64:0/483368099 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6774054010 con 0x7f67780b4830 2023-11-24T08:08:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.272+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f677806b450 con 0x7f67780b4830 2023-11-24T08:08:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.273+0000 7f676dffb640 1 -- 172.21.15.64:0/483368099 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1431 (secure 0 0 0) 0x7f6774057630 con 0x7f67780b4830 2023-11-24T08:08:30.276 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:30.276 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":299,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1700813023,"num_in_osds":8,"osd_in_since":1700813014,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":265261056,"bytes_avail":772828852224,"bytes_total":773094113280,"write_bytes_sec":2042,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":53,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7146}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-11-24T08:08:10.107159+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-24T08:08:30.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f675c06b3e0 msgr2=0x7f675c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f675c06b3e0 0x7f675c06d8a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6768006fd0 tx=0x7f6768006bc0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 msgr2=0x7f6778152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f6778152e70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f677400c500 tx=0x7f677400c9c0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 shutdown_connections 2023-11-24T08:08:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f675c06b3e0 0x7f675c06d8a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f677815bfd0 0x7f67780cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6778158af0 0x7f67780ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 --2- 172.21.15.64:0/483368099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67780b4830 0x7f6778152e70 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 >> 172.21.15.64:0/483368099 conn(0x7f67780b0b10 msgr2=0x7f6778159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 shutdown_connections 2023-11-24T08:08:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.276+0000 7f677e121640 1 -- 172.21.15.64:0/483368099 wait complete. 2023-11-24T08:08:30.298 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T08:08:30.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:30.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.519+0000 7f78f27a7640 1 Processor -- start 2023-11-24T08:08:30.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.519+0000 7f78f27a7640 1 -- start start 2023-11-24T08:08:30.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f27a7640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec157770 0x7f78ec06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f27a7640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f78ec06d080 0x7f78ec15d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f27a7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f78ec15ddb0 0x7f78ec0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f27a7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ec06d500 con 0x7f78ec15ddb0 2023-11-24T08:08:30.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f27a7640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ec161670 con 0x7f78ec06d080 2023-11-24T08:08:30.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f27a7640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec15e610 con 0x7f78ec157770 2023-11-24T08:08:30.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f1fa6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f78ec15ddb0 0x7f78ec0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f1fa6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f78ec15ddb0 0x7f78ec0ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60484/0 (socket says 172.21.15.64:60484) 2023-11-24T08:08:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f1fa6640 1 -- 172.21.15.64:0/939213912 learned_addr learned my addr 172.21.15.64:0/939213912 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.521+0000 7f78f0fa4640 1 --2- 172.21.15.64:0/939213912 >> v2:172.21.15.91:3300/0 conn(0x7f78ec06d080 0x7f78ec15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.522+0000 7f78f1fa6640 1 -- 172.21.15.64:0/939213912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec157770 msgr2=0x7f78ec06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.522+0000 7f78f17a5640 1 --2- 172.21.15.64:0/939213912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec157770 0x7f78ec06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.522+0000 7f78f1fa6640 1 --2- 172.21.15.64:0/939213912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec157770 0x7f78ec06cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.522+0000 7f78f1fa6640 1 -- 172.21.15.64:0/939213912 >> v2:172.21.15.91:3300/0 conn(0x7f78ec06d080 msgr2=0x7f78ec15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.522+0000 7f78f1fa6640 1 --2- 172.21.15.64:0/939213912 >> v2:172.21.15.91:3300/0 conn(0x7f78ec06d080 0x7f78ec15d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.522+0000 7f78f1fa6640 1 -- 172.21.15.64:0/939213912 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec160d40 con 0x7f78ec15ddb0 2023-11-24T08:08:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.522+0000 7f78f17a5640 1 --2- 172.21.15.64:0/939213912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec157770 0x7f78ec06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:30.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.523+0000 7f78f1fa6640 1 --2- 172.21.15.64:0/939213912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec15ddb0 0x7f78ec0ac870 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0ae680 tx=0x7f78e402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eeb8935574d6184 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.523+0000 7f78e3fff640 1 -- 172.21.15.64:0/939213912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e402ca80 con 0x7f78ec15ddb0 2023-11-24T08:08:30.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78e3fff640 1 -- 172.21.15.64:0/939213912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e402cbe0 con 0x7f78ec15ddb0 2023-11-24T08:08:30.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78e3fff640 1 -- 172.21.15.64:0/939213912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e40343f0 con 0x7f78ec15ddb0 2023-11-24T08:08:30.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 -- 172.21.15.64:0/939213912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec15ddb0 msgr2=0x7f78ec0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 --2- 172.21.15.64:0/939213912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec15ddb0 0x7f78ec0ac870 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0ae680 tx=0x7f78e402bee0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 -- 172.21.15.64:0/939213912 shutdown_connections 2023-11-24T08:08:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 --2- 172.21.15.64:0/939213912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec15ddb0 0x7f78ec0ac870 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 --2- 172.21.15.64:0/939213912 >> v2:172.21.15.91:3300/0 conn(0x7f78ec06d080 0x7f78ec15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 --2- 172.21.15.64:0/939213912 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec157770 0x7f78ec06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 -- 172.21.15.64:0/939213912 >> 172.21.15.64:0/939213912 conn(0x7f78ec06a860 msgr2=0x7f78ec06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 -- 172.21.15.64:0/939213912 shutdown_connections 2023-11-24T08:08:30.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.524+0000 7f78f27a7640 1 -- 172.21.15.64:0/939213912 wait complete. 2023-11-24T08:08:30.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.526+0000 7f78f27a7640 1 Processor -- start 2023-11-24T08:08:30.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.526+0000 7f78f27a7640 1 -- start start 2023-11-24T08:08:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.527+0000 7f78f27a7640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78ec06d080 0x7f78ec14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.527+0000 7f78f27a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec157770 0x7f78ec15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.527+0000 7f78f27a7640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec15ddb0 0x7f78ec0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.527+0000 7f78f27a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0cc030 con 0x7f78ec157770 2023-11-24T08:08:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.527+0000 7f78f27a7640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0cc310 con 0x7f78ec06d080 2023-11-24T08:08:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.527+0000 7f78f27a7640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0cc5f0 con 0x7f78ec15ddb0 2023-11-24T08:08:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f0fa4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec157770 0x7f78ec15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f0fa4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec157770 0x7f78ec15a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60490/0 (socket says 172.21.15.64:60490) 2023-11-24T08:08:30.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f0fa4640 1 -- 172.21.15.64:0/162364673 learned_addr learned my addr 172.21.15.64:0/162364673 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f17a5640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78ec06d080 0x7f78ec14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f1fa6640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec15ddb0 0x7f78ec0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f0fa4640 1 -- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec15ddb0 msgr2=0x7f78ec0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f0fa4640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec15ddb0 0x7f78ec0cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f0fa4640 1 -- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78ec06d080 msgr2=0x7f78ec14e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f0fa4640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78ec06d080 0x7f78ec14e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.528+0000 7f78f0fa4640 1 -- 172.21.15.64:0/162364673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e4009b30 con 0x7f78ec157770 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.529+0000 7f78f0fa4640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec157770 0x7f78ec15a400 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00d5f0 tx=0x7f78dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.529+0000 7f78e1ffb640 1 -- 172.21.15.64:0/162364673 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc0125f0 con 0x7f78ec157770 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.529+0000 7f78e1ffb640 1 -- 172.21.15.64:0/162364673 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc012750 con 0x7f78ec157770 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.529+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ec0cc8d0 con 0x7f78ec157770 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.529+0000 7f78e1ffb640 1 -- 172.21.15.64:0/162364673 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc012b00 con 0x7f78ec157770 2023-11-24T08:08:30.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.529+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ec14ec70 con 0x7f78ec157770 2023-11-24T08:08:30.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.530+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ec14ee30 con 0x7f78ec157770 2023-11-24T08:08:30.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.531+0000 7f78e1ffb640 1 -- 172.21.15.64:0/162364673 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f78dc012c60 con 0x7f78ec157770 2023-11-24T08:08:30.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.532+0000 7f78e1ffb640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78d006b3e0 0x7f78d006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.532+0000 7f78e1ffb640 1 -- 172.21.15.64:0/162364673 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78dc08a4e0 con 0x7f78ec157770 2023-11-24T08:08:30.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.533+0000 7f78f17a5640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78d006b3e0 0x7f78d006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.534+0000 7f78f17a5640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78d006b3e0 0x7f78d006d8a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f78e800a910 tx=0x7f78e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.536+0000 7f78e1ffb640 1 -- 172.21.15.64:0/162364673 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78dc054db0 con 0x7f78ec157770 2023-11-24T08:08:30.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.728+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78ec157b50 con 0x7f78d006b3e0 2023-11-24T08:08:30.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.751+0000 7f78e1ffb640 1 -- 172.21.15.64:0/162364673 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286994 (secure 0 0 0) 0x7f78e800a680 con 0x7f78d006b3e0 2023-11-24T08:08:30.757 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:30.759 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:08:30.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78d006b3e0 msgr2=0x7f78d006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78d006b3e0 0x7f78d006d8a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f78e800a910 tx=0x7f78e8008040 comp rx=0 tx=0).stop 2023-11-24T08:08:30.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec157770 msgr2=0x7f78ec15a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec157770 0x7f78ec15a400 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00d5f0 tx=0x7f78dc00dab0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 shutdown_connections 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78d006b3e0 0x7f78d006d8a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78ec15ddb0 0x7f78ec0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78ec157770 0x7f78ec15a400 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 --2- 172.21.15.64:0/162364673 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78ec06d080 0x7f78ec14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.758+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 >> 172.21.15.64:0/162364673 conn(0x7f78ec06a860 msgr2=0x7f78ec0b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.759+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 shutdown_connections 2023-11-24T08:08:30.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:30.759+0000 7f78f27a7640 1 -- 172.21.15.64:0/162364673 wait complete. 2023-11-24T08:08:30.782 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2023-11-24T08:08:30.114205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259044,"kb_used_data":9572,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715676,"statfs":{"total":773094113280,"available":772828852224,"internally_reserved":0,"allocated":9801728,"data_stored":4858129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.017720"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481944+0000","last_change":"2023-11-24T08:08:24.165952+0000","last_active":"2023-11-24T08:08:26.481944+0000","last_peered":"2023-11-24T08:08:26.481944+0000","last_clean":"2023-11-24T08:08:26.481944+0000","last_became_active":"2023-11-24T08:08:24.165644+0000","last_became_peered":"2023-11-24T08:08:24.165644+0000","last_unstale":"2023-11-24T08:08:26.481944+0000","last_undegraded":"2023-11-24T08:08:26.481944+0000","last_fullsized":"2023-11-24T08:08:26.481944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:56.571469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143998+0000","last_change":"2023-11-24T08:08:24.171684+0000","last_active":"2023-11-24T08:08:25.143998+0000","last_peered":"2023-11-24T08:08:25.143998+0000","last_clean":"2023-11-24T08:08:25.143998+0000","last_became_active":"2023-11-24T08:08:24.171424+0000","last_became_peered":"2023-11-24T08:08:24.171424+0000","last_unstale":"2023-11-24T08:08:25.143998+0000","last_undegraded":"2023-11-24T08:08:25.143998+0000","last_fullsized":"2023-11-24T08:08:25.143998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:04.722906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147330+0000","last_change":"2023-11-24T08:08:24.155196+0000","last_active":"2023-11-24T08:08:25.147330+0000","last_peered":"2023-11-24T08:08:25.147330+0000","last_clean":"2023-11-24T08:08:25.147330+0000","last_became_active":"2023-11-24T08:08:24.154911+0000","last_became_peered":"2023-11-24T08:08:24.154911+0000","last_unstale":"2023-11-24T08:08:25.147330+0000","last_undegraded":"2023-11-24T08:08:25.147330+0000","last_fullsized":"2023-11-24T08:08:25.147330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:24.342324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479948+0000","last_change":"2023-11-24T08:08:24.157950+0000","last_active":"2023-11-24T08:08:26.479948+0000","last_peered":"2023-11-24T08:08:26.479948+0000","last_clean":"2023-11-24T08:08:26.479948+0000","last_became_active":"2023-11-24T08:08:24.156255+0000","last_became_peered":"2023-11-24T08:08:24.156255+0000","last_unstale":"2023-11-24T08:08:26.479948+0000","last_undegraded":"2023-11-24T08:08:26.479948+0000","last_fullsized":"2023-11-24T08:08:26.479948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:03.428547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381159+0000","last_change":"2023-11-24T08:08:24.156314+0000","last_active":"2023-11-24T08:08:25.381159+0000","last_peered":"2023-11-24T08:08:25.381159+0000","last_clean":"2023-11-24T08:08:25.381159+0000","last_became_active":"2023-11-24T08:08:24.155860+0000","last_became_peered":"2023-11-24T08:08:24.155860+0000","last_unstale":"2023-11-24T08:08:25.381159+0000","last_undegraded":"2023-11-24T08:08:25.381159+0000","last_fullsized":"2023-11-24T08:08:25.381159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:48.755951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.155236+0000","last_change":"2023-11-24T08:08:24.157969+0000","last_active":"2023-11-24T08:08:25.155236+0000","last_peered":"2023-11-24T08:08:25.155236+0000","last_clean":"2023-11-24T08:08:25.155236+0000","last_became_active":"2023-11-24T08:08:24.157703+0000","last_became_peered":"2023-11-24T08:08:24.157703+0000","last_unstale":"2023-11-24T08:08:25.155236+0000","last_undegraded":"2023-11-24T08:08:25.155236+0000","last_fullsized":"2023-11-24T08:08:25.155236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:25.927000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479397+0000","last_change":"2023-11-24T08:08:24.144658+0000","last_active":"2023-11-24T08:08:26.479397+0000","last_peered":"2023-11-24T08:08:26.479397+0000","last_clean":"2023-11-24T08:08:26.479397+0000","last_became_active":"2023-11-24T08:08:24.143892+0000","last_became_peered":"2023-11-24T08:08:24.143892+0000","last_unstale":"2023-11-24T08:08:26.479397+0000","last_undegraded":"2023-11-24T08:08:26.479397+0000","last_fullsized":"2023-11-24T08:08:26.479397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:59.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.154774+0000","last_change":"2023-11-24T08:08:24.155734+0000","last_active":"2023-11-24T08:08:25.154774+0000","last_peered":"2023-11-24T08:08:25.154774+0000","last_clean":"2023-11-24T08:08:25.154774+0000","last_became_active":"2023-11-24T08:08:24.155360+0000","last_became_peered":"2023-11-24T08:08:24.155360+0000","last_unstale":"2023-11-24T08:08:25.154774+0000","last_undegraded":"2023-11-24T08:08:25.154774+0000","last_fullsized":"2023-11-24T08:08:25.154774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:00.707183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.155239+0000","last_change":"2023-11-24T08:08:24.160153+0000","last_active":"2023-11-24T08:08:25.155239+0000","last_peered":"2023-11-24T08:08:25.155239+0000","last_clean":"2023-11-24T08:08:25.155239+0000","last_became_active":"2023-11-24T08:08:24.159785+0000","last_became_peered":"2023-11-24T08:08:24.159785+0000","last_unstale":"2023-11-24T08:08:25.155239+0000","last_undegraded":"2023-11-24T08:08:25.155239+0000","last_fullsized":"2023-11-24T08:08:25.155239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:07.120244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.482137+0000","last_change":"2023-11-24T08:08:24.160757+0000","last_active":"2023-11-24T08:08:26.482137+0000","last_peered":"2023-11-24T08:08:26.482137+0000","last_clean":"2023-11-24T08:08:26.482137+0000","last_became_active":"2023-11-24T08:08:24.160413+0000","last_became_peered":"2023-11-24T08:08:24.160413+0000","last_unstale":"2023-11-24T08:08:26.482137+0000","last_undegraded":"2023-11-24T08:08:26.482137+0000","last_fullsized":"2023-11-24T08:08:26.482137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:07.690092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143888+0000","last_change":"2023-11-24T08:08:24.159225+0000","last_active":"2023-11-24T08:08:25.143888+0000","last_peered":"2023-11-24T08:08:25.143888+0000","last_clean":"2023-11-24T08:08:25.143888+0000","last_became_active":"2023-11-24T08:08:24.157678+0000","last_became_peered":"2023-11-24T08:08:24.157678+0000","last_unstale":"2023-11-24T08:08:25.143888+0000","last_undegraded":"2023-11-24T08:08:25.143888+0000","last_fullsized":"2023-11-24T08:08:25.143888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:44.112415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481415+0000","last_change":"2023-11-24T08:08:24.161646+0000","last_active":"2023-11-24T08:08:26.481415+0000","last_peered":"2023-11-24T08:08:26.481415+0000","last_clean":"2023-11-24T08:08:26.481415+0000","last_became_active":"2023-11-24T08:08:24.161236+0000","last_became_peered":"2023-11-24T08:08:24.161236+0000","last_unstale":"2023-11-24T08:08:26.481415+0000","last_undegraded":"2023-11-24T08:08:26.481415+0000","last_fullsized":"2023-11-24T08:08:26.481415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:53.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147198+0000","last_change":"2023-11-24T08:08:24.158083+0000","last_active":"2023-11-24T08:08:25.147198+0000","last_peered":"2023-11-24T08:08:25.147198+0000","last_clean":"2023-11-24T08:08:25.147198+0000","last_became_active":"2023-11-24T08:08:24.157605+0000","last_became_peered":"2023-11-24T08:08:24.157605+0000","last_unstale":"2023-11-24T08:08:25.147198+0000","last_undegraded":"2023-11-24T08:08:25.147198+0000","last_fullsized":"2023-11-24T08:08:25.147198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:18.389048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479804+0000","last_change":"2023-11-24T08:08:24.153686+0000","last_active":"2023-11-24T08:08:26.479804+0000","last_peered":"2023-11-24T08:08:26.479804+0000","last_clean":"2023-11-24T08:08:26.479804+0000","last_became_active":"2023-11-24T08:08:24.153258+0000","last_became_peered":"2023-11-24T08:08:24.153258+0000","last_unstale":"2023-11-24T08:08:26.479804+0000","last_undegraded":"2023-11-24T08:08:26.479804+0000","last_fullsized":"2023-11-24T08:08:26.479804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:16:40.809261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.145497+0000","last_change":"2023-11-24T08:08:24.154066+0000","last_active":"2023-11-24T08:08:25.145497+0000","last_peered":"2023-11-24T08:08:25.145497+0000","last_clean":"2023-11-24T08:08:25.145497+0000","last_became_active":"2023-11-24T08:08:24.147975+0000","last_became_peered":"2023-11-24T08:08:24.147975+0000","last_unstale":"2023-11-24T08:08:25.145497+0000","last_undegraded":"2023-11-24T08:08:25.145497+0000","last_fullsized":"2023-11-24T08:08:25.145497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:08:03.312879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.380437+0000","last_change":"2023-11-24T08:08:24.160030+0000","last_active":"2023-11-24T08:08:25.380437+0000","last_peered":"2023-11-24T08:08:25.380437+0000","last_clean":"2023-11-24T08:08:25.380437+0000","last_became_active":"2023-11-24T08:08:24.159643+0000","last_became_peered":"2023-11-24T08:08:24.159643+0000","last_unstale":"2023-11-24T08:08:25.380437+0000","last_undegraded":"2023-11-24T08:08:25.380437+0000","last_fullsized":"2023-11-24T08:08:25.380437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:44:04.154197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381322+0000","last_change":"2023-11-24T08:08:24.168692+0000","last_active":"2023-11-24T08:08:25.381322+0000","last_peered":"2023-11-24T08:08:25.381322+0000","last_clean":"2023-11-24T08:08:25.381322+0000","last_became_active":"2023-11-24T08:08:24.168371+0000","last_became_peered":"2023-11-24T08:08:24.168371+0000","last_unstale":"2023-11-24T08:08:25.381322+0000","last_undegraded":"2023-11-24T08:08:25.381322+0000","last_fullsized":"2023-11-24T08:08:25.381322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:07.436585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.480136+0000","last_change":"2023-11-24T08:08:24.162955+0000","last_active":"2023-11-24T08:08:26.480136+0000","last_peered":"2023-11-24T08:08:26.480136+0000","last_clean":"2023-11-24T08:08:26.480136+0000","last_became_active":"2023-11-24T08:08:24.160349+0000","last_became_peered":"2023-11-24T08:08:24.160349+0000","last_unstale":"2023-11-24T08:08:26.480136+0000","last_undegraded":"2023-11-24T08:08:26.480136+0000","last_fullsized":"2023-11-24T08:08:26.480136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:28.755176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.085380+0000","last_change":"2023-11-24T08:08:24.153755+0000","last_active":"2023-11-24T08:08:26.085380+0000","last_peered":"2023-11-24T08:08:26.085380+0000","last_clean":"2023-11-24T08:08:26.085380+0000","last_became_active":"2023-11-24T08:08:24.153449+0000","last_became_peered":"2023-11-24T08:08:24.153449+0000","last_unstale":"2023-11-24T08:08:26.085380+0000","last_undegraded":"2023-11-24T08:08:26.085380+0000","last_fullsized":"2023-11-24T08:08:26.085380+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:18.389048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.480151+0000","last_change":"2023-11-24T08:08:24.157609+0000","last_active":"2023-11-24T08:08:26.480151+0000","last_peered":"2023-11-24T08:08:26.480151+0000","last_clean":"2023-11-24T08:08:26.480151+0000","last_became_active":"2023-11-24T08:08:24.157259+0000","last_became_peered":"2023-11-24T08:08:24.157259+0000","last_unstale":"2023-11-24T08:08:26.480151+0000","last_undegraded":"2023-11-24T08:08:26.480151+0000","last_fullsized":"2023-11-24T08:08:26.480151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:26.636238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479749+0000","last_change":"2023-11-24T08:08:24.159323+0000","last_active":"2023-11-24T08:08:26.479749+0000","last_peered":"2023-11-24T08:08:26.479749+0000","last_clean":"2023-11-24T08:08:26.479749+0000","last_became_active":"2023-11-24T08:08:24.158210+0000","last_became_peered":"2023-11-24T08:08:24.158210+0000","last_unstale":"2023-11-24T08:08:26.479749+0000","last_undegraded":"2023-11-24T08:08:26.479749+0000","last_fullsized":"2023-11-24T08:08:26.479749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:25.760456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143478+0000","last_change":"2023-11-24T08:08:24.172898+0000","last_active":"2023-11-24T08:08:25.143478+0000","last_peered":"2023-11-24T08:08:25.143478+0000","last_clean":"2023-11-24T08:08:25.143478+0000","last_became_active":"2023-11-24T08:08:24.172404+0000","last_became_peered":"2023-11-24T08:08:24.172404+0000","last_unstale":"2023-11-24T08:08:25.143478+0000","last_undegraded":"2023-11-24T08:08:25.143478+0000","last_fullsized":"2023-11-24T08:08:25.143478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:26.636238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481363+0000","last_change":"2023-11-24T08:08:24.168086+0000","last_active":"2023-11-24T08:08:26.481363+0000","last_peered":"2023-11-24T08:08:26.481363+0000","last_clean":"2023-11-24T08:08:26.481363+0000","last_became_active":"2023-11-24T08:08:24.167760+0000","last_became_peered":"2023-11-24T08:08:24.167760+0000","last_unstale":"2023-11-24T08:08:26.481363+0000","last_undegraded":"2023-11-24T08:08:26.481363+0000","last_fullsized":"2023-11-24T08:08:26.481363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:48.755951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479418+0000","last_change":"2023-11-24T08:08:24.151462+0000","last_active":"2023-11-24T08:08:26.479418+0000","last_peered":"2023-11-24T08:08:26.479418+0000","last_clean":"2023-11-24T08:08:26.479418+0000","last_became_active":"2023-11-24T08:08:24.151171+0000","last_became_peered":"2023-11-24T08:08:24.151171+0000","last_unstale":"2023-11-24T08:08:26.479418+0000","last_undegraded":"2023-11-24T08:08:26.479418+0000","last_fullsized":"2023-11-24T08:08:26.479418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:12.841961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143595+0000","last_change":"2023-11-24T08:08:24.157638+0000","last_active":"2023-11-24T08:08:25.143595+0000","last_peered":"2023-11-24T08:08:25.143595+0000","last_clean":"2023-11-24T08:08:25.143595+0000","last_became_active":"2023-11-24T08:08:24.157323+0000","last_became_peered":"2023-11-24T08:08:24.157323+0000","last_unstale":"2023-11-24T08:08:25.143595+0000","last_undegraded":"2023-11-24T08:08:25.143595+0000","last_fullsized":"2023-11-24T08:08:25.143595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:49.519552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147444+0000","last_change":"2023-11-24T08:08:24.157286+0000","last_active":"2023-11-24T08:08:25.147444+0000","last_peered":"2023-11-24T08:08:25.147444+0000","last_clean":"2023-11-24T08:08:25.147444+0000","last_became_active":"2023-11-24T08:08:24.156644+0000","last_became_peered":"2023-11-24T08:08:24.156644+0000","last_unstale":"2023-11-24T08:08:25.147444+0000","last_undegraded":"2023-11-24T08:08:25.147444+0000","last_fullsized":"2023-11-24T08:08:25.147444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:25.200721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.380782+0000","last_change":"2023-11-24T08:08:24.165845+0000","last_active":"2023-11-24T08:08:25.380782+0000","last_peered":"2023-11-24T08:08:25.380782+0000","last_clean":"2023-11-24T08:08:25.380782+0000","last_became_active":"2023-11-24T08:08:24.165281+0000","last_became_peered":"2023-11-24T08:08:24.165281+0000","last_unstale":"2023-11-24T08:08:25.380782+0000","last_undegraded":"2023-11-24T08:08:25.380782+0000","last_fullsized":"2023-11-24T08:08:25.380782+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:07.120244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481664+0000","last_change":"2023-11-24T08:08:24.166045+0000","last_active":"2023-11-24T08:08:26.481664+0000","last_peered":"2023-11-24T08:08:26.481664+0000","last_clean":"2023-11-24T08:08:26.481664+0000","last_became_active":"2023-11-24T08:08:24.165768+0000","last_became_peered":"2023-11-24T08:08:24.165768+0000","last_unstale":"2023-11-24T08:08:26.481664+0000","last_undegraded":"2023-11-24T08:08:26.481664+0000","last_fullsized":"2023-11-24T08:08:26.481664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:58.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143386+0000","last_change":"2023-11-24T08:08:24.160321+0000","last_active":"2023-11-24T08:08:25.143386+0000","last_peered":"2023-11-24T08:08:25.143386+0000","last_clean":"2023-11-24T08:08:25.143386+0000","last_became_active":"2023-11-24T08:08:24.159480+0000","last_became_peered":"2023-11-24T08:08:24.159480+0000","last_unstale":"2023-11-24T08:08:25.143386+0000","last_undegraded":"2023-11-24T08:08:25.143386+0000","last_fullsized":"2023-11-24T08:08:25.143386+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:28.755176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143509+0000","last_change":"2023-11-24T08:08:24.172705+0000","last_active":"2023-11-24T08:08:25.143509+0000","last_peered":"2023-11-24T08:08:25.143509+0000","last_clean":"2023-11-24T08:08:25.143509+0000","last_became_active":"2023-11-24T08:08:24.172036+0000","last_became_peered":"2023-11-24T08:08:24.172036+0000","last_unstale":"2023-11-24T08:08:25.143509+0000","last_undegraded":"2023-11-24T08:08:25.143509+0000","last_fullsized":"2023-11-24T08:08:25.143509+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:26.534233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.380487+0000","last_change":"2023-11-24T08:08:24.163948+0000","last_active":"2023-11-24T08:08:25.380487+0000","last_peered":"2023-11-24T08:08:25.380487+0000","last_clean":"2023-11-24T08:08:25.380487+0000","last_became_active":"2023-11-24T08:08:24.163195+0000","last_became_peered":"2023-11-24T08:08:24.163195+0000","last_unstale":"2023-11-24T08:08:25.380487+0000","last_undegraded":"2023-11-24T08:08:25.380487+0000","last_fullsized":"2023-11-24T08:08:25.380487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:25.927000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481152+0000","last_change":"2023-11-24T08:08:24.155774+0000","last_active":"2023-11-24T08:08:26.481152+0000","last_peered":"2023-11-24T08:08:26.481152+0000","last_clean":"2023-11-24T08:08:26.481152+0000","last_became_active":"2023-11-24T08:08:24.152989+0000","last_became_peered":"2023-11-24T08:08:24.152989+0000","last_unstale":"2023-11-24T08:08:26.481152+0000","last_undegraded":"2023-11-24T08:08:26.481152+0000","last_fullsized":"2023-11-24T08:08:26.481152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:46.952196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381176+0000","last_change":"2023-11-24T08:08:24.162972+0000","last_active":"2023-11-24T08:08:25.381176+0000","last_peered":"2023-11-24T08:08:25.381176+0000","last_clean":"2023-11-24T08:08:25.381176+0000","last_became_active":"2023-11-24T08:08:24.162511+0000","last_became_peered":"2023-11-24T08:08:24.162511+0000","last_unstale":"2023-11-24T08:08:25.381176+0000","last_undegraded":"2023-11-24T08:08:25.381176+0000","last_fullsized":"2023-11-24T08:08:25.381176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:53.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.185819+0000","last_change":"2023-11-24T08:08:23.134349+0000","last_active":"2023-11-24T08:08:24.185819+0000","last_peered":"2023-11-24T08:08:24.185819+0000","last_clean":"2023-11-24T08:08:24.185819+0000","last_became_active":"2023-11-24T08:08:23.134026+0000","last_became_peered":"2023-11-24T08:08:23.134026+0000","last_unstale":"2023-11-24T08:08:24.185819+0000","last_undegraded":"2023-11-24T08:08:24.185819+0000","last_fullsized":"2023-11-24T08:08:24.185819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:44:03.138688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.482231+0000","last_change":"2023-11-24T08:08:24.163316+0000","last_active":"2023-11-24T08:08:26.482231+0000","last_peered":"2023-11-24T08:08:26.482231+0000","last_clean":"2023-11-24T08:08:26.482231+0000","last_became_active":"2023-11-24T08:08:24.162797+0000","last_became_peered":"2023-11-24T08:08:24.162797+0000","last_unstale":"2023-11-24T08:08:26.482231+0000","last_undegraded":"2023-11-24T08:08:26.482231+0000","last_fullsized":"2023-11-24T08:08:26.482231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:00.199367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.186650+0000","last_change":"2023-11-24T08:08:23.125215+0000","last_active":"2023-11-24T08:08:24.186650+0000","last_peered":"2023-11-24T08:08:24.186650+0000","last_clean":"2023-11-24T08:08:24.186650+0000","last_became_active":"2023-11-24T08:08:23.124899+0000","last_became_peered":"2023-11-24T08:08:23.124899+0000","last_unstale":"2023-11-24T08:08:24.186650+0000","last_undegraded":"2023-11-24T08:08:24.186650+0000","last_fullsized":"2023-11-24T08:08:24.186650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:44:59.691673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481802+0000","last_change":"2023-11-24T08:08:24.159889+0000","last_active":"2023-11-24T08:08:26.481802+0000","last_peered":"2023-11-24T08:08:26.481802+0000","last_clean":"2023-11-24T08:08:26.481802+0000","last_became_active":"2023-11-24T08:08:24.159591+0000","last_became_peered":"2023-11-24T08:08:24.159591+0000","last_unstale":"2023-11-24T08:08:26.481802+0000","last_undegraded":"2023-11-24T08:08:26.481802+0000","last_fullsized":"2023-11-24T08:08:26.481802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:07.436585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.186159+0000","last_change":"2023-11-24T08:08:23.132484+0000","last_active":"2023-11-24T08:08:24.186159+0000","last_peered":"2023-11-24T08:08:24.186159+0000","last_clean":"2023-11-24T08:08:24.186159+0000","last_became_active":"2023-11-24T08:08:23.132220+0000","last_became_peered":"2023-11-24T08:08:23.132220+0000","last_unstale":"2023-11-24T08:08:24.186159+0000","last_undegraded":"2023-11-24T08:08:24.186159+0000","last_fullsized":"2023-11-24T08:08:24.186159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:45.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147477+0000","last_change":"2023-11-24T08:08:24.142620+0000","last_active":"2023-11-24T08:08:25.147477+0000","last_peered":"2023-11-24T08:08:25.147477+0000","last_clean":"2023-11-24T08:08:25.147477+0000","last_became_active":"2023-11-24T08:08:24.142250+0000","last_became_peered":"2023-11-24T08:08:24.142250+0000","last_unstale":"2023-11-24T08:08:25.147477+0000","last_undegraded":"2023-11-24T08:08:25.147477+0000","last_fullsized":"2023-11-24T08:08:25.147477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:39.720328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.181025+0000","last_change":"2023-11-24T08:08:23.132020+0000","last_active":"2023-11-24T08:08:24.181025+0000","last_peered":"2023-11-24T08:08:24.181025+0000","last_clean":"2023-11-24T08:08:24.181025+0000","last_became_active":"2023-11-24T08:08:23.131713+0000","last_became_peered":"2023-11-24T08:08:23.131713+0000","last_unstale":"2023-11-24T08:08:24.181025+0000","last_undegraded":"2023-11-24T08:08:24.181025+0000","last_fullsized":"2023-11-24T08:08:24.181025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:51.802097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143693+0000","last_change":"2023-11-24T08:08:24.158190+0000","last_active":"2023-11-24T08:08:25.143693+0000","last_peered":"2023-11-24T08:08:25.143693+0000","last_clean":"2023-11-24T08:08:25.143693+0000","last_became_active":"2023-11-24T08:08:24.157811+0000","last_became_peered":"2023-11-24T08:08:24.157811+0000","last_unstale":"2023-11-24T08:08:25.143693+0000","last_undegraded":"2023-11-24T08:08:25.143693+0000","last_fullsized":"2023-11-24T08:08:25.143693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:30:05.629564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187598+0000","last_change":"2023-11-24T08:08:23.127933+0000","last_active":"2023-11-24T08:08:24.187598+0000","last_peered":"2023-11-24T08:08:24.187598+0000","last_clean":"2023-11-24T08:08:24.187598+0000","last_became_active":"2023-11-24T08:08:23.127651+0000","last_became_peered":"2023-11-24T08:08:23.127651+0000","last_unstale":"2023-11-24T08:08:24.187598+0000","last_undegraded":"2023-11-24T08:08:24.187598+0000","last_fullsized":"2023-11-24T08:08:24.187598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:52.811200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381163+0000","last_change":"2023-11-24T08:08:24.169839+0000","last_active":"2023-11-24T08:08:25.381163+0000","last_peered":"2023-11-24T08:08:25.381163+0000","last_clean":"2023-11-24T08:08:25.381163+0000","last_became_active":"2023-11-24T08:08:24.169423+0000","last_became_peered":"2023-11-24T08:08:24.169423+0000","last_unstale":"2023-11-24T08:08:25.381163+0000","last_undegraded":"2023-11-24T08:08:25.381163+0000","last_fullsized":"2023-11-24T08:08:25.381163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:46.952196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.186656+0000","last_change":"2023-11-24T08:08:23.120911+0000","last_active":"2023-11-24T08:08:24.186656+0000","last_peered":"2023-11-24T08:08:24.186656+0000","last_clean":"2023-11-24T08:08:24.186656+0000","last_became_active":"2023-11-24T08:08:23.117203+0000","last_became_peered":"2023-11-24T08:08:23.117203+0000","last_unstale":"2023-11-24T08:08:24.186656+0000","last_undegraded":"2023-11-24T08:08:24.186656+0000","last_fullsized":"2023-11-24T08:08:24.186656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:07:02.695698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.146335+0000","last_change":"2023-11-24T08:08:24.150918+0000","last_active":"2023-11-24T08:08:25.146335+0000","last_peered":"2023-11-24T08:08:25.146335+0000","last_clean":"2023-11-24T08:08:25.146335+0000","last_became_active":"2023-11-24T08:08:24.150172+0000","last_became_peered":"2023-11-24T08:08:24.150172+0000","last_unstale":"2023-11-24T08:08:25.146335+0000","last_undegraded":"2023-11-24T08:08:25.146335+0000","last_fullsized":"2023-11-24T08:08:25.146335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:27.063688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.142397+0000","last_change":"2023-11-24T08:08:23.124305+0000","last_active":"2023-11-24T08:08:24.142397+0000","last_peered":"2023-11-24T08:08:24.142397+0000","last_clean":"2023-11-24T08:08:24.142397+0000","last_became_active":"2023-11-24T08:08:23.123993+0000","last_became_peered":"2023-11-24T08:08:23.123993+0000","last_unstale":"2023-11-24T08:08:24.142397+0000","last_undegraded":"2023-11-24T08:08:24.142397+0000","last_fullsized":"2023-11-24T08:08:24.142397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:42:41.890853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.145569+0000","last_change":"2023-11-24T08:08:24.148229+0000","last_active":"2023-11-24T08:08:25.145569+0000","last_peered":"2023-11-24T08:08:25.145569+0000","last_clean":"2023-11-24T08:08:25.145569+0000","last_became_active":"2023-11-24T08:08:24.147640+0000","last_became_peered":"2023-11-24T08:08:24.147640+0000","last_unstale":"2023-11-24T08:08:25.145569+0000","last_undegraded":"2023-11-24T08:08:25.145569+0000","last_fullsized":"2023-11-24T08:08:25.145569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:24.342324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.144498+0000","last_change":"2023-11-24T08:08:23.118333+0000","last_active":"2023-11-24T08:08:24.144498+0000","last_peered":"2023-11-24T08:08:24.144498+0000","last_clean":"2023-11-24T08:08:24.144498+0000","last_became_active":"2023-11-24T08:08:23.116900+0000","last_became_peered":"2023-11-24T08:08:23.116900+0000","last_unstale":"2023-11-24T08:08:24.144498+0000","last_undegraded":"2023-11-24T08:08:24.144498+0000","last_fullsized":"2023-11-24T08:08:24.144498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:56:58.919609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.145965+0000","last_change":"2023-11-24T08:08:24.159290+0000","last_active":"2023-11-24T08:08:25.145965+0000","last_peered":"2023-11-24T08:08:25.145965+0000","last_clean":"2023-11-24T08:08:25.145965+0000","last_became_active":"2023-11-24T08:08:24.159027+0000","last_became_peered":"2023-11-24T08:08:24.159027+0000","last_unstale":"2023-11-24T08:08:25.145965+0000","last_undegraded":"2023-11-24T08:08:25.145965+0000","last_fullsized":"2023-11-24T08:08:25.145965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:25.200721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.153958+0000","last_change":"2023-11-24T08:08:23.130844+0000","last_active":"2023-11-24T08:08:24.153958+0000","last_peered":"2023-11-24T08:08:24.153958+0000","last_clean":"2023-11-24T08:08:24.153958+0000","last_became_active":"2023-11-24T08:08:23.130493+0000","last_became_peered":"2023-11-24T08:08:23.130493+0000","last_unstale":"2023-11-24T08:08:24.153958+0000","last_undegraded":"2023-11-24T08:08:24.153958+0000","last_fullsized":"2023-11-24T08:08:24.153958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:24.911490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.155611+0000","last_change":"2023-11-24T08:08:24.156298+0000","last_active":"2023-11-24T08:08:25.155611+0000","last_peered":"2023-11-24T08:08:25.155611+0000","last_clean":"2023-11-24T08:08:25.155611+0000","last_became_active":"2023-11-24T08:08:24.156004+0000","last_became_peered":"2023-11-24T08:08:24.156004+0000","last_unstale":"2023-11-24T08:08:25.155611+0000","last_undegraded":"2023-11-24T08:08:25.155611+0000","last_fullsized":"2023-11-24T08:08:25.155611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:46.696543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.154524+0000","last_change":"2023-11-24T08:08:23.129847+0000","last_active":"2023-11-24T08:08:24.154524+0000","last_peered":"2023-11-24T08:08:24.154524+0000","last_clean":"2023-11-24T08:08:24.154524+0000","last_became_active":"2023-11-24T08:08:23.128603+0000","last_became_peered":"2023-11-24T08:08:23.128603+0000","last_unstale":"2023-11-24T08:08:24.154524+0000","last_undegraded":"2023-11-24T08:08:24.154524+0000","last_fullsized":"2023-11-24T08:08:24.154524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:45.681033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.084913+0000","last_change":"2023-11-24T08:08:24.160546+0000","last_active":"2023-11-24T08:08:26.084913+0000","last_peered":"2023-11-24T08:08:26.084913+0000","last_clean":"2023-11-24T08:08:26.084913+0000","last_became_active":"2023-11-24T08:08:24.160002+0000","last_became_peered":"2023-11-24T08:08:24.160002+0000","last_unstale":"2023-11-24T08:08:26.084913+0000","last_undegraded":"2023-11-24T08:08:26.084913+0000","last_fullsized":"2023-11-24T08:08:26.084913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:24.342324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187940+0000","last_change":"2023-11-24T08:08:23.133517+0000","last_active":"2023-11-24T08:08:24.187940+0000","last_peered":"2023-11-24T08:08:24.187940+0000","last_clean":"2023-11-24T08:08:24.187940+0000","last_became_active":"2023-11-24T08:08:23.132959+0000","last_became_peered":"2023-11-24T08:08:23.132959+0000","last_unstale":"2023-11-24T08:08:24.187940+0000","last_undegraded":"2023-11-24T08:08:24.187940+0000","last_fullsized":"2023-11-24T08:08:24.187940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:58.479137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.380985+0000","last_change":"2023-11-24T08:08:24.167931+0000","last_active":"2023-11-24T08:08:25.380985+0000","last_peered":"2023-11-24T08:08:25.380985+0000","last_clean":"2023-11-24T08:08:25.380985+0000","last_became_active":"2023-11-24T08:08:24.167708+0000","last_became_peered":"2023-11-24T08:08:24.167708+0000","last_unstale":"2023-11-24T08:08:25.380985+0000","last_undegraded":"2023-11-24T08:08:25.380985+0000","last_fullsized":"2023-11-24T08:08:25.380985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:46.696543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.189502+0000","last_change":"2023-11-24T08:08:23.126238+0000","last_active":"2023-11-24T08:08:24.189502+0000","last_peered":"2023-11-24T08:08:24.189502+0000","last_clean":"2023-11-24T08:08:24.189502+0000","last_became_active":"2023-11-24T08:08:23.125933+0000","last_became_peered":"2023-11-24T08:08:23.125933+0000","last_unstale":"2023-11-24T08:08:24.189502+0000","last_undegraded":"2023-11-24T08:08:24.189502+0000","last_fullsized":"2023-11-24T08:08:24.189502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:07:02.695698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.084541+0000","last_change":"2023-11-24T08:08:24.157485+0000","last_active":"2023-11-24T08:08:26.084541+0000","last_peered":"2023-11-24T08:08:26.084541+0000","last_clean":"2023-11-24T08:08:26.084541+0000","last_became_active":"2023-11-24T08:08:24.156777+0000","last_became_peered":"2023-11-24T08:08:24.156777+0000","last_unstale":"2023-11-24T08:08:26.084541+0000","last_undegraded":"2023-11-24T08:08:26.084541+0000","last_fullsized":"2023-11-24T08:08:26.084541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:27.063688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.188654+0000","last_change":"2023-11-24T08:08:23.132368+0000","last_active":"2023-11-24T08:08:24.188654+0000","last_peered":"2023-11-24T08:08:24.188654+0000","last_clean":"2023-11-24T08:08:24.188654+0000","last_became_active":"2023-11-24T08:08:23.132111+0000","last_became_peered":"2023-11-24T08:08:23.132111+0000","last_unstale":"2023-11-24T08:08:24.188654+0000","last_undegraded":"2023-11-24T08:08:24.188654+0000","last_fullsized":"2023-11-24T08:08:24.188654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:54.838055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147385+0000","last_change":"2023-11-24T08:08:24.157898+0000","last_active":"2023-11-24T08:08:25.147385+0000","last_peered":"2023-11-24T08:08:25.147385+0000","last_clean":"2023-11-24T08:08:25.147385+0000","last_became_active":"2023-11-24T08:08:24.153426+0000","last_became_peered":"2023-11-24T08:08:24.153426+0000","last_unstale":"2023-11-24T08:08:25.147385+0000","last_undegraded":"2023-11-24T08:08:25.147385+0000","last_fullsized":"2023-11-24T08:08:25.147385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:08:03.312879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.135634+0000","last_change":"2023-11-24T08:08:23.116189+0000","last_active":"2023-11-24T08:08:24.135634+0000","last_peered":"2023-11-24T08:08:24.135634+0000","last_clean":"2023-11-24T08:08:24.135634+0000","last_became_active":"2023-11-24T08:08:23.115181+0000","last_became_peered":"2023-11-24T08:08:23.115181+0000","last_unstale":"2023-11-24T08:08:24.135634+0000","last_undegraded":"2023-11-24T08:08:24.135634+0000","last_fullsized":"2023-11-24T08:08:24.135634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:17.373539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.144027+0000","last_change":"2023-11-24T08:08:24.167507+0000","last_active":"2023-11-24T08:08:25.144027+0000","last_peered":"2023-11-24T08:08:25.144027+0000","last_clean":"2023-11-24T08:08:25.144027+0000","last_became_active":"2023-11-24T08:08:24.167203+0000","last_became_peered":"2023-11-24T08:08:24.167203+0000","last_unstale":"2023-11-24T08:08:25.144027+0000","last_undegraded":"2023-11-24T08:08:25.144027+0000","last_fullsized":"2023-11-24T08:08:25.144027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:29.562657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.139938+0000","last_change":"2023-11-24T08:08:23.127485+0000","last_active":"2023-11-24T08:08:24.139938+0000","last_peered":"2023-11-24T08:08:24.139938+0000","last_clean":"2023-11-24T08:08:24.139938+0000","last_became_active":"2023-11-24T08:08:23.127216+0000","last_became_peered":"2023-11-24T08:08:23.127216+0000","last_unstale":"2023-11-24T08:08:24.139938+0000","last_undegraded":"2023-11-24T08:08:24.139938+0000","last_fullsized":"2023-11-24T08:08:24.139938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:01.712392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143226+0000","last_change":"2023-11-24T08:08:24.169526+0000","last_active":"2023-11-24T08:08:25.143226+0000","last_peered":"2023-11-24T08:08:25.143226+0000","last_clean":"2023-11-24T08:08:25.143226+0000","last_became_active":"2023-11-24T08:08:24.169250+0000","last_became_peered":"2023-11-24T08:08:24.169250+0000","last_unstale":"2023-11-24T08:08:25.143226+0000","last_undegraded":"2023-11-24T08:08:25.143226+0000","last_fullsized":"2023-11-24T08:08:25.143226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:03.428547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.188209+0000","last_change":"2023-11-24T08:08:23.135139+0000","last_active":"2023-11-24T08:08:24.188209+0000","last_peered":"2023-11-24T08:08:24.188209+0000","last_clean":"2023-11-24T08:08:24.188209+0000","last_became_active":"2023-11-24T08:08:23.134748+0000","last_became_peered":"2023-11-24T08:08:23.134748+0000","last_unstale":"2023-11-24T08:08:24.188209+0000","last_undegraded":"2023-11-24T08:08:24.188209+0000","last_fullsized":"2023-11-24T08:08:24.188209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:36:59.183857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147479+0000","last_change":"2023-11-24T08:08:24.159551+0000","last_active":"2023-11-24T08:08:25.147479+0000","last_peered":"2023-11-24T08:08:25.147479+0000","last_clean":"2023-11-24T08:08:25.147479+0000","last_became_active":"2023-11-24T08:08:24.158654+0000","last_became_peered":"2023-11-24T08:08:24.158654+0000","last_unstale":"2023-11-24T08:08:25.147479+0000","last_undegraded":"2023-11-24T08:08:25.147479+0000","last_fullsized":"2023-11-24T08:08:25.147479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:24:01.962288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.180990+0000","last_change":"2023-11-24T08:08:23.132828+0000","last_active":"2023-11-24T08:08:24.180990+0000","last_peered":"2023-11-24T08:08:24.180990+0000","last_clean":"2023-11-24T08:08:24.180990+0000","last_became_active":"2023-11-24T08:08:23.132502+0000","last_became_peered":"2023-11-24T08:08:23.132502+0000","last_unstale":"2023-11-24T08:08:24.180990+0000","last_undegraded":"2023-11-24T08:08:24.180990+0000","last_fullsized":"2023-11-24T08:08:24.180990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:06.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.482381+0000","last_change":"2023-11-24T08:08:24.159612+0000","last_active":"2023-11-24T08:08:26.482381+0000","last_peered":"2023-11-24T08:08:26.482381+0000","last_clean":"2023-11-24T08:08:26.482381+0000","last_became_active":"2023-11-24T08:08:24.159238+0000","last_became_peered":"2023-11-24T08:08:24.159238+0000","last_unstale":"2023-11-24T08:08:26.482381+0000","last_undegraded":"2023-11-24T08:08:26.482381+0000","last_fullsized":"2023-11-24T08:08:26.482381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:42:42.906363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.182133+0000","last_change":"2023-11-24T08:08:23.128247+0000","last_active":"2023-11-24T08:08:24.182133+0000","last_peered":"2023-11-24T08:08:24.182133+0000","last_clean":"2023-11-24T08:08:24.182133+0000","last_became_active":"2023-11-24T08:08:23.127889+0000","last_became_peered":"2023-11-24T08:08:23.127889+0000","last_unstale":"2023-11-24T08:08:24.182133+0000","last_undegraded":"2023-11-24T08:08:24.182133+0000","last_fullsized":"2023-11-24T08:08:24.182133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:31.442780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381229+0000","last_change":"2023-11-24T08:08:24.172548+0000","last_active":"2023-11-24T08:08:25.381229+0000","last_peered":"2023-11-24T08:08:25.381229+0000","last_clean":"2023-11-24T08:08:25.381229+0000","last_became_active":"2023-11-24T08:08:24.172213+0000","last_became_peered":"2023-11-24T08:08:24.172213+0000","last_unstale":"2023-11-24T08:08:25.381229+0000","last_undegraded":"2023-11-24T08:08:25.381229+0000","last_fullsized":"2023-11-24T08:08:25.381229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:03.975015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.145497+0000","last_change":"2023-11-24T08:08:23.130898+0000","last_active":"2023-11-24T08:08:24.145497+0000","last_peered":"2023-11-24T08:08:24.145497+0000","last_clean":"2023-11-24T08:08:24.145497+0000","last_became_active":"2023-11-24T08:08:23.130435+0000","last_became_peered":"2023-11-24T08:08:23.130435+0000","last_unstale":"2023-11-24T08:08:24.145497+0000","last_undegraded":"2023-11-24T08:08:24.145497+0000","last_fullsized":"2023-11-24T08:08:24.145497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:43:41.791702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479818+0000","last_change":"2023-11-24T08:08:24.153512+0000","last_active":"2023-11-24T08:08:26.479818+0000","last_peered":"2023-11-24T08:08:26.479818+0000","last_clean":"2023-11-24T08:08:26.479818+0000","last_became_active":"2023-11-24T08:08:24.152901+0000","last_became_peered":"2023-11-24T08:08:24.152901+0000","last_unstale":"2023-11-24T08:08:26.479818+0000","last_undegraded":"2023-11-24T08:08:26.479818+0000","last_fullsized":"2023-11-24T08:08:26.479818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:49.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187463+0000","last_change":"2023-11-24T08:08:23.129858+0000","last_active":"2023-11-24T08:08:24.187463+0000","last_peered":"2023-11-24T08:08:24.187463+0000","last_clean":"2023-11-24T08:08:24.187463+0000","last_became_active":"2023-11-24T08:08:23.129534+0000","last_became_peered":"2023-11-24T08:08:23.129534+0000","last_unstale":"2023-11-24T08:08:24.187463+0000","last_undegraded":"2023-11-24T08:08:24.187463+0000","last_fullsized":"2023-11-24T08:08:24.187463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:31.557899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.155208+0000","last_change":"2023-11-24T08:08:24.159215+0000","last_active":"2023-11-24T08:08:25.155208+0000","last_peered":"2023-11-24T08:08:25.155208+0000","last_clean":"2023-11-24T08:08:25.155208+0000","last_became_active":"2023-11-24T08:08:24.158496+0000","last_became_peered":"2023-11-24T08:08:24.158496+0000","last_unstale":"2023-11-24T08:08:25.155208+0000","last_undegraded":"2023-11-24T08:08:25.155208+0000","last_fullsized":"2023-11-24T08:08:25.155208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:44:04.154197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.126385+0000","last_change":"2023-11-24T08:08:23.129448+0000","last_active":"2023-11-24T08:08:24.126385+0000","last_peered":"2023-11-24T08:08:24.126385+0000","last_clean":"2023-11-24T08:08:24.126385+0000","last_became_active":"2023-11-24T08:08:23.129016+0000","last_became_peered":"2023-11-24T08:08:23.129016+0000","last_unstale":"2023-11-24T08:08:24.126385+0000","last_undegraded":"2023-11-24T08:08:24.126385+0000","last_fullsized":"2023-11-24T08:08:24.126385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:11.062051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479397+0000","last_change":"2023-11-24T08:08:24.163315+0000","last_active":"2023-11-24T08:08:26.479397+0000","last_peered":"2023-11-24T08:08:26.479397+0000","last_clean":"2023-11-24T08:08:26.479397+0000","last_became_active":"2023-11-24T08:08:24.160313+0000","last_became_peered":"2023-11-24T08:08:24.160313+0000","last_unstale":"2023-11-24T08:08:26.479397+0000","last_undegraded":"2023-11-24T08:08:26.479397+0000","last_fullsized":"2023-11-24T08:08:26.479397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:38.051974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.127144+0000","last_change":"2023-11-24T08:08:23.134387+0000","last_active":"2023-11-24T08:08:24.127144+0000","last_peered":"2023-11-24T08:08:24.127144+0000","last_clean":"2023-11-24T08:08:24.127144+0000","last_became_active":"2023-11-24T08:08:23.134082+0000","last_became_peered":"2023-11-24T08:08:23.134082+0000","last_unstale":"2023-11-24T08:08:24.127144+0000","last_undegraded":"2023-11-24T08:08:24.127144+0000","last_fullsized":"2023-11-24T08:08:24.127144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:56:58.919609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.154858+0000","last_change":"2023-11-24T08:08:24.160436+0000","last_active":"2023-11-24T08:08:25.154858+0000","last_peered":"2023-11-24T08:08:25.154858+0000","last_clean":"2023-11-24T08:08:25.154858+0000","last_became_active":"2023-11-24T08:08:24.159741+0000","last_became_peered":"2023-11-24T08:08:24.159741+0000","last_unstale":"2023-11-24T08:08:25.154858+0000","last_undegraded":"2023-11-24T08:08:25.154858+0000","last_fullsized":"2023-11-24T08:08:25.154858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:59.494647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.130897+0000","last_change":"2023-11-24T08:08:23.127111+0000","last_active":"2023-11-24T08:08:24.130897+0000","last_peered":"2023-11-24T08:08:24.130897+0000","last_clean":"2023-11-24T08:08:24.130897+0000","last_became_active":"2023-11-24T08:08:23.126828+0000","last_became_peered":"2023-11-24T08:08:23.126828+0000","last_unstale":"2023-11-24T08:08:24.130897+0000","last_undegraded":"2023-11-24T08:08:24.130897+0000","last_fullsized":"2023-11-24T08:08:24.130897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:36.749530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381438+0000","last_change":"2023-11-24T08:08:24.165565+0000","last_active":"2023-11-24T08:08:25.381438+0000","last_peered":"2023-11-24T08:08:25.381438+0000","last_clean":"2023-11-24T08:08:25.381438+0000","last_became_active":"2023-11-24T08:08:24.164800+0000","last_became_peered":"2023-11-24T08:08:24.164800+0000","last_unstale":"2023-11-24T08:08:25.381438+0000","last_undegraded":"2023-11-24T08:08:25.381438+0000","last_fullsized":"2023-11-24T08:08:25.381438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:56.571469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187833+0000","last_change":"2023-11-24T08:08:23.125635+0000","last_active":"2023-11-24T08:08:24.187833+0000","last_peered":"2023-11-24T08:08:24.187833+0000","last_clean":"2023-11-24T08:08:24.187833+0000","last_became_active":"2023-11-24T08:08:23.125335+0000","last_became_peered":"2023-11-24T08:08:23.125335+0000","last_unstale":"2023-11-24T08:08:24.187833+0000","last_undegraded":"2023-11-24T08:08:24.187833+0000","last_fullsized":"2023-11-24T08:08:24.187833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:11.826451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147616+0000","last_change":"2023-11-24T08:08:24.155244+0000","last_active":"2023-11-24T08:08:25.147616+0000","last_peered":"2023-11-24T08:08:25.147616+0000","last_clean":"2023-11-24T08:08:25.147616+0000","last_became_active":"2023-11-24T08:08:24.154864+0000","last_became_peered":"2023-11-24T08:08:24.154864+0000","last_unstale":"2023-11-24T08:08:25.147616+0000","last_undegraded":"2023-11-24T08:08:25.147616+0000","last_fullsized":"2023-11-24T08:08:25.147616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:35.561433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.144219+0000","last_change":"2023-11-24T08:08:23.125737+0000","last_active":"2023-11-24T08:08:25.144219+0000","last_peered":"2023-11-24T08:08:25.144219+0000","last_clean":"2023-11-24T08:08:25.144219+0000","last_became_active":"2023-11-24T08:08:23.125403+0000","last_became_peered":"2023-11-24T08:08:23.125403+0000","last_unstale":"2023-11-24T08:08:25.144219+0000","last_undegraded":"2023-11-24T08:08:25.144219+0000","last_fullsized":"2023-11-24T08:08:25.144219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:16:39.793751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.482176+0000","last_change":"2023-11-24T08:08:24.155128+0000","last_active":"2023-11-24T08:08:26.482176+0000","last_peered":"2023-11-24T08:08:26.482176+0000","last_clean":"2023-11-24T08:08:26.482176+0000","last_became_active":"2023-11-24T08:08:24.154712+0000","last_became_peered":"2023-11-24T08:08:24.154712+0000","last_unstale":"2023-11-24T08:08:26.482176+0000","last_undegraded":"2023-11-24T08:08:26.482176+0000","last_fullsized":"2023-11-24T08:08:26.482176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:20.381334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.153469+0000","last_change":"2023-11-24T08:08:23.134364+0000","last_active":"2023-11-24T08:08:24.153469+0000","last_peered":"2023-11-24T08:08:24.153469+0000","last_clean":"2023-11-24T08:08:24.153469+0000","last_became_active":"2023-11-24T08:08:23.134008+0000","last_became_peered":"2023-11-24T08:08:23.134008+0000","last_unstale":"2023-11-24T08:08:24.153469+0000","last_undegraded":"2023-11-24T08:08:24.153469+0000","last_fullsized":"2023-11-24T08:08:24.153469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:32.432401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381328+0000","last_change":"2023-11-24T08:08:24.167879+0000","last_active":"2023-11-24T08:08:25.381328+0000","last_peered":"2023-11-24T08:08:25.381328+0000","last_clean":"2023-11-24T08:08:25.381328+0000","last_became_active":"2023-11-24T08:08:24.167516+0000","last_became_peered":"2023-11-24T08:08:24.167516+0000","last_unstale":"2023-11-24T08:08:25.381328+0000","last_undegraded":"2023-11-24T08:08:25.381328+0000","last_fullsized":"2023-11-24T08:08:25.381328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:58.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.152650+0000","last_change":"2023-11-24T08:08:23.130703+0000","last_active":"2023-11-24T08:08:24.152650+0000","last_peered":"2023-11-24T08:08:24.152650+0000","last_clean":"2023-11-24T08:08:24.152650+0000","last_became_active":"2023-11-24T08:08:23.130442+0000","last_became_peered":"2023-11-24T08:08:23.130442+0000","last_unstale":"2023-11-24T08:08:24.152650+0000","last_undegraded":"2023-11-24T08:08:24.152650+0000","last_fullsized":"2023-11-24T08:08:24.152650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:44:59.691673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.154970+0000","last_change":"2023-11-24T08:08:24.158290+0000","last_active":"2023-11-24T08:08:25.154970+0000","last_peered":"2023-11-24T08:08:25.154970+0000","last_clean":"2023-11-24T08:08:25.154970+0000","last_became_active":"2023-11-24T08:08:24.157918+0000","last_became_peered":"2023-11-24T08:08:24.157918+0000","last_unstale":"2023-11-24T08:08:25.154970+0000","last_undegraded":"2023-11-24T08:08:25.154970+0000","last_fullsized":"2023-11-24T08:08:25.154970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:33.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.148264+0000","last_change":"2023-11-24T08:08:23.134436+0000","last_active":"2023-11-24T08:08:24.148264+0000","last_peered":"2023-11-24T08:08:24.148264+0000","last_clean":"2023-11-24T08:08:24.148264+0000","last_became_active":"2023-11-24T08:08:23.133914+0000","last_became_peered":"2023-11-24T08:08:23.133914+0000","last_unstale":"2023-11-24T08:08:24.148264+0000","last_undegraded":"2023-11-24T08:08:24.148264+0000","last_fullsized":"2023-11-24T08:08:24.148264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:18.540450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.084576+0000","last_change":"2023-11-24T08:08:24.158421+0000","last_active":"2023-11-24T08:08:26.084576+0000","last_peered":"2023-11-24T08:08:26.084576+0000","last_clean":"2023-11-24T08:08:26.084576+0000","last_became_active":"2023-11-24T08:08:24.157781+0000","last_became_peered":"2023-11-24T08:08:24.157781+0000","last_unstale":"2023-11-24T08:08:26.084576+0000","last_undegraded":"2023-11-24T08:08:26.084576+0000","last_fullsized":"2023-11-24T08:08:26.084576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:32.573408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.155636+0000","last_change":"2023-11-24T08:08:23.124236+0000","last_active":"2023-11-24T08:08:24.155636+0000","last_peered":"2023-11-24T08:08:24.155636+0000","last_clean":"2023-11-24T08:08:24.155636+0000","last_became_active":"2023-11-24T08:08:23.123917+0000","last_became_peered":"2023-11-24T08:08:23.123917+0000","last_unstale":"2023-11-24T08:08:24.155636+0000","last_undegraded":"2023-11-24T08:08:24.155636+0000","last_fullsized":"2023-11-24T08:08:24.155636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:19.350345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.146516+0000","last_change":"2023-11-24T08:08:24.152581+0000","last_active":"2023-11-24T08:08:25.146516+0000","last_peered":"2023-11-24T08:08:25.146516+0000","last_clean":"2023-11-24T08:08:25.146516+0000","last_became_active":"2023-11-24T08:08:24.151683+0000","last_became_peered":"2023-11-24T08:08:24.151683+0000","last_unstale":"2023-11-24T08:08:25.146516+0000","last_undegraded":"2023-11-24T08:08:25.146516+0000","last_fullsized":"2023-11-24T08:08:25.146516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:32.573408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.151903+0000","last_change":"2023-11-24T08:08:23.128663+0000","last_active":"2023-11-24T08:08:24.151903+0000","last_peered":"2023-11-24T08:08:24.151903+0000","last_clean":"2023-11-24T08:08:24.151903+0000","last_became_active":"2023-11-24T08:08:23.128380+0000","last_became_peered":"2023-11-24T08:08:23.128380+0000","last_unstale":"2023-11-24T08:08:24.151903+0000","last_undegraded":"2023-11-24T08:08:24.151903+0000","last_fullsized":"2023-11-24T08:08:24.151903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:28.883402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481449+0000","last_change":"2023-11-24T08:08:24.158422+0000","last_active":"2023-11-24T08:08:26.481449+0000","last_peered":"2023-11-24T08:08:26.481449+0000","last_clean":"2023-11-24T08:08:26.481449+0000","last_became_active":"2023-11-24T08:08:24.156197+0000","last_became_peered":"2023-11-24T08:08:24.156197+0000","last_unstale":"2023-11-24T08:08:26.481449+0000","last_undegraded":"2023-11-24T08:08:26.481449+0000","last_fullsized":"2023-11-24T08:08:26.481449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:03.975015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.157655+0000","last_change":"2023-11-24T08:08:23.133278+0000","last_active":"2023-11-24T08:08:24.157655+0000","last_peered":"2023-11-24T08:08:24.157655+0000","last_clean":"2023-11-24T08:08:24.157655+0000","last_became_active":"2023-11-24T08:08:23.132849+0000","last_became_peered":"2023-11-24T08:08:23.132849+0000","last_unstale":"2023-11-24T08:08:24.157655+0000","last_undegraded":"2023-11-24T08:08:24.157655+0000","last_fullsized":"2023-11-24T08:08:24.157655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:28.883402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143658+0000","last_change":"2023-11-24T08:08:24.168197+0000","last_active":"2023-11-24T08:08:25.143658+0000","last_peered":"2023-11-24T08:08:25.143658+0000","last_clean":"2023-11-24T08:08:25.143658+0000","last_became_active":"2023-11-24T08:08:24.162649+0000","last_became_peered":"2023-11-24T08:08:24.162649+0000","last_unstale":"2023-11-24T08:08:25.143658+0000","last_undegraded":"2023-11-24T08:08:25.143658+0000","last_fullsized":"2023-11-24T08:08:25.143658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:56.759572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187828+0000","last_change":"2023-11-24T08:08:23.136137+0000","last_active":"2023-11-24T08:08:24.187828+0000","last_peered":"2023-11-24T08:08:24.187828+0000","last_clean":"2023-11-24T08:08:24.187828+0000","last_became_active":"2023-11-24T08:08:23.135553+0000","last_became_peered":"2023-11-24T08:08:23.135553+0000","last_unstale":"2023-11-24T08:08:24.187828+0000","last_undegraded":"2023-11-24T08:08:24.187828+0000","last_fullsized":"2023-11-24T08:08:24.187828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:45.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.129735+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:24.129735+0000","last_peered":"2023-11-24T08:08:24.129735+0000","last_clean":"2023-11-24T08:08:24.129735+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:24.129735+0000","last_undegraded":"2023-11-24T08:08:24.129735+0000","last_fullsized":"2023-11-24T08:08:24.129735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1049637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.995,"15min":1.0009999999999999},"min":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.516,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.046,"15min":1.079},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.403,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.3320000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.747,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6799999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.276},{"interface":"front","average":{"1min":1.232,"5min":1.3069999999999999,"15min":1.3720000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.712,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.117}]},{"osd":2,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.875},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.268,"15min":1.3029999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.5}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.26,"15min":1.3129999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8859999999999999,"5min":2.524,"15min":2.524},"last":1.081},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3959999999999999,"15min":1.498},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7889999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.585}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.391,"15min":1.5109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8520000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.323,"15min":1.365},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6779999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4510000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.5189999999999999,"15min":1.599},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.403},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9750000000000001,"5min":2.944,"15min":2.944},"last":1.538}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.631,"15min":1.7350000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.915,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.607},{"interface":"front","average":{"1min":1.359,"5min":1.5389999999999999,"15min":1.643},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.962,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.22}]}]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1081344,"data_stored":459893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.319},"min":{"1min":0.73099999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.6579999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.262,"15min":1.335},"min":{"1min":0.81899999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.3380000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.63300000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.593,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2270000000000001,"15min":1.3029999999999999},"min":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7090000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.204,"15min":1.2869999999999999},"min":{"1min":0.86699999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.825,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3260000000000001,"15min":1.476},"min":{"1min":0.84999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5169999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.91700000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3680000000000001,"15min":1.4370000000000001},"min":{"1min":1.0980000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.8089999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.45,"15min":1.492},"min":{"1min":1.024,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.821,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.4219999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.526,"15min":1.724},"min":{"1min":0.995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.627,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.1279999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9299999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.5760000000000001,"15min":1.778},"min":{"1min":0.84399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.784,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.298},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.714,"15min":1.9179999999999999},"min":{"1min":1.3640000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8660000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.5700000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.599,"15min":1.6759999999999999},"min":{"1min":1.175,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0190000000000001,"5min":2.98,"15min":2.98},"last":1.72},{"interface":"front","average":{"1min":1.536,"5min":1.5800000000000001,"15min":1.6379999999999999},"min":{"1min":1.0549999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9810000000000001,"5min":2.29,"15min":2.29},"last":1.605}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1085440,"data_stored":460833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0129999999999999,"15min":1.0489999999999999},"min":{"1min":0.68899999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6419999999999999,"5min":1.764,"15min":1.764},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.016},"min":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5369999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.80000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9530000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.1000000000000001,"15min":1.0580000000000001},"min":{"1min":0.78600000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.589,"5min":1.635,"15min":1.635},"last":1.208}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.214,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.252,"15min":1.224},"min":{"1min":0.81100000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.2690000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.381,"15min":1.4259999999999999},"min":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.7400000000000002,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0740000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.302,"15min":1.304},"min":{"1min":0.67700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.885,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.448,"15min":1.4239999999999999},"min":{"1min":0.91700000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.6459999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.4790000000000001,"15min":1.48},"min":{"1min":0.621,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.502,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":0.76500000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.321}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.55,"15min":1.538},"min":{"1min":0.78900000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.9370000000000003,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.74},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.5640000000000001,"15min":1.538},"min":{"1min":0.74299999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.323,"5min":2.859,"15min":2.859},"last":1.4379999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1069056,"data_stored":459199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.034,"15min":1.04},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.534,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.163},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.68500000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.617,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0700000000000001,"15min":1.081},"min":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5660000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1970000000000001,"15min":1.2330000000000001},"min":{"1min":0.72499999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5760000000000001,"5min":1.903,"15min":1.903},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1910000000000001,"15min":1.2010000000000001},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7390000000000001,"5min":1.823,"15min":1.823},"last":1.006},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.248},"min":{"1min":0.72999999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.534,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.248}]},{"osd":3,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.278,"15min":1.296},"min":{"1min":0.76300000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7829999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.286,"15min":1.306},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.478,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.361}]},{"osd":5,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.306,"15min":1.304},"min":{"1min":1.095,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8120000000000001,"5min":1.944,"15min":1.944},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.496,"5min":1.365,"15min":1.367},"min":{"1min":1.0960000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0049999999999999,"5min":2.012,"15min":2.012},"last":1.5800000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.385,"15min":1.371},"min":{"1min":1.0229999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.736,"5min":2.105,"15min":2.105},"last":1.512},{"interface":"front","average":{"1min":1.615,"5min":1.508,"15min":1.514},"min":{"1min":1.306,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.798}]},{"osd":7,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.452,"15min":1.4370000000000001},"min":{"1min":1.1599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0379999999999998,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":1.2430000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.827,"5min":2.722,"15min":2.722},"last":1.6060000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1081344,"data_stored":460811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.3560000000000001,"5min":1.585,"15min":1.585},"last":1.054},{"interface":"front","average":{"1min":0.87,"5min":0.90500000000000003,"15min":0.94199999999999995},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.488,"5min":1.51,"15min":1.51},"last":0.99199999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.01,"15min":1.129},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4299999999999999,"5min":3.246,"15min":3.246},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.97999999999999998,"15min":0.96899999999999997},"min":{"1min":0.51400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.2909999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1140000000000001,"15min":1.1739999999999999},"min":{"1min":0.63800000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.988,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.1120000000000001,"15min":1.1539999999999999},"min":{"1min":0.68999999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8009999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.5609999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2390000000000001,"15min":1.262},"min":{"1min":0.94999999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6950000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.325,"15min":1.3799999999999999},"min":{"1min":0.83799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8919999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.516}]},{"osd":5,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3939999999999999,"15min":1.45},"min":{"1min":0.96799999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.839,"5min":2.395,"15min":2.395},"last":1.611},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.89800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.833}]},{"osd":6,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4219999999999999,"15min":1.4319999999999999},"min":{"1min":0.95999999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0230000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.9390000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.393,"15min":1.4159999999999999},"min":{"1min":1.0269999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.6439999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.417,"15min":1.4419999999999999},"min":{"1min":0.98199999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.5029999999999999,"15min":1.573},"min":{"1min":1.1120000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0510000000000002,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.8899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":459781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83699999999999997},"min":{"1min":0.44500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.369,"5min":1.524,"15min":1.524},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95799999999999996,"15min":1.01},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.694,"5min":2.133,"15min":2.133},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.087,"15min":1.2070000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4359999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.872},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0569999999999999},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.875,"5min":1.998,"15min":1.998},"last":0.751}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.258,"15min":1.3160000000000001},"min":{"1min":0.63700000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.722,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.1220000000000001,"15min":1.1859999999999999},"min":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.9350000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.1200000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.248,"15min":1.321},"min":{"1min":0.82799999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.843,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.411,"15min":1.4930000000000001},"min":{"1min":0.75900000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8280000000000001,"5min":3.161,"15min":3.161},"last":1.1990000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.325,"15min":1.387},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.1629999999999998,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.355},{"interface":"front","average":{"1min":1.266,"5min":1.3200000000000001,"15min":1.3999999999999999},"min":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6220000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.5149999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4750000000000001,"15min":1.579},"min":{"1min":0.73299999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8420000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.335},{"interface":"front","average":{"1min":1.405,"5min":1.4470000000000001,"15min":1.552},"min":{"1min":0.93100000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0510000000000002,"5min":2.702,"15min":2.702},"last":1.754}]},{"osd":7,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.504},"min":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.996,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.53,"15min":1.6319999999999999},"min":{"1min":0.81999999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.907,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.6359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1081344,"data_stored":459393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.88800000000000001},"min":{"1min":0.53900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.95599999999999996,"15min":1.042},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6459999999999999,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94599999999999995,"15min":1.018},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.724,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1399999999999999,"15min":1.216},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8959999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1990000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.395},{"interface":"front","average":{"1min":1.01,"5min":1.127,"15min":1.2150000000000001},"min":{"1min":0.60399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.0899999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.157}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.198,"15min":1.24},"min":{"1min":0.81799999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.645,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.379,"15min":1.4410000000000001},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7909999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.96999999999999997}]},{"osd":5,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.393,"15min":1.446},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.8740000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.415,"15min":1.466},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.855,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":0.877}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.4059999999999999,"15min":1.45},"min":{"1min":0.92900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9630000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.3939999999999999,"15min":1.466},"min":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.669,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.2230000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.48,"15min":1.548},"min":{"1min":1.0940000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6739999999999999,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.536,"15min":1.5840000000000001},"min":{"1min":1.0209999999999999,"5min":0.872,"15min":0.872},"max":{"1min":2.335,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.423}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1662976,"data_stored":1048582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.876,"15min":0.88},"min":{"1min":0.504,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2110000000000001,"5min":3.423,"15min":3.423},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.997,"15min":0.95399999999999996},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.661,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.081,"15min":1.0649999999999999},"min":{"1min":0.60899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5589999999999999,"5min":3.899,"15min":3.899},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.99399999999999999,"15min":0.95799999999999996},"min":{"1min":0.46899999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.3140000000000001,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.073}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.282,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.244},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.2250000000000001},"min":{"1min":0.65300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.621,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.143}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3,"15min":1.2689999999999999},"min":{"1min":0.81999999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6459999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.585},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.482,"15min":1.476},"min":{"1min":0.99199999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8959999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.4910000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4510000000000001,"15min":1.393},"min":{"1min":0.96599999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4550000000000001,"15min":1.4379999999999999},"min":{"1min":0.80000000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.0110000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.4330000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.399,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.786,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":0.84899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9359999999999999,"5min":3.879,"15min":3.879},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.54,"15min":1.54},"min":{"1min":0.83499999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":3.823,"15min":3.823},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.5049999999999999,"15min":1.492},"min":{"1min":0.88300000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8740000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.371}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:33.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:34.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.024+0000 7f32b807c640 1 Processor -- start 2023-11-24T08:08:34.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.024+0000 7f32b807c640 1 -- start start 2023-11-24T08:08:34.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.026+0000 7f32b807c640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f32b0070780 0x7f32b006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.026+0000 7f32b807c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0070cd0 0x7f32b006da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.026+0000 7f32b807c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b0071cd0 con 0x7f32b0160970 2023-11-24T08:08:34.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.026+0000 7f32b807c640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b00b2120 con 0x7f32b0070780 2023-11-24T08:08:34.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.026+0000 7f32b807c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00b2400 con 0x7f32b0070cd0 2023-11-24T08:08:34.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.027+0000 7f32b5617640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f32b0070780 0x7f32b006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.027+0000 7f32b6619640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0070cd0 0x7f32b006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.027+0000 7f32b5e18640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f32b0160970 0x7f32b00b2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:41754/0 (socket says 172.21.15.64:41754) 2023-11-24T08:08:34.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.027+0000 7f32b5e18640 1 -- 172.21.15.64:0/1995867543 learned_addr learned my addr 172.21.15.64:0/1995867543 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:34.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.027+0000 7f32b6619640 1 -- 172.21.15.64:0/1995867543 >> v2:172.21.15.91:3300/0 conn(0x7f32b0070780 msgr2=0x7f32b006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:34.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.027+0000 7f32b6619640 1 --2- 172.21.15.64:0/1995867543 >> v2:172.21.15.91:3300/0 conn(0x7f32b0070780 0x7f32b006d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.027+0000 7f32b4e16640 1 -- 172.21.15.64:0/1995867543 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456698938 0 0) 0x7f32b0071cd0 con 0x7f32b0160970 2023-11-24T08:08:34.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.027+0000 7f32b6619640 1 -- 172.21.15.64:0/1995867543 >> v1:172.21.15.91:6789/0 conn(0x7f32b0160970 legacy=0x7f32b00b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.028+0000 7f32b6619640 1 -- 172.21.15.64:0/1995867543 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00b26e0 con 0x7f32b0070cd0 2023-11-24T08:08:34.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.029+0000 7f32b6619640 1 --2- 172.21.15.64:0/1995867543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0070cd0 0x7f32b006da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009d90 tx=0x7f32a802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8a4b476281c85c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.029+0000 7f32b4e16640 1 -- 172.21.15.64:0/1995867543 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a802cbc0 con 0x7f32b0070cd0 2023-11-24T08:08:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.029+0000 7f32b4e16640 1 -- 172.21.15.64:0/1995867543 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a802cd20 con 0x7f32b0070cd0 2023-11-24T08:08:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.029+0000 7f32b4e16640 1 -- 172.21.15.64:0/1995867543 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8034660 con 0x7f32b0070cd0 2023-11-24T08:08:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.030+0000 7f32b807c640 1 -- 172.21.15.64:0/1995867543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0070cd0 msgr2=0x7f32b006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.030+0000 7f32b807c640 1 --2- 172.21.15.64:0/1995867543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0070cd0 0x7f32b006da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009d90 tx=0x7f32a802be10 comp rx=0 tx=0).stop 2023-11-24T08:08:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.030+0000 7f32b807c640 1 -- 172.21.15.64:0/1995867543 shutdown_connections 2023-11-24T08:08:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.030+0000 7f32b807c640 1 --2- 172.21.15.64:0/1995867543 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0070cd0 0x7f32b006da60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.030+0000 7f32b807c640 1 --2- 172.21.15.64:0/1995867543 >> v2:172.21.15.91:3300/0 conn(0x7f32b0070780 0x7f32b006d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.030+0000 7f32b807c640 1 -- 172.21.15.64:0/1995867543 >> 172.21.15.64:0/1995867543 conn(0x7f32b00b0d80 msgr2=0x7f32b00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:34.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.030+0000 7f32b807c640 1 -- 172.21.15.64:0/1995867543 shutdown_connections 2023-11-24T08:08:34.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.030+0000 7f32b807c640 1 -- 172.21.15.64:0/1995867543 wait complete. 2023-11-24T08:08:34.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.033+0000 7f32b807c640 1 Processor -- start 2023-11-24T08:08:34.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.033+0000 7f32b807c640 1 -- start start 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.034+0000 7f32b807c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32b0070780 0x7f32b0148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.034+0000 7f32b807c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0070cd0 0x7f32b01f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b807c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0160970 0x7f32b01f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b807c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01f74c0 con 0x7f32b0070cd0 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b807c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01f6f00 con 0x7f32b0070780 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b807c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01f71e0 con 0x7f32b0160970 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5617640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0070cd0 0x7f32b01f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5617640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0070cd0 0x7f32b01f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60500/0 (socket says 172.21.15.64:60500) 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5617640 1 -- 172.21.15.64:0/3335137684 learned_addr learned my addr 172.21.15.64:0/3335137684 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5e18640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32b0070780 0x7f32b0148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b6619640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0160970 0x7f32b01f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5617640 1 -- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0160970 msgr2=0x7f32b01f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5617640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0160970 0x7f32b01f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5617640 1 -- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32b0070780 msgr2=0x7f32b0148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5617640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32b0070780 0x7f32b0148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.035+0000 7f32b5617640 1 -- 172.21.15.64:0/3335137684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a8009a70 con 0x7f32b0070cd0 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.036+0000 7f32b6619640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0160970 0x7f32b01f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.036+0000 7f32b5617640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0070cd0 0x7f32b01f3020 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f32a000e9b0 tx=0x7f32a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.036+0000 7f32a67fc640 1 -- 172.21.15.64:0/3335137684 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009670 con 0x7f32b0070cd0 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.036+0000 7f32a67fc640 1 -- 172.21.15.64:0/3335137684 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a000c9a0 con 0x7f32b0070cd0 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.036+0000 7f32a67fc640 1 -- 172.21.15.64:0/3335137684 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000cd50 con 0x7f32b0070cd0 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.036+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b0099770 con 0x7f32b0070cd0 2023-11-24T08:08:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.036+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b0148770 con 0x7f32b0070cd0 2023-11-24T08:08:34.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.038+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b0067ee0 con 0x7f32b0070cd0 2023-11-24T08:08:34.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.039+0000 7f32a67fc640 1 -- 172.21.15.64:0/3335137684 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f32a0013470 con 0x7f32b0070cd0 2023-11-24T08:08:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.040+0000 7f32a67fc640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f329406b3e0 0x7f329406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.040+0000 7f32a67fc640 1 -- 172.21.15.64:0/3335137684 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32a008a190 con 0x7f32b0070cd0 2023-11-24T08:08:34.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.040+0000 7f32b5e18640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f329406b3e0 0x7f329406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.041+0000 7f32b5e18640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f329406b3e0 0x7f329406d8a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f32ac000f60 tx=0x7f32ac007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:34.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.044+0000 7f32a67fc640 1 -- 172.21.15.64:0/3335137684 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32a0054ad0 con 0x7f32b0070cd0 2023-11-24T08:08:34.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.240+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32b01f7350 con 0x7f329406b3e0 2023-11-24T08:08:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.264+0000 7f32a67fc640 1 -- 172.21.15.64:0/3335137684 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286789 (secure 0 0 0) 0x7f32b01f7350 con 0x7f329406b3e0 2023-11-24T08:08:34.271 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:08:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f329406b3e0 msgr2=0x7f329406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f329406b3e0 0x7f329406d8a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f32ac000f60 tx=0x7f32ac007e40 comp rx=0 tx=0).stop 2023-11-24T08:08:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0070cd0 msgr2=0x7f32b01f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0070cd0 0x7f32b01f3020 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f32a000e9b0 tx=0x7f32a000ee70 comp rx=0 tx=0).stop 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 shutdown_connections 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f329406b3e0 0x7f329406d8a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f32b0160970 0x7f32b01f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0070cd0 0x7f32b01f3020 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 --2- 172.21.15.64:0/3335137684 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f32b0070780 0x7f32b0148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 >> 172.21.15.64:0/3335137684 conn(0x7f32b00b0d80 msgr2=0x7f32b00ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 shutdown_connections 2023-11-24T08:08:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.271+0000 7f32b807c640 1 -- 172.21.15.64:0/3335137684 wait complete. 2023-11-24T08:08:34.294 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2023-11-24T08:08:34.115817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259044,"kb_used_data":9572,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715676,"statfs":{"total":773094113280,"available":772828852224,"internally_reserved":0,"allocated":9801728,"data_stored":4858129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.994013"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481944+0000","last_change":"2023-11-24T08:08:24.165952+0000","last_active":"2023-11-24T08:08:26.481944+0000","last_peered":"2023-11-24T08:08:26.481944+0000","last_clean":"2023-11-24T08:08:26.481944+0000","last_became_active":"2023-11-24T08:08:24.165644+0000","last_became_peered":"2023-11-24T08:08:24.165644+0000","last_unstale":"2023-11-24T08:08:26.481944+0000","last_undegraded":"2023-11-24T08:08:26.481944+0000","last_fullsized":"2023-11-24T08:08:26.481944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:56.571469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143998+0000","last_change":"2023-11-24T08:08:24.171684+0000","last_active":"2023-11-24T08:08:25.143998+0000","last_peered":"2023-11-24T08:08:25.143998+0000","last_clean":"2023-11-24T08:08:25.143998+0000","last_became_active":"2023-11-24T08:08:24.171424+0000","last_became_peered":"2023-11-24T08:08:24.171424+0000","last_unstale":"2023-11-24T08:08:25.143998+0000","last_undegraded":"2023-11-24T08:08:25.143998+0000","last_fullsized":"2023-11-24T08:08:25.143998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:04.722906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147330+0000","last_change":"2023-11-24T08:08:24.155196+0000","last_active":"2023-11-24T08:08:25.147330+0000","last_peered":"2023-11-24T08:08:25.147330+0000","last_clean":"2023-11-24T08:08:25.147330+0000","last_became_active":"2023-11-24T08:08:24.154911+0000","last_became_peered":"2023-11-24T08:08:24.154911+0000","last_unstale":"2023-11-24T08:08:25.147330+0000","last_undegraded":"2023-11-24T08:08:25.147330+0000","last_fullsized":"2023-11-24T08:08:25.147330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:24.342324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479948+0000","last_change":"2023-11-24T08:08:24.157950+0000","last_active":"2023-11-24T08:08:26.479948+0000","last_peered":"2023-11-24T08:08:26.479948+0000","last_clean":"2023-11-24T08:08:26.479948+0000","last_became_active":"2023-11-24T08:08:24.156255+0000","last_became_peered":"2023-11-24T08:08:24.156255+0000","last_unstale":"2023-11-24T08:08:26.479948+0000","last_undegraded":"2023-11-24T08:08:26.479948+0000","last_fullsized":"2023-11-24T08:08:26.479948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:03.428547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381159+0000","last_change":"2023-11-24T08:08:24.156314+0000","last_active":"2023-11-24T08:08:25.381159+0000","last_peered":"2023-11-24T08:08:25.381159+0000","last_clean":"2023-11-24T08:08:25.381159+0000","last_became_active":"2023-11-24T08:08:24.155860+0000","last_became_peered":"2023-11-24T08:08:24.155860+0000","last_unstale":"2023-11-24T08:08:25.381159+0000","last_undegraded":"2023-11-24T08:08:25.381159+0000","last_fullsized":"2023-11-24T08:08:25.381159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:48.755951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.155236+0000","last_change":"2023-11-24T08:08:24.157969+0000","last_active":"2023-11-24T08:08:25.155236+0000","last_peered":"2023-11-24T08:08:25.155236+0000","last_clean":"2023-11-24T08:08:25.155236+0000","last_became_active":"2023-11-24T08:08:24.157703+0000","last_became_peered":"2023-11-24T08:08:24.157703+0000","last_unstale":"2023-11-24T08:08:25.155236+0000","last_undegraded":"2023-11-24T08:08:25.155236+0000","last_fullsized":"2023-11-24T08:08:25.155236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:25.927000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479397+0000","last_change":"2023-11-24T08:08:24.144658+0000","last_active":"2023-11-24T08:08:26.479397+0000","last_peered":"2023-11-24T08:08:26.479397+0000","last_clean":"2023-11-24T08:08:26.479397+0000","last_became_active":"2023-11-24T08:08:24.143892+0000","last_became_peered":"2023-11-24T08:08:24.143892+0000","last_unstale":"2023-11-24T08:08:26.479397+0000","last_undegraded":"2023-11-24T08:08:26.479397+0000","last_fullsized":"2023-11-24T08:08:26.479397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:59.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.154774+0000","last_change":"2023-11-24T08:08:24.155734+0000","last_active":"2023-11-24T08:08:25.154774+0000","last_peered":"2023-11-24T08:08:25.154774+0000","last_clean":"2023-11-24T08:08:25.154774+0000","last_became_active":"2023-11-24T08:08:24.155360+0000","last_became_peered":"2023-11-24T08:08:24.155360+0000","last_unstale":"2023-11-24T08:08:25.154774+0000","last_undegraded":"2023-11-24T08:08:25.154774+0000","last_fullsized":"2023-11-24T08:08:25.154774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:00.707183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.155239+0000","last_change":"2023-11-24T08:08:24.160153+0000","last_active":"2023-11-24T08:08:25.155239+0000","last_peered":"2023-11-24T08:08:25.155239+0000","last_clean":"2023-11-24T08:08:25.155239+0000","last_became_active":"2023-11-24T08:08:24.159785+0000","last_became_peered":"2023-11-24T08:08:24.159785+0000","last_unstale":"2023-11-24T08:08:25.155239+0000","last_undegraded":"2023-11-24T08:08:25.155239+0000","last_fullsized":"2023-11-24T08:08:25.155239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:07.120244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.482137+0000","last_change":"2023-11-24T08:08:24.160757+0000","last_active":"2023-11-24T08:08:26.482137+0000","last_peered":"2023-11-24T08:08:26.482137+0000","last_clean":"2023-11-24T08:08:26.482137+0000","last_became_active":"2023-11-24T08:08:24.160413+0000","last_became_peered":"2023-11-24T08:08:24.160413+0000","last_unstale":"2023-11-24T08:08:26.482137+0000","last_undegraded":"2023-11-24T08:08:26.482137+0000","last_fullsized":"2023-11-24T08:08:26.482137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:07.690092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143888+0000","last_change":"2023-11-24T08:08:24.159225+0000","last_active":"2023-11-24T08:08:25.143888+0000","last_peered":"2023-11-24T08:08:25.143888+0000","last_clean":"2023-11-24T08:08:25.143888+0000","last_became_active":"2023-11-24T08:08:24.157678+0000","last_became_peered":"2023-11-24T08:08:24.157678+0000","last_unstale":"2023-11-24T08:08:25.143888+0000","last_undegraded":"2023-11-24T08:08:25.143888+0000","last_fullsized":"2023-11-24T08:08:25.143888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:44.112415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481415+0000","last_change":"2023-11-24T08:08:24.161646+0000","last_active":"2023-11-24T08:08:26.481415+0000","last_peered":"2023-11-24T08:08:26.481415+0000","last_clean":"2023-11-24T08:08:26.481415+0000","last_became_active":"2023-11-24T08:08:24.161236+0000","last_became_peered":"2023-11-24T08:08:24.161236+0000","last_unstale":"2023-11-24T08:08:26.481415+0000","last_undegraded":"2023-11-24T08:08:26.481415+0000","last_fullsized":"2023-11-24T08:08:26.481415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:53.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147198+0000","last_change":"2023-11-24T08:08:24.158083+0000","last_active":"2023-11-24T08:08:25.147198+0000","last_peered":"2023-11-24T08:08:25.147198+0000","last_clean":"2023-11-24T08:08:25.147198+0000","last_became_active":"2023-11-24T08:08:24.157605+0000","last_became_peered":"2023-11-24T08:08:24.157605+0000","last_unstale":"2023-11-24T08:08:25.147198+0000","last_undegraded":"2023-11-24T08:08:25.147198+0000","last_fullsized":"2023-11-24T08:08:25.147198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:18.389048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479804+0000","last_change":"2023-11-24T08:08:24.153686+0000","last_active":"2023-11-24T08:08:26.479804+0000","last_peered":"2023-11-24T08:08:26.479804+0000","last_clean":"2023-11-24T08:08:26.479804+0000","last_became_active":"2023-11-24T08:08:24.153258+0000","last_became_peered":"2023-11-24T08:08:24.153258+0000","last_unstale":"2023-11-24T08:08:26.479804+0000","last_undegraded":"2023-11-24T08:08:26.479804+0000","last_fullsized":"2023-11-24T08:08:26.479804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:16:40.809261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.145497+0000","last_change":"2023-11-24T08:08:24.154066+0000","last_active":"2023-11-24T08:08:25.145497+0000","last_peered":"2023-11-24T08:08:25.145497+0000","last_clean":"2023-11-24T08:08:25.145497+0000","last_became_active":"2023-11-24T08:08:24.147975+0000","last_became_peered":"2023-11-24T08:08:24.147975+0000","last_unstale":"2023-11-24T08:08:25.145497+0000","last_undegraded":"2023-11-24T08:08:25.145497+0000","last_fullsized":"2023-11-24T08:08:25.145497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:08:03.312879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.380437+0000","last_change":"2023-11-24T08:08:24.160030+0000","last_active":"2023-11-24T08:08:25.380437+0000","last_peered":"2023-11-24T08:08:25.380437+0000","last_clean":"2023-11-24T08:08:25.380437+0000","last_became_active":"2023-11-24T08:08:24.159643+0000","last_became_peered":"2023-11-24T08:08:24.159643+0000","last_unstale":"2023-11-24T08:08:25.380437+0000","last_undegraded":"2023-11-24T08:08:25.380437+0000","last_fullsized":"2023-11-24T08:08:25.380437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:44:04.154197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381322+0000","last_change":"2023-11-24T08:08:24.168692+0000","last_active":"2023-11-24T08:08:25.381322+0000","last_peered":"2023-11-24T08:08:25.381322+0000","last_clean":"2023-11-24T08:08:25.381322+0000","last_became_active":"2023-11-24T08:08:24.168371+0000","last_became_peered":"2023-11-24T08:08:24.168371+0000","last_unstale":"2023-11-24T08:08:25.381322+0000","last_undegraded":"2023-11-24T08:08:25.381322+0000","last_fullsized":"2023-11-24T08:08:25.381322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:07.436585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.480136+0000","last_change":"2023-11-24T08:08:24.162955+0000","last_active":"2023-11-24T08:08:26.480136+0000","last_peered":"2023-11-24T08:08:26.480136+0000","last_clean":"2023-11-24T08:08:26.480136+0000","last_became_active":"2023-11-24T08:08:24.160349+0000","last_became_peered":"2023-11-24T08:08:24.160349+0000","last_unstale":"2023-11-24T08:08:26.480136+0000","last_undegraded":"2023-11-24T08:08:26.480136+0000","last_fullsized":"2023-11-24T08:08:26.480136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:28.755176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.085380+0000","last_change":"2023-11-24T08:08:24.153755+0000","last_active":"2023-11-24T08:08:26.085380+0000","last_peered":"2023-11-24T08:08:26.085380+0000","last_clean":"2023-11-24T08:08:26.085380+0000","last_became_active":"2023-11-24T08:08:24.153449+0000","last_became_peered":"2023-11-24T08:08:24.153449+0000","last_unstale":"2023-11-24T08:08:26.085380+0000","last_undegraded":"2023-11-24T08:08:26.085380+0000","last_fullsized":"2023-11-24T08:08:26.085380+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:18.389048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.480151+0000","last_change":"2023-11-24T08:08:24.157609+0000","last_active":"2023-11-24T08:08:26.480151+0000","last_peered":"2023-11-24T08:08:26.480151+0000","last_clean":"2023-11-24T08:08:26.480151+0000","last_became_active":"2023-11-24T08:08:24.157259+0000","last_became_peered":"2023-11-24T08:08:24.157259+0000","last_unstale":"2023-11-24T08:08:26.480151+0000","last_undegraded":"2023-11-24T08:08:26.480151+0000","last_fullsized":"2023-11-24T08:08:26.480151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:26.636238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479749+0000","last_change":"2023-11-24T08:08:24.159323+0000","last_active":"2023-11-24T08:08:26.479749+0000","last_peered":"2023-11-24T08:08:26.479749+0000","last_clean":"2023-11-24T08:08:26.479749+0000","last_became_active":"2023-11-24T08:08:24.158210+0000","last_became_peered":"2023-11-24T08:08:24.158210+0000","last_unstale":"2023-11-24T08:08:26.479749+0000","last_undegraded":"2023-11-24T08:08:26.479749+0000","last_fullsized":"2023-11-24T08:08:26.479749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:25.760456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143478+0000","last_change":"2023-11-24T08:08:24.172898+0000","last_active":"2023-11-24T08:08:25.143478+0000","last_peered":"2023-11-24T08:08:25.143478+0000","last_clean":"2023-11-24T08:08:25.143478+0000","last_became_active":"2023-11-24T08:08:24.172404+0000","last_became_peered":"2023-11-24T08:08:24.172404+0000","last_unstale":"2023-11-24T08:08:25.143478+0000","last_undegraded":"2023-11-24T08:08:25.143478+0000","last_fullsized":"2023-11-24T08:08:25.143478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:26.636238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481363+0000","last_change":"2023-11-24T08:08:24.168086+0000","last_active":"2023-11-24T08:08:26.481363+0000","last_peered":"2023-11-24T08:08:26.481363+0000","last_clean":"2023-11-24T08:08:26.481363+0000","last_became_active":"2023-11-24T08:08:24.167760+0000","last_became_peered":"2023-11-24T08:08:24.167760+0000","last_unstale":"2023-11-24T08:08:26.481363+0000","last_undegraded":"2023-11-24T08:08:26.481363+0000","last_fullsized":"2023-11-24T08:08:26.481363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:48.755951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479418+0000","last_change":"2023-11-24T08:08:24.151462+0000","last_active":"2023-11-24T08:08:26.479418+0000","last_peered":"2023-11-24T08:08:26.479418+0000","last_clean":"2023-11-24T08:08:26.479418+0000","last_became_active":"2023-11-24T08:08:24.151171+0000","last_became_peered":"2023-11-24T08:08:24.151171+0000","last_unstale":"2023-11-24T08:08:26.479418+0000","last_undegraded":"2023-11-24T08:08:26.479418+0000","last_fullsized":"2023-11-24T08:08:26.479418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:12.841961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143595+0000","last_change":"2023-11-24T08:08:24.157638+0000","last_active":"2023-11-24T08:08:25.143595+0000","last_peered":"2023-11-24T08:08:25.143595+0000","last_clean":"2023-11-24T08:08:25.143595+0000","last_became_active":"2023-11-24T08:08:24.157323+0000","last_became_peered":"2023-11-24T08:08:24.157323+0000","last_unstale":"2023-11-24T08:08:25.143595+0000","last_undegraded":"2023-11-24T08:08:25.143595+0000","last_fullsized":"2023-11-24T08:08:25.143595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:49.519552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147444+0000","last_change":"2023-11-24T08:08:24.157286+0000","last_active":"2023-11-24T08:08:25.147444+0000","last_peered":"2023-11-24T08:08:25.147444+0000","last_clean":"2023-11-24T08:08:25.147444+0000","last_became_active":"2023-11-24T08:08:24.156644+0000","last_became_peered":"2023-11-24T08:08:24.156644+0000","last_unstale":"2023-11-24T08:08:25.147444+0000","last_undegraded":"2023-11-24T08:08:25.147444+0000","last_fullsized":"2023-11-24T08:08:25.147444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:25.200721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.380782+0000","last_change":"2023-11-24T08:08:24.165845+0000","last_active":"2023-11-24T08:08:25.380782+0000","last_peered":"2023-11-24T08:08:25.380782+0000","last_clean":"2023-11-24T08:08:25.380782+0000","last_became_active":"2023-11-24T08:08:24.165281+0000","last_became_peered":"2023-11-24T08:08:24.165281+0000","last_unstale":"2023-11-24T08:08:25.380782+0000","last_undegraded":"2023-11-24T08:08:25.380782+0000","last_fullsized":"2023-11-24T08:08:25.380782+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:07.120244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481664+0000","last_change":"2023-11-24T08:08:24.166045+0000","last_active":"2023-11-24T08:08:26.481664+0000","last_peered":"2023-11-24T08:08:26.481664+0000","last_clean":"2023-11-24T08:08:26.481664+0000","last_became_active":"2023-11-24T08:08:24.165768+0000","last_became_peered":"2023-11-24T08:08:24.165768+0000","last_unstale":"2023-11-24T08:08:26.481664+0000","last_undegraded":"2023-11-24T08:08:26.481664+0000","last_fullsized":"2023-11-24T08:08:26.481664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:58.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143386+0000","last_change":"2023-11-24T08:08:24.160321+0000","last_active":"2023-11-24T08:08:25.143386+0000","last_peered":"2023-11-24T08:08:25.143386+0000","last_clean":"2023-11-24T08:08:25.143386+0000","last_became_active":"2023-11-24T08:08:24.159480+0000","last_became_peered":"2023-11-24T08:08:24.159480+0000","last_unstale":"2023-11-24T08:08:25.143386+0000","last_undegraded":"2023-11-24T08:08:25.143386+0000","last_fullsized":"2023-11-24T08:08:25.143386+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:28.755176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143509+0000","last_change":"2023-11-24T08:08:24.172705+0000","last_active":"2023-11-24T08:08:25.143509+0000","last_peered":"2023-11-24T08:08:25.143509+0000","last_clean":"2023-11-24T08:08:25.143509+0000","last_became_active":"2023-11-24T08:08:24.172036+0000","last_became_peered":"2023-11-24T08:08:24.172036+0000","last_unstale":"2023-11-24T08:08:25.143509+0000","last_undegraded":"2023-11-24T08:08:25.143509+0000","last_fullsized":"2023-11-24T08:08:25.143509+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:26.534233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.380487+0000","last_change":"2023-11-24T08:08:24.163948+0000","last_active":"2023-11-24T08:08:25.380487+0000","last_peered":"2023-11-24T08:08:25.380487+0000","last_clean":"2023-11-24T08:08:25.380487+0000","last_became_active":"2023-11-24T08:08:24.163195+0000","last_became_peered":"2023-11-24T08:08:24.163195+0000","last_unstale":"2023-11-24T08:08:25.380487+0000","last_undegraded":"2023-11-24T08:08:25.380487+0000","last_fullsized":"2023-11-24T08:08:25.380487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:25.927000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481152+0000","last_change":"2023-11-24T08:08:24.155774+0000","last_active":"2023-11-24T08:08:26.481152+0000","last_peered":"2023-11-24T08:08:26.481152+0000","last_clean":"2023-11-24T08:08:26.481152+0000","last_became_active":"2023-11-24T08:08:24.152989+0000","last_became_peered":"2023-11-24T08:08:24.152989+0000","last_unstale":"2023-11-24T08:08:26.481152+0000","last_undegraded":"2023-11-24T08:08:26.481152+0000","last_fullsized":"2023-11-24T08:08:26.481152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:46.952196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381176+0000","last_change":"2023-11-24T08:08:24.162972+0000","last_active":"2023-11-24T08:08:25.381176+0000","last_peered":"2023-11-24T08:08:25.381176+0000","last_clean":"2023-11-24T08:08:25.381176+0000","last_became_active":"2023-11-24T08:08:24.162511+0000","last_became_peered":"2023-11-24T08:08:24.162511+0000","last_unstale":"2023-11-24T08:08:25.381176+0000","last_undegraded":"2023-11-24T08:08:25.381176+0000","last_fullsized":"2023-11-24T08:08:25.381176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:26:53.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.185819+0000","last_change":"2023-11-24T08:08:23.134349+0000","last_active":"2023-11-24T08:08:24.185819+0000","last_peered":"2023-11-24T08:08:24.185819+0000","last_clean":"2023-11-24T08:08:24.185819+0000","last_became_active":"2023-11-24T08:08:23.134026+0000","last_became_peered":"2023-11-24T08:08:23.134026+0000","last_unstale":"2023-11-24T08:08:24.185819+0000","last_undegraded":"2023-11-24T08:08:24.185819+0000","last_fullsized":"2023-11-24T08:08:24.185819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:44:03.138688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.482231+0000","last_change":"2023-11-24T08:08:24.163316+0000","last_active":"2023-11-24T08:08:26.482231+0000","last_peered":"2023-11-24T08:08:26.482231+0000","last_clean":"2023-11-24T08:08:26.482231+0000","last_became_active":"2023-11-24T08:08:24.162797+0000","last_became_peered":"2023-11-24T08:08:24.162797+0000","last_unstale":"2023-11-24T08:08:26.482231+0000","last_undegraded":"2023-11-24T08:08:26.482231+0000","last_fullsized":"2023-11-24T08:08:26.482231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:00.199367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.186650+0000","last_change":"2023-11-24T08:08:23.125215+0000","last_active":"2023-11-24T08:08:24.186650+0000","last_peered":"2023-11-24T08:08:24.186650+0000","last_clean":"2023-11-24T08:08:24.186650+0000","last_became_active":"2023-11-24T08:08:23.124899+0000","last_became_peered":"2023-11-24T08:08:23.124899+0000","last_unstale":"2023-11-24T08:08:24.186650+0000","last_undegraded":"2023-11-24T08:08:24.186650+0000","last_fullsized":"2023-11-24T08:08:24.186650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:44:59.691673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481802+0000","last_change":"2023-11-24T08:08:24.159889+0000","last_active":"2023-11-24T08:08:26.481802+0000","last_peered":"2023-11-24T08:08:26.481802+0000","last_clean":"2023-11-24T08:08:26.481802+0000","last_became_active":"2023-11-24T08:08:24.159591+0000","last_became_peered":"2023-11-24T08:08:24.159591+0000","last_unstale":"2023-11-24T08:08:26.481802+0000","last_undegraded":"2023-11-24T08:08:26.481802+0000","last_fullsized":"2023-11-24T08:08:26.481802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:07.436585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.186159+0000","last_change":"2023-11-24T08:08:23.132484+0000","last_active":"2023-11-24T08:08:24.186159+0000","last_peered":"2023-11-24T08:08:24.186159+0000","last_clean":"2023-11-24T08:08:24.186159+0000","last_became_active":"2023-11-24T08:08:23.132220+0000","last_became_peered":"2023-11-24T08:08:23.132220+0000","last_unstale":"2023-11-24T08:08:24.186159+0000","last_undegraded":"2023-11-24T08:08:24.186159+0000","last_fullsized":"2023-11-24T08:08:24.186159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:45.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147477+0000","last_change":"2023-11-24T08:08:24.142620+0000","last_active":"2023-11-24T08:08:25.147477+0000","last_peered":"2023-11-24T08:08:25.147477+0000","last_clean":"2023-11-24T08:08:25.147477+0000","last_became_active":"2023-11-24T08:08:24.142250+0000","last_became_peered":"2023-11-24T08:08:24.142250+0000","last_unstale":"2023-11-24T08:08:25.147477+0000","last_undegraded":"2023-11-24T08:08:25.147477+0000","last_fullsized":"2023-11-24T08:08:25.147477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:39.720328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.181025+0000","last_change":"2023-11-24T08:08:23.132020+0000","last_active":"2023-11-24T08:08:24.181025+0000","last_peered":"2023-11-24T08:08:24.181025+0000","last_clean":"2023-11-24T08:08:24.181025+0000","last_became_active":"2023-11-24T08:08:23.131713+0000","last_became_peered":"2023-11-24T08:08:23.131713+0000","last_unstale":"2023-11-24T08:08:24.181025+0000","last_undegraded":"2023-11-24T08:08:24.181025+0000","last_fullsized":"2023-11-24T08:08:24.181025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:51.802097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143693+0000","last_change":"2023-11-24T08:08:24.158190+0000","last_active":"2023-11-24T08:08:25.143693+0000","last_peered":"2023-11-24T08:08:25.143693+0000","last_clean":"2023-11-24T08:08:25.143693+0000","last_became_active":"2023-11-24T08:08:24.157811+0000","last_became_peered":"2023-11-24T08:08:24.157811+0000","last_unstale":"2023-11-24T08:08:25.143693+0000","last_undegraded":"2023-11-24T08:08:25.143693+0000","last_fullsized":"2023-11-24T08:08:25.143693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:30:05.629564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187598+0000","last_change":"2023-11-24T08:08:23.127933+0000","last_active":"2023-11-24T08:08:24.187598+0000","last_peered":"2023-11-24T08:08:24.187598+0000","last_clean":"2023-11-24T08:08:24.187598+0000","last_became_active":"2023-11-24T08:08:23.127651+0000","last_became_peered":"2023-11-24T08:08:23.127651+0000","last_unstale":"2023-11-24T08:08:24.187598+0000","last_undegraded":"2023-11-24T08:08:24.187598+0000","last_fullsized":"2023-11-24T08:08:24.187598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:52.811200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381163+0000","last_change":"2023-11-24T08:08:24.169839+0000","last_active":"2023-11-24T08:08:25.381163+0000","last_peered":"2023-11-24T08:08:25.381163+0000","last_clean":"2023-11-24T08:08:25.381163+0000","last_became_active":"2023-11-24T08:08:24.169423+0000","last_became_peered":"2023-11-24T08:08:24.169423+0000","last_unstale":"2023-11-24T08:08:25.381163+0000","last_undegraded":"2023-11-24T08:08:25.381163+0000","last_fullsized":"2023-11-24T08:08:25.381163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:46.952196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.186656+0000","last_change":"2023-11-24T08:08:23.120911+0000","last_active":"2023-11-24T08:08:24.186656+0000","last_peered":"2023-11-24T08:08:24.186656+0000","last_clean":"2023-11-24T08:08:24.186656+0000","last_became_active":"2023-11-24T08:08:23.117203+0000","last_became_peered":"2023-11-24T08:08:23.117203+0000","last_unstale":"2023-11-24T08:08:24.186656+0000","last_undegraded":"2023-11-24T08:08:24.186656+0000","last_fullsized":"2023-11-24T08:08:24.186656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:07:02.695698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.146335+0000","last_change":"2023-11-24T08:08:24.150918+0000","last_active":"2023-11-24T08:08:25.146335+0000","last_peered":"2023-11-24T08:08:25.146335+0000","last_clean":"2023-11-24T08:08:25.146335+0000","last_became_active":"2023-11-24T08:08:24.150172+0000","last_became_peered":"2023-11-24T08:08:24.150172+0000","last_unstale":"2023-11-24T08:08:25.146335+0000","last_undegraded":"2023-11-24T08:08:25.146335+0000","last_fullsized":"2023-11-24T08:08:25.146335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:27.063688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.142397+0000","last_change":"2023-11-24T08:08:23.124305+0000","last_active":"2023-11-24T08:08:24.142397+0000","last_peered":"2023-11-24T08:08:24.142397+0000","last_clean":"2023-11-24T08:08:24.142397+0000","last_became_active":"2023-11-24T08:08:23.123993+0000","last_became_peered":"2023-11-24T08:08:23.123993+0000","last_unstale":"2023-11-24T08:08:24.142397+0000","last_undegraded":"2023-11-24T08:08:24.142397+0000","last_fullsized":"2023-11-24T08:08:24.142397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:42:41.890853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.145569+0000","last_change":"2023-11-24T08:08:24.148229+0000","last_active":"2023-11-24T08:08:25.145569+0000","last_peered":"2023-11-24T08:08:25.145569+0000","last_clean":"2023-11-24T08:08:25.145569+0000","last_became_active":"2023-11-24T08:08:24.147640+0000","last_became_peered":"2023-11-24T08:08:24.147640+0000","last_unstale":"2023-11-24T08:08:25.145569+0000","last_undegraded":"2023-11-24T08:08:25.145569+0000","last_fullsized":"2023-11-24T08:08:25.145569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:24.342324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.144498+0000","last_change":"2023-11-24T08:08:23.118333+0000","last_active":"2023-11-24T08:08:24.144498+0000","last_peered":"2023-11-24T08:08:24.144498+0000","last_clean":"2023-11-24T08:08:24.144498+0000","last_became_active":"2023-11-24T08:08:23.116900+0000","last_became_peered":"2023-11-24T08:08:23.116900+0000","last_unstale":"2023-11-24T08:08:24.144498+0000","last_undegraded":"2023-11-24T08:08:24.144498+0000","last_fullsized":"2023-11-24T08:08:24.144498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:56:58.919609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.145965+0000","last_change":"2023-11-24T08:08:24.159290+0000","last_active":"2023-11-24T08:08:25.145965+0000","last_peered":"2023-11-24T08:08:25.145965+0000","last_clean":"2023-11-24T08:08:25.145965+0000","last_became_active":"2023-11-24T08:08:24.159027+0000","last_became_peered":"2023-11-24T08:08:24.159027+0000","last_unstale":"2023-11-24T08:08:25.145965+0000","last_undegraded":"2023-11-24T08:08:25.145965+0000","last_fullsized":"2023-11-24T08:08:25.145965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:25.200721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.153958+0000","last_change":"2023-11-24T08:08:23.130844+0000","last_active":"2023-11-24T08:08:24.153958+0000","last_peered":"2023-11-24T08:08:24.153958+0000","last_clean":"2023-11-24T08:08:24.153958+0000","last_became_active":"2023-11-24T08:08:23.130493+0000","last_became_peered":"2023-11-24T08:08:23.130493+0000","last_unstale":"2023-11-24T08:08:24.153958+0000","last_undegraded":"2023-11-24T08:08:24.153958+0000","last_fullsized":"2023-11-24T08:08:24.153958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:24.911490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.155611+0000","last_change":"2023-11-24T08:08:24.156298+0000","last_active":"2023-11-24T08:08:25.155611+0000","last_peered":"2023-11-24T08:08:25.155611+0000","last_clean":"2023-11-24T08:08:25.155611+0000","last_became_active":"2023-11-24T08:08:24.156004+0000","last_became_peered":"2023-11-24T08:08:24.156004+0000","last_unstale":"2023-11-24T08:08:25.155611+0000","last_undegraded":"2023-11-24T08:08:25.155611+0000","last_fullsized":"2023-11-24T08:08:25.155611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:46.696543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.154524+0000","last_change":"2023-11-24T08:08:23.129847+0000","last_active":"2023-11-24T08:08:24.154524+0000","last_peered":"2023-11-24T08:08:24.154524+0000","last_clean":"2023-11-24T08:08:24.154524+0000","last_became_active":"2023-11-24T08:08:23.128603+0000","last_became_peered":"2023-11-24T08:08:23.128603+0000","last_unstale":"2023-11-24T08:08:24.154524+0000","last_undegraded":"2023-11-24T08:08:24.154524+0000","last_fullsized":"2023-11-24T08:08:24.154524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:45.681033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.084913+0000","last_change":"2023-11-24T08:08:24.160546+0000","last_active":"2023-11-24T08:08:26.084913+0000","last_peered":"2023-11-24T08:08:26.084913+0000","last_clean":"2023-11-24T08:08:26.084913+0000","last_became_active":"2023-11-24T08:08:24.160002+0000","last_became_peered":"2023-11-24T08:08:24.160002+0000","last_unstale":"2023-11-24T08:08:26.084913+0000","last_undegraded":"2023-11-24T08:08:26.084913+0000","last_fullsized":"2023-11-24T08:08:26.084913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:24.342324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187940+0000","last_change":"2023-11-24T08:08:23.133517+0000","last_active":"2023-11-24T08:08:24.187940+0000","last_peered":"2023-11-24T08:08:24.187940+0000","last_clean":"2023-11-24T08:08:24.187940+0000","last_became_active":"2023-11-24T08:08:23.132959+0000","last_became_peered":"2023-11-24T08:08:23.132959+0000","last_unstale":"2023-11-24T08:08:24.187940+0000","last_undegraded":"2023-11-24T08:08:24.187940+0000","last_fullsized":"2023-11-24T08:08:24.187940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:58.479137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.380985+0000","last_change":"2023-11-24T08:08:24.167931+0000","last_active":"2023-11-24T08:08:25.380985+0000","last_peered":"2023-11-24T08:08:25.380985+0000","last_clean":"2023-11-24T08:08:25.380985+0000","last_became_active":"2023-11-24T08:08:24.167708+0000","last_became_peered":"2023-11-24T08:08:24.167708+0000","last_unstale":"2023-11-24T08:08:25.380985+0000","last_undegraded":"2023-11-24T08:08:25.380985+0000","last_fullsized":"2023-11-24T08:08:25.380985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:46.696543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.189502+0000","last_change":"2023-11-24T08:08:23.126238+0000","last_active":"2023-11-24T08:08:24.189502+0000","last_peered":"2023-11-24T08:08:24.189502+0000","last_clean":"2023-11-24T08:08:24.189502+0000","last_became_active":"2023-11-24T08:08:23.125933+0000","last_became_peered":"2023-11-24T08:08:23.125933+0000","last_unstale":"2023-11-24T08:08:24.189502+0000","last_undegraded":"2023-11-24T08:08:24.189502+0000","last_fullsized":"2023-11-24T08:08:24.189502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:07:02.695698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.084541+0000","last_change":"2023-11-24T08:08:24.157485+0000","last_active":"2023-11-24T08:08:26.084541+0000","last_peered":"2023-11-24T08:08:26.084541+0000","last_clean":"2023-11-24T08:08:26.084541+0000","last_became_active":"2023-11-24T08:08:24.156777+0000","last_became_peered":"2023-11-24T08:08:24.156777+0000","last_unstale":"2023-11-24T08:08:26.084541+0000","last_undegraded":"2023-11-24T08:08:26.084541+0000","last_fullsized":"2023-11-24T08:08:26.084541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:27.063688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.188654+0000","last_change":"2023-11-24T08:08:23.132368+0000","last_active":"2023-11-24T08:08:24.188654+0000","last_peered":"2023-11-24T08:08:24.188654+0000","last_clean":"2023-11-24T08:08:24.188654+0000","last_became_active":"2023-11-24T08:08:23.132111+0000","last_became_peered":"2023-11-24T08:08:23.132111+0000","last_unstale":"2023-11-24T08:08:24.188654+0000","last_undegraded":"2023-11-24T08:08:24.188654+0000","last_fullsized":"2023-11-24T08:08:24.188654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:54.838055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147385+0000","last_change":"2023-11-24T08:08:24.157898+0000","last_active":"2023-11-24T08:08:25.147385+0000","last_peered":"2023-11-24T08:08:25.147385+0000","last_clean":"2023-11-24T08:08:25.147385+0000","last_became_active":"2023-11-24T08:08:24.153426+0000","last_became_peered":"2023-11-24T08:08:24.153426+0000","last_unstale":"2023-11-24T08:08:25.147385+0000","last_undegraded":"2023-11-24T08:08:25.147385+0000","last_fullsized":"2023-11-24T08:08:25.147385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:08:03.312879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.135634+0000","last_change":"2023-11-24T08:08:23.116189+0000","last_active":"2023-11-24T08:08:24.135634+0000","last_peered":"2023-11-24T08:08:24.135634+0000","last_clean":"2023-11-24T08:08:24.135634+0000","last_became_active":"2023-11-24T08:08:23.115181+0000","last_became_peered":"2023-11-24T08:08:23.115181+0000","last_unstale":"2023-11-24T08:08:24.135634+0000","last_undegraded":"2023-11-24T08:08:24.135634+0000","last_fullsized":"2023-11-24T08:08:24.135634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:17.373539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.144027+0000","last_change":"2023-11-24T08:08:24.167507+0000","last_active":"2023-11-24T08:08:25.144027+0000","last_peered":"2023-11-24T08:08:25.144027+0000","last_clean":"2023-11-24T08:08:25.144027+0000","last_became_active":"2023-11-24T08:08:24.167203+0000","last_became_peered":"2023-11-24T08:08:24.167203+0000","last_unstale":"2023-11-24T08:08:25.144027+0000","last_undegraded":"2023-11-24T08:08:25.144027+0000","last_fullsized":"2023-11-24T08:08:25.144027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:29.562657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.139938+0000","last_change":"2023-11-24T08:08:23.127485+0000","last_active":"2023-11-24T08:08:24.139938+0000","last_peered":"2023-11-24T08:08:24.139938+0000","last_clean":"2023-11-24T08:08:24.139938+0000","last_became_active":"2023-11-24T08:08:23.127216+0000","last_became_peered":"2023-11-24T08:08:23.127216+0000","last_unstale":"2023-11-24T08:08:24.139938+0000","last_undegraded":"2023-11-24T08:08:24.139938+0000","last_fullsized":"2023-11-24T08:08:24.139938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:48:01.712392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143226+0000","last_change":"2023-11-24T08:08:24.169526+0000","last_active":"2023-11-24T08:08:25.143226+0000","last_peered":"2023-11-24T08:08:25.143226+0000","last_clean":"2023-11-24T08:08:25.143226+0000","last_became_active":"2023-11-24T08:08:24.169250+0000","last_became_peered":"2023-11-24T08:08:24.169250+0000","last_unstale":"2023-11-24T08:08:25.143226+0000","last_undegraded":"2023-11-24T08:08:25.143226+0000","last_fullsized":"2023-11-24T08:08:25.143226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:03.428547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.188209+0000","last_change":"2023-11-24T08:08:23.135139+0000","last_active":"2023-11-24T08:08:24.188209+0000","last_peered":"2023-11-24T08:08:24.188209+0000","last_clean":"2023-11-24T08:08:24.188209+0000","last_became_active":"2023-11-24T08:08:23.134748+0000","last_became_peered":"2023-11-24T08:08:23.134748+0000","last_unstale":"2023-11-24T08:08:24.188209+0000","last_undegraded":"2023-11-24T08:08:24.188209+0000","last_fullsized":"2023-11-24T08:08:24.188209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:36:59.183857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147479+0000","last_change":"2023-11-24T08:08:24.159551+0000","last_active":"2023-11-24T08:08:25.147479+0000","last_peered":"2023-11-24T08:08:25.147479+0000","last_clean":"2023-11-24T08:08:25.147479+0000","last_became_active":"2023-11-24T08:08:24.158654+0000","last_became_peered":"2023-11-24T08:08:24.158654+0000","last_unstale":"2023-11-24T08:08:25.147479+0000","last_undegraded":"2023-11-24T08:08:25.147479+0000","last_fullsized":"2023-11-24T08:08:25.147479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:24:01.962288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.180990+0000","last_change":"2023-11-24T08:08:23.132828+0000","last_active":"2023-11-24T08:08:24.180990+0000","last_peered":"2023-11-24T08:08:24.180990+0000","last_clean":"2023-11-24T08:08:24.180990+0000","last_became_active":"2023-11-24T08:08:23.132502+0000","last_became_peered":"2023-11-24T08:08:23.132502+0000","last_unstale":"2023-11-24T08:08:24.180990+0000","last_undegraded":"2023-11-24T08:08:24.180990+0000","last_fullsized":"2023-11-24T08:08:24.180990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:06.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.482381+0000","last_change":"2023-11-24T08:08:24.159612+0000","last_active":"2023-11-24T08:08:26.482381+0000","last_peered":"2023-11-24T08:08:26.482381+0000","last_clean":"2023-11-24T08:08:26.482381+0000","last_became_active":"2023-11-24T08:08:24.159238+0000","last_became_peered":"2023-11-24T08:08:24.159238+0000","last_unstale":"2023-11-24T08:08:26.482381+0000","last_undegraded":"2023-11-24T08:08:26.482381+0000","last_fullsized":"2023-11-24T08:08:26.482381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:42:42.906363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.182133+0000","last_change":"2023-11-24T08:08:23.128247+0000","last_active":"2023-11-24T08:08:24.182133+0000","last_peered":"2023-11-24T08:08:24.182133+0000","last_clean":"2023-11-24T08:08:24.182133+0000","last_became_active":"2023-11-24T08:08:23.127889+0000","last_became_peered":"2023-11-24T08:08:23.127889+0000","last_unstale":"2023-11-24T08:08:24.182133+0000","last_undegraded":"2023-11-24T08:08:24.182133+0000","last_fullsized":"2023-11-24T08:08:24.182133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:46:31.442780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381229+0000","last_change":"2023-11-24T08:08:24.172548+0000","last_active":"2023-11-24T08:08:25.381229+0000","last_peered":"2023-11-24T08:08:25.381229+0000","last_clean":"2023-11-24T08:08:25.381229+0000","last_became_active":"2023-11-24T08:08:24.172213+0000","last_became_peered":"2023-11-24T08:08:24.172213+0000","last_unstale":"2023-11-24T08:08:25.381229+0000","last_undegraded":"2023-11-24T08:08:25.381229+0000","last_fullsized":"2023-11-24T08:08:25.381229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:03.975015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.145497+0000","last_change":"2023-11-24T08:08:23.130898+0000","last_active":"2023-11-24T08:08:24.145497+0000","last_peered":"2023-11-24T08:08:24.145497+0000","last_clean":"2023-11-24T08:08:24.145497+0000","last_became_active":"2023-11-24T08:08:23.130435+0000","last_became_peered":"2023-11-24T08:08:23.130435+0000","last_unstale":"2023-11-24T08:08:24.145497+0000","last_undegraded":"2023-11-24T08:08:24.145497+0000","last_fullsized":"2023-11-24T08:08:24.145497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:43:41.791702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479818+0000","last_change":"2023-11-24T08:08:24.153512+0000","last_active":"2023-11-24T08:08:26.479818+0000","last_peered":"2023-11-24T08:08:26.479818+0000","last_clean":"2023-11-24T08:08:26.479818+0000","last_became_active":"2023-11-24T08:08:24.152901+0000","last_became_peered":"2023-11-24T08:08:24.152901+0000","last_unstale":"2023-11-24T08:08:26.479818+0000","last_undegraded":"2023-11-24T08:08:26.479818+0000","last_fullsized":"2023-11-24T08:08:26.479818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:49.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187463+0000","last_change":"2023-11-24T08:08:23.129858+0000","last_active":"2023-11-24T08:08:24.187463+0000","last_peered":"2023-11-24T08:08:24.187463+0000","last_clean":"2023-11-24T08:08:24.187463+0000","last_became_active":"2023-11-24T08:08:23.129534+0000","last_became_peered":"2023-11-24T08:08:23.129534+0000","last_unstale":"2023-11-24T08:08:24.187463+0000","last_undegraded":"2023-11-24T08:08:24.187463+0000","last_fullsized":"2023-11-24T08:08:24.187463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:31.557899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.155208+0000","last_change":"2023-11-24T08:08:24.159215+0000","last_active":"2023-11-24T08:08:25.155208+0000","last_peered":"2023-11-24T08:08:25.155208+0000","last_clean":"2023-11-24T08:08:25.155208+0000","last_became_active":"2023-11-24T08:08:24.158496+0000","last_became_peered":"2023-11-24T08:08:24.158496+0000","last_unstale":"2023-11-24T08:08:25.155208+0000","last_undegraded":"2023-11-24T08:08:25.155208+0000","last_fullsized":"2023-11-24T08:08:25.155208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:44:04.154197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.126385+0000","last_change":"2023-11-24T08:08:23.129448+0000","last_active":"2023-11-24T08:08:24.126385+0000","last_peered":"2023-11-24T08:08:24.126385+0000","last_clean":"2023-11-24T08:08:24.126385+0000","last_became_active":"2023-11-24T08:08:23.129016+0000","last_became_peered":"2023-11-24T08:08:23.129016+0000","last_unstale":"2023-11-24T08:08:24.126385+0000","last_undegraded":"2023-11-24T08:08:24.126385+0000","last_fullsized":"2023-11-24T08:08:24.126385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:11.062051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.479397+0000","last_change":"2023-11-24T08:08:24.163315+0000","last_active":"2023-11-24T08:08:26.479397+0000","last_peered":"2023-11-24T08:08:26.479397+0000","last_clean":"2023-11-24T08:08:26.479397+0000","last_became_active":"2023-11-24T08:08:24.160313+0000","last_became_peered":"2023-11-24T08:08:24.160313+0000","last_unstale":"2023-11-24T08:08:26.479397+0000","last_undegraded":"2023-11-24T08:08:26.479397+0000","last_fullsized":"2023-11-24T08:08:26.479397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:38.051974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.127144+0000","last_change":"2023-11-24T08:08:23.134387+0000","last_active":"2023-11-24T08:08:24.127144+0000","last_peered":"2023-11-24T08:08:24.127144+0000","last_clean":"2023-11-24T08:08:24.127144+0000","last_became_active":"2023-11-24T08:08:23.134082+0000","last_became_peered":"2023-11-24T08:08:23.134082+0000","last_unstale":"2023-11-24T08:08:24.127144+0000","last_undegraded":"2023-11-24T08:08:24.127144+0000","last_fullsized":"2023-11-24T08:08:24.127144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:56:58.919609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.154858+0000","last_change":"2023-11-24T08:08:24.160436+0000","last_active":"2023-11-24T08:08:25.154858+0000","last_peered":"2023-11-24T08:08:25.154858+0000","last_clean":"2023-11-24T08:08:25.154858+0000","last_became_active":"2023-11-24T08:08:24.159741+0000","last_became_peered":"2023-11-24T08:08:24.159741+0000","last_unstale":"2023-11-24T08:08:25.154858+0000","last_undegraded":"2023-11-24T08:08:25.154858+0000","last_fullsized":"2023-11-24T08:08:25.154858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:59.494647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.130897+0000","last_change":"2023-11-24T08:08:23.127111+0000","last_active":"2023-11-24T08:08:24.130897+0000","last_peered":"2023-11-24T08:08:24.130897+0000","last_clean":"2023-11-24T08:08:24.130897+0000","last_became_active":"2023-11-24T08:08:23.126828+0000","last_became_peered":"2023-11-24T08:08:23.126828+0000","last_unstale":"2023-11-24T08:08:24.130897+0000","last_undegraded":"2023-11-24T08:08:24.130897+0000","last_fullsized":"2023-11-24T08:08:24.130897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:36.749530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381438+0000","last_change":"2023-11-24T08:08:24.165565+0000","last_active":"2023-11-24T08:08:25.381438+0000","last_peered":"2023-11-24T08:08:25.381438+0000","last_clean":"2023-11-24T08:08:25.381438+0000","last_became_active":"2023-11-24T08:08:24.164800+0000","last_became_peered":"2023-11-24T08:08:24.164800+0000","last_unstale":"2023-11-24T08:08:25.381438+0000","last_undegraded":"2023-11-24T08:08:25.381438+0000","last_fullsized":"2023-11-24T08:08:25.381438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:56.571469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187833+0000","last_change":"2023-11-24T08:08:23.125635+0000","last_active":"2023-11-24T08:08:24.187833+0000","last_peered":"2023-11-24T08:08:24.187833+0000","last_clean":"2023-11-24T08:08:24.187833+0000","last_became_active":"2023-11-24T08:08:23.125335+0000","last_became_peered":"2023-11-24T08:08:23.125335+0000","last_unstale":"2023-11-24T08:08:24.187833+0000","last_undegraded":"2023-11-24T08:08:24.187833+0000","last_fullsized":"2023-11-24T08:08:24.187833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:54:11.826451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.147616+0000","last_change":"2023-11-24T08:08:24.155244+0000","last_active":"2023-11-24T08:08:25.147616+0000","last_peered":"2023-11-24T08:08:25.147616+0000","last_clean":"2023-11-24T08:08:25.147616+0000","last_became_active":"2023-11-24T08:08:24.154864+0000","last_became_peered":"2023-11-24T08:08:24.154864+0000","last_unstale":"2023-11-24T08:08:25.147616+0000","last_undegraded":"2023-11-24T08:08:25.147616+0000","last_fullsized":"2023-11-24T08:08:25.147616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:35.561433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.144219+0000","last_change":"2023-11-24T08:08:23.125737+0000","last_active":"2023-11-24T08:08:25.144219+0000","last_peered":"2023-11-24T08:08:25.144219+0000","last_clean":"2023-11-24T08:08:25.144219+0000","last_became_active":"2023-11-24T08:08:23.125403+0000","last_became_peered":"2023-11-24T08:08:23.125403+0000","last_unstale":"2023-11-24T08:08:25.144219+0000","last_undegraded":"2023-11-24T08:08:25.144219+0000","last_fullsized":"2023-11-24T08:08:25.144219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:16:39.793751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.482176+0000","last_change":"2023-11-24T08:08:24.155128+0000","last_active":"2023-11-24T08:08:26.482176+0000","last_peered":"2023-11-24T08:08:26.482176+0000","last_clean":"2023-11-24T08:08:26.482176+0000","last_became_active":"2023-11-24T08:08:24.154712+0000","last_became_peered":"2023-11-24T08:08:24.154712+0000","last_unstale":"2023-11-24T08:08:26.482176+0000","last_undegraded":"2023-11-24T08:08:26.482176+0000","last_fullsized":"2023-11-24T08:08:26.482176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:20.381334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.153469+0000","last_change":"2023-11-24T08:08:23.134364+0000","last_active":"2023-11-24T08:08:24.153469+0000","last_peered":"2023-11-24T08:08:24.153469+0000","last_clean":"2023-11-24T08:08:24.153469+0000","last_became_active":"2023-11-24T08:08:23.134008+0000","last_became_peered":"2023-11-24T08:08:23.134008+0000","last_unstale":"2023-11-24T08:08:24.153469+0000","last_undegraded":"2023-11-24T08:08:24.153469+0000","last_fullsized":"2023-11-24T08:08:24.153469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:32.432401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.381328+0000","last_change":"2023-11-24T08:08:24.167879+0000","last_active":"2023-11-24T08:08:25.381328+0000","last_peered":"2023-11-24T08:08:25.381328+0000","last_clean":"2023-11-24T08:08:25.381328+0000","last_became_active":"2023-11-24T08:08:24.167516+0000","last_became_peered":"2023-11-24T08:08:24.167516+0000","last_unstale":"2023-11-24T08:08:25.381328+0000","last_undegraded":"2023-11-24T08:08:25.381328+0000","last_fullsized":"2023-11-24T08:08:25.381328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:58.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.152650+0000","last_change":"2023-11-24T08:08:23.130703+0000","last_active":"2023-11-24T08:08:24.152650+0000","last_peered":"2023-11-24T08:08:24.152650+0000","last_clean":"2023-11-24T08:08:24.152650+0000","last_became_active":"2023-11-24T08:08:23.130442+0000","last_became_peered":"2023-11-24T08:08:23.130442+0000","last_unstale":"2023-11-24T08:08:24.152650+0000","last_undegraded":"2023-11-24T08:08:24.152650+0000","last_fullsized":"2023-11-24T08:08:24.152650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:44:59.691673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.154970+0000","last_change":"2023-11-24T08:08:24.158290+0000","last_active":"2023-11-24T08:08:25.154970+0000","last_peered":"2023-11-24T08:08:25.154970+0000","last_clean":"2023-11-24T08:08:25.154970+0000","last_became_active":"2023-11-24T08:08:24.157918+0000","last_became_peered":"2023-11-24T08:08:24.157918+0000","last_unstale":"2023-11-24T08:08:25.154970+0000","last_undegraded":"2023-11-24T08:08:25.154970+0000","last_fullsized":"2023-11-24T08:08:25.154970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:33.447911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.148264+0000","last_change":"2023-11-24T08:08:23.134436+0000","last_active":"2023-11-24T08:08:24.148264+0000","last_peered":"2023-11-24T08:08:24.148264+0000","last_clean":"2023-11-24T08:08:24.148264+0000","last_became_active":"2023-11-24T08:08:23.133914+0000","last_became_peered":"2023-11-24T08:08:23.133914+0000","last_unstale":"2023-11-24T08:08:24.148264+0000","last_undegraded":"2023-11-24T08:08:24.148264+0000","last_fullsized":"2023-11-24T08:08:24.148264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:18.540450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.084576+0000","last_change":"2023-11-24T08:08:24.158421+0000","last_active":"2023-11-24T08:08:26.084576+0000","last_peered":"2023-11-24T08:08:26.084576+0000","last_clean":"2023-11-24T08:08:26.084576+0000","last_became_active":"2023-11-24T08:08:24.157781+0000","last_became_peered":"2023-11-24T08:08:24.157781+0000","last_unstale":"2023-11-24T08:08:26.084576+0000","last_undegraded":"2023-11-24T08:08:26.084576+0000","last_fullsized":"2023-11-24T08:08:26.084576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:32.573408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.155636+0000","last_change":"2023-11-24T08:08:23.124236+0000","last_active":"2023-11-24T08:08:24.155636+0000","last_peered":"2023-11-24T08:08:24.155636+0000","last_clean":"2023-11-24T08:08:24.155636+0000","last_became_active":"2023-11-24T08:08:23.123917+0000","last_became_peered":"2023-11-24T08:08:23.123917+0000","last_unstale":"2023-11-24T08:08:24.155636+0000","last_undegraded":"2023-11-24T08:08:24.155636+0000","last_fullsized":"2023-11-24T08:08:24.155636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:19.350345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.146516+0000","last_change":"2023-11-24T08:08:24.152581+0000","last_active":"2023-11-24T08:08:25.146516+0000","last_peered":"2023-11-24T08:08:25.146516+0000","last_clean":"2023-11-24T08:08:25.146516+0000","last_became_active":"2023-11-24T08:08:24.151683+0000","last_became_peered":"2023-11-24T08:08:24.151683+0000","last_unstale":"2023-11-24T08:08:25.146516+0000","last_undegraded":"2023-11-24T08:08:25.146516+0000","last_fullsized":"2023-11-24T08:08:25.146516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:32.573408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.151903+0000","last_change":"2023-11-24T08:08:23.128663+0000","last_active":"2023-11-24T08:08:24.151903+0000","last_peered":"2023-11-24T08:08:24.151903+0000","last_clean":"2023-11-24T08:08:24.151903+0000","last_became_active":"2023-11-24T08:08:23.128380+0000","last_became_peered":"2023-11-24T08:08:23.128380+0000","last_unstale":"2023-11-24T08:08:24.151903+0000","last_undegraded":"2023-11-24T08:08:24.151903+0000","last_fullsized":"2023-11-24T08:08:24.151903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:28.883402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:26.481449+0000","last_change":"2023-11-24T08:08:24.158422+0000","last_active":"2023-11-24T08:08:26.481449+0000","last_peered":"2023-11-24T08:08:26.481449+0000","last_clean":"2023-11-24T08:08:26.481449+0000","last_became_active":"2023-11-24T08:08:24.156197+0000","last_became_peered":"2023-11-24T08:08:24.156197+0000","last_unstale":"2023-11-24T08:08:26.481449+0000","last_undegraded":"2023-11-24T08:08:26.481449+0000","last_fullsized":"2023-11-24T08:08:26.481449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:40:03.975015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.157655+0000","last_change":"2023-11-24T08:08:23.133278+0000","last_active":"2023-11-24T08:08:24.157655+0000","last_peered":"2023-11-24T08:08:24.157655+0000","last_clean":"2023-11-24T08:08:24.157655+0000","last_became_active":"2023-11-24T08:08:23.132849+0000","last_became_peered":"2023-11-24T08:08:23.132849+0000","last_unstale":"2023-11-24T08:08:24.157655+0000","last_undegraded":"2023-11-24T08:08:24.157655+0000","last_fullsized":"2023-11-24T08:08:24.157655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:28.883402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T08:08:25.143658+0000","last_change":"2023-11-24T08:08:24.168197+0000","last_active":"2023-11-24T08:08:25.143658+0000","last_peered":"2023-11-24T08:08:25.143658+0000","last_clean":"2023-11-24T08:08:25.143658+0000","last_became_active":"2023-11-24T08:08:24.162649+0000","last_became_peered":"2023-11-24T08:08:24.162649+0000","last_unstale":"2023-11-24T08:08:25.143658+0000","last_undegraded":"2023-11-24T08:08:25.143658+0000","last_fullsized":"2023-11-24T08:08:25.143658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:23.087219+0000","last_clean_scrub_stamp":"2023-11-24T08:08:23.087219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:56.759572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.187828+0000","last_change":"2023-11-24T08:08:23.136137+0000","last_active":"2023-11-24T08:08:24.187828+0000","last_peered":"2023-11-24T08:08:24.187828+0000","last_clean":"2023-11-24T08:08:24.187828+0000","last_became_active":"2023-11-24T08:08:23.135553+0000","last_became_peered":"2023-11-24T08:08:23.135553+0000","last_unstale":"2023-11-24T08:08:24.187828+0000","last_undegraded":"2023-11-24T08:08:24.187828+0000","last_fullsized":"2023-11-24T08:08:24.187828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:22.071709+0000","last_clean_scrub_stamp":"2023-11-24T08:08:22.071709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:45.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T08:08:24.129735+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:24.129735+0000","last_peered":"2023-11-24T08:08:24.129735+0000","last_clean":"2023-11-24T08:08:24.129735+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:24.129735+0000","last_undegraded":"2023-11-24T08:08:24.129735+0000","last_fullsized":"2023-11-24T08:08:24.129735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1049637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.995,"15min":1.0009999999999999},"min":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.516,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.046,"15min":1.079},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.403,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.999}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.747,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6799999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.073},{"interface":"front","average":{"1min":1.232,"5min":1.3069999999999999,"15min":1.3720000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.712,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.355}]},{"osd":2,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.268,"15min":1.3029999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.96699999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.26,"15min":1.3129999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8859999999999999,"5min":2.524,"15min":2.524},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3959999999999999,"15min":1.498},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7889999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.391,"15min":1.5109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8520000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.129},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.323,"15min":1.365},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6779999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.3069999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.5189999999999999,"15min":1.599},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.544,"15min":1.6419999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9750000000000001,"5min":2.944,"15min":2.944},"last":1.609}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.631,"15min":1.7350000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.915,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.458},{"interface":"front","average":{"1min":1.359,"5min":1.5389999999999999,"15min":1.643},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.962,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.5529999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1081344,"data_stored":459893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.319},"min":{"1min":0.73099999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.6579999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.276},{"interface":"front","average":{"1min":1.173,"5min":1.262,"15min":1.335},"min":{"1min":0.81899999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.45}]},{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.63300000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.593,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.117},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2270000000000001,"15min":1.3029999999999999},"min":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7090000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.5189999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.204,"15min":1.2869999999999999},"min":{"1min":0.86699999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.825,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3260000000000001,"15min":1.476},"min":{"1min":0.84999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5169999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.3109999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3680000000000001,"15min":1.4370000000000001},"min":{"1min":1.0980000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.8089999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.45,"15min":1.492},"min":{"1min":1.024,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.821,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.6180000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.526,"15min":1.724},"min":{"1min":0.995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.627,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.389},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.1279999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9299999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.728}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.5760000000000001,"15min":1.778},"min":{"1min":0.84399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.784,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.218},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.714,"15min":1.9179999999999999},"min":{"1min":1.3640000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8660000000000001,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.151}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.599,"15min":1.6759999999999999},"min":{"1min":1.175,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0190000000000001,"5min":2.98,"15min":2.98},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.536,"5min":1.5800000000000001,"15min":1.6379999999999999},"min":{"1min":1.0549999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9810000000000001,"5min":2.29,"15min":2.29},"last":1.4870000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1085440,"data_stored":460833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0129999999999999,"15min":1.0489999999999999},"min":{"1min":0.68899999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6419999999999999,"5min":1.764,"15min":1.764},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.016},"min":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5369999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.5109999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.80000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9530000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.407},{"interface":"front","average":{"1min":1.149,"5min":1.1000000000000001,"15min":1.0580000000000001},"min":{"1min":0.78600000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.589,"5min":1.635,"15min":1.635},"last":1.345}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.214,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.252,"15min":1.224},"min":{"1min":0.81100000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.2690000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.3049999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.381,"15min":1.4259999999999999},"min":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.7400000000000002,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.054},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.4930000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.302,"15min":1.304},"min":{"1min":0.67700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.885,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.244},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.448,"15min":1.4239999999999999},"min":{"1min":0.91700000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":0.95399999999999996}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.4790000000000001,"15min":1.48},"min":{"1min":0.621,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":0.76500000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.98899999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.55,"15min":1.538},"min":{"1min":0.78900000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.9370000000000003,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.369},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.5640000000000001,"15min":1.538},"min":{"1min":0.74299999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.323,"5min":2.859,"15min":2.859},"last":1.544}]}]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1069056,"data_stored":459199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.034,"15min":1.04},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.534,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.68500000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.617,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.2589999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0700000000000001,"15min":1.081},"min":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5660000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.274},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1970000000000001,"15min":1.2330000000000001},"min":{"1min":0.72499999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5760000000000001,"5min":1.903,"15min":1.903},"last":1.125}]},{"osd":2,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1910000000000001,"15min":1.2010000000000001},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7390000000000001,"5min":1.823,"15min":1.823},"last":1.046},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.248},"min":{"1min":0.72999999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.534,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.2250000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.278,"15min":1.296},"min":{"1min":0.76300000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7829999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.286,"15min":1.306},"min":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.478,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.167}]},{"osd":5,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.306,"15min":1.304},"min":{"1min":1.095,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8120000000000001,"5min":1.944,"15min":1.944},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.365,"15min":1.367},"min":{"1min":1.0960000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0049999999999999,"5min":2.012,"15min":2.012},"last":1.288}]},{"osd":6,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.385,"15min":1.371},"min":{"1min":1.0229999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.736,"5min":2.105,"15min":2.105},"last":1.093},{"interface":"front","average":{"1min":1.615,"5min":1.508,"15min":1.514},"min":{"1min":1.306,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.01}]},{"osd":7,"last update":"Fri Nov 24 08:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.452,"15min":1.4370000000000001},"min":{"1min":1.1599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0379999999999998,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.151},{"interface":"front","average":{"1min":1.494,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":1.2430000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.827,"5min":2.722,"15min":2.722},"last":1.3400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1081344,"data_stored":460811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.3560000000000001,"5min":1.585,"15min":1.585},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.90500000000000003,"15min":0.94199999999999995},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.488,"5min":1.51,"15min":1.51},"last":0.65600000000000003}]},{"osd":1,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.01,"15min":1.129},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4299999999999999,"5min":3.246,"15min":3.246},"last":0.62},{"interface":"front","average":{"1min":0.995,"5min":0.97999999999999998,"15min":0.96899999999999997},"min":{"1min":0.51400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1140000000000001,"15min":1.1739999999999999},"min":{"1min":0.63800000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.988,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.75},{"interface":"front","average":{"1min":1.155,"5min":1.1120000000000001,"15min":1.1539999999999999},"min":{"1min":0.68999999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8009999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2390000000000001,"15min":1.262},"min":{"1min":0.94999999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6950000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.325,"15min":1.3799999999999999},"min":{"1min":0.83799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8919999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0369999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3939999999999999,"15min":1.45},"min":{"1min":0.96799999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.839,"5min":2.395,"15min":2.395},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.89800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.1539999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4219999999999999,"15min":1.4319999999999999},"min":{"1min":0.95999999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0230000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.208},{"interface":"front","average":{"1min":1.407,"5min":1.393,"15min":1.4159999999999999},"min":{"1min":1.0269999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.872}]},{"osd":7,"last update":"Fri Nov 24 08:07:57 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.417,"15min":1.4419999999999999},"min":{"1min":0.98199999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.242},{"interface":"front","average":{"1min":1.486,"5min":1.5029999999999999,"15min":1.573},"min":{"1min":1.1120000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0510000000000002,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.1950000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":459781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83699999999999997},"min":{"1min":0.44500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.369,"5min":1.524,"15min":1.524},"last":1.3560000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95799999999999996,"15min":1.01},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.694,"5min":2.133,"15min":2.133},"last":0.80400000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.087,"15min":1.2070000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4359999999999999,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.006},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0569999999999999},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.875,"5min":1.998,"15min":1.998},"last":0.877}]},{"osd":3,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.258,"15min":1.3160000000000001},"min":{"1min":0.63700000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.722,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.732},{"interface":"front","average":{"1min":1.095,"5min":1.1220000000000001,"15min":1.1859999999999999},"min":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.9350000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3140000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.248,"15min":1.321},"min":{"1min":0.82799999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.843,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.411,"15min":1.4930000000000001},"min":{"1min":0.75900000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8280000000000001,"5min":3.161,"15min":3.161},"last":1.518}]},{"osd":5,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.325,"15min":1.387},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.1629999999999998,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.115},{"interface":"front","average":{"1min":1.266,"5min":1.3200000000000001,"15min":1.3999999999999999},"min":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6220000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.4510000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4750000000000001,"15min":1.579},"min":{"1min":0.73299999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8420000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.405},{"interface":"front","average":{"1min":1.405,"5min":1.4470000000000001,"15min":1.552},"min":{"1min":0.93100000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0510000000000002,"5min":2.702,"15min":2.702},"last":1.4770000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.504},"min":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.996,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.53,"15min":1.6319999999999999},"min":{"1min":0.81999999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.907,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.6379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1081344,"data_stored":459393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.88800000000000001},"min":{"1min":0.53900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.95599999999999996,"15min":1.042},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6459999999999999,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0720000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94599999999999995,"15min":1.018},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.724,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.278},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1399999999999999,"15min":1.216},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8959999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.2509999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1990000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.496,"15min":0.496},"max":{"1min":1.9079999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.359},{"interface":"front","average":{"1min":1.01,"5min":1.127,"15min":1.2150000000000001},"min":{"1min":0.60399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.0899999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.3999999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.198,"15min":1.24},"min":{"1min":0.81799999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.645,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.379,"15min":1.4410000000000001},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7909999999999999,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.169}]},{"osd":5,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.393,"15min":1.446},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.8740000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.415,"15min":1.466},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.855,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.476}]},{"osd":6,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.4059999999999999,"15min":1.45},"min":{"1min":0.92900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9630000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.972},{"interface":"front","average":{"1min":1.256,"5min":1.3939999999999999,"15min":1.466},"min":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.669,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.8220000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:07:51 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.48,"15min":1.548},"min":{"1min":1.0940000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6739999999999999,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.536,"15min":1.5840000000000001},"min":{"1min":1.0209999999999999,"5min":0.872,"15min":0.872},"max":{"1min":2.335,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":2.0779999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1662976,"data_stored":1048582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.876,"15min":0.88},"min":{"1min":0.504,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2110000000000001,"5min":3.423,"15min":3.423},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.997,"15min":0.95399999999999996},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.661,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.081,"15min":1.0649999999999999},"min":{"1min":0.60899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5589999999999999,"5min":3.899,"15min":3.899},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.99399999999999999,"15min":0.95799999999999996},"min":{"1min":0.46899999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.3140000000000001,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.97699999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.282,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.2250000000000001},"min":{"1min":0.65300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.621,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":0.82799999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3,"15min":1.2689999999999999},"min":{"1min":0.81999999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6459999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.242},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.482,"15min":1.476},"min":{"1min":0.99199999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8959999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4510000000000001,"15min":1.393},"min":{"1min":0.96599999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.4550000000000001,"15min":1.4379999999999999},"min":{"1min":0.80000000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.0110000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.0329999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.399,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.786,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.5469999999999999,"15min":1.51},"min":{"1min":0.84899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9359999999999999,"5min":3.879,"15min":3.879},"last":1.099}]},{"osd":7,"last update":"Fri Nov 24 08:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.54,"15min":1.54},"min":{"1min":0.83499999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":3.823,"15min":3.823},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.409,"5min":1.5049999999999999,"15min":1.492},"min":{"1min":0.88300000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8740000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.337}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:34.302 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:08:34.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:08:34.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.575+0000 7ff1365b3640 1 Processor -- start 2023-11-24T08:08:34.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.576+0000 7ff1365b3640 1 -- start start 2023-11-24T08:08:34.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.577+0000 7ff1365b3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff13015d340 0x7ff1300d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.577+0000 7ff1365b3640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff1300cdbb0 0x7ff1300d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.577+0000 7ff1365b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1300d1bc0 0x7ff1300b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.577+0000 7ff1365b3640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1300d2050 con 0x7ff13015d340 2023-11-24T08:08:34.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.577+0000 7ff1365b3640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1300cef30 con 0x7ff1300cdbb0 2023-11-24T08:08:34.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.577+0000 7ff1365b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300d11a0 con 0x7ff1300d1bc0 2023-11-24T08:08:34.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.577+0000 7ff1355b1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff13015d340 0x7ff1300d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff1355b1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff13015d340 0x7ff1300d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60516/0 (socket says 172.21.15.64:60516) 2023-11-24T08:08:34.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff1355b1640 1 -- 172.21.15.64:0/3408375431 learned_addr learned my addr 172.21.15.64:0/3408375431 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff135db2640 1 --2- 172.21.15.64:0/3408375431 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1300d1bc0 0x7ff1300b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff1355b1640 1 -- 172.21.15.64:0/3408375431 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1300d1bc0 msgr2=0x7ff1300b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff1355b1640 1 --2- 172.21.15.64:0/3408375431 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1300d1bc0 0x7ff1300b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff134db0640 1 --2- 172.21.15.64:0/3408375431 >> v2:172.21.15.91:3300/0 conn(0x7ff1300cdbb0 0x7ff1300d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff1355b1640 1 -- 172.21.15.64:0/3408375431 >> v2:172.21.15.91:3300/0 conn(0x7ff1300cdbb0 msgr2=0x7ff1300d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff1355b1640 1 --2- 172.21.15.64:0/3408375431 >> v2:172.21.15.91:3300/0 conn(0x7ff1300cdbb0 0x7ff1300d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff1355b1640 1 -- 172.21.15.64:0/3408375431 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1300ce320 con 0x7ff13015d340 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.578+0000 7ff134db0640 1 --2- 172.21.15.64:0/3408375431 >> v2:172.21.15.91:3300/0 conn(0x7ff1300cdbb0 0x7ff1300d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:34.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.579+0000 7ff1355b1640 1 --2- 172.21.15.64:0/3408375431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff13015d340 0x7ff1300d00b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff1300b5ba0 tx=0x7ff12002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da137284fd46db9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.579+0000 7ff12ffff640 1 -- 172.21.15.64:0/3408375431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12002ca80 con 0x7ff13015d340 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff12ffff640 1 -- 172.21.15.64:0/3408375431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12002cbe0 con 0x7ff13015d340 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff12ffff640 1 -- 172.21.15.64:0/3408375431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1200343f0 con 0x7ff13015d340 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 -- 172.21.15.64:0/3408375431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff13015d340 msgr2=0x7ff1300d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3408375431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff13015d340 0x7ff1300d00b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff1300b5ba0 tx=0x7ff12002bee0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 -- 172.21.15.64:0/3408375431 shutdown_connections 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3408375431 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1300d1bc0 0x7ff1300b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3408375431 >> v2:172.21.15.91:3300/0 conn(0x7ff1300cdbb0 0x7ff1300d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3408375431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff13015d340 0x7ff1300d00b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 -- 172.21.15.64:0/3408375431 >> 172.21.15.64:0/3408375431 conn(0x7ff13009c370 msgr2=0x7ff1300d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:34.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 -- 172.21.15.64:0/3408375431 shutdown_connections 2023-11-24T08:08:34.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.580+0000 7ff1365b3640 1 -- 172.21.15.64:0/3408375431 wait complete. 2023-11-24T08:08:34.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.582+0000 7ff1365b3640 1 Processor -- start 2023-11-24T08:08:34.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.582+0000 7ff1365b3640 1 -- start start 2023-11-24T08:08:34.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.583+0000 7ff1365b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1300cdbb0 0x7ff13015e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.583+0000 7ff1365b3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1300d1bc0 0x7ff130159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.583+0000 7ff1365b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff13015d340 0x7ff13015fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.583+0000 7ff1365b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300c8050 con 0x7ff1300cdbb0 2023-11-24T08:08:34.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.583+0000 7ff1365b3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130159690 con 0x7ff1300d1bc0 2023-11-24T08:08:34.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.583+0000 7ff1365b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300b85d0 con 0x7ff13015d340 2023-11-24T08:08:34.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.583+0000 7ff1355b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1300cdbb0 0x7ff13015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff1355b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1300cdbb0 0x7ff13015e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60532/0 (socket says 172.21.15.64:60532) 2023-11-24T08:08:34.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff1355b1640 1 -- 172.21.15.64:0/3082872290 learned_addr learned my addr 172.21.15.64:0/3082872290 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:34.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff135db2640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff13015d340 0x7ff13015fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff1355b1640 1 -- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff13015d340 msgr2=0x7ff13015fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff134db0640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1300d1bc0 0x7ff130159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff1355b1640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff13015d340 0x7ff13015fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff1355b1640 1 -- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1300d1bc0 msgr2=0x7ff130159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff1355b1640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1300d1bc0 0x7ff130159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff1355b1640 1 -- 172.21.15.64:0/3082872290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120009b30 con 0x7ff1300cdbb0 2023-11-24T08:08:34.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff135db2640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff13015d340 0x7ff13015fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:34.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.584+0000 7ff1355b1640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1300cdbb0 0x7ff13015e240 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff120000f80 tx=0x7ff120004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:34.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.585+0000 7ff12dffb640 1 -- 172.21.15.64:0/3082872290 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120009510 con 0x7ff1300cdbb0 2023-11-24T08:08:34.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.585+0000 7ff12dffb640 1 -- 172.21.15.64:0/3082872290 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12003e450 con 0x7ff1300cdbb0 2023-11-24T08:08:34.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.585+0000 7ff12dffb640 1 -- 172.21.15.64:0/3082872290 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12003e800 con 0x7ff1300cdbb0 2023-11-24T08:08:34.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.585+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1300b88b0 con 0x7ff1300cdbb0 2023-11-24T08:08:34.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.585+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1300c7c70 con 0x7ff1300cdbb0 2023-11-24T08:08:34.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.586+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff130067d70 con 0x7ff1300cdbb0 2023-11-24T08:08:34.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.587+0000 7ff12dffb640 1 -- 172.21.15.64:0/3082872290 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff12002cd50 con 0x7ff1300cdbb0 2023-11-24T08:08:34.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.588+0000 7ff12dffb640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff11c06b3e0 0x7ff11c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:34.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.588+0000 7ff12dffb640 1 -- 172.21.15.64:0/3082872290 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1200acca0 con 0x7ff1300cdbb0 2023-11-24T08:08:34.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.589+0000 7ff134db0640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff11c06b3e0 0x7ff11c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:34.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.590+0000 7ff134db0640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff11c06b3e0 0x7ff11c06d8a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff1301f6e00 tx=0x7ff124008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:34.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.592+0000 7ff12dffb640 1 -- 172.21.15.64:0/3082872290 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1200774a0 con 0x7ff1300cdbb0 2023-11-24T08:08:34.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.854+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff13004e840 con 0x7ff1300cdbb0 2023-11-24T08:08:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.859+0000 7ff12dffb640 1 -- 172.21.15.64:0/3082872290 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff12007aac0 con 0x7ff1300cdbb0 2023-11-24T08:08:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-11-24T08:08:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpWGBlG4y+MBAAEnC4ZTfmxdeazdd8S94UZA== 2023-11-24T08:08:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow" 2023-11-24T08:08:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow r" 2023-11-24T08:08:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:08:34.862 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-11-24T08:08:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff11c06b3e0 msgr2=0x7ff11c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff11c06b3e0 0x7ff11c06d8a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff1301f6e00 tx=0x7ff124008040 comp rx=0 tx=0).stop 2023-11-24T08:08:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1300cdbb0 msgr2=0x7ff13015e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1300cdbb0 0x7ff13015e240 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff120000f80 tx=0x7ff120004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 shutdown_connections 2023-11-24T08:08:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff11c06b3e0 0x7ff11c06d8a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff13015d340 0x7ff13015fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff1300d1bc0 0x7ff130159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 --2- 172.21.15.64:0/3082872290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1300cdbb0 0x7ff13015e240 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.862+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 >> 172.21.15.64:0/3082872290 conn(0x7ff13009c370 msgr2=0x7ff13006bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.863+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 shutdown_connections 2023-11-24T08:08:34.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:34.863+0000 7ff1365b3640 1 -- 172.21.15.64:0/3082872290 wait complete. 2023-11-24T08:08:34.885 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.101+0000 7f0115859640 1 Processor -- start 2023-11-24T08:08:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.101+0000 7f0115859640 1 -- start start 2023-11-24T08:08:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.103+0000 7f0115859640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0110143950 0x7f0110164e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.103+0000 7f0115859640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0110071700 con 0x7f0110143570 2023-11-24T08:08:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.103+0000 7f0115859640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011015b410 con 0x7f01100cf760 2023-11-24T08:08:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.103+0000 7f0115859640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011015b130 con 0x7f0110143950 2023-11-24T08:08:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.103+0000 7f0114857640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0110143570 0x7f01100cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46098/0 (socket says 172.21.15.64:46098) 2023-11-24T08:08:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.103+0000 7f0114857640 1 -- 172.21.15.64:0/3669378088 learned_addr learned my addr 172.21.15.64:0/3669378088 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.103+0000 7f0115058640 1 --2- 172.21.15.64:0/3669378088 >> v2:172.21.15.64:3300/0 conn(0x7f0110143950 0x7f0110164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.104+0000 7f010f7fe640 1 -- 172.21.15.64:0/3669378088 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344518176 0 0) 0x7f0104001110 con 0x7f0110143570 2023-11-24T08:08:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.104+0000 7f010f7fe640 1 -- 172.21.15.64:0/3669378088 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0100005730 con 0x7f0110143570 2023-11-24T08:08:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.104+0000 7f0115058640 1 -- 172.21.15.64:0/3669378088 >> v1:172.21.15.91:6789/0 conn(0x7f01100cf760 legacy=0x7f011015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.104+0000 7f0115058640 1 -- 172.21.15.64:0/3669378088 >> v1:172.21.15.64:6789/0 conn(0x7f0110143570 legacy=0x7f01100cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.104+0000 7f0115058640 1 -- 172.21.15.64:0/3669378088 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011015dbd0 con 0x7f0110143950 2023-11-24T08:08:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.105+0000 7f0115058640 1 --2- 172.21.15.64:0/3669378088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0110143950 0x7f0110164e30 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0108009d90 tx=0x7f010802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179a3d72c00a450 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.105+0000 7f010f7fe640 1 -- 172.21.15.64:0/3669378088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010802cbd0 con 0x7f0110143950 2023-11-24T08:08:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.105+0000 7f010f7fe640 1 -- 172.21.15.64:0/3669378088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010802cd30 con 0x7f0110143950 2023-11-24T08:08:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.105+0000 7f010f7fe640 1 -- 172.21.15.64:0/3669378088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010802cea0 con 0x7f0110143950 2023-11-24T08:08:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.106+0000 7f0115859640 1 -- 172.21.15.64:0/3669378088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0110143950 msgr2=0x7f0110164e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.106+0000 7f0115859640 1 --2- 172.21.15.64:0/3669378088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0110143950 0x7f0110164e30 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0108009d90 tx=0x7f010802be20 comp rx=0 tx=0).stop 2023-11-24T08:08:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.106+0000 7f0115859640 1 -- 172.21.15.64:0/3669378088 shutdown_connections 2023-11-24T08:08:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.106+0000 7f0115859640 1 --2- 172.21.15.64:0/3669378088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0110143950 0x7f0110164e30 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.106+0000 7f0115859640 1 -- 172.21.15.64:0/3669378088 >> 172.21.15.64:0/3669378088 conn(0x7f011006a860 msgr2=0x7f011006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.106+0000 7f0115859640 1 -- 172.21.15.64:0/3669378088 shutdown_connections 2023-11-24T08:08:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.106+0000 7f0115859640 1 -- 172.21.15.64:0/3669378088 wait complete. 2023-11-24T08:08:35.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.108+0000 7f0115859640 1 Processor -- start 2023-11-24T08:08:35.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.109+0000 7f0115859640 1 -- start start 2023-11-24T08:08:35.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0115859640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01100cf760 0x7f01101597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:35.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0115859640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0110143570 0x7f0110141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:35.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0115859640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0110143950 0x7f0110141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:35.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0115859640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100ce430 con 0x7f01100cf760 2023-11-24T08:08:35.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0115859640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01101312f0 con 0x7f0110143570 2023-11-24T08:08:35.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0115859640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01101315d0 con 0x7f0110143950 2023-11-24T08:08:35.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0114857640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01100cf760 0x7f01101597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:35.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0114857640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01100cf760 0x7f01101597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60544/0 (socket says 172.21.15.64:60544) 2023-11-24T08:08:35.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.110+0000 7f0114857640 1 -- 172.21.15.64:0/1943479268 learned_addr learned my addr 172.21.15.64:0/1943479268 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:35.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f0115058640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0110143950 0x7f0110141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:35.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f0114857640 1 -- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0110143950 msgr2=0x7f0110141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:35.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f010ffff640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0110143570 0x7f0110141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:35.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f0114857640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0110143950 0x7f0110141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:35.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f0114857640 1 -- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0110143570 msgr2=0x7f0110141780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f0114857640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0110143570 0x7f0110141780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f0114857640 1 -- 172.21.15.64:0/1943479268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0108009a70 con 0x7f01100cf760 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f0115058640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0110143950 0x7f0110141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f0114857640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01100cf760 0x7f01101597c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f010400c700 tx=0x7f010400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.111+0000 7f010d7fa640 1 -- 172.21.15.64:0/1943479268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104005ce0 con 0x7f01100cf760 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.112+0000 7f010d7fa640 1 -- 172.21.15.64:0/1943479268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0104005e40 con 0x7f01100cf760 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.112+0000 7f010d7fa640 1 -- 172.21.15.64:0/1943479268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104007290 con 0x7f01100cf760 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.112+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01101318b0 con 0x7f01100cf760 2023-11-24T08:08:35.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.112+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0110159d00 con 0x7f01100cf760 2023-11-24T08:08:35.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.113+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011004e840 con 0x7f01100cf760 2023-11-24T08:08:35.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.114+0000 7f010d7fa640 1 -- 172.21.15.64:0/1943479268 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f01040074b0 con 0x7f01100cf760 2023-11-24T08:08:35.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.115+0000 7f010d7fa640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f406b3e0 0x7f00f406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:35.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.115+0000 7f010d7fa640 1 -- 172.21.15.64:0/1943479268 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0104089580 con 0x7f01100cf760 2023-11-24T08:08:35.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.116+0000 7f010ffff640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f406b3e0 0x7f00f406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:35.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.117+0000 7f010ffff640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f406b3e0 0x7f00f406d8a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0110137960 tx=0x7f00fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:35.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.119+0000 7f010d7fa640 1 -- 172.21.15.64:0/1943479268 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01040577c0 con 0x7f01100cf760 2023-11-24T08:08:35.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.352+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0110071700 con 0x7f01100cf760 2023-11-24T08:08:35.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.353+0000 7f010d7fa640 1 -- 172.21.15.64:0/1943479268 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f0110071700 con 0x7f01100cf760 2023-11-24T08:08:35.355 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:35.356 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/970726691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":970726691},{"type":"v1","addr":"172.21.15.64:6835","nonce":970726691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2338720158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2338720158},{"type":"v1","addr":"172.21.15.91:6836","nonce":2338720158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2942920208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2942920208},{"type":"v1","addr":"172.21.15.91:6837","nonce":2942920208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:23.583381+0000","modified":"2023-11-24T08:08:26.696160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/3674316997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3674316997},{"type":"v1","addr":"172.21.15.64:6834","nonce":3674316997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:08:35.356 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 53 2023-11-24T08:08:35.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f406b3e0 msgr2=0x7f00f406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:35.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f406b3e0 0x7f00f406d8a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0110137960 tx=0x7f00fc009040 comp rx=0 tx=0).stop 2023-11-24T08:08:35.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01100cf760 msgr2=0x7f01101597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:35.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01100cf760 0x7f01101597c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f010400c700 tx=0x7f010400cb00 comp rx=0 tx=0).stop 2023-11-24T08:08:35.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 shutdown_connections 2023-11-24T08:08:35.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f00f406b3e0 0x7f00f406d8a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:35.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0110143950 0x7f0110141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:35.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0110143570 0x7f0110141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:35.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 --2- 172.21.15.64:0/1943479268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01100cf760 0x7f01101597c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:35.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.356+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 >> 172.21.15.64:0/1943479268 conn(0x7f011006a860 msgr2=0x7f011013ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:35.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.357+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 shutdown_connections 2023-11-24T08:08:35.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:35.357+0000 7f0115859640 1 -- 172.21.15.64:0/1943479268 wait complete. 2023-11-24T08:08:35.780 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:08:23.583381+0000', 'modified': '2023-11-24T08:08:26.696160+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7146}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7146': {'gid': 7146, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6834/3674316997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3674316997}, {'type': 'v1', 'addr': '172.21.15.64:6834', 'nonce': 3674316997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:08:35.781 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:08:35.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T08:08:36.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.015+0000 7f19ae609640 1 Processor -- start 2023-11-24T08:08:36.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.015+0000 7f19ae609640 1 -- start start 2023-11-24T08:08:36.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.017+0000 7f19ae609640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19a815f460 0x7f19a806cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.017+0000 7f19ae609640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f19a8141e80 0x7f19a806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.017+0000 7f19ae609640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a8070bf0 con 0x7f19a81416f0 2023-11-24T08:08:36.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.017+0000 7f19ae609640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a806db30 con 0x7f19a815f460 2023-11-24T08:08:36.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.017+0000 7f19ae609640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a806d5a0 con 0x7f19a8141e80 2023-11-24T08:08:36.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.017+0000 7f19ad607640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f19a81416f0 0x7f19a815ed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46112/0 (socket says 172.21.15.64:46112) 2023-11-24T08:08:36.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.017+0000 7f19ad607640 1 -- 172.21.15.64:0/3163399949 learned_addr learned my addr 172.21.15.64:0/3163399949 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:36.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.017+0000 7f19ace06640 1 --2- 172.21.15.64:0/3163399949 >> v2:172.21.15.64:3300/0 conn(0x7f19a815f460 0x7f19a806cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.018+0000 7f19ade08640 1 --2- 172.21.15.64:0/3163399949 >> v2:172.21.15.91:3300/0 conn(0x7f19a8141e80 0x7f19a806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.018+0000 7f199ffff640 1 -- 172.21.15.64:0/3163399949 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505760659 0 0) 0x7f19a8070bf0 con 0x7f19a81416f0 2023-11-24T08:08:36.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.018+0000 7f199ffff640 1 -- 172.21.15.64:0/3163399949 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1994003520 con 0x7f19a81416f0 2023-11-24T08:08:36.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.018+0000 7f19ace06640 1 -- 172.21.15.64:0/3163399949 >> v2:172.21.15.91:3300/0 conn(0x7f19a8141e80 msgr2=0x7f19a806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.018+0000 7f19ace06640 1 --2- 172.21.15.64:0/3163399949 >> v2:172.21.15.91:3300/0 conn(0x7f19a8141e80 0x7f19a806d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.018+0000 7f19ace06640 1 -- 172.21.15.64:0/3163399949 >> v1:172.21.15.64:6789/0 conn(0x7f19a81416f0 legacy=0x7f19a815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.018+0000 7f19ace06640 1 -- 172.21.15.64:0/3163399949 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a806d850 con 0x7f19a815f460 2023-11-24T08:08:36.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.019+0000 7f19ace06640 1 --2- 172.21.15.64:0/3163399949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a815f460 0x7f19a806cb20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f19a80ae750 tx=0x7f199802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104fb41c8034f81f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f199ffff640 1 -- 172.21.15.64:0/3163399949 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998031830 con 0x7f19a815f460 2023-11-24T08:08:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f199ffff640 1 -- 172.21.15.64:0/3163399949 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1998031990 con 0x7f19a815f460 2023-11-24T08:08:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f199ffff640 1 -- 172.21.15.64:0/3163399949 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998031d20 con 0x7f19a815f460 2023-11-24T08:08:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f19ae609640 1 -- 172.21.15.64:0/3163399949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a815f460 msgr2=0x7f19a806cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f19ae609640 1 --2- 172.21.15.64:0/3163399949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a815f460 0x7f19a806cb20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f19a80ae750 tx=0x7f199802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f19ae609640 1 -- 172.21.15.64:0/3163399949 shutdown_connections 2023-11-24T08:08:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f19ae609640 1 --2- 172.21.15.64:0/3163399949 >> v2:172.21.15.91:3300/0 conn(0x7f19a8141e80 0x7f19a806d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f19ae609640 1 --2- 172.21.15.64:0/3163399949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a815f460 0x7f19a806cb20 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f19ae609640 1 -- 172.21.15.64:0/3163399949 >> 172.21.15.64:0/3163399949 conn(0x7f19a80b0840 msgr2=0x7f19a80b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f19ae609640 1 -- 172.21.15.64:0/3163399949 shutdown_connections 2023-11-24T08:08:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.020+0000 7f19ae609640 1 -- 172.21.15.64:0/3163399949 wait complete. 2023-11-24T08:08:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.022+0000 7f19ae609640 1 Processor -- start 2023-11-24T08:08:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.022+0000 7f19ae609640 1 -- start start 2023-11-24T08:08:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.023+0000 7f19ae609640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a81416f0 0x7f19a8133110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.023+0000 7f19ae609640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a8141e80 0x7f19a80cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.023+0000 7f19ae609640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19a815f460 0x7f19a8131cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.023+0000 7f19ae609640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a813eaf0 con 0x7f19a81416f0 2023-11-24T08:08:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.023+0000 7f19ae609640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a8131820 con 0x7f19a8141e80 2023-11-24T08:08:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.023+0000 7f19ae609640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a8133650 con 0x7f19a815f460 2023-11-24T08:08:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ad607640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a81416f0 0x7f19a8133110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ad607640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a81416f0 0x7f19a8133110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60560/0 (socket says 172.21.15.64:60560) 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ad607640 1 -- 172.21.15.64:0/3020778112 learned_addr learned my addr 172.21.15.64:0/3020778112 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ace06640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a8141e80 0x7f19a80cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ade08640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19a815f460 0x7f19a8131cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ad607640 1 -- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19a815f460 msgr2=0x7f19a8131cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ad607640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19a815f460 0x7f19a8131cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ad607640 1 -- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a8141e80 msgr2=0x7f19a80cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ad607640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a8141e80 0x7f19a80cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ad607640 1 -- 172.21.15.64:0/3020778112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998009400 con 0x7f19a81416f0 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.024+0000 7f19ade08640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19a815f460 0x7f19a8131cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.025+0000 7f19ad607640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a81416f0 0x7f19a8133110 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f19a400c500 tx=0x7f19a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.025+0000 7f199dffb640 1 -- 172.21.15.64:0/3020778112 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4005ce0 con 0x7f19a81416f0 2023-11-24T08:08:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.025+0000 7f199dffb640 1 -- 172.21.15.64:0/3020778112 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4005e40 con 0x7f19a81416f0 2023-11-24T08:08:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.025+0000 7f19ae609640 1 -- 172.21.15.64:0/3020778112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a8130c10 con 0x7f19a81416f0 2023-11-24T08:08:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.025+0000 7f199dffb640 1 -- 172.21.15.64:0/3020778112 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4007330 con 0x7f19a81416f0 2023-11-24T08:08:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.025+0000 7f19ae609640 1 -- 172.21.15.64:0/3020778112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a81311d0 con 0x7f19a81416f0 2023-11-24T08:08:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.028+0000 7f199dffb640 1 -- 172.21.15.64:0/3020778112 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f19a4007490 con 0x7f19a81416f0 2023-11-24T08:08:36.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.029+0000 7f199dffb640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f198c06b410 0x7f198c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.029+0000 7f199dffb640 1 -- 172.21.15.64:0/3020778112 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19a4089630 con 0x7f19a81416f0 2023-11-24T08:08:36.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.029+0000 7f19b2404740 1 Processor -- start 2023-11-24T08:08:36.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.029+0000 7f19ace06640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f198c06b410 0x7f198c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.030+0000 7f19b2404740 1 -- start start 2023-11-24T08:08:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.030+0000 7f19b2404740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd13070 0x55bf9cd13450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.030+0000 7f19b2404740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd12270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19b2404740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd127b0 0x55bf9cd56730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19b2404740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cc71880 con 0x55bf9cd13070 2023-11-24T08:08:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19b2404740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cc80ae0 con 0x55bf9cd127b0 2023-11-24T08:08:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19b2404740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cc7e1d0 con 0x55bf9cd0dd70 2023-11-24T08:08:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ad607640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd13070 0x55bf9cd13450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ad607640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd13070 0x55bf9cd13450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60566/0 (socket says 172.21.15.64:60566) 2023-11-24T08:08:36.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ad607640 1 -- 172.21.15.64:0/1495952150 learned_addr learned my addr 172.21.15.64:0/1495952150 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ade08640 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd127b0 0x55bf9cd56730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ace06640 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd12270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ad607640 1 -- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 msgr2=0x55bf9cd12270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ad607640 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd12270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ad607640 1 -- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd127b0 msgr2=0x55bf9cd56730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ad607640 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd127b0 0x55bf9cd56730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ad607640 1 -- 172.21.15.64:0/1495952150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf9cc41860 con 0x55bf9cd13070 2023-11-24T08:08:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ace06640 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd12270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ade08640 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd127b0 0x55bf9cd56730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.031+0000 7f19ace06640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f198c06b410 0x7f198c06d8d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1998036000 tx=0x7f199803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.032+0000 7f19ad607640 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd13070 0x55bf9cd13450 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f19a4054180 tx=0x7f19a4058440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8f0d0653078d2f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.032+0000 7f1992ffd640 1 -- 172.21.15.64:0/1495952150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4087250 con 0x55bf9cd13070 2023-11-24T08:08:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.032+0000 7f1992ffd640 1 -- 172.21.15.64:0/1495952150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a405ff80 con 0x55bf9cd13070 2023-11-24T08:08:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.032+0000 7f1992ffd640 1 -- 172.21.15.64:0/1495952150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4057170 con 0x55bf9cd13070 2023-11-24T08:08:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 -- 172.21.15.64:0/1495952150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd13070 msgr2=0x55bf9cd13450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd13070 0x55bf9cd13450 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f19a4054180 tx=0x7f19a4058440 comp rx=0 tx=0).stop 2023-11-24T08:08:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 -- 172.21.15.64:0/1495952150 shutdown_connections 2023-11-24T08:08:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd127b0 0x55bf9cd56730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd12270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 --2- 172.21.15.64:0/1495952150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd13070 0x55bf9cd13450 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 -- 172.21.15.64:0/1495952150 >> 172.21.15.64:0/1495952150 conn(0x55bf9ccd4c70 msgr2=0x55bf9cd58b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 -- 172.21.15.64:0/1495952150 shutdown_connections 2023-11-24T08:08:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.033+0000 7f19b2404740 1 -- 172.21.15.64:0/1495952150 wait complete. 2023-11-24T08:08:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.034+0000 7f19b2404740 1 -- start start 2023-11-24T08:08:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.034+0000 7f19b2404740 1 Processor -- start 2023-11-24T08:08:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19b2404740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd60a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19b2404740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19b2404740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd13070 0x55bf9cd370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19b2404740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cc79ac0 con 0x55bf9cd127b0 2023-11-24T08:08:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19b2404740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cc162f0 con 0x55bf9cd13070 2023-11-24T08:08:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19b2404740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cc33c70 con 0x55bf9cd0dd70 2023-11-24T08:08:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19b2404740 10 client.0 _ll_register_callbacks cb 0x55bf9cc383b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:08:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19ace06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19ace06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60582/0 (socket says 172.21.15.64:60582) 2023-11-24T08:08:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.035+0000 7f19ace06640 1 -- 172.21.15.64:0/1308046134 learned_addr learned my addr 172.21.15.64:0/1308046134 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ade08640 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd13070 0x55bf9cd370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ad607640 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd60a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ace06640 1 -- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 msgr2=0x55bf9cd60a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ace06640 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd60a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ace06640 1 -- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd13070 msgr2=0x55bf9cd370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ace06640 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd13070 0x55bf9cd370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ace06640 1 -- 172.21.15.64:0/1308046134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a81311d0 con 0x55bf9cd127b0 2023-11-24T08:08:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ade08640 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd13070 0x55bf9cd370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f19ace06640 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36ba0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1998000c00 tx=0x7f199802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36d76a0c15c2750 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.036+0000 7f1987fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:08:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f1987fff640 1 -- 172.21.15.64:0/1308046134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998049040 con 0x55bf9cd127b0 2023-11-24T08:08:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f1987fff640 1 -- 172.21.15.64:0/1308046134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f199803f070 con 0x55bf9cd127b0 2023-11-24T08:08:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f19b2404740 1 -- 172.21.15.64:0/1308046134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf9cc72aa0 con 0x55bf9cd127b0 2023-11-24T08:08:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f1987fff640 1 -- 172.21.15.64:0/1308046134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f199803a830 con 0x55bf9cd127b0 2023-11-24T08:08:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f19b2404740 10 client.7317 fetch_fsmap learned FSMap version 53 2023-11-24T08:08:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f19b2404740 1 -- 172.21.15.64:0/1308046134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55bf9cc78140 con 0x55bf9cd127b0 2023-11-24T08:08:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f1987fff640 1 -- 172.21.15.64:0/1308046134 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf9cc72aa0 con 0x55bf9cd127b0 2023-11-24T08:08:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f1987fff640 1 -- 172.21.15.64:0/1308046134 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f199803a990 con 0x55bf9cd127b0 2023-11-24T08:08:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f19b2404740 10 client.7317 fetch_fsmap finished waiting for FSMap version 53 2023-11-24T08:08:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f19b2404740 10 client.7317 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:08:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f19b2404740 20 client.7317 populate_metadata read hostname 'smithi064' 2023-11-24T08:08:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.037+0000 7f19b2404740 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] conn(0x55bf9cd5f580 0x55bf9cd75e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.038+0000 7f19b2404740 4 client.7317 mds_command: new command op to 7146 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:08:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.038+0000 7f19b2404740 1 -- 172.21.15.64:0/1308046134 --> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf9cc727c0 con 0x55bf9cd5f580 2023-11-24T08:08:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.038+0000 7f19ad607640 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] conn(0x55bf9cd5f580 0x55bf9cd75e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.038+0000 7f19ad607640 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] conn(0x55bf9cd5f580 0x55bf9cd75e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92a5eec14d4eeee server_cookie=73b5c441f5fa4003 in_seq=0 out_seq=0 2023-11-24T08:08:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.038+0000 7f1987fff640 10 client.7317 ms_handle_connect on v2:172.21.15.64:6832/3674316997 2023-11-24T08:08:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f1987fff640 1 -- 172.21.15.64:0/1308046134 <== mds.0 v2:172.21.15.64:6832/3674316997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf9cc727c0 con 0x55bf9cd5f580 2023-11-24T08:08:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f1987fff640 10 client.7317 handle_command_reply: tid=0 2023-11-24T08:08:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 client.7317 shutdown 2023-11-24T08:08:36.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 -- 172.21.15.64:0/1308046134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 msgr2=0x55bf9cd36ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36ba0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1998000c00 tx=0x7f199802e000 comp rx=0 tx=0).stop 2023-11-24T08:08:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 -- 172.21.15.64:0/1308046134 shutdown_connections 2023-11-24T08:08:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] conn(0x55bf9cd5f580 0x55bf9cd75e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd13070 0x55bf9cd370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36ba0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 --2- 172.21.15.64:0/1308046134 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd0dd70 0x55bf9cd60a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f1987fff640 0 client.7317 ms_handle_reset on v2:172.21.15.64:6832/3674316997 2023-11-24T08:08:36.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.039+0000 7f19b2404740 1 -- 172.21.15.64:0/1308046134 >> 172.21.15.64:0/1308046134 conn(0x55bf9ccd4c70 msgr2=0x55bf9cd0cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:36.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.040+0000 7f19b2404740 1 -- 172.21.15.64:0/1308046134 shutdown_connections 2023-11-24T08:08:36.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.040+0000 7f19b2404740 1 -- 172.21.15.64:0/1308046134 wait complete. 2023-11-24T08:08:36.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.040+0000 7f19b2404740 20 client.7317 trim_cache size 0 max 16384 2023-11-24T08:08:36.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.057+0000 7f19b2404740 1 Processor -- start 2023-11-24T08:08:36.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.057+0000 7f19b2404740 1 -- start start 2023-11-24T08:08:36.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.058+0000 7f19b2404740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cd36aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.058+0000 7f19b2404740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.058+0000 7f19b2404740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd5c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.058+0000 7f19b2404740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9c9dcf20 con 0x55bf9cd127b0 2023-11-24T08:08:36.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.058+0000 7f19b2404740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cd0a0b0 con 0x55bf9cd0dd70 2023-11-24T08:08:36.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.058+0000 7f19b2404740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cd5e770 con 0x55bf9cd13070 2023-11-24T08:08:36.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.058+0000 7f19ace06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ace06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60598/0 (socket says 172.21.15.64:60598) 2023-11-24T08:08:36.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ace06640 1 -- 172.21.15.64:0/1608488868 learned_addr learned my addr 172.21.15.64:0/1608488868 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:36.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ad607640 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cd36aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ade08640 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd5c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ace06640 1 -- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 msgr2=0x55bf9cd5c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ace06640 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd5c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ace06640 1 -- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 msgr2=0x55bf9cd36aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ace06640 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cd36aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ace06640 1 -- 172.21.15.64:0/1608488868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf9cc15b80 con 0x55bf9cd127b0 2023-11-24T08:08:36.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ad607640 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cd36aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:36.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ade08640 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd5c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:36.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19ace06640 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36fe0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1998009650 tx=0x7f1998031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e2d7f9f41ad69 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.059+0000 7f19937fe640 1 -- 172.21.15.64:0/1608488868 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998049070 con 0x55bf9cd127b0 2023-11-24T08:08:36.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19937fe640 1 -- 172.21.15.64:0/1608488868 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199803a830 con 0x55bf9cd127b0 2023-11-24T08:08:36.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19937fe640 1 -- 172.21.15.64:0/1608488868 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998004030 con 0x55bf9cd127b0 2023-11-24T08:08:36.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 -- 172.21.15.64:0/1608488868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 msgr2=0x55bf9cd36fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36fe0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1998009650 tx=0x7f1998031000 comp rx=0 tx=0).stop 2023-11-24T08:08:36.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 -- 172.21.15.64:0/1608488868 shutdown_connections 2023-11-24T08:08:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd5c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd36fe0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 --2- 172.21.15.64:0/1608488868 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cd36aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 -- 172.21.15.64:0/1608488868 >> 172.21.15.64:0/1608488868 conn(0x55bf9ccd4c70 msgr2=0x55bf9cd6b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 -- 172.21.15.64:0/1608488868 shutdown_connections 2023-11-24T08:08:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.060+0000 7f19b2404740 1 -- 172.21.15.64:0/1608488868 wait complete. 2023-11-24T08:08:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.061+0000 7f19b2404740 1 -- start start 2023-11-24T08:08:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.061+0000 7f19b2404740 1 Processor -- start 2023-11-24T08:08:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.062+0000 7f19b2404740 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cdc2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.062+0000 7f19b2404740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.062+0000 7f19b2404740 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd79e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.062+0000 7f19b2404740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cd5d050 con 0x55bf9cd127b0 2023-11-24T08:08:36.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.062+0000 7f19b2404740 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cd5a2f0 con 0x55bf9cd0dd70 2023-11-24T08:08:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.062+0000 7f19b2404740 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf9cd4f9b0 con 0x55bf9cd13070 2023-11-24T08:08:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.062+0000 7f19b2404740 10 client.0 _ll_register_callbacks cb 0x7f19940036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:08:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ace06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd34220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ace06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd34220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60604/0 (socket says 172.21.15.64:60604) 2023-11-24T08:08:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ace06640 1 -- 172.21.15.64:0/183498838 learned_addr learned my addr 172.21.15.64:0/183498838 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ad607640 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cdc2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ade08640 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd79e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ace06640 1 -- 172.21.15.64:0/183498838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 msgr2=0x55bf9cd79e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ace06640 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd79e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ace06640 1 -- 172.21.15.64:0/183498838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 msgr2=0x55bf9cdc2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ace06640 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cdc2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.063+0000 7f19ace06640 1 -- 172.21.15.64:0/183498838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf9cc78140 con 0x55bf9cd127b0 2023-11-24T08:08:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.064+0000 7f19ace06640 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd34220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f199805f040 tx=0x7f1998004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b07b86b43b38ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.064+0000 7f1987fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-11-24T08:08:36.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.064+0000 7f1987fff640 1 -- 172.21.15.64:0/183498838 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998049040 con 0x55bf9cd127b0 2023-11-24T08:08:36.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.064+0000 7f1987fff640 1 -- 172.21.15.64:0/183498838 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f199803f040 con 0x55bf9cd127b0 2023-11-24T08:08:36.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.064+0000 7f19b2404740 1 -- 172.21.15.64:0/183498838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19a405ff80 con 0x55bf9cd127b0 2023-11-24T08:08:36.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.064+0000 7f1987fff640 1 -- 172.21.15.64:0/183498838 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1998003610 con 0x55bf9cd127b0 2023-11-24T08:08:36.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.064+0000 7f19b2404740 10 client.7323 fetch_fsmap learned FSMap version 53 2023-11-24T08:08:36.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.064+0000 7f19b2404740 1 -- 172.21.15.64:0/183498838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55bf9cc3b170 con 0x55bf9cd127b0 2023-11-24T08:08:36.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f1987fff640 1 -- 172.21.15.64:0/183498838 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1998076080 con 0x55bf9cd127b0 2023-11-24T08:08:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f1987fff640 1 -- 172.21.15.64:0/183498838 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1998003770 con 0x55bf9cd127b0 2023-11-24T08:08:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f19b2404740 10 client.7323 fetch_fsmap finished waiting for FSMap version 53 2023-11-24T08:08:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f19b2404740 10 client.7323 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:08:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f19b2404740 20 client.7323 populate_metadata read hostname 'smithi064' 2023-11-24T08:08:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f19b2404740 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] conn(0x55bf9cd71f50 0x55bf9cd7a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f19b2404740 4 client.7323 mds_command: new command op to 7146 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:08:36.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f19b2404740 1 -- 172.21.15.64:0/183498838 --> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bf9c97a6c0 con 0x55bf9cd71f50 2023-11-24T08:08:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.065+0000 7f19ad607640 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] conn(0x55bf9cd71f50 0x55bf9cd7a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19ad607640 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] conn(0x55bf9cd71f50 0x55bf9cd7a370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a163fa4ad019a2c server_cookie=957416359aedad9 in_seq=0 out_seq=0 2023-11-24T08:08:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f1987fff640 10 client.7323 ms_handle_connect on v2:172.21.15.64:6832/3674316997 2023-11-24T08:08:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f1987fff640 1 -- 172.21.15.64:0/183498838 <== mds.0 v2:172.21.15.64:6832/3674316997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55bf9c97a6c0 con 0x55bf9cd71f50 2023-11-24T08:08:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f1987fff640 10 client.7323 handle_command_reply: tid=0 2023-11-24T08:08:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 client.7323 shutdown 2023-11-24T08:08:36.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 -- 172.21.15.64:0/183498838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 msgr2=0x55bf9cd34220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd34220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f199805f040 tx=0x7f1998004730 comp rx=0 tx=0).stop 2023-11-24T08:08:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 -- 172.21.15.64:0/183498838 shutdown_connections 2023-11-24T08:08:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.64:6832/3674316997,v1:172.21.15.64:6834/3674316997] conn(0x55bf9cd71f50 0x55bf9cd7a370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x55bf9cd13070 0x55bf9cd79e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf9cd127b0 0x55bf9cd34220 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f1987fff640 0 client.7323 ms_handle_reset on v2:172.21.15.64:6832/3674316997 2023-11-24T08:08:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 --2- 172.21.15.64:0/183498838 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x55bf9cd0dd70 0x55bf9cdc2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 -- 172.21.15.64:0/183498838 >> 172.21.15.64:0/183498838 conn(0x55bf9ccd4c70 msgr2=0x55bf9cd60040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 -- 172.21.15.64:0/183498838 shutdown_connections 2023-11-24T08:08:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.066+0000 7f19b2404740 1 -- 172.21.15.64:0/183498838 wait complete. 2023-11-24T08:08:36.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19b2404740 20 client.7323 trim_cache size 0 max 16384 2023-11-24T08:08:36.078 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:08:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918", 2023-11-24T08:08:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-11-24T08:08:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 7146, 2023-11-24T08:08:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-11-24T08:08:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-11-24T08:08:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-11-24T08:08:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 11.926772720000001, 2023-11-24T08:08:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 53, 2023-11-24T08:08:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 87, 2023-11-24T08:08:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 87, 2023-11-24T08:08:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 17.298102129 2023-11-24T08:08:36.080 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:08:36.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 -- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f198c06b410 msgr2=0x7f198c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f198c06b410 0x7f198c06d8d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1998036000 tx=0x7f199803c040 comp rx=0 tx=0).stop 2023-11-24T08:08:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 -- 172.21.15.64:0/3020778112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a81416f0 msgr2=0x7f19a8133110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a81416f0 0x7f19a8133110 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f19a400c500 tx=0x7f19a400c9c0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 -- 172.21.15.64:0/3020778112 shutdown_connections 2023-11-24T08:08:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f198c06b410 0x7f198c06d8d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f19a815f460 0x7f19a8131cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f19a8141e80 0x7f19a80cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 --2- 172.21.15.64:0/3020778112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19a81416f0 0x7f19a8133110 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 -- 172.21.15.64:0/3020778112 >> 172.21.15.64:0/3020778112 conn(0x7f19a80b0840 msgr2=0x7f19a80ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:36.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 -- 172.21.15.64:0/3020778112 shutdown_connections 2023-11-24T08:08:36.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.067+0000 7f19ae609640 1 -- 172.21.15.64:0/3020778112 wait complete. 2023-11-24T08:08:36.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:08:36.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.293+0000 7f21755c1640 1 Processor -- start 2023-11-24T08:08:36.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.294+0000 7f21755c1640 1 -- start start 2023-11-24T08:08:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.295+0000 7f21755c1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f21700d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.295+0000 7f21755c1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21700d3990 con 0x7f217015d620 2023-11-24T08:08:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.295+0000 7f21755c1640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21700d3c70 con 0x7f217015d240 2023-11-24T08:08:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.295+0000 7f21755c1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700d3f50 con 0x7f21700b3f00 2023-11-24T08:08:36.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.296+0000 7f216f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f217015d620 0x7f21700d1500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46124/0 (socket says 172.21.15.64:46124) 2023-11-24T08:08:36.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.296+0000 7f216f7fe640 1 -- 172.21.15.64:0/2850419505 learned_addr learned my addr 172.21.15.64:0/2850419505 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:36.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.296+0000 7f2174dc0640 1 --2- 172.21.15.64:0/2850419505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f21700d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.296+0000 7f216effd640 1 -- 172.21.15.64:0/2850419505 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629493043 0 0) 0x7f21700d3990 con 0x7f217015d620 2023-11-24T08:08:36.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.296+0000 7f216effd640 1 -- 172.21.15.64:0/2850419505 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2158003520 con 0x7f217015d620 2023-11-24T08:08:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.297+0000 7f216effd640 1 -- 172.21.15.64:0/2850419505 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377517384 0 0) 0x7f21700d3c70 con 0x7f217015d240 2023-11-24T08:08:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.297+0000 7f216effd640 1 -- 172.21.15.64:0/2850419505 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21700d3990 con 0x7f217015d240 2023-11-24T08:08:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.297+0000 7f2174dc0640 1 -- 172.21.15.64:0/2850419505 >> v1:172.21.15.91:6789/0 conn(0x7f217015d240 legacy=0x7f21700d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.297+0000 7f2174dc0640 1 -- 172.21.15.64:0/2850419505 >> v1:172.21.15.64:6789/0 conn(0x7f217015d620 legacy=0x7f21700d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.297+0000 7f216effd640 1 -- 172.21.15.64:0/2850419505 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3751089403 0 0) 0x7f2158003520 con 0x7f217015d620 2023-11-24T08:08:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.297+0000 7f2174dc0640 1 -- 172.21.15.64:0/2850419505 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700b29f0 con 0x7f21700b3f00 2023-11-24T08:08:36.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.298+0000 7f2174dc0640 1 --2- 172.21.15.64:0/2850419505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f21700d1c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2168009dc0 tx=0x7f216802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb342d2cd036e9d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f216effd640 1 -- 172.21.15.64:0/2850419505 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164001670 con 0x7f21700b3f00 2023-11-24T08:08:36.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f216effd640 1 -- 172.21.15.64:0/2850419505 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216802c910 con 0x7f21700b3f00 2023-11-24T08:08:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f216effd640 1 -- 172.21.15.64:0/2850419505 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216802cc80 con 0x7f21700b3f00 2023-11-24T08:08:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f21755c1640 1 -- 172.21.15.64:0/2850419505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 msgr2=0x7f21700d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f21755c1640 1 --2- 172.21.15.64:0/2850419505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f21700d1c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2168009dc0 tx=0x7f216802bde0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f21755c1640 1 -- 172.21.15.64:0/2850419505 shutdown_connections 2023-11-24T08:08:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f21755c1640 1 --2- 172.21.15.64:0/2850419505 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f21700d1c10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f21755c1640 1 -- 172.21.15.64:0/2850419505 >> 172.21.15.64:0/2850419505 conn(0x7f217009c3f0 msgr2=0x7f21700d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f21755c1640 1 -- 172.21.15.64:0/2850419505 shutdown_connections 2023-11-24T08:08:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.299+0000 7f21755c1640 1 -- 172.21.15.64:0/2850419505 wait complete. 2023-11-24T08:08:36.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.301+0000 7f21755c1640 1 Processor -- start 2023-11-24T08:08:36.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.301+0000 7f21755c1640 1 -- start start 2023-11-24T08:08:36.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.302+0000 7f21755c1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f2170157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.302+0000 7f21755c1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f217015d240 0x7f217014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.302+0000 7f21755c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217015d620 0x7f21700ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.302+0000 7f21755c1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c6ec0 con 0x7f217015d620 2023-11-24T08:08:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.302+0000 7f21755c1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217014be00 con 0x7f217015d240 2023-11-24T08:08:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.302+0000 7f21755c1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c6be0 con 0x7f21700b3f00 2023-11-24T08:08:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f2174dc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217015d620 0x7f21700ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f2174dc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217015d620 0x7f21700ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60614/0 (socket says 172.21.15.64:60614) 2023-11-24T08:08:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f2174dc0640 1 -- 172.21.15.64:0/2431922792 learned_addr learned my addr 172.21.15.64:0/2431922792 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f216f7fe640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f217015d240 0x7f217014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f216ffff640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f2170157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f2174dc0640 1 -- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 msgr2=0x7f2170157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f2174dc0640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f2170157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f2174dc0640 1 -- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f217015d240 msgr2=0x7f217014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f2174dc0640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f217015d240 0x7f217014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.303+0000 7f2174dc0640 1 -- 172.21.15.64:0/2431922792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168009a70 con 0x7f217015d620 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.304+0000 7f2174dc0640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217015d620 0x7f21700ca340 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f216802c3f0 tx=0x7f2168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.304+0000 7f216cff9640 1 -- 172.21.15.64:0/2431922792 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168009450 con 0x7f217015d620 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.304+0000 7f216cff9640 1 -- 172.21.15.64:0/2431922792 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2168004250 con 0x7f217015d620 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.304+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2170156ca0 con 0x7f217015d620 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.304+0000 7f216cff9640 1 -- 172.21.15.64:0/2431922792 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21680045c0 con 0x7f217015d620 2023-11-24T08:08:36.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.304+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2170157260 con 0x7f217015d620 2023-11-24T08:08:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.305+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217014b860 con 0x7f217015d620 2023-11-24T08:08:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.307+0000 7f216cff9640 1 -- 172.21.15.64:0/2431922792 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f216803b070 con 0x7f217015d620 2023-11-24T08:08:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.307+0000 7f216cff9640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f215c06b410 0x7f215c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:36.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.308+0000 7f216cff9640 1 -- 172.21.15.64:0/2431922792 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21680ac370 con 0x7f217015d620 2023-11-24T08:08:36.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.308+0000 7f216ffff640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f215c06b410 0x7f215c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.309+0000 7f216ffff640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f215c06b410 0x7f215c06d8d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f21700ca2d0 tx=0x7f2160009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.312+0000 7f216cff9640 1 -- 172.21.15.64:0/2431922792 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2168076b70 con 0x7f217015d620 2023-11-24T08:08:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.543+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f217004e840 con 0x7f217015d620 2023-11-24T08:08:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.544+0000 7f216cff9640 1 -- 172.21.15.64:0/2431922792 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f216807a190 con 0x7f217015d620 2023-11-24T08:08:36.546 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.546+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f215c06b410 msgr2=0x7f215c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.546+0000 7f21755c1640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f215c06b410 0x7f215c06d8d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f21700ca2d0 tx=0x7f2160009040 comp rx=0 tx=0).stop 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.546+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217015d620 msgr2=0x7f21700ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.546+0000 7f21755c1640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217015d620 0x7f21700ca340 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f216802c3f0 tx=0x7f2168004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.547+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 shutdown_connections 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.547+0000 7f21755c1640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f215c06b410 0x7f215c06d8d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.547+0000 7f21755c1640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217015d620 0x7f21700ca340 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.547+0000 7f21755c1640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f217015d240 0x7f217014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.547+0000 7f21755c1640 1 --2- 172.21.15.64:0/2431922792 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f21700b3f00 0x7f2170157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.547+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 >> 172.21.15.64:0/2431922792 conn(0x7f217009c3f0 msgr2=0x7f2170154fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.547+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 shutdown_connections 2023-11-24T08:08:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:36.547+0000 7f21755c1640 1 -- 172.21.15.64:0/2431922792 wait complete. 2023-11-24T08:08:36.568 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:08:36.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:08:36.569 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-11-24T08:08:36.569 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:08:36.569 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:08:36.569 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:08:36.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:08:36.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:08:36.570 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:08:36.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:36.571 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-11-24T08:08:36.628 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:08:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42066sec preferred_lft 42066sec 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3c83:ffff:fee3:836f/64 scope link 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 56:d4:76:60:98:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::54d4:76ff:fe60:984a/64 scope link 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether ae:e6:20:dc:18:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ace6:20ff:fedc:18d9/64 scope link 2023-11-24T08:08:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:36.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:36.632 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:08:36.725 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-11-24T08:08:36.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:36.730 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-11-24T08:08:36.790 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-11-24T08:08:36.790 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:08:36.790 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:08:36.790 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:08:36.790 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:08:36.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:36.791 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-11-24T08:08:36.791 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-11-24T08:08:36.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:08:36.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:08:36.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:08:36.791 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-11-24T08:08:36.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:36.987 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:08:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:08:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:08:37.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:08:37.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:37.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-11-24T08:08:37.146 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T08:08:37.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:08:37.147 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:08:37.206 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:08:37.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:08:37.206 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:08:37.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:08:37.366 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:08:37.428 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:08:37.429 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:08:37.430 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:08:37.431 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:08:37.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:37.432 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:08:37.490 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:08:37.490 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T08:08:37.496 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-11-24T08:08:37.575 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:08:37.619 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2023-11-24T08:08:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:08:37.621 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:08:37.622 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:08:37.623 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:08:37.623 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:08:37.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:37.624 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-11-24T08:08:37.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-11-24T08:08:37.683+0000 7f5b043f6480 -1 init, newargv = 0x556ba4fe9b30 newargc=15 2023-11-24T08:08:37.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[115893]: starting ceph client 2023-11-24T08:08:37.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[115893]: starting fuse 2023-11-24T08:08:37.712 INFO:teuthology.orchestra.run.smithi064.stdout:40 2023-11-24T08:08:37.713 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T08:08:37.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:08:37.713 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-11-24T08:08:37.714 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-11-24T08:08:37.714 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-11-24T08:08:37.714 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-11-24T08:08:37.714 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:08:37.714 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-11-24T08:08:37.715 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T08:08:37.715 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-11-24T08:08:37.715 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T08:08:37.715 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:08:37.715 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-11-24T08:08:37.716 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-11-24T08:08:37.717 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:08:37.717 DEBUG:teuthology.orchestra.run.smithi064:> 2023-11-24T08:08:37.717 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-11-24T08:08:37.717 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-11-24T08:08:37.938 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.115893.asok 2023-11-24T08:08:37.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:08:37.957 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.115893.asok 2023-11-24T08:08:37.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:37.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.115893.asok status 2023-11-24T08:08:38.207 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:08:38.207 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-11-24T08:08:38.207 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:08:38.207 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:08:38.207 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-11-24T08:08:38.207 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-11-24T08:08:38.207 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T08:08:38.207 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "115893", 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 7335, 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-11-24T08:08:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 7335 2023-11-24T08:08:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:08:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:08:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:08:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1487101303 2023-11-24T08:08:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:38.209 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-11-24T08:08:38.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-11-24T08:08:38.210 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:08:38.210 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1487101303 2023-11-24T08:08:38.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-11-24T08:08:38.210 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.7335 192.168.0.1:0/1487101303", 2023-11-24T08:08:38.210 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/1487101303", 2023-11-24T08:08:38.210 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-11-24T08:08:38.211 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 53, 2023-11-24T08:08:38.211 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 87, 2023-11-24T08:08:38.211 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:08:38.211 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-11-24T08:08:38.211 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-11-24T08:08:38.211 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:08:38.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:38.222 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-11-24T08:08:38.222 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T08:08:38.277 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-11-24T08:08:38.277 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T08:08:38.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:38.277 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:08:38.376 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:08:38.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.602+0000 7f78d6106640 1 Processor -- start 2023-11-24T08:08:38.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.602+0000 7f78d6106640 1 -- start start 2023-11-24T08:08:38.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.604+0000 7f78d6106640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f78d013dfd0 0x7f78d015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.604+0000 7f78d6106640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d00cf500 0x7f78d006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.604+0000 7f78d6106640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d00ce7c0 con 0x7f78d00cf050 2023-11-24T08:08:38.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.604+0000 7f78d6106640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d0140900 con 0x7f78d013dfd0 2023-11-24T08:08:38.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.604+0000 7f78d6106640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d015fa70 con 0x7f78d00cf500 2023-11-24T08:08:38.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.604+0000 7f78d4903640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f78d00cf050 0x7f78d006c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57540/0 (socket says 172.21.15.64:57540) 2023-11-24T08:08:38.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.604+0000 7f78d4903640 1 -- 172.21.15.64:0/2546649195 learned_addr learned my addr 172.21.15.64:0/2546649195 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78d5104640 1 --2- 172.21.15.64:0/2546649195 >> v2:172.21.15.91:3300/0 conn(0x7f78d013dfd0 0x7f78d015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78d5905640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d00cf500 0x7f78d006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596356644 0 0) 0x7f78d00ce7c0 con 0x7f78d00cf050 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78bc003520 con 0x7f78d00cf050 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4216524413 0 0) 0x7f78bc003520 con 0x7f78d00cf050 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d00cf500 msgr2=0x7f78d006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d00cf500 0x7f78d006c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 >> v2:172.21.15.91:3300/0 conn(0x7f78d013dfd0 msgr2=0x7f78d015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 --2- 172.21.15.64:0/2546649195 >> v2:172.21.15.91:3300/0 conn(0x7f78d013dfd0 0x7f78d015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00cf950 con 0x7f78d00cf050 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f78c0001690 con 0x7f78d00cf050 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 >> v1:172.21.15.64:6789/0 conn(0x7f78d00cf050 legacy=0x7f78d006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.605+0000 7f78c7fff640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78c7fff640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d00cf500 0x7f78bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78c7fff640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78bc004d00 0x7f78bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d0140900 con 0x7f78bc0039e0 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d015fa70 con 0x7f78bc004d00 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc00b980 con 0x7f78d00cf500 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78d5104640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78d4903640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d00cf500 0x7f78bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78d5905640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78bc004d00 0x7f78bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78d5104640 1 -- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d00cf500 msgr2=0x7f78bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78d5104640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d00cf500 0x7f78bc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78d5104640 1 -- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78bc004d00 msgr2=0x7f78bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78d5104640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78bc004d00 0x7f78bc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.606+0000 7f78d5104640 1 -- 172.21.15.64:0/2546649195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc003520 con 0x7f78bc0039e0 2023-11-24T08:08:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.607+0000 7f78d5104640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78bc003e00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009ac0 tx=0x7f78bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5275e6ee30549 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.608+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc02c710 con 0x7f78bc0039e0 2023-11-24T08:08:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.608+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc02c870 con 0x7f78bc0039e0 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.608+0000 7f78c7fff640 1 -- 172.21.15.64:0/2546649195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc02cc00 con 0x7f78bc0039e0 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.608+0000 7f78d6106640 1 -- 172.21.15.64:0/2546649195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 msgr2=0x7f78bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.608+0000 7f78d6106640 1 --2- 172.21.15.64:0/2546649195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78bc003e00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009ac0 tx=0x7f78bc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.608+0000 7f78d5905640 1 -- 172.21.15.64:0/2546649195 reap_dead start 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.608+0000 7f78d6106640 1 -- 172.21.15.64:0/2546649195 shutdown_connections 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.608+0000 7f78d6106640 1 -- 172.21.15.64:0/2546649195 >> 172.21.15.64:0/2546649195 conn(0x7f78d006a860 msgr2=0x7f78d006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.609+0000 7f78d6106640 1 -- 172.21.15.64:0/2546649195 shutdown_connections 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.609+0000 7f78d6106640 1 -- 172.21.15.64:0/2546649195 wait complete. 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.610+0000 7f78d6106640 1 Processor -- start 2023-11-24T08:08:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.611+0000 7f78d6106640 1 -- start start 2023-11-24T08:08:38.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.611+0000 7f78d6106640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78d0159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.611+0000 7f78d6106640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d01590d0 0x7f78d0154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.611+0000 7f78d6106640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d0159550 0x7f78d0155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d6106640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d015ad70 con 0x7f78bc0039e0 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d6106640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d0155650 con 0x7f78d0159550 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d6106640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00b23e0 con 0x7f78d01590d0 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5104640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78d0159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5104640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78d0159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33588/0 (socket says 172.21.15.64:33588) 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5104640 1 -- 172.21.15.64:0/2564178074 learned_addr learned my addr 172.21.15.64:0/2564178074 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5905640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d0159550 0x7f78d0155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d4903640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d01590d0 0x7f78d0154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5104640 1 -- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d01590d0 msgr2=0x7f78d0154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5104640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d01590d0 0x7f78d0154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5104640 1 -- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d0159550 msgr2=0x7f78d0155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:38.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5104640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d0159550 0x7f78d0155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5104640 1 -- 172.21.15.64:0/2564178074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00b2680 con 0x7f78bc0039e0 2023-11-24T08:08:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.612+0000 7f78d5905640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d0159550 0x7f78d0155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.613+0000 7f78d5104640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78d0159bc0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f78cc02c2b0 tx=0x7f78cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.613+0000 7f78c5ffb640 1 -- 172.21.15.64:0/2564178074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc003810 con 0x7f78bc0039e0 2023-11-24T08:08:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.613+0000 7f78c5ffb640 1 -- 172.21.15.64:0/2564178074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc02de90 con 0x7f78bc0039e0 2023-11-24T08:08:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.613+0000 7f78c5ffb640 1 -- 172.21.15.64:0/2564178074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc003bb0 con 0x7f78bc0039e0 2023-11-24T08:08:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.613+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d00b2960 con 0x7f78bc0039e0 2023-11-24T08:08:38.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.613+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d013fbe0 con 0x7f78bc0039e0 2023-11-24T08:08:38.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.614+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d004e840 con 0x7f78bc0039e0 2023-11-24T08:08:38.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.615+0000 7f78c5ffb640 1 -- 172.21.15.64:0/2564178074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f78cc003d10 con 0x7f78bc0039e0 2023-11-24T08:08:38.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.616+0000 7f78c5ffb640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78b406b3e0 0x7f78b406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:38.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.616+0000 7f78c5ffb640 1 -- 172.21.15.64:0/2564178074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f78cc0ac040 con 0x7f78bc0039e0 2023-11-24T08:08:38.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.617+0000 7f78d4903640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78b406b3e0 0x7f78b406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:38.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.618+0000 7f78d4903640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78b406b3e0 0x7f78b406d8a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f78c0001880 tx=0x7f78c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:38.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.621+0000 7f78c5ffb640 1 -- 172.21.15.64:0/2564178074 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78cc076860 con 0x7f78bc0039e0 2023-11-24T08:08:38.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.853+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f78d00ce9a0 con 0x7f78bc0039e0 2023-11-24T08:08:38.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.854+0000 7f78c5ffb640 1 -- 172.21.15.64:0/2564178074 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f78cc079e80 con 0x7f78bc0039e0 2023-11-24T08:08:38.856 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:08:38.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78b406b3e0 msgr2=0x7f78b406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:38.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78b406b3e0 0x7f78b406d8a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f78c0001880 tx=0x7f78c0009040 comp rx=0 tx=0).stop 2023-11-24T08:08:38.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 msgr2=0x7f78d0159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:38.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78d0159bc0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f78cc02c2b0 tx=0x7f78cc004730 comp rx=0 tx=0).stop 2023-11-24T08:08:38.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 shutdown_connections 2023-11-24T08:08:38.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f78b406b3e0 0x7f78b406d8a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f78d0159550 0x7f78d0155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f78d01590d0 0x7f78d0154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 --2- 172.21.15.64:0/2564178074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc0039e0 0x7f78d0159bc0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:38.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 >> 172.21.15.64:0/2564178074 conn(0x7f78d006a860 msgr2=0x7f78d00709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:38.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 shutdown_connections 2023-11-24T08:08:38.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:38.857+0000 7f78d6106640 1 -- 172.21.15.64:0/2564178074 wait complete. 2023-11-24T08:08:38.878 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:08:38.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:08:38.878 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi091.front.sepia.ceph.com 2023-11-24T08:08:38.878 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:08:38.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:08:38.879 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:08:38.879 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:08:38.880 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:08:38.880 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:08:38.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:38.880 DEBUG:teuthology.orchestra.run.smithi091:> ip addr 2023-11-24T08:08:38.943 INFO:teuthology.orchestra.run.smithi091.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:08:38.943 INFO:teuthology.orchestra.run.smithi091.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:08:38.943 INFO:teuthology.orchestra.run.smithi091.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:08:38.943 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:38.943 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 ::1/128 scope host 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f0 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:d8:d9:c5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout: altname enp7s0f1 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d4 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 0c:c4:7a:8f:d3:d5 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:38.944 INFO:teuthology.orchestra.run.smithi091.stdout: inet 172.21.15.91/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft 42053sec preferred_lft 42053sec 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::ec4:7aff:fe8f:d3d5/64 scope link 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::5073:3fff:fef1:71bb/64 scope link 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 26:97:37:0c:85:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:08:38.945 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::2497:37ff:fe0c:852a/64 scope link 2023-11-24T08:08:38.946 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:38.946 INFO:teuthology.orchestra.run.smithi091.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:08:38.946 INFO:teuthology.orchestra.run.smithi091.stdout: link/ether 32:d9:7a:07:6c:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:08:38.946 INFO:teuthology.orchestra.run.smithi091.stdout: inet6 fe80::30d9:7aff:fe07:6ce0/64 scope link 2023-11-24T08:08:38.946 INFO:teuthology.orchestra.run.smithi091.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:08:38.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:38.946 DEBUG:teuthology.orchestra.run.smithi091:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:08:39.038 INFO:teuthology.orchestra.run.smithi091.stdout:1 2023-11-24T08:08:39.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:39.045 DEBUG:teuthology.orchestra.run.smithi091:> route 2023-11-24T08:08:39.097 INFO:teuthology.orchestra.run.smithi091.stdout:Kernel IP routing table 2023-11-24T08:08:39.097 INFO:teuthology.orchestra.run.smithi091.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:08:39.097 INFO:teuthology.orchestra.run.smithi091.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:08:39.097 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:08:39.097 INFO:teuthology.orchestra.run.smithi091.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:08:39.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:39.098 DEBUG:teuthology.orchestra.run.smithi091:> (cd / && exec bash -c ' 2023-11-24T08:08:39.098 DEBUG:teuthology.orchestra.run.smithi091:> set -e 2023-11-24T08:08:39.098 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:08:39.098 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:08:39.098 DEBUG:teuthology.orchestra.run.smithi091:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:08:39.098 DEBUG:teuthology.orchestra.run.smithi091:> ') 2023-11-24T08:08:39.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:39.279 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:08:39.329 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:08:39.329 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:08:39.330 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:08:39.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:39.330 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link set brx.0 up 2023-11-24T08:08:39.414 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T08:08:39.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:08:39.414 DEBUG:teuthology.orchestra.run.smithi091:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:08:39.466 INFO:teuthology.orchestra.run.smithi091.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:08:39.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:08:39.467 DEBUG:teuthology.orchestra.run.smithi091:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:08:39.517 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:08:39.608 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:08:39.668 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:08:39.668 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:08:39.668 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:08:39.668 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:08:39.669 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:08:39.670 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:08:39.671 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:08:39.671 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:08:39.671 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:08:39.671 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:08:39.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:39.672 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:08:39.723 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T08:08:39.724 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T08:08:39.729 DEBUG:teuthology.orchestra.run.smithi091:> sudo modprobe fuse 2023-11-24T08:08:39.804 DEBUG:teuthology.orchestra.run.smithi091:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T08:08:39.845 INFO:teuthology.orchestra.run.smithi091.stdout:/proc 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/sys 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/dev 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/security 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/shm 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/pts 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/run 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/cgroup 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/pstore 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/ 2023-11-24T08:08:39.846 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/selinux 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/mqueue 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/dev/hugepages 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/debug 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/tracing 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/fs/fuse/connections 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/sys/kernel/config 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T08:08:39.847 INFO:teuthology.orchestra.run.smithi091.stdout:/run/user/1000 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:08:39.848 INFO:teuthology.orchestra.run.smithi091.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:08:39.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:39.849 DEBUG:teuthology.orchestra.run.smithi091:> ls /sys/fs/fuse/connections 2023-11-24T08:08:39.916 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:2023-11-24T08:08:39.914+0000 7fd289a43480 -1 init, newargv = 0x5631d9a629f0 newargc=15 2023-11-24T08:08:39.916 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[98580]: starting ceph client 2023-11-24T08:08:39.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[98580]: starting fuse 2023-11-24T08:08:39.940 INFO:teuthology.orchestra.run.smithi091.stdout:41 2023-11-24T08:08:39.940 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T08:08:39.940 DEBUG:teuthology.orchestra.run.smithi091:> sudo stdin-killer -- python3 -c ' 2023-11-24T08:08:39.940 DEBUG:teuthology.orchestra.run.smithi091:> import glob 2023-11-24T08:08:39.940 DEBUG:teuthology.orchestra.run.smithi091:> import re 2023-11-24T08:08:39.940 DEBUG:teuthology.orchestra.run.smithi091:> import os 2023-11-24T08:08:39.940 DEBUG:teuthology.orchestra.run.smithi091:> import subprocess 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> def _find_admin_socket(client_name): 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> files = glob.glob(asok_path) 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> # Given a non-glob path, it better be there 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> if "*" not in asok_path: 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> assert(len(files) == 1) 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> return files[0] 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:08:39.941 DEBUG:teuthology.orchestra.run.smithi091:> for f in files: 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> contents = proc_f.read() 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> if mountpoint in contents: 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> return f 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> print(_find_admin_socket("client.1")) 2023-11-24T08:08:39.942 DEBUG:teuthology.orchestra.run.smithi091:> ' 2023-11-24T08:08:40.142 INFO:teuthology.orchestra.run.smithi091.stdout:/var/run/ceph/ceph-client.1.98580.asok 2023-11-24T08:08:40.147 INFO:teuthology.orchestra.run.smithi091.stderr:2023-11-24T08:08:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T08:08:40.161 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.98580.asok 2023-11-24T08:08:40.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:40.161 DEBUG:teuthology.orchestra.run.smithi091:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.98580.asok status 2023-11-24T08:08:40.397 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2023-11-24T08:08:40.397 INFO:teuthology.orchestra.run.smithi091.stdout: "metadata": { 2023-11-24T08:08:40.397 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: "entity_id": "1", 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: "hostname": "smithi091", 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: "pid": "98580", 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: "root": "/" 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_count": 0, 2023-11-24T08:08:40.398 INFO:teuthology.orchestra.run.smithi091.stdout: "dentry_pinned_count": 0, 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "id": 6109, 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "inst": { 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "name": { 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "client", 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "num": 6109 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 2058571346 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: } 2023-11-24T08:08:40.399 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": { 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "type": "v1", 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "addr": "192.168.0.1:0", 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "nonce": 2058571346 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: }, 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "inst_str": "client.6109 192.168.0.1:0/2058571346", 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "addr_str": "192.168.0.1:0/2058571346", 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "inode_count": 1, 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_epoch": 53, 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch": 89, 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_epoch_barrier": 0, 2023-11-24T08:08:40.400 INFO:teuthology.orchestra.run.smithi091.stdout: "blocklisted": false, 2023-11-24T08:08:40.401 INFO:teuthology.orchestra.run.smithi091.stdout: "fs_name": "cephfs" 2023-11-24T08:08:40.401 INFO:teuthology.orchestra.run.smithi091.stdout:} 2023-11-24T08:08:40.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:40.412 DEBUG:teuthology.orchestra.run.smithi091:> stat --file-system '--printf=%T 2023-11-24T08:08:40.412 DEBUG:teuthology.orchestra.run.smithi091:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T08:08:40.464 INFO:teuthology.orchestra.run.smithi091.stdout:fuseblk 2023-11-24T08:08:40.464 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T08:08:40.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:40.464 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:08:40.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:08:40.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.783+0000 7f983eb78640 1 Processor -- start 2023-11-24T08:08:40.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.783+0000 7f983eb78640 1 -- start start 2023-11-24T08:08:40.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.785+0000 7f983eb78640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f983815b350 0x7f98380abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.785+0000 7f983eb78640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983815b730 con 0x7f983809cb30 2023-11-24T08:08:40.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.785+0000 7f983eb78640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98380ac740 con 0x7f98380adb60 2023-11-24T08:08:40.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.785+0000 7f983eb78640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98380ac460 con 0x7f983815b350 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.785+0000 7f983c914640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f983815b350 0x7f98380abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.785+0000 7f983c914640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f983815b350 0x7f98380abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33598/0 (socket says 172.21.15.64:33598) 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.785+0000 7f983c914640 1 -- 172.21.15.64:0/195754841 learned_addr learned my addr 172.21.15.64:0/195754841 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.785+0000 7f9837fff640 1 --1- 172.21.15.64:0/195754841 >> v1:172.21.15.64:6789/0 conn(0x7f983809cb30 0x7f98380b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57554/0 (socket says 172.21.15.64:57554) 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.786+0000 7f983c914640 1 -- 172.21.15.64:0/195754841 >> v1:172.21.15.91:6789/0 conn(0x7f98380adb60 legacy=0x7f98380adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.786+0000 7f983c914640 1 -- 172.21.15.64:0/195754841 >> v1:172.21.15.64:6789/0 conn(0x7f983809cb30 legacy=0x7f98380b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.786+0000 7f98377fe640 1 -- 172.21.15.64:0/195754841 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630092502 0 0) 0x7f9824001110 con 0x7f983809cb30 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.786+0000 7f983c914640 1 -- 172.21.15.64:0/195754841 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380b2a50 con 0x7f983815b350 2023-11-24T08:08:40.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.787+0000 7f983c914640 1 --2- 172.21.15.64:0/195754841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983815b350 0x7f98380abe80 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f982c009870 tx=0x7f982c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866869b3c847bdcc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.787+0000 7f98377fe640 1 -- 172.21.15.64:0/195754841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c030860 con 0x7f983815b350 2023-11-24T08:08:40.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.787+0000 7f98377fe640 1 -- 172.21.15.64:0/195754841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c0309c0 con 0x7f983815b350 2023-11-24T08:08:40.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.787+0000 7f98377fe640 1 -- 172.21.15.64:0/195754841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c030d50 con 0x7f983815b350 2023-11-24T08:08:40.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.788+0000 7f983eb78640 1 -- 172.21.15.64:0/195754841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983815b350 msgr2=0x7f98380abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.788+0000 7f983eb78640 1 --2- 172.21.15.64:0/195754841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983815b350 0x7f98380abe80 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f982c009870 tx=0x7f982c02b9a0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.788+0000 7f983eb78640 1 -- 172.21.15.64:0/195754841 shutdown_connections 2023-11-24T08:08:40.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.788+0000 7f983eb78640 1 --2- 172.21.15.64:0/195754841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983815b350 0x7f98380abe80 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.788+0000 7f983eb78640 1 -- 172.21.15.64:0/195754841 >> 172.21.15.64:0/195754841 conn(0x7f98380b0ba0 msgr2=0x7f98380b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:40.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.788+0000 7f983eb78640 1 -- 172.21.15.64:0/195754841 shutdown_connections 2023-11-24T08:08:40.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.788+0000 7f983eb78640 1 -- 172.21.15.64:0/195754841 wait complete. 2023-11-24T08:08:40.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.790+0000 7f983eb78640 1 Processor -- start 2023-11-24T08:08:40.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.790+0000 7f983eb78640 1 -- start start 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983eb78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983809cb30 0x7f98380afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983eb78640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98380adb60 0x7f983813e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983eb78640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f983815b350 0x7f983813eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983eb78640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983813f020 con 0x7f983809cb30 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983eb78640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838072330 con 0x7f98380adb60 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983eb78640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983814ffc0 con 0x7f983815b350 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983c914640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983809cb30 0x7f98380afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983c914640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983809cb30 0x7f98380afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33602/0 (socket says 172.21.15.64:33602) 2023-11-24T08:08:40.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.791+0000 7f983c914640 1 -- 172.21.15.64:0/4090085775 learned_addr learned my addr 172.21.15.64:0/4090085775 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.792+0000 7f983c914640 1 -- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f983815b350 msgr2=0x7f983813eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.792+0000 7f983c914640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f983815b350 0x7f983813eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.792+0000 7f983c914640 1 -- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98380adb60 msgr2=0x7f983813e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.792+0000 7f9837fff640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98380adb60 0x7f983813e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.792+0000 7f983c914640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98380adb60 0x7f983813e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.792+0000 7f983c914640 1 -- 172.21.15.64:0/4090085775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0094d0 con 0x7f983809cb30 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.792+0000 7f9837fff640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98380adb60 0x7f983813e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.792+0000 7f983c914640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983809cb30 0x7f98380afdd0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.793+0000 7f98357fa640 1 -- 172.21.15.64:0/4090085775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c039b60 con 0x7f983809cb30 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.793+0000 7f98357fa640 1 -- 172.21.15.64:0/4090085775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c039cc0 con 0x7f983809cb30 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.793+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98381502a0 con 0x7f983809cb30 2023-11-24T08:08:40.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.793+0000 7f98357fa640 1 -- 172.21.15.64:0/4090085775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c043650 con 0x7f983809cb30 2023-11-24T08:08:40.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.793+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9838150860 con 0x7f983809cb30 2023-11-24T08:08:40.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.794+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9838150a50 con 0x7f983809cb30 2023-11-24T08:08:40.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.795+0000 7f98357fa640 1 -- 172.21.15.64:0/4090085775 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f982c04c3b0 con 0x7f983809cb30 2023-11-24T08:08:40.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.796+0000 7f98357fa640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f981c06b3e0 0x7f981c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.796+0000 7f98357fa640 1 -- 172.21.15.64:0/4090085775 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f982c0ac590 con 0x7f983809cb30 2023-11-24T08:08:40.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.796+0000 7f9837fff640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f981c06b3e0 0x7f981c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.798+0000 7f9837fff640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f981c06b3e0 0x7f981c06d8a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f983809b970 tx=0x7f9824009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:40.800+0000 7f98357fa640 1 -- 172.21.15.64:0/4090085775 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f982c076c10 con 0x7f983809cb30 2023-11-24T08:08:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:41.037+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f98380ac740 con 0x7f983809cb30 2023-11-24T08:08:42.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.036+0000 7f98357fa640 1 -- 172.21.15.64:0/4090085775 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 136+0+0 (secure 0 0 0) 0x7f98380ac740 con 0x7f983809cb30 2023-11-24T08:08:42.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.039+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f981c06b3e0 msgr2=0x7f981c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.039+0000 7f983eb78640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f981c06b3e0 0x7f981c06d8a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f983809b970 tx=0x7f9824009040 comp rx=0 tx=0).stop 2023-11-24T08:08:42.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.039+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983809cb30 msgr2=0x7f98380afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.039+0000 7f983eb78640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983809cb30 0x7f98380afdd0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c004730 comp rx=0 tx=0).stop 2023-11-24T08:08:42.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.040+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 shutdown_connections 2023-11-24T08:08:42.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.040+0000 7f983eb78640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f981c06b3e0 0x7f981c06d8a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.040+0000 7f983eb78640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f983815b350 0x7f983813eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.040+0000 7f983eb78640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f98380adb60 0x7f983813e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.040+0000 7f983eb78640 1 --2- 172.21.15.64:0/4090085775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f983809cb30 0x7f98380afdd0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.040+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 >> 172.21.15.64:0/4090085775 conn(0x7f98380b0ba0 msgr2=0x7f9838157260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:42.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.040+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 shutdown_connections 2023-11-24T08:08:42.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.040+0000 7f983eb78640 1 -- 172.21.15.64:0/4090085775 wait complete. 2023-11-24T08:08:42.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:08:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.274+0000 7f618f9c4640 1 Processor -- start 2023-11-24T08:08:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.274+0000 7f618f9c4640 1 -- start start 2023-11-24T08:08:42.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618f9c4640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f618806dc30 0x7f6188071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618f9c4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f61880715d0 0x7f61880ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618f9c4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61880ad400 con 0x7f61880b3b60 2023-11-24T08:08:42.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618f9c4640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61880acb50 con 0x7f618806dc30 2023-11-24T08:08:42.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618f9c4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880acdf0 con 0x7f61880715d0 2023-11-24T08:08:42.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618d760640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f61880b3b60 0x7f618806d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57568/0 (socket says 172.21.15.64:57568) 2023-11-24T08:08:42.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618d760640 1 -- 172.21.15.64:0/1867161910 learned_addr learned my addr 172.21.15.64:0/1867161910 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618cf5f640 1 --2- 172.21.15.64:0/1867161910 >> v2:172.21.15.91:3300/0 conn(0x7f618806dc30 0x7f6188071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135428525 0 0) 0x7f61880ad400 con 0x7f61880b3b60 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f618df61640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f61880715d0 0x7f61880ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.276+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6174003520 con 0x7f61880b3b60 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3556834015 0 0) 0x7f6174003520 con 0x7f61880b3b60 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f61880715d0 msgr2=0x7f61880ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f61880715d0 0x7f61880ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 >> v2:172.21.15.91:3300/0 conn(0x7f618806dc30 msgr2=0x7f6188071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 --2- 172.21.15.64:0/1867161910 >> v2:172.21.15.91:3300/0 conn(0x7f618806dc30 0x7f6188071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880720a0 con 0x7f61880b3b60 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f6184001670 con 0x7f61880b3b60 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 >> v1:172.21.15.64:6789/0 conn(0x7f61880b3b60 legacy=0x7f618806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6174003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f61740047c0 0x7f6174004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6174005180 0x7f617400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880acb50 con 0x7f6174003970 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880acdf0 con 0x7f6174005180 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617400bef0 con 0x7f61740047c0 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f618df61640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f61880715d0 0x7f61880ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.277+0000 7f618d760640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6174003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.278+0000 7f618d760640 1 -- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f61740047c0 msgr2=0x7f6174004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:42.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.278+0000 7f618d760640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f61740047c0 0x7f6174004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.278+0000 7f618d760640 1 -- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6174005180 msgr2=0x7f617400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:42.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.278+0000 7f618d760640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6174005180 0x7f617400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.278+0000 7f618d760640 1 -- 172.21.15.64:0/1867161910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6174003520 con 0x7f6174003970 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.279+0000 7f618d760640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6174003d90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f618400c240 tx=0x7f61740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5136b575f8ad3003 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.279+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618402cb00 con 0x7f6174003970 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.279+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618402cc60 con 0x7f6174003970 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.279+0000 7f617ffff640 1 -- 172.21.15.64:0/1867161910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184005af0 con 0x7f6174003970 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.279+0000 7f618f9c4640 1 -- 172.21.15.64:0/1867161910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 msgr2=0x7f6174003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.279+0000 7f618f9c4640 1 --2- 172.21.15.64:0/1867161910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6174003d90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f618400c240 tx=0x7f61740057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.279+0000 7f618df61640 1 -- 172.21.15.64:0/1867161910 reap_dead start 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.280+0000 7f618f9c4640 1 -- 172.21.15.64:0/1867161910 shutdown_connections 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.280+0000 7f618f9c4640 1 -- 172.21.15.64:0/1867161910 >> 172.21.15.64:0/1867161910 conn(0x7f61880b0d80 msgr2=0x7f61880b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.280+0000 7f618f9c4640 1 -- 172.21.15.64:0/1867161910 shutdown_connections 2023-11-24T08:08:42.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.280+0000 7f618f9c4640 1 -- 172.21.15.64:0/1867161910 wait complete. 2023-11-24T08:08:42.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.281+0000 7f618f9c4640 1 Processor -- start 2023-11-24T08:08:42.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.282+0000 7f618f9c4640 1 -- start start 2023-11-24T08:08:42.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.282+0000 7f618f9c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6188151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.282+0000 7f618f9c4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6188152410 0x7f61880c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.282+0000 7f618f9c4640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6188154740 0x7f6188154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618f9c4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880cd460 con 0x7f6174003970 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618f9c4640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6188153730 con 0x7f6188154740 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618f9c4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61881542b0 con 0x7f6188152410 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6188151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6188151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33622/0 (socket says 172.21.15.64:33622) 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 -- 172.21.15.64:0/2845680052 learned_addr learned my addr 172.21.15.64:0/2845680052 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 -- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6188152410 msgr2=0x7f61880c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6188152410 0x7f61880c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 -- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6188154740 msgr2=0x7f6188154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6188154740 0x7f6188154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 -- 172.21.15.64:0/2845680052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188152170 con 0x7f6174003970 2023-11-24T08:08:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.283+0000 7f618d760640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6188151c30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f618402c570 tx=0x7f6184001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:42.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.284+0000 7f617dffb640 1 -- 172.21.15.64:0/2845680052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618403c040 con 0x7f6174003970 2023-11-24T08:08:42.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.284+0000 7f617dffb640 1 -- 172.21.15.64:0/2845680052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184001630 con 0x7f6174003970 2023-11-24T08:08:42.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.284+0000 7f617dffb640 1 -- 172.21.15.64:0/2845680052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184007270 con 0x7f6174003970 2023-11-24T08:08:42.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.284+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6188153980 con 0x7f6174003970 2023-11-24T08:08:42.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.284+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6188153f40 con 0x7f6174003970 2023-11-24T08:08:42.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.285+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6188074770 con 0x7f6174003970 2023-11-24T08:08:42.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.286+0000 7f617dffb640 1 -- 172.21.15.64:0/2845680052 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6184032070 con 0x7f6174003970 2023-11-24T08:08:42.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.287+0000 7f617dffb640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616c06b410 0x7f616c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.287+0000 7f617dffb640 1 -- 172.21.15.64:0/2845680052 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f61840abb30 con 0x7f6174003970 2023-11-24T08:08:42.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.287+0000 7f618cf5f640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616c06b410 0x7f616c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.289+0000 7f618cf5f640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616c06b410 0x7f616c06d8d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f61780093d0 tx=0x7f6178009300 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:42.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.291+0000 7f617dffb640 1 -- 172.21.15.64:0/2845680052 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6184076310 con 0x7f6174003970 2023-11-24T08:08:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.511+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6188071a40 con 0x7f616c06b410 2023-11-24T08:08:42.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.515+0000 7f617dffb640 1 -- 172.21.15.64:0/2845680052 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f617800e720 con 0x7f616c06b410 2023-11-24T08:08:42.516 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-11-24T08:08:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-11-24T08:08:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-11-24T08:08:42.517 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-11-24T08:08:42.517 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-11-24T08:08:42.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616c06b410 msgr2=0x7f616c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616c06b410 0x7f616c06d8d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f61780093d0 tx=0x7f6178009300 comp rx=0 tx=0).stop 2023-11-24T08:08:42.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 msgr2=0x7f6188151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6188151c30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f618402c570 tx=0x7f6184001a00 comp rx=0 tx=0).stop 2023-11-24T08:08:42.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 shutdown_connections 2023-11-24T08:08:42.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f616c06b410 0x7f616c06d8d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6188154740 0x7f6188154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6188152410 0x7f61880c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 --2- 172.21.15.64:0/2845680052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174003970 0x7f6188151c30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 >> 172.21.15.64:0/2845680052 conn(0x7f61880b0d80 msgr2=0x7f618806c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:42.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 shutdown_connections 2023-11-24T08:08:42.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.517+0000 7f618f9c4640 1 -- 172.21.15.64:0/2845680052 wait complete. 2023-11-24T08:08:42.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:08:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.767+0000 7f8e1dadc640 1 Processor -- start 2023-11-24T08:08:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.768+0000 7f8e1dadc640 1 -- start start 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.769+0000 7f8e1dadc640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8e1806dae0 0x7f8e1815b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.770+0000 7f8e1dadc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e1815bbd0 0x7f8e18070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.770+0000 7f8e1dadc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e18144ca0 con 0x7f8e181448c0 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.770+0000 7f8e1dadc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1815c450 con 0x7f8e1815bbd0 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.770+0000 7f8e1dadc640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e180715b0 con 0x7f8e1806dae0 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.770+0000 7f8e1d2db640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e1815bbd0 0x7f8e18070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.770+0000 7f8e1d2db640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e1815bbd0 0x7f8e18070070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33632/0 (socket says 172.21.15.64:33632) 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.770+0000 7f8e1d2db640 1 -- 172.21.15.64:0/1113834716 learned_addr learned my addr 172.21.15.64:0/1113834716 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.770+0000 7f8e17fff640 1 --2- 172.21.15.64:0/1113834716 >> v2:172.21.15.91:3300/0 conn(0x7f8e1806dae0 0x7f8e1815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.771+0000 7f8e1d2db640 1 -- 172.21.15.64:0/1113834716 >> v2:172.21.15.91:3300/0 conn(0x7f8e1806dae0 msgr2=0x7f8e1815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.771+0000 7f8e1d2db640 1 --2- 172.21.15.64:0/1113834716 >> v2:172.21.15.91:3300/0 conn(0x7f8e1806dae0 0x7f8e1815b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.771+0000 7f8e1d2db640 1 -- 172.21.15.64:0/1113834716 >> v1:172.21.15.64:6789/0 conn(0x7f8e181448c0 legacy=0x7f8e1806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.771+0000 7f8e1d2db640 1 -- 172.21.15.64:0/1113834716 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18071890 con 0x7f8e1815bbd0 2023-11-24T08:08:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.772+0000 7f8e1d2db640 1 --2- 172.21.15.64:0/1113834716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e1815bbd0 0x7f8e18070070 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8e180b1a90 tx=0x7f8e1002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf07623fff7e6e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.772+0000 7f8e177fe640 1 -- 172.21.15.64:0/1113834716 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1002cb10 con 0x7f8e1815bbd0 2023-11-24T08:08:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.772+0000 7f8e177fe640 1 -- 172.21.15.64:0/1113834716 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1002cc70 con 0x7f8e1815bbd0 2023-11-24T08:08:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.772+0000 7f8e177fe640 1 -- 172.21.15.64:0/1113834716 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100343f0 con 0x7f8e1815bbd0 2023-11-24T08:08:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.773+0000 7f8e1dadc640 1 -- 172.21.15.64:0/1113834716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e1815bbd0 msgr2=0x7f8e18070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.773+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/1113834716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e1815bbd0 0x7f8e18070070 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8e180b1a90 tx=0x7f8e1002bf70 comp rx=0 tx=0).stop 2023-11-24T08:08:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.773+0000 7f8e1dadc640 1 -- 172.21.15.64:0/1113834716 shutdown_connections 2023-11-24T08:08:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.773+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/1113834716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e1815bbd0 0x7f8e18070070 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.773+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/1113834716 >> v2:172.21.15.91:3300/0 conn(0x7f8e1806dae0 0x7f8e1815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.773+0000 7f8e1dadc640 1 -- 172.21.15.64:0/1113834716 >> 172.21.15.64:0/1113834716 conn(0x7f8e180b0aa0 msgr2=0x7f8e180b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.773+0000 7f8e1dadc640 1 -- 172.21.15.64:0/1113834716 shutdown_connections 2023-11-24T08:08:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.773+0000 7f8e1dadc640 1 -- 172.21.15.64:0/1113834716 wait complete. 2023-11-24T08:08:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.775+0000 7f8e1dadc640 1 Processor -- start 2023-11-24T08:08:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.775+0000 7f8e1dadc640 1 -- start start 2023-11-24T08:08:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.776+0000 7f8e1dadc640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8e1806dae0 0x7f8e180701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.776+0000 7f8e1dadc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e181448c0 0x7f8e181f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.776+0000 7f8e1dadc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8e1815bbd0 0x7f8e181f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.776+0000 7f8e1dadc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181f6900 con 0x7f8e181448c0 2023-11-24T08:08:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.776+0000 7f8e1dadc640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181f6be0 con 0x7f8e1806dae0 2023-11-24T08:08:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.776+0000 7f8e1dadc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1813c7e0 con 0x7f8e1815bbd0 2023-11-24T08:08:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.776+0000 7f8e17fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e181448c0 0x7f8e181f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e17fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e181448c0 0x7f8e181f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33646/0 (socket says 172.21.15.64:33646) 2023-11-24T08:08:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e17fff640 1 -- 172.21.15.64:0/3405006096 learned_addr learned my addr 172.21.15.64:0/3405006096 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e1cada640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8e1806dae0 0x7f8e180701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e1d2db640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8e1815bbd0 0x7f8e181f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e17fff640 1 -- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8e1815bbd0 msgr2=0x7f8e181f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e17fff640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8e1815bbd0 0x7f8e181f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e17fff640 1 -- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8e1806dae0 msgr2=0x7f8e180701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e17fff640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8e1806dae0 0x7f8e180701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e17fff640 1 -- 172.21.15.64:0/3405006096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009b30 con 0x7f8e181448c0 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.777+0000 7f8e17fff640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e181448c0 0x7f8e181f5f90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400d800 tx=0x7f8e0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.778+0000 7f8e157fa640 1 -- 172.21.15.64:0/3405006096 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400b6c0 con 0x7f8e181448c0 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.778+0000 7f8e157fa640 1 -- 172.21.15.64:0/3405006096 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0400b820 con 0x7f8e181448c0 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.778+0000 7f8e157fa640 1 -- 172.21.15.64:0/3405006096 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400bbd0 con 0x7f8e181448c0 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.778+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e180724f0 con 0x7f8e181448c0 2023-11-24T08:08:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.778+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e18070720 con 0x7f8e181448c0 2023-11-24T08:08:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.779+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1813c640 con 0x7f8e181448c0 2023-11-24T08:08:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.780+0000 7f8e157fa640 1 -- 172.21.15.64:0/3405006096 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8e0400bd30 con 0x7f8e181448c0 2023-11-24T08:08:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.781+0000 7f8e157fa640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dfc06b3e0 0x7f8dfc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.781+0000 7f8e157fa640 1 -- 172.21.15.64:0/3405006096 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8e0408a610 con 0x7f8e181448c0 2023-11-24T08:08:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.782+0000 7f8e1cada640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dfc06b3e0 0x7f8dfc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:42.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.783+0000 7f8e1cada640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dfc06b3e0 0x7f8dfc06d8a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c000f60 tx=0x7f8e0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:42.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.785+0000 7f8e157fa640 1 -- 172.21.15.64:0/3405006096 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e1813c640 con 0x7f8e181448c0 2023-11-24T08:08:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.979+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8e1813c640 con 0x7f8e181448c0 2023-11-24T08:08:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.980+0000 7f8e157fa640 1 -- 172.21.15.64:0/3405006096 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8e1813c640 con 0x7f8e181448c0 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.982+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dfc06b3e0 msgr2=0x7f8dfc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.982+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dfc06b3e0 0x7f8dfc06d8a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c000f60 tx=0x7f8e0c009040 comp rx=0 tx=0).stop 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.982+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e181448c0 msgr2=0x7f8e181f5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.982+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e181448c0 0x7f8e181f5f90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400d800 tx=0x7f8e0400dcc0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.983+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 shutdown_connections 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.983+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8dfc06b3e0 0x7f8dfc06d8a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.983+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8e1815bbd0 0x7f8e181f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.983+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e181448c0 0x7f8e181f5f90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.983+0000 7f8e1dadc640 1 --2- 172.21.15.64:0/3405006096 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8e1806dae0 0x7f8e180701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.983+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 >> 172.21.15.64:0/3405006096 conn(0x7f8e180b0aa0 msgr2=0x7f8e180b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.983+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 shutdown_connections 2023-11-24T08:08:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:42.983+0000 7f8e1dadc640 1 -- 172.21.15.64:0/3405006096 wait complete. 2023-11-24T08:08:43.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp4507 2023-11-24T08:08:43.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.224+0000 7fa5d1f7a640 1 Processor -- start 2023-11-24T08:08:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.225+0000 7fa5d1f7a640 1 -- start start 2023-11-24T08:08:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.226+0000 7fa5d1f7a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 0x7fa5cc15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.226+0000 7fa5d1f7a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc15f930 con 0x7fa5cc15f0a0 2023-11-24T08:08:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.226+0000 7fa5d1f7a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc0b3b10 con 0x7fa5cc0b35a0 2023-11-24T08:08:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.226+0000 7fa5d1f7a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b43b0 con 0x7fa5cc144770 2023-11-24T08:08:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.226+0000 7fa5d1779640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa5cc15f0a0 0x7fa5cc0ce4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57586/0 (socket says 172.21.15.64:57586) 2023-11-24T08:08:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.226+0000 7fa5d1779640 1 -- 172.21.15.64:0/3522432170 learned_addr learned my addr 172.21.15.64:0/3522432170 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5d0f78640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 0x7fa5cc15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218868558 0 0) 0x7fa5cc15f930 con 0x7fa5cc15f0a0 2023-11-24T08:08:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b4003520 con 0x7fa5cc15f0a0 2023-11-24T08:08:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232142453 0 0) 0x7fa5b4003520 con 0x7fa5cc15f0a0 2023-11-24T08:08:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 msgr2=0x7fa5cc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 0x7fa5cc15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 >> v1:172.21.15.91:6789/0 conn(0x7fa5cc0b35a0 legacy=0x7fa5cc06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0b3df0 con 0x7fa5cc15f0a0 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fa5c4001670 con 0x7fa5cc15f0a0 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 >> v1:172.21.15.64:6789/0 conn(0x7fa5cc15f0a0 legacy=0x7fa5cc0ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5b4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 0x7fa5b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5b4004d00 0x7fa5b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b43b0 con 0x7fa5b4003a10 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b400ba90 con 0x7fa5b4004d00 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.227+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b400bc50 con 0x7fa5cc144770 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5d0f78640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5b4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5d1779640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5b4004d00 0x7fa5b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5d0f78640 1 -- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 msgr2=0x7fa5b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5c3fff640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 0x7fa5b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5d0f78640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 0x7fa5b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5d0f78640 1 -- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5b4004d00 msgr2=0x7fa5b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5d0f78640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5b4004d00 0x7fa5b4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5d0f78640 1 -- 172.21.15.64:0/3522432170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4003520 con 0x7fa5b4003a10 2023-11-24T08:08:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5c3fff640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc144770 0x7fa5b40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.228+0000 7fa5d1779640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5b4004d00 0x7fa5b4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:43.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.229+0000 7fa5d0f78640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5b4003e30 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009f90 tx=0x7fa5b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d988df7bebf07d78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c802cb30 con 0x7fa5b4003a10 2023-11-24T08:08:43.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c800f040 con 0x7fa5b4003a10 2023-11-24T08:08:43.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5c37fe640 1 -- 172.21.15.64:0/3522432170 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8034440 con 0x7fa5b4003a10 2023-11-24T08:08:43.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/3522432170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 msgr2=0x7fa5b4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5d1f7a640 1 --2- 172.21.15.64:0/3522432170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5b4003e30 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009f90 tx=0x7fa5b40057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5d1779640 1 -- 172.21.15.64:0/3522432170 reap_dead start 2023-11-24T08:08:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/3522432170 shutdown_connections 2023-11-24T08:08:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/3522432170 >> 172.21.15.64:0/3522432170 conn(0x7fa5cc0cec10 msgr2=0x7fa5cc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.230+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/3522432170 shutdown_connections 2023-11-24T08:08:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.231+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/3522432170 wait complete. 2023-11-24T08:08:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.232+0000 7fa5d1f7a640 1 Processor -- start 2023-11-24T08:08:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.233+0000 7fa5d1f7a640 1 -- start start 2023-11-24T08:08:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.233+0000 7fa5d1f7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5cc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.233+0000 7fa5d1f7a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc15aa10 0x7fa5cc139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d1f7a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc1424e0 0x7fa5cc142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d1f7a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc1f2e40 con 0x7fa5b4003a10 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d1f7a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc142ea0 con 0x7fa5cc15aa10 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d1f7a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc139860 con 0x7fa5cc1424e0 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d0f78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5cc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d0f78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5cc070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33666/0 (socket says 172.21.15.64:33666) 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d0f78640 1 -- 172.21.15.64:0/410478299 learned_addr learned my addr 172.21.15.64:0/410478299 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5c3fff640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc15aa10 0x7fa5cc139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d1779640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc1424e0 0x7fa5cc142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d0f78640 1 -- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc1424e0 msgr2=0x7fa5cc142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d0f78640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc1424e0 0x7fa5cc142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d0f78640 1 -- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc15aa10 msgr2=0x7fa5cc139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d0f78640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc15aa10 0x7fa5cc139010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d0f78640 1 -- 172.21.15.64:0/410478299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc139b40 con 0x7fa5b4003a10 2023-11-24T08:08:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5c3fff640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc15aa10 0x7fa5cc139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:43.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.234+0000 7fa5d1779640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc1424e0 0x7fa5cc142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:43.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.235+0000 7fa5d0f78640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5cc070d40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa5c802c4d0 tx=0x7fa5c8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.235+0000 7fa5c17fa640 1 -- 172.21.15.64:0/410478299 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8034d40 con 0x7fa5b4003a10 2023-11-24T08:08:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.235+0000 7fa5c17fa640 1 -- 172.21.15.64:0/410478299 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8040450 con 0x7fa5b4003a10 2023-11-24T08:08:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.235+0000 7fa5c17fa640 1 -- 172.21.15.64:0/410478299 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8040800 con 0x7fa5b4003a10 2023-11-24T08:08:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.235+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc139e20 con 0x7fa5b4003a10 2023-11-24T08:08:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.235+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc15ae90 con 0x7fa5b4003a10 2023-11-24T08:08:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.236+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc04e840 con 0x7fa5b4003a10 2023-11-24T08:08:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.237+0000 7fa5c17fa640 1 -- 172.21.15.64:0/410478299 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fa5c8040960 con 0x7fa5b4003a10 2023-11-24T08:08:43.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.239+0000 7fa5c17fa640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa5b806b410 0x7fa5b806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.239+0000 7fa5c17fa640 1 -- 172.21.15.64:0/410478299 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa5c80ac6a0 con 0x7fa5b4003a10 2023-11-24T08:08:43.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.239+0000 7fa5c3fff640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa5b806b410 0x7fa5b806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.241+0000 7fa5c3fff640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa5b806b410 0x7fa5b806d8d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0716a0 tx=0x7fa5bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.243+0000 7fa5c17fa640 1 -- 172.21.15.64:0/410478299 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa5c8077050 con 0x7fa5b4003a10 2023-11-24T08:08:43.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.466+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp4507", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5cc144b50 con 0x7fa5b806b410 2023-11-24T08:08:43.481 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:08:43.479+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:08:43.481 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:08:43.479+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:08:43.481 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:08:43.479+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:08:43.481 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:08:43.479+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:08:43.481 INFO:tasks.ceph.mgr.x.smithi091.stderr:2023-11-24T08:08:43.479+0000 7fe78a34d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:08:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.512+0000 7fa5c17fa640 1 -- 172.21.15.64:0/410478299 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa5bc00d040 con 0x7fa5b806b410 2023-11-24T08:08:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.515+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa5b806b410 msgr2=0x7fa5b806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.515+0000 7fa5d1f7a640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa5b806b410 0x7fa5b806d8d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0716a0 tx=0x7fa5bc009040 comp rx=0 tx=0).stop 2023-11-24T08:08:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.515+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 msgr2=0x7fa5cc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.515+0000 7fa5d1f7a640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5cc070d40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa5c802c4d0 tx=0x7fa5c8009750 comp rx=0 tx=0).stop 2023-11-24T08:08:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.516+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 shutdown_connections 2023-11-24T08:08:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.516+0000 7fa5d1f7a640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa5b806b410 0x7fa5b806d8d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.516+0000 7fa5d1f7a640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa5cc1424e0 0x7fa5cc142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.516+0000 7fa5d1f7a640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa5cc15aa10 0x7fa5cc139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.516+0000 7fa5d1f7a640 1 --2- 172.21.15.64:0/410478299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b4003a10 0x7fa5cc070d40 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.516+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 >> 172.21.15.64:0/410478299 conn(0x7fa5cc0cec10 msgr2=0x7fa5cc15d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.516+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 shutdown_connections 2023-11-24T08:08:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.516+0000 7fa5d1f7a640 1 -- 172.21.15.64:0/410478299 wait complete. 2023-11-24T08:08:43.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol6487 --group_name subvol_grp4507 2023-11-24T08:08:43.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.774+0000 7f1d3f59e640 1 Processor -- start 2023-11-24T08:08:43.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.774+0000 7f1d3f59e640 1 -- start start 2023-11-24T08:08:43.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.776+0000 7f1d3f59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d40144770 0x7f1d4015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.776+0000 7f1d3f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d400b35a0 0x7f1d4015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.776+0000 7f1d3f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4015f930 con 0x7f1d4006c9d0 2023-11-24T08:08:43.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.776+0000 7f1d3f59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400b43b0 con 0x7f1d40144770 2023-11-24T08:08:43.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.776+0000 7f1d3f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400b3b10 con 0x7f1d400b35a0 2023-11-24T08:08:43.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.776+0000 7f1d3ed9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1d4006c9d0 0x7f1d4006ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57588/0 (socket says 172.21.15.64:57588) 2023-11-24T08:08:43.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.776+0000 7f1d3ed9d640 1 -- 172.21.15.64:0/1683618072 learned_addr learned my addr 172.21.15.64:0/1683618072 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:43.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.776+0000 7f1d3e59c640 1 --2- 172.21.15.64:0/1683618072 >> v2:172.21.15.91:3300/0 conn(0x7f1d40144770 0x7f1d4015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3dd9b640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d400b35a0 0x7f1d4015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17202395 0 0) 0x7f1d4015f930 con 0x7f1d4006c9d0 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d28003520 con 0x7f1d4006c9d0 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3940092221 0 0) 0x7f1d28003520 con 0x7f1d4006c9d0 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d400b35a0 msgr2=0x7f1d4015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d400b35a0 0x7f1d4015f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 >> v2:172.21.15.91:3300/0 conn(0x7f1d40144770 msgr2=0x7f1d4015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 --2- 172.21.15.64:0/1683618072 >> v2:172.21.15.91:3300/0 conn(0x7f1d40144770 0x7f1d4015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400b3df0 con 0x7f1d4006c9d0 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f1d380019f0 con 0x7f1d4006c9d0 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 >> v1:172.21.15.64:6789/0 conn(0x7f1d4006c9d0 legacy=0x7f1d4006ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d280039e0 0x7f1d28003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d400b35a0 0x7f1d28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400b43b0 con 0x7f1d280047c0 2023-11-24T08:08:43.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400b3b10 con 0x7f1d280039e0 2023-11-24T08:08:43.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.777+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2800b980 con 0x7f1d400b35a0 2023-11-24T08:08:43.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.778+0000 7f1d3dd9b640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.778+0000 7f1d3dd9b640 1 -- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d400b35a0 msgr2=0x7f1d28005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.778+0000 7f1d3dd9b640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d400b35a0 0x7f1d28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.778+0000 7f1d3e59c640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d280039e0 0x7f1d28003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.778+0000 7f1d3dd9b640 1 -- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d280039e0 msgr2=0x7f1d28003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.778+0000 7f1d3dd9b640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d280039e0 0x7f1d28003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.778+0000 7f1d3dd9b640 1 -- 172.21.15.64:0/1683618072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28003520 con 0x7f1d280047c0 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.779+0000 7f1d3dd9b640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d28004c40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009af0 tx=0x7f1d280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a410dfc212fabe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.779+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3402c710 con 0x7f1d280047c0 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.779+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3402c870 con 0x7f1d280047c0 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.779+0000 7f1d3d59a640 1 -- 172.21.15.64:0/1683618072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3402cc00 con 0x7f1d280047c0 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.780+0000 7f1d3f59e640 1 -- 172.21.15.64:0/1683618072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 msgr2=0x7f1d28004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.780+0000 7f1d3f59e640 1 --2- 172.21.15.64:0/1683618072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d28004c40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009af0 tx=0x7f1d280057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.780+0000 7f1d3ed9d640 1 -- 172.21.15.64:0/1683618072 reap_dead start 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.780+0000 7f1d3f59e640 1 -- 172.21.15.64:0/1683618072 shutdown_connections 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.780+0000 7f1d3f59e640 1 -- 172.21.15.64:0/1683618072 >> 172.21.15.64:0/1683618072 conn(0x7f1d400cec10 msgr2=0x7f1d400cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.780+0000 7f1d3f59e640 1 -- 172.21.15.64:0/1683618072 shutdown_connections 2023-11-24T08:08:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.780+0000 7f1d3f59e640 1 -- 172.21.15.64:0/1683618072 wait complete. 2023-11-24T08:08:43.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.782+0000 7f1d3f59e640 1 Processor -- start 2023-11-24T08:08:43.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.782+0000 7f1d3f59e640 1 -- start start 2023-11-24T08:08:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d4015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d401e49c0 0x7f1d40159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d401f2ff0 0x7f1d4015d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40070360 con 0x7f1d280047c0 2023-11-24T08:08:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401e5740 con 0x7f1d401f2ff0 2023-11-24T08:08:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401e4ea0 con 0x7f1d401e49c0 2023-11-24T08:08:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d4015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d4015a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33672/0 (socket says 172.21.15.64:33672) 2023-11-24T08:08:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.783+0000 7f1d3e59c640 1 -- 172.21.15.64:0/886611855 learned_addr learned my addr 172.21.15.64:0/886611855 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d3e59c640 1 -- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d401e49c0 msgr2=0x7f1d40159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d3e59c640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d401e49c0 0x7f1d40159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d3e59c640 1 -- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d401f2ff0 msgr2=0x7f1d4015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d3ed9d640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d401f2ff0 0x7f1d4015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d3e59c640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d401f2ff0 0x7f1d4015d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d3e59c640 1 -- 172.21.15.64:0/886611855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d401e5180 con 0x7f1d280047c0 2023-11-24T08:08:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d3ed9d640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d401f2ff0 0x7f1d4015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d3e59c640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d4015a590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1d300108e0 tx=0x7f1d30010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.784+0000 7f1d2effd640 1 -- 172.21.15.64:0/886611855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000eb60 con 0x7f1d280047c0 2023-11-24T08:08:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.785+0000 7f1d2effd640 1 -- 172.21.15.64:0/886611855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3000ecc0 con 0x7f1d280047c0 2023-11-24T08:08:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.785+0000 7f1d2effd640 1 -- 172.21.15.64:0/886611855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3001c400 con 0x7f1d280047c0 2023-11-24T08:08:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.785+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d401e5460 con 0x7f1d280047c0 2023-11-24T08:08:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.785+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d400ae110 con 0x7f1d280047c0 2023-11-24T08:08:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.786+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4004e840 con 0x7f1d280047c0 2023-11-24T08:08:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.787+0000 7f1d2effd640 1 -- 172.21.15.64:0/886611855 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1d3001c560 con 0x7f1d280047c0 2023-11-24T08:08:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.788+0000 7f1d2effd640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d2006b410 0x7f1d2006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.788+0000 7f1d2effd640 1 -- 172.21.15.64:0/886611855 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1d3008a810 con 0x7f1d280047c0 2023-11-24T08:08:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.789+0000 7f1d3dd9b640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d2006b410 0x7f1d2006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.790+0000 7f1d3dd9b640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d2006b410 0x7f1d2006d8d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1d34036000 tx=0x7f1d3402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:43.792+0000 7f1d2effd640 1 -- 172.21.15.64:0/886611855 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d300550a0 con 0x7f1d280047c0 2023-11-24T08:08:44.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.013+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol6487", "group_name": "subvol_grp4507", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d40144b50 con 0x7f1d2006b410 2023-11-24T08:08:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.080+0000 7f1d2effd640 1 -- 172.21.15.64:0/886611855 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1d3402c9e0 con 0x7f1d2006b410 2023-11-24T08:08:44.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d2006b410 msgr2=0x7f1d2006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d2006b410 0x7f1d2006d8d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1d34036000 tx=0x7f1d3402e000 comp rx=0 tx=0).stop 2023-11-24T08:08:44.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 msgr2=0x7f1d4015a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d4015a590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1d300108e0 tx=0x7f1d30010da0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 shutdown_connections 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d2006b410 0x7f1d2006d8d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d401f2ff0 0x7f1d4015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d401e49c0 0x7f1d40159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 --2- 172.21.15.64:0/886611855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d280047c0 0x7f1d4015a590 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 >> 172.21.15.64:0/886611855 conn(0x7f1d400cec10 msgr2=0x7f1d400afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.083+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 shutdown_connections 2023-11-24T08:08:44.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.084+0000 7f1d3f59e640 1 -- 172.21.15.64:0/886611855 wait complete. 2023-11-24T08:08:44.105 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:08:44.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:08:44.229 DEBUG:teuthology.orchestra.run.smithi091:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:08:44.375 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:08:44.375 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:08:44.375 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:44.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:44.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.650+0000 7f803ace7640 1 Processor -- start 2023-11-24T08:08:44.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.651+0000 7f803ace7640 1 -- start start 2023-11-24T08:08:44.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.652+0000 7f803ace7640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80340abff0 0x7f8034071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.652+0000 7f803ace7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803413da30 con 0x7f80340b39a0 2023-11-24T08:08:44.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.652+0000 7f803ace7640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340ac560 con 0x7f803415a100 2023-11-24T08:08:44.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.652+0000 7f803ace7640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80340ac840 con 0x7f80340abff0 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.653+0000 7f8033fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f80340b39a0 0x7f8034071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57596/0 (socket says 172.21.15.64:57596) 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.653+0000 7f8033fff640 1 -- 172.21.15.64:0/1797352247 learned_addr learned my addr 172.21.15.64:0/1797352247 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.653+0000 7f8039284640 1 --2- 172.21.15.64:0/1797352247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80340abff0 0x7f8034071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.654+0000 7f80337fe640 1 -- 172.21.15.64:0/1797352247 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311113586 0 0) 0x7f803413da30 con 0x7f80340b39a0 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.654+0000 7f80337fe640 1 -- 172.21.15.64:0/1797352247 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8024003520 con 0x7f80340b39a0 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.654+0000 7f80337fe640 1 -- 172.21.15.64:0/1797352247 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136662107 0 0) 0x7f80340ac560 con 0x7f803415a100 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.654+0000 7f80337fe640 1 -- 172.21.15.64:0/1797352247 --> v1:172.21.15.91:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f803413da30 con 0x7f803415a100 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.654+0000 7f80337fe640 1 -- 172.21.15.64:0/1797352247 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2086449169 0 0) 0x7f8024003520 con 0x7f80340b39a0 2023-11-24T08:08:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.654+0000 7f8039284640 1 -- 172.21.15.64:0/1797352247 >> v1:172.21.15.91:6789/0 conn(0x7f803415a100 legacy=0x7f80340b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.654+0000 7f8039284640 1 -- 172.21.15.64:0/1797352247 >> v1:172.21.15.64:6789/0 conn(0x7f80340b39a0 legacy=0x7f8034071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.654+0000 7f8039284640 1 -- 172.21.15.64:0/1797352247 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8034070650 con 0x7f80340abff0 2023-11-24T08:08:44.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.655+0000 7f8039284640 1 --2- 172.21.15.64:0/1797352247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80340abff0 0x7f8034071840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f802c009dc0 tx=0x7f802c02bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c25edb25b992e134 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.656+0000 7f80337fe640 1 -- 172.21.15.64:0/1797352247 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020001670 con 0x7f80340abff0 2023-11-24T08:08:44.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.656+0000 7f80337fe640 1 -- 172.21.15.64:0/1797352247 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c02c910 con 0x7f80340abff0 2023-11-24T08:08:44.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.656+0000 7f80337fe640 1 -- 172.21.15.64:0/1797352247 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c02cca0 con 0x7f80340abff0 2023-11-24T08:08:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.657+0000 7f803ace7640 1 -- 172.21.15.64:0/1797352247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80340abff0 msgr2=0x7f8034071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.657+0000 7f803ace7640 1 --2- 172.21.15.64:0/1797352247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80340abff0 0x7f8034071840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f802c009dc0 tx=0x7f802c02bde0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.657+0000 7f803ace7640 1 -- 172.21.15.64:0/1797352247 shutdown_connections 2023-11-24T08:08:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.657+0000 7f803ace7640 1 --2- 172.21.15.64:0/1797352247 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f80340abff0 0x7f8034071840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.657+0000 7f803ace7640 1 -- 172.21.15.64:0/1797352247 >> 172.21.15.64:0/1797352247 conn(0x7f80340b0d80 msgr2=0x7f80340b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.657+0000 7f803ace7640 1 -- 172.21.15.64:0/1797352247 shutdown_connections 2023-11-24T08:08:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.657+0000 7f803ace7640 1 -- 172.21.15.64:0/1797352247 wait complete. 2023-11-24T08:08:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.659+0000 7f803ace7640 1 Processor -- start 2023-11-24T08:08:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.659+0000 7f803ace7640 1 -- start start 2023-11-24T08:08:44.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f803ace7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80340abff0 0x7f80340cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f803ace7640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80340b39a0 0x7f80340c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f803ace7640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f803415a100 0x7f80340ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f803ace7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340ced10 con 0x7f80340abff0 2023-11-24T08:08:44.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f803ace7640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340c8fa0 con 0x7f80340b39a0 2023-11-24T08:08:44.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f803ace7640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803413f590 con 0x7f803415a100 2023-11-24T08:08:44.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f8038a83640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80340abff0 0x7f80340cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f8038a83640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80340abff0 0x7f80340cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33684/0 (socket says 172.21.15.64:33684) 2023-11-24T08:08:44.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.660+0000 7f8038a83640 1 -- 172.21.15.64:0/3013273236 learned_addr learned my addr 172.21.15.64:0/3013273236 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8033fff640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80340b39a0 0x7f80340c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8039284640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f803415a100 0x7f80340ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8038a83640 1 -- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f803415a100 msgr2=0x7f80340ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8038a83640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f803415a100 0x7f80340ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8038a83640 1 -- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80340b39a0 msgr2=0x7f80340c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8038a83640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80340b39a0 0x7f80340c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8038a83640 1 -- 172.21.15.64:0/3013273236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c009a70 con 0x7f80340abff0 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8033fff640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80340b39a0 0x7f80340c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f8038a83640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80340abff0 0x7f80340cba40 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f802800c490 tx=0x7f802800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f80317fa640 1 -- 172.21.15.64:0/3013273236 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280052d0 con 0x7f80340abff0 2023-11-24T08:08:44.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f80317fa640 1 -- 172.21.15.64:0/3013273236 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8028005430 con 0x7f80340abff0 2023-11-24T08:08:44.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803406e800 con 0x7f80340abff0 2023-11-24T08:08:44.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f80317fa640 1 -- 172.21.15.64:0/3013273236 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028012870 con 0x7f80340abff0 2023-11-24T08:08:44.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.661+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803406e520 con 0x7f80340abff0 2023-11-24T08:08:44.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.663+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8034067960 con 0x7f80340abff0 2023-11-24T08:08:44.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.664+0000 7f80317fa640 1 -- 172.21.15.64:0/3013273236 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f80280129d0 con 0x7f80340abff0 2023-11-24T08:08:44.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.665+0000 7f80317fa640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f801806b3e0 0x7f801806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.666+0000 7f80317fa640 1 -- 172.21.15.64:0/3013273236 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8028089bd0 con 0x7f80340abff0 2023-11-24T08:08:44.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.666+0000 7f8033fff640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f801806b3e0 0x7f801806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.668+0000 7f8033fff640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f801806b3e0 0x7f801806d8a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f80200011e0 tx=0x7f8020009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.671+0000 7f80317fa640 1 -- 172.21.15.64:0/3013273236 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8028054480 con 0x7f80340abff0 2023-11-24T08:08:44.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.867+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80340c90e0 con 0x7f80340abff0 2023-11-24T08:08:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.870+0000 7f80317fa640 1 -- 172.21.15.64:0/3013273236 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16562 (secure 0 0 0) 0x7f8028057aa0 con 0x7f80340abff0 2023-11-24T08:08:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:44.873 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":89,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:39.037812+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:21.467760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:22.606730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.2a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f801806b3e0 msgr2=0x7f801806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f801806b3e0 0x7f801806d8a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f80200011e0 tx=0x7f8020009040 comp rx=0 tx=0).stop 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80340abff0 msgr2=0x7f80340cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80340abff0 0x7f80340cba40 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f802800c490 tx=0x7f802800c950 comp rx=0 tx=0).stop 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 shutdown_connections 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f801806b3e0 0x7f801806d8a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f803415a100 0x7f80340ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f80340b39a0 0x7f80340c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.873+0000 7f803ace7640 1 --2- 172.21.15.64:0/3013273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80340abff0 0x7f80340cba40 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.874+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 >> 172.21.15.64:0/3013273236 conn(0x7f80340b0d80 msgr2=0x7f80340b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:44.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.874+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 shutdown_connections 2023-11-24T08:08:44.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:44.874+0000 7f803ace7640 1 -- 172.21.15.64:0/3013273236 wait complete. 2023-11-24T08:08:44.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:21.467760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:22.606730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:44.896 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:45.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.127+0000 7f69da852640 1 Processor -- start 2023-11-24T08:08:45.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.127+0000 7f69da852640 1 -- start start 2023-11-24T08:08:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.129+0000 7f69da852640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4153ea0 0x7f69d40ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.129+0000 7f69da852640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f69d4154280 0x7f69d415d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.129+0000 7f69da852640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d40ce650 con 0x7f69d4070850 2023-11-24T08:08:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.129+0000 7f69da852640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d415ddd0 con 0x7f69d4154280 2023-11-24T08:08:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.129+0000 7f69da852640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d415e0a0 con 0x7f69d4153ea0 2023-11-24T08:08:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.130+0000 7f69d9850640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4153ea0 0x7f69d40ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.130+0000 7f69d9850640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4153ea0 0x7f69d40ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:55522/0 (socket says 172.21.15.64:55522) 2023-11-24T08:08:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.130+0000 7f69d9850640 1 -- 172.21.15.64:0/353521215 learned_addr learned my addr 172.21.15.64:0/353521215 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.130+0000 7f69d904f640 1 --2- 172.21.15.64:0/353521215 >> v2:172.21.15.91:3300/0 conn(0x7f69d4154280 0x7f69d415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.130+0000 7f69d9850640 1 -- 172.21.15.64:0/353521215 >> v2:172.21.15.91:3300/0 conn(0x7f69d4154280 msgr2=0x7f69d415d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.130+0000 7f69d9850640 1 --2- 172.21.15.64:0/353521215 >> v2:172.21.15.91:3300/0 conn(0x7f69d4154280 0x7f69d415d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.131+0000 7f69d9850640 1 -- 172.21.15.64:0/353521215 >> v1:172.21.15.91:6789/0 conn(0x7f69d4070850 legacy=0x7f69d4071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.131+0000 7f69d9850640 1 -- 172.21.15.64:0/353521215 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d415e380 con 0x7f69d4153ea0 2023-11-24T08:08:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.132+0000 7f69d9850640 1 --2- 172.21.15.64:0/353521215 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4153ea0 0x7f69d40ad450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009870 tx=0x7f69d002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e984e0cc7f9a4fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.133+0000 7f69d884e640 1 -- 172.21.15.64:0/353521215 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0030840 con 0x7f69d4153ea0 2023-11-24T08:08:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.133+0000 7f69d884e640 1 -- 172.21.15.64:0/353521215 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d00309a0 con 0x7f69d4153ea0 2023-11-24T08:08:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.133+0000 7f69d884e640 1 -- 172.21.15.64:0/353521215 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0030d30 con 0x7f69d4153ea0 2023-11-24T08:08:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.134+0000 7f69da852640 1 -- 172.21.15.64:0/353521215 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4153ea0 msgr2=0x7f69d40ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.134+0000 7f69da852640 1 --2- 172.21.15.64:0/353521215 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4153ea0 0x7f69d40ad450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009870 tx=0x7f69d002b980 comp rx=0 tx=0).stop 2023-11-24T08:08:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.134+0000 7f69da852640 1 -- 172.21.15.64:0/353521215 shutdown_connections 2023-11-24T08:08:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.134+0000 7f69da852640 1 --2- 172.21.15.64:0/353521215 >> v2:172.21.15.91:3300/0 conn(0x7f69d4154280 0x7f69d415d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.134+0000 7f69da852640 1 --2- 172.21.15.64:0/353521215 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4153ea0 0x7f69d40ad450 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.134+0000 7f69da852640 1 -- 172.21.15.64:0/353521215 >> 172.21.15.64:0/353521215 conn(0x7f69d40cec10 msgr2=0x7f69d40cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.134+0000 7f69da852640 1 -- 172.21.15.64:0/353521215 shutdown_connections 2023-11-24T08:08:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.134+0000 7f69da852640 1 -- 172.21.15.64:0/353521215 wait complete. 2023-11-24T08:08:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.136+0000 7f69da852640 1 Processor -- start 2023-11-24T08:08:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.136+0000 7f69da852640 1 -- start start 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69da852640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d4070850 0x7f69d41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69da852640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69d4153ea0 0x7f69d4158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69da852640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4154280 0x7f69d40b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69da852640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4155b30 con 0x7f69d4070850 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69da852640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40b10a0 con 0x7f69d4153ea0 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69da852640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4155dd0 con 0x7f69d4154280 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69d9850640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d4070850 0x7f69d41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69d9850640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d4070850 0x7f69d41587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33698/0 (socket says 172.21.15.64:33698) 2023-11-24T08:08:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69d9850640 1 -- 172.21.15.64:0/14605886 learned_addr learned my addr 172.21.15.64:0/14605886 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.138+0000 7f69da051640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4154280 0x7f69d40b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.139+0000 7f69d904f640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69d4153ea0 0x7f69d4158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.139+0000 7f69d9850640 1 -- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4154280 msgr2=0x7f69d40b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.139+0000 7f69d9850640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4154280 0x7f69d40b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.139+0000 7f69d9850640 1 -- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69d4153ea0 msgr2=0x7f69d4158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.139+0000 7f69d9850640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69d4153ea0 0x7f69d4158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.139+0000 7f69d9850640 1 -- 172.21.15.64:0/14605886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00094d0 con 0x7f69d4070850 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.139+0000 7f69d9850640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d4070850 0x7f69d41587d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f69d0000c00 tx=0x7f69d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.140+0000 7f69ca7fc640 1 -- 172.21.15.64:0/14605886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0039c90 con 0x7f69d4070850 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.140+0000 7f69ca7fc640 1 -- 172.21.15.64:0/14605886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d0039e10 con 0x7f69d4070850 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.140+0000 7f69ca7fc640 1 -- 172.21.15.64:0/14605886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0043640 con 0x7f69d4070850 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.140+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d41560b0 con 0x7f69d4070850 2023-11-24T08:08:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.140+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d41f3a40 con 0x7f69d4070850 2023-11-24T08:08:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.141+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d41565d0 con 0x7f69d4070850 2023-11-24T08:08:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.142+0000 7f69ca7fc640 1 -- 172.21.15.64:0/14605886 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f69d00437a0 con 0x7f69d4070850 2023-11-24T08:08:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.143+0000 7f69ca7fc640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806b470 0x7f69b806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.143+0000 7f69ca7fc640 1 -- 172.21.15.64:0/14605886 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f69d003e070 con 0x7f69d4070850 2023-11-24T08:08:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.144+0000 7f69d904f640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806b470 0x7f69b806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.145+0000 7f69d904f640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806b470 0x7f69b806d930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f69d41558b0 tx=0x7f69c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.148+0000 7f69ca7fc640 1 -- 172.21.15.64:0/14605886 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69d41565d0 con 0x7f69d4070850 2023-11-24T08:08:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.342+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69d41565d0 con 0x7f69d4070850 2023-11-24T08:08:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.343+0000 7f69ca7fc640 1 -- 172.21.15.64:0/14605886 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69d0077900 con 0x7f69d4070850 2023-11-24T08:08:45.344 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:08:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806b470 msgr2=0x7f69b806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806b470 0x7f69b806d930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f69d41558b0 tx=0x7f69c4008040 comp rx=0 tx=0).stop 2023-11-24T08:08:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d4070850 msgr2=0x7f69d41587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d4070850 0x7f69d41587d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f69d0000c00 tx=0x7f69d0004730 comp rx=0 tx=0).stop 2023-11-24T08:08:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 shutdown_connections 2023-11-24T08:08:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f69b806b470 0x7f69b806d930 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f69d4154280 0x7f69d40b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f69d4153ea0 0x7f69d4158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 --2- 172.21.15.64:0/14605886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d4070850 0x7f69d41587d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.345+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 >> 172.21.15.64:0/14605886 conn(0x7f69d40cec10 msgr2=0x7f69d415eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.346+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 shutdown_connections 2023-11-24T08:08:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.346+0000 7f69da852640 1 -- 172.21.15.64:0/14605886 wait complete. 2023-11-24T08:08:45.369 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:08:45.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.584+0000 7f9559451640 1 Processor -- start 2023-11-24T08:08:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.585+0000 7f9559451640 1 -- start start 2023-11-24T08:08:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.587+0000 7f9559451640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9554162600 0x7f9554162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.587+0000 7f9559451640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f955406c3c0 0x7f955406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.587+0000 7f9559451640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9554162fa0 con 0x7f9554162220 2023-11-24T08:08:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.587+0000 7f9559451640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95540706a0 con 0x7f9554162600 2023-11-24T08:08:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.587+0000 7f9559451640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9554070c60 con 0x7f955406c3c0 2023-11-24T08:08:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.587+0000 7f954bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9554162600 0x7f9554162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.587+0000 7f954bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9554162600 0x7f9554162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33702/0 (socket says 172.21.15.64:33702) 2023-11-24T08:08:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.587+0000 7f954bfff640 1 -- 172.21.15.64:0/1982231035 learned_addr learned my addr 172.21.15.64:0/1982231035 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.588+0000 7f954bfff640 1 -- 172.21.15.64:0/1982231035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f955406c3c0 msgr2=0x7f955406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.588+0000 7f95537fe640 1 --2- 172.21.15.64:0/1982231035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f955406c3c0 0x7f955406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.588+0000 7f954bfff640 1 --2- 172.21.15.64:0/1982231035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f955406c3c0 0x7f955406c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.588+0000 7f954bfff640 1 -- 172.21.15.64:0/1982231035 >> v1:172.21.15.91:6789/0 conn(0x7f9554162220 legacy=0x7f95540b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.588+0000 7f954bfff640 1 -- 172.21.15.64:0/1982231035 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955406cd60 con 0x7f9554162600 2023-11-24T08:08:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.588+0000 7f95537fe640 1 --2- 172.21.15.64:0/1982231035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f955406c3c0 0x7f955406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.589+0000 7f954bfff640 1 --2- 172.21.15.64:0/1982231035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162600 0x7f9554162a60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f953c009870 tx=0x7f953c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2f3d90e82a52a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.589+0000 7f95527fc640 1 -- 172.21.15.64:0/1982231035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c031640 con 0x7f9554162600 2023-11-24T08:08:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.589+0000 7f95527fc640 1 -- 172.21.15.64:0/1982231035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c0317a0 con 0x7f9554162600 2023-11-24T08:08:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.589+0000 7f95527fc640 1 -- 172.21.15.64:0/1982231035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c031b30 con 0x7f9554162600 2023-11-24T08:08:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.590+0000 7f9559451640 1 -- 172.21.15.64:0/1982231035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162600 msgr2=0x7f9554162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.590+0000 7f9559451640 1 --2- 172.21.15.64:0/1982231035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162600 0x7f9554162a60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f953c009870 tx=0x7f953c02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.590+0000 7f9559451640 1 -- 172.21.15.64:0/1982231035 shutdown_connections 2023-11-24T08:08:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.590+0000 7f9559451640 1 --2- 172.21.15.64:0/1982231035 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f955406c3c0 0x7f955406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.590+0000 7f9559451640 1 --2- 172.21.15.64:0/1982231035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162600 0x7f9554162a60 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.590+0000 7f9559451640 1 -- 172.21.15.64:0/1982231035 >> 172.21.15.64:0/1982231035 conn(0x7f95540b0ba0 msgr2=0x7f95540b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:45.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.590+0000 7f9559451640 1 -- 172.21.15.64:0/1982231035 shutdown_connections 2023-11-24T08:08:45.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.590+0000 7f9559451640 1 -- 172.21.15.64:0/1982231035 wait complete. 2023-11-24T08:08:45.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.592+0000 7f9559451640 1 Processor -- start 2023-11-24T08:08:45.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.592+0000 7f9559451640 1 -- start start 2023-11-24T08:08:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f9559451640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f955406c3c0 0x7f95541ebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f9559451640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162220 0x7f955414f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f9559451640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9554162600 0x7f955409bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f9559451640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955409c890 con 0x7f9554162220 2023-11-24T08:08:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f9559451640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95541ec330 con 0x7f955406c3c0 2023-11-24T08:08:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f9559451640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955409c5b0 con 0x7f9554162600 2023-11-24T08:08:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f954bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162220 0x7f955414f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f954bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162220 0x7f955414f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33712/0 (socket says 172.21.15.64:33712) 2023-11-24T08:08:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f954bfff640 1 -- 172.21.15.64:0/2084935169 learned_addr learned my addr 172.21.15.64:0/2084935169 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f9552ffd640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f955406c3c0 0x7f95541ebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f95537fe640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9554162600 0x7f955409bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.593+0000 7f954bfff640 1 -- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9554162600 msgr2=0x7f955409bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f954bfff640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9554162600 0x7f955409bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f954bfff640 1 -- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f955406c3c0 msgr2=0x7f95541ebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f954bfff640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f955406c3c0 0x7f95541ebdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f954bfff640 1 -- 172.21.15.64:0/2084935169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c0094d0 con 0x7f9554162220 2023-11-24T08:08:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f95537fe640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9554162600 0x7f955409bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f9552ffd640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f955406c3c0 0x7f95541ebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f954bfff640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162220 0x7f955414f130 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f953c009870 tx=0x7f953c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f954b7fe640 1 -- 172.21.15.64:0/2084935169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c03ac60 con 0x7f9554162220 2023-11-24T08:08:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f954b7fe640 1 -- 172.21.15.64:0/2084935169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c03ade0 con 0x7f9554162220 2023-11-24T08:08:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95541eb310 con 0x7f9554162220 2023-11-24T08:08:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.594+0000 7f954b7fe640 1 -- 172.21.15.64:0/2084935169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c044630 con 0x7f9554162220 2023-11-24T08:08:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.595+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95541eb8d0 con 0x7f9554162220 2023-11-24T08:08:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.596+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95541eb170 con 0x7f9554162220 2023-11-24T08:08:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.597+0000 7f954b7fe640 1 -- 172.21.15.64:0/2084935169 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f953c02f050 con 0x7f9554162220 2023-11-24T08:08:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.598+0000 7f954b7fe640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f953406b470 0x7f953406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.598+0000 7f954b7fe640 1 -- 172.21.15.64:0/2084935169 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f953c0ac3c0 con 0x7f9554162220 2023-11-24T08:08:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.598+0000 7f9552ffd640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f953406b470 0x7f953406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.599+0000 7f9552ffd640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f953406b470 0x7f953406d930 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f954400a830 tx=0x7f9544008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.603+0000 7f954b7fe640 1 -- 172.21.15.64:0/2084935169 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95541eb170 con 0x7f9554162220 2023-11-24T08:08:45.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.795+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95541eb170 con 0x7f9554162220 2023-11-24T08:08:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.797+0000 7f954b7fe640 1 -- 172.21.15.64:0/2084935169 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95541eb170 con 0x7f9554162220 2023-11-24T08:08:45.798 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:08:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f953406b470 msgr2=0x7f953406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f953406b470 0x7f953406d930 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f954400a830 tx=0x7f9544008040 comp rx=0 tx=0).stop 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162220 msgr2=0x7f955414f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162220 0x7f955414f130 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f953c009870 tx=0x7f953c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 shutdown_connections 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f953406b470 0x7f953406d930 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9554162600 0x7f955409bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554162220 0x7f955414f130 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 --2- 172.21.15.64:0/2084935169 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f955406c3c0 0x7f95541ebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.799+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 >> 172.21.15.64:0/2084935169 conn(0x7f95540b0ba0 msgr2=0x7f955415f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.800+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 shutdown_connections 2023-11-24T08:08:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:45.800+0000 7f9559451640 1 -- 172.21.15.64:0/2084935169 wait complete. 2023-11-24T08:08:45.821 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:08:46.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:08:46.046 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:08:46.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.054+0000 7f8448fea640 1 Processor -- start 2023-11-24T08:08:46.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.054+0000 7f8448fea640 1 -- start start 2023-11-24T08:08:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f8448fea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8444160bd0 0x7f84440b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f8448fea640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84440b48d0 0x7f84440b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f8448fea640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f84440b3040 0x7f8444166280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f8448fea640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84440cdc70 con 0x7f84440b48d0 2023-11-24T08:08:46.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f8448fea640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84440b2870 con 0x7f84440b3040 2023-11-24T08:08:46.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f8448fea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84440b23e0 con 0x7f8444160bd0 2023-11-24T08:08:46.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f843bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84440b48d0 0x7f84440b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f843bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84440b48d0 0x7f84440b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33720/0 (socket says 172.21.15.64:33720) 2023-11-24T08:08:46.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f843bfff640 1 -- 172.21.15.64:0/2451949324 learned_addr learned my addr 172.21.15.64:0/2451949324 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:46.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.055+0000 7f8442d9d640 1 --2- 172.21.15.64:0/2451949324 >> v2:172.21.15.91:3300/0 conn(0x7f84440b3040 0x7f8444166280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.056+0000 7f844259c640 1 --2- 172.21.15.64:0/2451949324 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8444160bd0 0x7f84440b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.056+0000 7f843bfff640 1 -- 172.21.15.64:0/2451949324 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8444160bd0 msgr2=0x7f84440b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:46.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.056+0000 7f843bfff640 1 --2- 172.21.15.64:0/2451949324 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8444160bd0 0x7f84440b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.056+0000 7f843bfff640 1 -- 172.21.15.64:0/2451949324 >> v2:172.21.15.91:3300/0 conn(0x7f84440b3040 msgr2=0x7f8444166280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:46.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.056+0000 7f843bfff640 1 --2- 172.21.15.64:0/2451949324 >> v2:172.21.15.91:3300/0 conn(0x7f84440b3040 0x7f8444166280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.056+0000 7f843bfff640 1 -- 172.21.15.64:0/2451949324 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440d41b0 con 0x7f84440b48d0 2023-11-24T08:08:46.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.057+0000 7f843bfff640 1 --2- 172.21.15.64:0/2451949324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84440b48d0 0x7f84440b2b00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f84440d08d0 tx=0x7f842c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9902960a0d88e562 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8441d9b640 1 -- 172.21.15.64:0/2451949324 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0317e0 con 0x7f84440b48d0 2023-11-24T08:08:46.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8441d9b640 1 -- 172.21.15.64:0/2451949324 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c031940 con 0x7f84440b48d0 2023-11-24T08:08:46.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8441d9b640 1 -- 172.21.15.64:0/2451949324 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c031cd0 con 0x7f84440b48d0 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8448fea640 1 -- 172.21.15.64:0/2451949324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84440b48d0 msgr2=0x7f84440b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8448fea640 1 --2- 172.21.15.64:0/2451949324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84440b48d0 0x7f84440b2b00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f84440d08d0 tx=0x7f842c02b880 comp rx=0 tx=0).stop 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8448fea640 1 -- 172.21.15.64:0/2451949324 shutdown_connections 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8448fea640 1 --2- 172.21.15.64:0/2451949324 >> v2:172.21.15.91:3300/0 conn(0x7f84440b3040 0x7f8444166280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8448fea640 1 --2- 172.21.15.64:0/2451949324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84440b48d0 0x7f84440b2b00 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8448fea640 1 --2- 172.21.15.64:0/2451949324 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8444160bd0 0x7f84440b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8448fea640 1 -- 172.21.15.64:0/2451949324 >> 172.21.15.64:0/2451949324 conn(0x7f844409c5a0 msgr2=0x7f84440d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.058+0000 7f8448fea640 1 -- 172.21.15.64:0/2451949324 shutdown_connections 2023-11-24T08:08:46.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.059+0000 7f8448fea640 1 -- 172.21.15.64:0/2451949324 wait complete. 2023-11-24T08:08:46.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.060+0000 7f8448fea640 1 Processor -- start 2023-11-24T08:08:46.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.061+0000 7f8448fea640 1 -- start start 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.061+0000 7f8448fea640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84440b3040 0x7f84440cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8448fea640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84440b48d0 0x7f8444158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8448fea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444160bd0 0x7f84440b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8448fea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84440cc2d0 con 0x7f8444160bd0 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8448fea640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84440c7920 con 0x7f84440b3040 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8448fea640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84440c7c00 con 0x7f84440b48d0 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8442d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444160bd0 0x7f84440b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8442d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444160bd0 0x7f84440b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33728/0 (socket says 172.21.15.64:33728) 2023-11-24T08:08:46.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8442d9d640 1 -- 172.21.15.64:0/930465982 learned_addr learned my addr 172.21.15.64:0/930465982 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:46.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f844259c640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84440b3040 0x7f84440cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.062+0000 7f8442d9d640 1 -- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84440b48d0 msgr2=0x7f8444158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f8442d9d640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84440b48d0 0x7f8444158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f8442d9d640 1 -- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84440b3040 msgr2=0x7f84440cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f8442d9d640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84440b3040 0x7f84440cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f8442d9d640 1 -- 172.21.15.64:0/930465982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0094d0 con 0x7f8444160bd0 2023-11-24T08:08:46.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f8442d9d640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444160bd0 0x7f84440b18c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f843c00eed0 tx=0x7f843c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f843affd640 1 -- 172.21.15.64:0/930465982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c010040 con 0x7f8444160bd0 2023-11-24T08:08:46.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f843affd640 1 -- 172.21.15.64:0/930465982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c00bd20 con 0x7f8444160bd0 2023-11-24T08:08:46.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84440c7ee0 con 0x7f8444160bd0 2023-11-24T08:08:46.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f843affd640 1 -- 172.21.15.64:0/930465982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c014640 con 0x7f8444160bd0 2023-11-24T08:08:46.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.063+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84440b1e00 con 0x7f8444160bd0 2023-11-24T08:08:46.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.065+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84440c7780 con 0x7f8444160bd0 2023-11-24T08:08:46.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.066+0000 7f843affd640 1 -- 172.21.15.64:0/930465982 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f843c0147a0 con 0x7f8444160bd0 2023-11-24T08:08:46.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.067+0000 7f843affd640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f842406b470 0x7f842406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.067+0000 7f843affd640 1 -- 172.21.15.64:0/930465982 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f843c08e410 con 0x7f8444160bd0 2023-11-24T08:08:46.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.068+0000 7f844259c640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f842406b470 0x7f842406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.069+0000 7f844259c640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f842406b470 0x7f842406d930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8434006fd0 tx=0x7f8434008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.073+0000 7f843affd640 1 -- 172.21.15.64:0/930465982 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84440c7780 con 0x7f8444160bd0 2023-11-24T08:08:46.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.269+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84440c7780 con 0x7f8444160bd0 2023-11-24T08:08:46.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.270+0000 7f843affd640 1 -- 172.21.15.64:0/930465982 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f84440c7780 con 0x7f8444160bd0 2023-11-24T08:08:46.272 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:08:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f842406b470 msgr2=0x7f842406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f842406b470 0x7f842406d930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8434006fd0 tx=0x7f8434008040 comp rx=0 tx=0).stop 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444160bd0 msgr2=0x7f84440b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444160bd0 0x7f84440b18c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f843c00eed0 tx=0x7f843c00b480 comp rx=0 tx=0).stop 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 shutdown_connections 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f842406b470 0x7f842406d930 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444160bd0 0x7f84440b18c0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f84440b48d0 0x7f8444158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 --2- 172.21.15.64:0/930465982 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f84440b3040 0x7f84440cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 >> 172.21.15.64:0/930465982 conn(0x7f844409c5a0 msgr2=0x7f844406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 shutdown_connections 2023-11-24T08:08:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.273+0000 7f8448fea640 1 -- 172.21.15.64:0/930465982 wait complete. 2023-11-24T08:08:46.295 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:08:46.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.511+0000 7fcdade1a640 1 Processor -- start 2023-11-24T08:08:46.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.512+0000 7fcdade1a640 1 -- start start 2023-11-24T08:08:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.513+0000 7fcdade1a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcda815d5d0 0x7fcda80ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.513+0000 7fcdade1a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80d1d80 0x7fcda80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.513+0000 7fcdade1a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda80d04b0 con 0x7fcda80cf710 2023-11-24T08:08:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.513+0000 7fcdade1a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda80ce4e0 con 0x7fcda815d5d0 2023-11-24T08:08:46.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.513+0000 7fcdade1a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80cdb80 con 0x7fcda80d1d80 2023-11-24T08:08:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.514+0000 7fcdad619640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80d1d80 0x7fcda80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.514+0000 7fcdad619640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80d1d80 0x7fcda80cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:55578/0 (socket says 172.21.15.64:55578) 2023-11-24T08:08:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.514+0000 7fcdad619640 1 -- 172.21.15.64:0/1584028329 learned_addr learned my addr 172.21.15.64:0/1584028329 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.514+0000 7fcdace18640 1 --2- 172.21.15.64:0/1584028329 >> v2:172.21.15.91:3300/0 conn(0x7fcda815d5d0 0x7fcda80ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.515+0000 7fcdad619640 1 -- 172.21.15.64:0/1584028329 >> v2:172.21.15.91:3300/0 conn(0x7fcda815d5d0 msgr2=0x7fcda80ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.515+0000 7fcdad619640 1 --2- 172.21.15.64:0/1584028329 >> v2:172.21.15.91:3300/0 conn(0x7fcda815d5d0 0x7fcda80ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.515+0000 7fcdad619640 1 -- 172.21.15.64:0/1584028329 >> v1:172.21.15.91:6789/0 conn(0x7fcda80cf710 legacy=0x7fcda80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.515+0000 7fcdad619640 1 -- 172.21.15.64:0/1584028329 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda80d01d0 con 0x7fcda80d1d80 2023-11-24T08:08:46.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.516+0000 7fcdad619640 1 --2- 172.21.15.64:0/1584028329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80d1d80 0x7fcda80cfb70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009870 tx=0x7fcda002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85047dbd3fa6639e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.517+0000 7fcda77fe640 1 -- 172.21.15.64:0/1584028329 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0031640 con 0x7fcda80d1d80 2023-11-24T08:08:46.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.517+0000 7fcda77fe640 1 -- 172.21.15.64:0/1584028329 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda00317a0 con 0x7fcda80d1d80 2023-11-24T08:08:46.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.517+0000 7fcda77fe640 1 -- 172.21.15.64:0/1584028329 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0031b30 con 0x7fcda80d1d80 2023-11-24T08:08:46.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.518+0000 7fcdade1a640 1 -- 172.21.15.64:0/1584028329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80d1d80 msgr2=0x7fcda80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.518+0000 7fcdade1a640 1 --2- 172.21.15.64:0/1584028329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80d1d80 0x7fcda80cfb70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009870 tx=0x7fcda002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.518+0000 7fcdade1a640 1 -- 172.21.15.64:0/1584028329 shutdown_connections 2023-11-24T08:08:46.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.518+0000 7fcdade1a640 1 --2- 172.21.15.64:0/1584028329 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80d1d80 0x7fcda80cfb70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.518+0000 7fcdade1a640 1 --2- 172.21.15.64:0/1584028329 >> v2:172.21.15.91:3300/0 conn(0x7fcda815d5d0 0x7fcda80ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.518+0000 7fcdade1a640 1 -- 172.21.15.64:0/1584028329 >> 172.21.15.64:0/1584028329 conn(0x7fcda809c3d0 msgr2=0x7fcda80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:46.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.518+0000 7fcdade1a640 1 -- 172.21.15.64:0/1584028329 shutdown_connections 2023-11-24T08:08:46.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.518+0000 7fcdade1a640 1 -- 172.21.15.64:0/1584028329 wait complete. 2023-11-24T08:08:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.519+0000 7fcdade1a640 1 Processor -- start 2023-11-24T08:08:46.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.520+0000 7fcdade1a640 1 -- start start 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.520+0000 7fcdade1a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80cf710 0x7fcda815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.520+0000 7fcdade1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcda80d1d80 0x7fcda815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcdade1a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcda815d5d0 0x7fcda815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcdade1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81609c0 con 0x7fcda80d1d80 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcdade1a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80caef0 con 0x7fcda815d5d0 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcdade1a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80cb1d0 con 0x7fcda80cf710 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcda7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcda80d1d80 0x7fcda815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcda7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcda80d1d80 0x7fcda815f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33732/0 (socket says 172.21.15.64:33732) 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcda7fff640 1 -- 172.21.15.64:0/310310901 learned_addr learned my addr 172.21.15.64:0/310310901 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcdace18640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80cf710 0x7fcda815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcdad619640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcda815d5d0 0x7fcda815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcda7fff640 1 -- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80cf710 msgr2=0x7fcda815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcda7fff640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80cf710 0x7fcda815d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcda7fff640 1 -- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcda815d5d0 msgr2=0x7fcda815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcda7fff640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcda815d5d0 0x7fcda815fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcda7fff640 1 -- 172.21.15.64:0/310310901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00094d0 con 0x7fcda80d1d80 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.521+0000 7fcdad619640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcda815d5d0 0x7fcda815fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.522+0000 7fcda7fff640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcda80d1d80 0x7fcda815f690 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00c400 tx=0x7fcd9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.522+0000 7fcda57fa640 1 -- 172.21.15.64:0/310310901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0052d0 con 0x7fcda80d1d80 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.522+0000 7fcda57fa640 1 -- 172.21.15.64:0/310310901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c005430 con 0x7fcda80d1d80 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.522+0000 7fcda57fa640 1 -- 172.21.15.64:0/310310901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c003830 con 0x7fcda80d1d80 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.522+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda80cb4b0 con 0x7fcda80d1d80 2023-11-24T08:08:46.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.522+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda80cba70 con 0x7fcda80d1d80 2023-11-24T08:08:46.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.523+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda804e840 con 0x7fcda80d1d80 2023-11-24T08:08:46.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.524+0000 7fcda57fa640 1 -- 172.21.15.64:0/310310901 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fcd9c003990 con 0x7fcda80d1d80 2023-11-24T08:08:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.525+0000 7fcda57fa640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd9406b4a0 0x7fcd9406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.525+0000 7fcda57fa640 1 -- 172.21.15.64:0/310310901 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcd9c089650 con 0x7fcda80d1d80 2023-11-24T08:08:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.526+0000 7fcdace18640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd9406b4a0 0x7fcd9406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.527+0000 7fcdace18640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd9406b4a0 0x7fcd9406d960 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcda80cc0c0 tx=0x7fcd9800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.530+0000 7fcda57fa640 1 -- 172.21.15.64:0/310310901 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd9c053df0 con 0x7fcda80d1d80 2023-11-24T08:08:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.736+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcda80678d0 con 0x7fcda80d1d80 2023-11-24T08:08:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.737+0000 7fcda57fa640 1 -- 172.21.15.64:0/310310901 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd9c057410 con 0x7fcda80d1d80 2023-11-24T08:08:46.739 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:08:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd9406b4a0 msgr2=0x7fcd9406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd9406b4a0 0x7fcd9406d960 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcda80cc0c0 tx=0x7fcd9800b040 comp rx=0 tx=0).stop 2023-11-24T08:08:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcda80d1d80 msgr2=0x7fcda815f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcda80d1d80 0x7fcda815f690 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00c400 tx=0x7fcd9c00c8c0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 shutdown_connections 2023-11-24T08:08:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcd9406b4a0 0x7fcd9406d960 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcda815d5d0 0x7fcda815fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcda80d1d80 0x7fcda815f690 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 --2- 172.21.15.64:0/310310901 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcda80cf710 0x7fcda815d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.739+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 >> 172.21.15.64:0/310310901 conn(0x7fcda809c3d0 msgr2=0x7fcda80d1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.740+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 shutdown_connections 2023-11-24T08:08:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:46.740+0000 7fcdade1a640 1 -- 172.21.15.64:0/310310901 wait complete. 2023-11-24T08:08:46.761 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-11-24T08:08:46.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:46.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T08:08:46.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[115893]: fuse finished with error 0 and tester_r 0 2023-11-24T08:08:46.867 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:08:46.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:46.868 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:08:47.050 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:08:47.050 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:47.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.340+0000 7fcf163bc640 1 Processor -- start 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.340+0000 7fcf163bc640 1 -- start start 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.342+0000 7fcf163bc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcf1015e4a0 0x7fcf100b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.342+0000 7fcf163bc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf1006d440 0x7fcf1006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.342+0000 7fcf163bc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1015e900 con 0x7fcf10143770 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.342+0000 7fcf163bc640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1006de10 con 0x7fcf1015e4a0 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.342+0000 7fcf163bc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100cd4d0 con 0x7fcf1006d440 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.342+0000 7fcf153ba640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcf10143770 0x7fcf100caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33312/0 (socket says 172.21.15.64:33312) 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf153ba640 1 -- 172.21.15.64:0/2433332764 learned_addr learned my addr 172.21.15.64:0/2433332764 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf14bb9640 1 --2- 172.21.15.64:0/2433332764 >> v2:172.21.15.64:3300/0 conn(0x7fcf1015e4a0 0x7fcf100b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf15bbb640 1 --2- 172.21.15.64:0/2433332764 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf1006d440 0x7fcf1006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf14bb9640 1 -- 172.21.15.64:0/2433332764 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf1006d440 msgr2=0x7fcf1006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf14bb9640 1 --2- 172.21.15.64:0/2433332764 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf1006d440 0x7fcf1006d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf07fff640 1 -- 172.21.15.64:0/2433332764 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120586185 0 0) 0x7fcf1015e900 con 0x7fcf10143770 2023-11-24T08:08:47.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf14bb9640 1 -- 172.21.15.64:0/2433332764 >> v1:172.21.15.64:6789/0 conn(0x7fcf10143770 legacy=0x7fcf100caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf14bb9640 1 -- 172.21.15.64:0/2433332764 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf100cb740 con 0x7fcf1015e4a0 2023-11-24T08:08:47.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.343+0000 7fcf15bbb640 1 --2- 172.21.15.64:0/2433332764 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf1006d440 0x7fcf1006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:47.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.344+0000 7fcf14bb9640 1 --2- 172.21.15.64:0/2433332764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1015e4a0 0x7fcf100b26b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fcf000097a0 tx=0x7fcf0002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37130b48aa4859e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.344+0000 7fcf07fff640 1 -- 172.21.15.64:0/2433332764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0002d040 con 0x7fcf1015e4a0 2023-11-24T08:08:47.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.345+0000 7fcf07fff640 1 -- 172.21.15.64:0/2433332764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00031810 con 0x7fcf1015e4a0 2023-11-24T08:08:47.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.345+0000 7fcf07fff640 1 -- 172.21.15.64:0/2433332764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00031ba0 con 0x7fcf1015e4a0 2023-11-24T08:08:47.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.345+0000 7fcf163bc640 1 -- 172.21.15.64:0/2433332764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1015e4a0 msgr2=0x7fcf100b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.345+0000 7fcf163bc640 1 --2- 172.21.15.64:0/2433332764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1015e4a0 0x7fcf100b26b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fcf000097a0 tx=0x7fcf0002b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.345+0000 7fcf163bc640 1 -- 172.21.15.64:0/2433332764 shutdown_connections 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.345+0000 7fcf163bc640 1 --2- 172.21.15.64:0/2433332764 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf1006d440 0x7fcf1006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.345+0000 7fcf163bc640 1 --2- 172.21.15.64:0/2433332764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1015e4a0 0x7fcf100b26b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.345+0000 7fcf163bc640 1 -- 172.21.15.64:0/2433332764 >> 172.21.15.64:0/2433332764 conn(0x7fcf100b0b10 msgr2=0x7fcf100b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.346+0000 7fcf163bc640 1 -- 172.21.15.64:0/2433332764 shutdown_connections 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.346+0000 7fcf163bc640 1 -- 172.21.15.64:0/2433332764 wait complete. 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.347+0000 7fcf163bc640 1 Processor -- start 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.347+0000 7fcf163bc640 1 -- start start 2023-11-24T08:08:47.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.348+0000 7fcf163bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1006d440 0x7fcf100ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.348+0000 7fcf163bc640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf10143770 0x7fcf101f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.348+0000 7fcf163bc640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf1015e4a0 0x7fcf101f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.348+0000 7fcf163bc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10156260 con 0x7fcf1006d440 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.348+0000 7fcf163bc640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10070f10 con 0x7fcf1015e4a0 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.348+0000 7fcf163bc640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10155c10 con 0x7fcf10143770 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1006d440 0x7fcf100ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1006d440 0x7fcf100ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34804/0 (socket says 172.21.15.64:34804) 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 -- 172.21.15.64:0/177568740 learned_addr learned my addr 172.21.15.64:0/177568740 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf15bbb640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf1015e4a0 0x7fcf101f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 -- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf10143770 msgr2=0x7fcf101f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf14bb9640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf10143770 0x7fcf101f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf10143770 0x7fcf101f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 -- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf1015e4a0 msgr2=0x7fcf101f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf1015e4a0 0x7fcf101f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 -- 172.21.15.64:0/177568740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00009400 con 0x7fcf1006d440 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf15bbb640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf1015e4a0 0x7fcf101f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.349+0000 7fcf153ba640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1006d440 0x7fcf100ad1e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c4d0 tx=0x7fcf0c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.350+0000 7fcf05ffb640 1 -- 172.21.15.64:0/177568740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0074e0 con 0x7fcf1006d440 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.350+0000 7fcf05ffb640 1 -- 172.21.15.64:0/177568740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c007660 con 0x7fcf1006d440 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.350+0000 7fcf05ffb640 1 -- 172.21.15.64:0/177568740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c004a20 con 0x7fcf1006d440 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.350+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf10155ef0 con 0x7fcf1006d440 2023-11-24T08:08:47.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.350+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf100ad720 con 0x7fcf1006d440 2023-11-24T08:08:47.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.351+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1004e840 con 0x7fcf1006d440 2023-11-24T08:08:47.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.352+0000 7fcf05ffb640 1 -- 172.21.15.64:0/177568740 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fcf0c004b80 con 0x7fcf1006d440 2023-11-24T08:08:47.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.353+0000 7fcf05ffb640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcef406b470 0x7fcef406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.353+0000 7fcf05ffb640 1 -- 172.21.15.64:0/177568740 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcf0c0893d0 con 0x7fcf1006d440 2023-11-24T08:08:47.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.354+0000 7fcf14bb9640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcef406b470 0x7fcef406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.355+0000 7fcf14bb9640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcef406b470 0x7fcef406d930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcf00036000 tx=0x7fcf0003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.358+0000 7fcf05ffb640 1 -- 172.21.15.64:0/177568740 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf0c053ae0 con 0x7fcf1006d440 2023-11-24T08:08:47.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.551+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf100b2490 con 0x7fcf1006d440 2023-11-24T08:08:47.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.555+0000 7fcf05ffb640 1 -- 172.21.15.64:0/177568740 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16562 (secure 0 0 0) 0x7fcf0c057100 con 0x7fcf1006d440 2023-11-24T08:08:47.557 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:47.557 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":89,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:39.037812+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:21.467760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:22.606730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.2a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcef406b470 msgr2=0x7fcef406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcef406b470 0x7fcef406d930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcf00036000 tx=0x7fcf0003c040 comp rx=0 tx=0).stop 2023-11-24T08:08:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1006d440 msgr2=0x7fcf100ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1006d440 0x7fcf100ad1e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c4d0 tx=0x7fcf0c00c990 comp rx=0 tx=0).stop 2023-11-24T08:08:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 shutdown_connections 2023-11-24T08:08:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcef406b470 0x7fcef406d930 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf1015e4a0 0x7fcf101f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf10143770 0x7fcf101f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.557+0000 7fcf163bc640 1 --2- 172.21.15.64:0/177568740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf1006d440 0x7fcf100ad1e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.558+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 >> 172.21.15.64:0/177568740 conn(0x7fcf100b0b10 msgr2=0x7fcf100cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.558+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 shutdown_connections 2023-11-24T08:08:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.558+0000 7fcf163bc640 1 -- 172.21.15.64:0/177568740 wait complete. 2023-11-24T08:08:47.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:21.467760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:22.606730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:47.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:47.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.811+0000 7efdcafb3640 1 Processor -- start 2023-11-24T08:08:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.811+0000 7efdcafb3640 1 -- start start 2023-11-24T08:08:47.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.813+0000 7efdcafb3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdc41439b0 0x7efdc4070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.813+0000 7efdcafb3640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc4070d40 con 0x7efdc40b4140 2023-11-24T08:08:47.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.813+0000 7efdcafb3640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc4071020 con 0x7efdc41435d0 2023-11-24T08:08:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.813+0000 7efdcafb3640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc4071300 con 0x7efdc41439b0 2023-11-24T08:08:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.813+0000 7efdca7b2640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdc41439b0 0x7efdc4070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.814+0000 7efdc97b0640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efdc40b4140 0x7efdc4070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33322/0 (socket says 172.21.15.64:33322) 2023-11-24T08:08:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.814+0000 7efdc97b0640 1 -- 172.21.15.64:0/3763933614 learned_addr learned my addr 172.21.15.64:0/3763933614 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:47.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.814+0000 7efdca7b2640 1 -- 172.21.15.64:0/3763933614 >> v1:172.21.15.91:6789/0 conn(0x7efdc41435d0 legacy=0x7efdc40b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.814+0000 7efdca7b2640 1 -- 172.21.15.64:0/3763933614 >> v1:172.21.15.64:6789/0 conn(0x7efdc40b4140 legacy=0x7efdc4070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.814+0000 7efdca7b2640 1 -- 172.21.15.64:0/3763933614 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc40715e0 con 0x7efdc41439b0 2023-11-24T08:08:47.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.815+0000 7efdca7b2640 1 --2- 172.21.15.64:0/3763933614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41439b0 0x7efdc4070760 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009d90 tx=0x7efdbc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643745eec0c4f17b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.816+0000 7efdc8faf640 1 -- 172.21.15.64:0/3763933614 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc02cc60 con 0x7efdc41439b0 2023-11-24T08:08:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.816+0000 7efdc8faf640 1 -- 172.21.15.64:0/3763933614 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc02cdc0 con 0x7efdc41439b0 2023-11-24T08:08:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.816+0000 7efdc8faf640 1 -- 172.21.15.64:0/3763933614 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc034660 con 0x7efdc41439b0 2023-11-24T08:08:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.816+0000 7efdcafb3640 1 -- 172.21.15.64:0/3763933614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41439b0 msgr2=0x7efdc4070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.816+0000 7efdcafb3640 1 --2- 172.21.15.64:0/3763933614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41439b0 0x7efdc4070760 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009d90 tx=0x7efdbc02beb0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.817+0000 7efdcafb3640 1 -- 172.21.15.64:0/3763933614 shutdown_connections 2023-11-24T08:08:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.817+0000 7efdcafb3640 1 --2- 172.21.15.64:0/3763933614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41439b0 0x7efdc4070760 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.817+0000 7efdcafb3640 1 -- 172.21.15.64:0/3763933614 >> 172.21.15.64:0/3763933614 conn(0x7efdc40cec10 msgr2=0x7efdc40cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.817+0000 7efdcafb3640 1 -- 172.21.15.64:0/3763933614 shutdown_connections 2023-11-24T08:08:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.817+0000 7efdcafb3640 1 -- 172.21.15.64:0/3763933614 wait complete. 2023-11-24T08:08:47.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.818+0000 7efdcafb3640 1 Processor -- start 2023-11-24T08:08:47.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.819+0000 7efdcafb3640 1 -- start start 2023-11-24T08:08:47.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdcafb3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc40b4140 0x7efdc40ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdcafb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41435d0 0x7efdc413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdcafb3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc41439b0 0x7efdc40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdcafb3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc4158020 con 0x7efdc41435d0 2023-11-24T08:08:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdcafb3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc413a640 con 0x7efdc40b4140 2023-11-24T08:08:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdcafb3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc413a920 con 0x7efdc41439b0 2023-11-24T08:08:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc97b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41435d0 0x7efdc413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc97b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41435d0 0x7efdc413cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34828/0 (socket says 172.21.15.64:34828) 2023-11-24T08:08:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc97b0640 1 -- 172.21.15.64:0/2019440485 learned_addr learned my addr 172.21.15.64:0/2019440485 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc9fb1640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc40b4140 0x7efdc40ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc97b0640 1 -- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc41439b0 msgr2=0x7efdc40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdca7b2640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc41439b0 0x7efdc40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc97b0640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc41439b0 0x7efdc40cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc97b0640 1 -- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc40b4140 msgr2=0x7efdc40ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc97b0640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc40b4140 0x7efdc40ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc97b0640 1 -- 172.21.15.64:0/2019440485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc009a70 con 0x7efdc41435d0 2023-11-24T08:08:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.820+0000 7efdc9fb1640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc40b4140 0x7efdc40ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.821+0000 7efdc97b0640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41435d0 0x7efdc413cbc0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efdb400c4a0 tx=0x7efdb400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.821+0000 7efdba7fc640 1 -- 172.21.15.64:0/2019440485 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb4005ce0 con 0x7efdc41435d0 2023-11-24T08:08:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.821+0000 7efdba7fc640 1 -- 172.21.15.64:0/2019440485 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb4005e40 con 0x7efdc41435d0 2023-11-24T08:08:47.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.821+0000 7efdba7fc640 1 -- 172.21.15.64:0/2019440485 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb4007270 con 0x7efdc41435d0 2023-11-24T08:08:47.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.821+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc413ac00 con 0x7efdc41435d0 2023-11-24T08:08:47.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.821+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc413aee0 con 0x7efdc41435d0 2023-11-24T08:08:47.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.823+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc414d4f0 con 0x7efdc41435d0 2023-11-24T08:08:47.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.823+0000 7efdba7fc640 1 -- 172.21.15.64:0/2019440485 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7efdb4007490 con 0x7efdc41435d0 2023-11-24T08:08:47.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.824+0000 7efdba7fc640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda806b470 0x7efda806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.824+0000 7efdba7fc640 1 -- 172.21.15.64:0/2019440485 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efdb4089640 con 0x7efdc41435d0 2023-11-24T08:08:47.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.825+0000 7efdc9fb1640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda806b470 0x7efda806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.826+0000 7efdc9fb1640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda806b470 0x7efda806d930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efdc0000f60 tx=0x7efdc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:47.829+0000 7efdba7fc640 1 -- 172.21.15.64:0/2019440485 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efdc414d4f0 con 0x7efdc41435d0 2023-11-24T08:08:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.023+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdc414d4f0 con 0x7efdc41435d0 2023-11-24T08:08:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.024+0000 7efdba7fc640 1 -- 172.21.15.64:0/2019440485 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdc414d4f0 con 0x7efdc41435d0 2023-11-24T08:08:48.026 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda806b470 msgr2=0x7efda806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda806b470 0x7efda806d930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efdc0000f60 tx=0x7efdc0009040 comp rx=0 tx=0).stop 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41435d0 msgr2=0x7efdc413cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41435d0 0x7efdc413cbc0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efdb400c4a0 tx=0x7efdb400c960 comp rx=0 tx=0).stop 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 shutdown_connections 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7efda806b470 0x7efda806d930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7efdc41439b0 0x7efdc40cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc41435d0 0x7efdc413cbc0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 --2- 172.21.15.64:0/2019440485 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7efdc40b4140 0x7efdc40ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 >> 172.21.15.64:0/2019440485 conn(0x7efdc40cec10 msgr2=0x7efdc406cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 shutdown_connections 2023-11-24T08:08:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.027+0000 7efdcafb3640 1 -- 172.21.15.64:0/2019440485 wait complete. 2023-11-24T08:08:48.047 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:08:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.280+0000 7f5a148c5640 1 Processor -- start 2023-11-24T08:08:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.280+0000 7f5a148c5640 1 -- start start 2023-11-24T08:08:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.282+0000 7f5a148c5640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f5a101435d0 0x7f5a10071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.282+0000 7f5a148c5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a100b4140 0x7f5a10070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.282+0000 7f5a148c5640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a10070d40 con 0x7f5a101439b0 2023-11-24T08:08:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.282+0000 7f5a148c5640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a10071020 con 0x7f5a101435d0 2023-11-24T08:08:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.282+0000 7f5a148c5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10071300 con 0x7f5a100b4140 2023-11-24T08:08:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a100b4140 0x7f5a10070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0e59c640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f5a101439b0 0x7f5a100b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:41824/0 (socket says 172.21.15.64:41824) 2023-11-24T08:08:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0e59c640 1 -- 172.21.15.64:0/1085404842 learned_addr learned my addr 172.21.15.64:0/1085404842 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0ed9d640 1 --2- 172.21.15.64:0/1085404842 >> v2:172.21.15.91:3300/0 conn(0x7f5a101435d0 0x7f5a10071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0f59e640 1 -- 172.21.15.64:0/1085404842 >> v2:172.21.15.91:3300/0 conn(0x7f5a101435d0 msgr2=0x7f5a10071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0f59e640 1 --2- 172.21.15.64:0/1085404842 >> v2:172.21.15.91:3300/0 conn(0x7f5a101435d0 0x7f5a10071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0dd9b640 1 -- 172.21.15.64:0/1085404842 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430418426 0 0) 0x7f5a10070d40 con 0x7f5a101439b0 2023-11-24T08:08:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0f59e640 1 -- 172.21.15.64:0/1085404842 >> v1:172.21.15.91:6789/0 conn(0x7f5a101439b0 legacy=0x7f5a100b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.283+0000 7f5a0f59e640 1 -- 172.21.15.64:0/1085404842 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a100715e0 con 0x7f5a100b4140 2023-11-24T08:08:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.285+0000 7f5a0f59e640 1 --2- 172.21.15.64:0/1085404842 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a100b4140 0x7f5a10070050 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009d90 tx=0x7f5a0802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e8f6e377ade0c9b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.285+0000 7f5a0dd9b640 1 -- 172.21.15.64:0/1085404842 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0802cc50 con 0x7f5a100b4140 2023-11-24T08:08:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.285+0000 7f5a0dd9b640 1 -- 172.21.15.64:0/1085404842 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0802cdb0 con 0x7f5a100b4140 2023-11-24T08:08:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.285+0000 7f5a0dd9b640 1 -- 172.21.15.64:0/1085404842 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08034660 con 0x7f5a100b4140 2023-11-24T08:08:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.286+0000 7f5a148c5640 1 -- 172.21.15.64:0/1085404842 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a100b4140 msgr2=0x7f5a10070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.286+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1085404842 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a100b4140 0x7f5a10070050 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009d90 tx=0x7f5a0802bea0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.286+0000 7f5a148c5640 1 -- 172.21.15.64:0/1085404842 shutdown_connections 2023-11-24T08:08:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.286+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1085404842 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a100b4140 0x7f5a10070050 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.286+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1085404842 >> v2:172.21.15.91:3300/0 conn(0x7f5a101435d0 0x7f5a10071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.286+0000 7f5a148c5640 1 -- 172.21.15.64:0/1085404842 >> 172.21.15.64:0/1085404842 conn(0x7f5a100cec10 msgr2=0x7f5a100cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.286+0000 7f5a148c5640 1 -- 172.21.15.64:0/1085404842 shutdown_connections 2023-11-24T08:08:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.286+0000 7f5a148c5640 1 -- 172.21.15.64:0/1085404842 wait complete. 2023-11-24T08:08:48.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.288+0000 7f5a148c5640 1 Processor -- start 2023-11-24T08:08:48.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.288+0000 7f5a148c5640 1 -- start start 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a148c5640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a100b4140 0x7f5a101524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a148c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a101435d0 0x7f5a100cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a148c5640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a101439b0 0x7f5a100cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a148c5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10152fa0 con 0x7f5a101435d0 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a148c5640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10153280 con 0x7f5a100b4140 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a148c5640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10153560 con 0x7f5a101439b0 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a101435d0 0x7f5a100cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a101435d0 0x7f5a100cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34840/0 (socket says 172.21.15.64:34840) 2023-11-24T08:08:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a0e59c640 1 -- 172.21.15.64:0/1659971841 learned_addr learned my addr 172.21.15.64:0/1659971841 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.289+0000 7f5a0ed9d640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a100b4140 0x7f5a101524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a0f59e640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a101439b0 0x7f5a100cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a0e59c640 1 -- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a101439b0 msgr2=0x7f5a100cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a0e59c640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a101439b0 0x7f5a100cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a0e59c640 1 -- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a100b4140 msgr2=0x7f5a101524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a0e59c640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a100b4140 0x7f5a101524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a0e59c640 1 -- 172.21.15.64:0/1659971841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08009a70 con 0x7f5a101435d0 2023-11-24T08:08:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a0ed9d640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a100b4140 0x7f5a101524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:48.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a0e59c640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a101435d0 0x7f5a100cd860 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c410 tx=0x7f5a0400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f59f77fe640 1 -- 172.21.15.64:0/1659971841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040074e0 con 0x7f5a101435d0 2023-11-24T08:08:48.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f59f77fe640 1 -- 172.21.15.64:0/1659971841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a04007660 con 0x7f5a101435d0 2023-11-24T08:08:48.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f59f77fe640 1 -- 172.21.15.64:0/1659971841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04004a20 con 0x7f5a101435d0 2023-11-24T08:08:48.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.290+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a10153840 con 0x7f5a101435d0 2023-11-24T08:08:48.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.291+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a10139d30 con 0x7f5a101435d0 2023-11-24T08:08:48.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.291+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a10153d60 con 0x7f5a101435d0 2023-11-24T08:08:48.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.293+0000 7f59f77fe640 1 -- 172.21.15.64:0/1659971841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f5a04004b80 con 0x7f5a101435d0 2023-11-24T08:08:48.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.293+0000 7f59f77fe640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f59fc06b4a0 0x7f59fc06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.293+0000 7f59f77fe640 1 -- 172.21.15.64:0/1659971841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5a04089eb0 con 0x7f5a101435d0 2023-11-24T08:08:48.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.294+0000 7f5a0ed9d640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f59fc06b4a0 0x7f59fc06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.295+0000 7f5a0ed9d640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f59fc06b4a0 0x7f59fc06d960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009ac0 tx=0x7f5a00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.299+0000 7f59f77fe640 1 -- 172.21.15.64:0/1659971841 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a10153d60 con 0x7f5a101435d0 2023-11-24T08:08:48.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.493+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a10153d60 con 0x7f5a101435d0 2023-11-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.494+0000 7f59f77fe640 1 -- 172.21.15.64:0/1659971841 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a10153d60 con 0x7f5a101435d0 2023-11-24T08:08:48.496 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:08:48.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f59fc06b4a0 msgr2=0x7f59fc06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f59fc06b4a0 0x7f59fc06d960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009ac0 tx=0x7f5a00008040 comp rx=0 tx=0).stop 2023-11-24T08:08:48.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a101435d0 msgr2=0x7f5a100cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a101435d0 0x7f5a100cd860 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c410 tx=0x7f5a0400c8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 shutdown_connections 2023-11-24T08:08:48.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f59fc06b4a0 0x7f59fc06d960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f5a101439b0 0x7f5a100cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a101435d0 0x7f5a100cd860 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 --2- 172.21.15.64:0/1659971841 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f5a100b4140 0x7f5a101524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 >> 172.21.15.64:0/1659971841 conn(0x7f5a100cec10 msgr2=0x7f5a1006cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 shutdown_connections 2023-11-24T08:08:48.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.497+0000 7f5a148c5640 1 -- 172.21.15.64:0/1659971841 wait complete. 2023-11-24T08:08:48.519 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:08:48.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.748+0000 7f946ffff640 1 Processor -- start 2023-11-24T08:08:48.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.748+0000 7f946ffff640 1 -- start start 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.749+0000 7f946ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f94700d1d90 0x7f94700ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946ffff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f94700d0fa0 0x7f94700b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946ffff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94700c6bf0 con 0x7f947015d3b0 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946ffff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94700d13f0 con 0x7f94700d1d90 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946ffff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94700cedc0 con 0x7f94700d0fa0 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946e7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f94700d1d90 0x7f94700ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946effd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f947015d3b0 0x7f94700d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33334/0 (socket says 172.21.15.64:33334) 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946effd640 1 -- 172.21.15.64:0/2453321301 learned_addr learned my addr 172.21.15.64:0/2453321301 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:48.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946f7fe640 1 --2- 172.21.15.64:0/2453321301 >> v2:172.21.15.91:3300/0 conn(0x7f94700d0fa0 0x7f94700b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946e7fc640 1 -- 172.21.15.64:0/2453321301 >> v2:172.21.15.91:3300/0 conn(0x7f94700d0fa0 msgr2=0x7f94700b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:48.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.750+0000 7f946e7fc640 1 --2- 172.21.15.64:0/2453321301 >> v2:172.21.15.91:3300/0 conn(0x7f94700d0fa0 0x7f94700b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.751+0000 7f946e7fc640 1 -- 172.21.15.64:0/2453321301 >> v1:172.21.15.64:6789/0 conn(0x7f947015d3b0 legacy=0x7f94700d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.751+0000 7f946e7fc640 1 -- 172.21.15.64:0/2453321301 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700c6e40 con 0x7f94700d1d90 2023-11-24T08:08:48.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.752+0000 7f946e7fc640 1 --2- 172.21.15.64:0/2453321301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d1d90 0x7f94700ce6c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9470072b10 tx=0x7f946402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd57f33ffcf1358 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.752+0000 7f946dffb640 1 -- 172.21.15.64:0/2453321301 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946402e070 con 0x7f94700d1d90 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.752+0000 7f946dffb640 1 -- 172.21.15.64:0/2453321301 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640315a0 con 0x7f94700d1d90 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.752+0000 7f946dffb640 1 -- 172.21.15.64:0/2453321301 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464031930 con 0x7f94700d1d90 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.752+0000 7f946ffff640 1 -- 172.21.15.64:0/2453321301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d1d90 msgr2=0x7f94700ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.752+0000 7f946ffff640 1 --2- 172.21.15.64:0/2453321301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d1d90 0x7f94700ce6c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9470072b10 tx=0x7f946402b840 comp rx=0 tx=0).stop 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.753+0000 7f946ffff640 1 -- 172.21.15.64:0/2453321301 shutdown_connections 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.753+0000 7f946ffff640 1 --2- 172.21.15.64:0/2453321301 >> v2:172.21.15.91:3300/0 conn(0x7f94700d0fa0 0x7f94700b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.753+0000 7f946ffff640 1 --2- 172.21.15.64:0/2453321301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d1d90 0x7f94700ce6c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.753+0000 7f946ffff640 1 -- 172.21.15.64:0/2453321301 >> 172.21.15.64:0/2453321301 conn(0x7f94700d30a0 msgr2=0x7f94700d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.753+0000 7f946ffff640 1 -- 172.21.15.64:0/2453321301 shutdown_connections 2023-11-24T08:08:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.753+0000 7f946ffff640 1 -- 172.21.15.64:0/2453321301 wait complete. 2023-11-24T08:08:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.754+0000 7f946ffff640 1 Processor -- start 2023-11-24T08:08:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.755+0000 7f946ffff640 1 -- start start 2023-11-24T08:08:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.755+0000 7f946ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d0fa0 0x7f94701f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.755+0000 7f946ffff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94700d1d90 0x7f94700cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946ffff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947015d3b0 0x7f94700b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94700ca170 con 0x7f94700d0fa0 2023-11-24T08:08:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946ffff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94701f30f0 con 0x7f947015d3b0 2023-11-24T08:08:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946ffff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94701f33d0 con 0x7f94700d1d90 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d0fa0 0x7f94701f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d0fa0 0x7f94701f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34866/0 (socket says 172.21.15.64:34866) 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946effd640 1 -- 172.21.15.64:0/1767866805 learned_addr learned my addr 172.21.15.64:0/1767866805 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946effd640 1 -- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94700d1d90 msgr2=0x7f94700cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946effd640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94700d1d90 0x7f94700cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946effd640 1 -- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947015d3b0 msgr2=0x7f94700b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946effd640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947015d3b0 0x7f94700b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.756+0000 7f946effd640 1 -- 172.21.15.64:0/1767866805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464009400 con 0x7f94700d0fa0 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.757+0000 7f946effd640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d0fa0 0x7f94701f2b10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f946000cbc0 tx=0x7f946000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.757+0000 7f9457fff640 1 -- 172.21.15.64:0/1767866805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000aca0 con 0x7f94700d0fa0 2023-11-24T08:08:48.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.757+0000 7f9457fff640 1 -- 172.21.15.64:0/1767866805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946000ae00 con 0x7f94700d0fa0 2023-11-24T08:08:48.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.757+0000 7f9457fff640 1 -- 172.21.15.64:0/1767866805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460012600 con 0x7f94700d0fa0 2023-11-24T08:08:48.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.757+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94701f36b0 con 0x7f94700d0fa0 2023-11-24T08:08:48.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.757+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94700cd3c0 con 0x7f94700d0fa0 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.758+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947004e840 con 0x7f94700d0fa0 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.759+0000 7f9457fff640 1 -- 172.21.15.64:0/1767866805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f9460007070 con 0x7f94700d0fa0 2023-11-24T08:08:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.760+0000 7f9457fff640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b470 0x7f945c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.760+0000 7f9457fff640 1 -- 172.21.15.64:0/1767866805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f946008a010 con 0x7f94700d0fa0 2023-11-24T08:08:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.761+0000 7f946e7fc640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b470 0x7f945c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.762+0000 7f946e7fc640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b470 0x7f945c06d930 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9464036000 tx=0x7f9464030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.765+0000 7f9457fff640 1 -- 172.21.15.64:0/1767866805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9460054810 con 0x7f94700d0fa0 2023-11-24T08:08:48.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.960+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f947006cab0 con 0x7f94700d0fa0 2023-11-24T08:08:48.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.961+0000 7f9457fff640 1 -- 172.21.15.64:0/1767866805 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9460017020 con 0x7f94700d0fa0 2023-11-24T08:08:48.962 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:08:48.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.963+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b470 msgr2=0x7f945c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.963+0000 7f946ffff640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b470 0x7f945c06d930 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9464036000 tx=0x7f9464030000 comp rx=0 tx=0).stop 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.963+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d0fa0 msgr2=0x7f94701f2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.963+0000 7f946ffff640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d0fa0 0x7f94701f2b10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f946000cbc0 tx=0x7f946000a060 comp rx=0 tx=0).stop 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.964+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 shutdown_connections 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.964+0000 7f946ffff640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f945c06b470 0x7f945c06d930 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.964+0000 7f946ffff640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f947015d3b0 0x7f94700b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.964+0000 7f946ffff640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f94700d1d90 0x7f94700cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.964+0000 7f946ffff640 1 --2- 172.21.15.64:0/1767866805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94700d0fa0 0x7f94701f2b10 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.964+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 >> 172.21.15.64:0/1767866805 conn(0x7f94700d30a0 msgr2=0x7f94700c87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.964+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 shutdown_connections 2023-11-24T08:08:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:48.964+0000 7f946ffff640 1 -- 172.21.15.64:0/1767866805 wait complete. 2023-11-24T08:08:48.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:08:49.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.214+0000 7f0caf44b640 1 Processor -- start 2023-11-24T08:08:49.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.215+0000 7f0caf44b640 1 -- start start 2023-11-24T08:08:49.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.216+0000 7f0caf44b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0ca81416f0 0x7f0ca815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.216+0000 7f0caf44b640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0ca806d230 0x7f0ca815f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.216+0000 7f0caf44b640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca8070bf0 con 0x7f0ca8141e80 2023-11-24T08:08:49.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.216+0000 7f0caf44b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca806db30 con 0x7f0ca81416f0 2023-11-24T08:08:49.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.216+0000 7f0caf44b640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca806d660 con 0x7f0ca806d230 2023-11-24T08:08:49.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.217+0000 7f0cae449640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0ca81416f0 0x7f0ca815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.217+0000 7f0cae449640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0ca81416f0 0x7f0ca815ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34868/0 (socket says 172.21.15.64:34868) 2023-11-24T08:08:49.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.217+0000 7f0cae449640 1 -- 172.21.15.64:0/2699822878 learned_addr learned my addr 172.21.15.64:0/2699822878 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.217+0000 7f0cae449640 1 -- 172.21.15.64:0/2699822878 >> v2:172.21.15.91:3300/0 conn(0x7f0ca806d230 msgr2=0x7f0ca815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.217+0000 7f0caec4a640 1 --2- 172.21.15.64:0/2699822878 >> v2:172.21.15.91:3300/0 conn(0x7f0ca806d230 0x7f0ca815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.218+0000 7f0cae449640 1 --2- 172.21.15.64:0/2699822878 >> v2:172.21.15.91:3300/0 conn(0x7f0ca806d230 0x7f0ca815f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.218+0000 7f0cae449640 1 -- 172.21.15.64:0/2699822878 >> v1:172.21.15.91:6789/0 conn(0x7f0ca8141e80 legacy=0x7f0ca806cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.218+0000 7f0cae449640 1 -- 172.21.15.64:0/2699822878 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca806d850 con 0x7f0ca81416f0 2023-11-24T08:08:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.219+0000 7f0cae449640 1 --2- 172.21.15.64:0/2699822878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca81416f0 0x7f0ca815ed50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0ca806e350 tx=0x7f0ca402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ded0be573d3e6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.219+0000 7f0cad447640 1 -- 172.21.15.64:0/2699822878 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4031870 con 0x7f0ca81416f0 2023-11-24T08:08:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.219+0000 7f0cad447640 1 -- 172.21.15.64:0/2699822878 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca40319d0 con 0x7f0ca81416f0 2023-11-24T08:08:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.219+0000 7f0cad447640 1 -- 172.21.15.64:0/2699822878 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4031d60 con 0x7f0ca81416f0 2023-11-24T08:08:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.220+0000 7f0caf44b640 1 -- 172.21.15.64:0/2699822878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca81416f0 msgr2=0x7f0ca815ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.220+0000 7f0caf44b640 1 --2- 172.21.15.64:0/2699822878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca81416f0 0x7f0ca815ed50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0ca806e350 tx=0x7f0ca402b910 comp rx=0 tx=0).stop 2023-11-24T08:08:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.220+0000 7f0caf44b640 1 -- 172.21.15.64:0/2699822878 shutdown_connections 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.220+0000 7f0caf44b640 1 --2- 172.21.15.64:0/2699822878 >> v2:172.21.15.91:3300/0 conn(0x7f0ca806d230 0x7f0ca815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.220+0000 7f0caf44b640 1 --2- 172.21.15.64:0/2699822878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca81416f0 0x7f0ca815ed50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.220+0000 7f0caf44b640 1 -- 172.21.15.64:0/2699822878 >> 172.21.15.64:0/2699822878 conn(0x7f0ca80b0840 msgr2=0x7f0ca80b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.220+0000 7f0caf44b640 1 -- 172.21.15.64:0/2699822878 shutdown_connections 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.220+0000 7f0caf44b640 1 -- 172.21.15.64:0/2699822878 wait complete. 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.222+0000 7f0caf44b640 1 Processor -- start 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.222+0000 7f0caf44b640 1 -- start start 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caf44b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ca806d230 0x7f0ca813c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caf44b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca81416f0 0x7f0ca80a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caf44b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca8141e80 0x7f0ca814d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caf44b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca815cd90 con 0x7f0ca8141e80 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caf44b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca815dbf0 con 0x7f0ca81416f0 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caf44b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca815cfe0 con 0x7f0ca806d230 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caec4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca8141e80 0x7f0ca814d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caec4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca8141e80 0x7f0ca814d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34880/0 (socket says 172.21.15.64:34880) 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0caec4a640 1 -- 172.21.15.64:0/95903763 learned_addr learned my addr 172.21.15.64:0/95903763 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.223+0000 7f0cadc48640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca81416f0 0x7f0ca80a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0cae449640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ca806d230 0x7f0ca813c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0caec4a640 1 -- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ca806d230 msgr2=0x7f0ca813c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0caec4a640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ca806d230 0x7f0ca813c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0caec4a640 1 -- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca81416f0 msgr2=0x7f0ca80a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0caec4a640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca81416f0 0x7f0ca80a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0caec4a640 1 -- 172.21.15.64:0/95903763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40094d0 con 0x7f0ca8141e80 2023-11-24T08:08:49.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0cadc48640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca81416f0 0x7f0ca80a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:49.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0caec4a640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca8141e80 0x7f0ca814d4b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000cda0 tx=0x7f0ca000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:49.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0c9affd640 1 -- 172.21.15.64:0/95903763 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0003b90 con 0x7f0ca8141e80 2023-11-24T08:08:49.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0c9affd640 1 -- 172.21.15.64:0/95903763 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca0003cf0 con 0x7f0ca8141e80 2023-11-24T08:08:49.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0c9affd640 1 -- 172.21.15.64:0/95903763 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0004240 con 0x7f0ca8141e80 2023-11-24T08:08:49.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.224+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca815d2c0 con 0x7f0ca8141e80 2023-11-24T08:08:49.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.225+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca815d880 con 0x7f0ca8141e80 2023-11-24T08:08:49.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.226+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca80cca10 con 0x7f0ca8141e80 2023-11-24T08:08:49.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.227+0000 7f0c9affd640 1 -- 172.21.15.64:0/95903763 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f0ca00043a0 con 0x7f0ca8141e80 2023-11-24T08:08:49.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.228+0000 7f0c9affd640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8c06b470 0x7f0c8c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.228+0000 7f0c9affd640 1 -- 172.21.15.64:0/95903763 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0ca0012070 con 0x7f0ca8141e80 2023-11-24T08:08:49.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.228+0000 7f0cae449640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8c06b470 0x7f0c8c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.230+0000 7f0cae449640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8c06b470 0x7f0c8c06d930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4036000 tx=0x7f0ca403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:49.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.232+0000 7f0c9affd640 1 -- 172.21.15.64:0/95903763 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ca00552e0 con 0x7f0ca8141e80 2023-11-24T08:08:49.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.440+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0ca804e840 con 0x7f0ca8141e80 2023-11-24T08:08:49.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.441+0000 7f0c9affd640 1 -- 172.21.15.64:0/95903763 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0ca0058900 con 0x7f0ca8141e80 2023-11-24T08:08:49.442 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2023-11-24T08:08:49.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8c06b470 msgr2=0x7f0c8c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:49.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8c06b470 0x7f0c8c06d930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4036000 tx=0x7f0ca403c040 comp rx=0 tx=0).stop 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca8141e80 msgr2=0x7f0ca814d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca8141e80 0x7f0ca814d4b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000cda0 tx=0x7f0ca000a660 comp rx=0 tx=0).stop 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 shutdown_connections 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0c8c06b470 0x7f0c8c06d930 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca8141e80 0x7f0ca814d4b0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0ca81416f0 0x7f0ca80a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 --2- 172.21.15.64:0/95903763 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0ca806d230 0x7f0ca813c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 >> 172.21.15.64:0/95903763 conn(0x7f0ca80b0840 msgr2=0x7f0ca80ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:49.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.443+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 shutdown_connections 2023-11-24T08:08:49.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:49.444+0000 7f0caf44b640 1 -- 172.21.15.64:0/95903763 wait complete. 2023-11-24T08:08:49.465 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi091.front.sepia.ceph.com... 2023-11-24T08:08:49.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:49.465 DEBUG:teuthology.orchestra.run.smithi091:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T08:08:49.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi091.stderr:ceph-fuse[98580]: fuse finished with error 0 and tester_r 0 2023-11-24T08:08:49.567 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:08:49.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:49.567 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:08:49.721 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:08:49.721 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:49.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:08:49.783 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:08:49.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:49.783 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T08:08:49.842 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T08:08:49.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:08:49.843 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:08:49.844 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:49.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:08:49.906 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:08:49.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:49.906 DEBUG:teuthology.orchestra.run.smithi091:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T08:08:49.958 INFO:teuthology.orchestra.run.smithi091.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T08:08:49.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:08:49.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:50.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.195+0000 7f0391644640 1 Processor -- start 2023-11-24T08:08:50.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.196+0000 7f0391644640 1 -- start start 2023-11-24T08:08:50.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.197+0000 7f0391644640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f038c070780 0x7f038c06d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.197+0000 7f0391644640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 0x7f038c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.197+0000 7f0391644640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c071cd0 con 0x7f038c160970 2023-11-24T08:08:50.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.198+0000 7f0391644640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c0b2120 con 0x7f038c070780 2023-11-24T08:08:50.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.198+0000 7f0391644640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0b2400 con 0x7f038c070cd0 2023-11-24T08:08:50.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.198+0000 7f0383fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f038c070780 0x7f038c06d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.198+0000 7f0383fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f038c070780 0x7f038c06d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34890/0 (socket says 172.21.15.64:34890) 2023-11-24T08:08:50.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.198+0000 7f0383fff640 1 -- 172.21.15.64:0/658518074 learned_addr learned my addr 172.21.15.64:0/658518074 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:50.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.198+0000 7f038b7fe640 1 --2- 172.21.15.64:0/658518074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 0x7f038c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.199+0000 7f0383fff640 1 -- 172.21.15.64:0/658518074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 msgr2=0x7f038c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:50.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.199+0000 7f0383fff640 1 --2- 172.21.15.64:0/658518074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 0x7f038c06da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.199+0000 7f0383fff640 1 -- 172.21.15.64:0/658518074 >> v1:172.21.15.64:6789/0 conn(0x7f038c160970 legacy=0x7f038c0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.199+0000 7f0383fff640 1 -- 172.21.15.64:0/658518074 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0b26e0 con 0x7f038c070780 2023-11-24T08:08:50.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.200+0000 7f0383fff640 1 --2- 172.21.15.64:0/658518074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c06d520 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f0374009870 tx=0x7f037402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8cce72ce58e52e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.200+0000 7f038a7fc640 1 -- 172.21.15.64:0/658518074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374031640 con 0x7f038c070780 2023-11-24T08:08:50.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.200+0000 7f038a7fc640 1 -- 172.21.15.64:0/658518074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03740317a0 con 0x7f038c070780 2023-11-24T08:08:50.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.200+0000 7f038a7fc640 1 -- 172.21.15.64:0/658518074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374031b30 con 0x7f038c070780 2023-11-24T08:08:50.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.201+0000 7f0391644640 1 -- 172.21.15.64:0/658518074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 msgr2=0x7f038c06d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.201+0000 7f0391644640 1 --2- 172.21.15.64:0/658518074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c06d520 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f0374009870 tx=0x7f037402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.201+0000 7f0391644640 1 -- 172.21.15.64:0/658518074 shutdown_connections 2023-11-24T08:08:50.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.201+0000 7f0391644640 1 --2- 172.21.15.64:0/658518074 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 0x7f038c06da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.201+0000 7f0391644640 1 --2- 172.21.15.64:0/658518074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c06d520 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.201+0000 7f0391644640 1 -- 172.21.15.64:0/658518074 >> 172.21.15.64:0/658518074 conn(0x7f038c0b0d80 msgr2=0x7f038c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:50.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.201+0000 7f0391644640 1 -- 172.21.15.64:0/658518074 shutdown_connections 2023-11-24T08:08:50.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.201+0000 7f0391644640 1 -- 172.21.15.64:0/658518074 wait complete. 2023-11-24T08:08:50.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.203+0000 7f0391644640 1 Processor -- start 2023-11-24T08:08:50.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.203+0000 7f0391644640 1 -- start start 2023-11-24T08:08:50.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f0391644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f0391644640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 0x7f038c158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f0391644640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f038c160970 0x7f038c159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f0391644640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c157460 con 0x7f038c070780 2023-11-24T08:08:50.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f0391644640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c1576b0 con 0x7f038c160970 2023-11-24T08:08:50.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f0391644640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c157990 con 0x7f038c070cd0 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f038affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f038affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34898/0 (socket says 172.21.15.64:34898) 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.204+0000 7f038affd640 1 -- 172.21.15.64:0/352969410 learned_addr learned my addr 172.21.15.64:0/352969410 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f0383fff640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 0x7f038c158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f038affd640 1 -- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 msgr2=0x7f038c158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f038affd640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 0x7f038c158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f038b7fe640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f038c160970 0x7f038c159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f038affd640 1 -- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f038c160970 msgr2=0x7f038c159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f038affd640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f038c160970 0x7f038c159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f038affd640 1 -- 172.21.15.64:0/352969410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03740094d0 con 0x7f038c070780 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f038affd640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c152e70 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f037c00c400 tx=0x7f037c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f03837fe640 1 -- 172.21.15.64:0/352969410 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c0052d0 con 0x7f038c070780 2023-11-24T08:08:50.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f03837fe640 1 -- 172.21.15.64:0/352969410 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c005430 con 0x7f038c070780 2023-11-24T08:08:50.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f03837fe640 1 -- 172.21.15.64:0/352969410 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c003830 con 0x7f038c070780 2023-11-24T08:08:50.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.205+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c157c70 con 0x7f038c070780 2023-11-24T08:08:50.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.206+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c158230 con 0x7f038c070780 2023-11-24T08:08:50.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.207+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c074770 con 0x7f038c070780 2023-11-24T08:08:50.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.208+0000 7f03837fe640 1 -- 172.21.15.64:0/352969410 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f037c003990 con 0x7f038c070780 2023-11-24T08:08:50.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.209+0000 7f03837fe640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f036c06b470 0x7f036c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.209+0000 7f03837fe640 1 -- 172.21.15.64:0/352969410 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f037c089680 con 0x7f038c070780 2023-11-24T08:08:50.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.209+0000 7f0383fff640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f036c06b470 0x7f036c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.210+0000 7f0383fff640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f036c06b470 0x7f036c06d930 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0374036000 tx=0x7f037403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.214+0000 7f03837fe640 1 -- 172.21.15.64:0/352969410 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f037c053e50 con 0x7f038c070780 2023-11-24T08:08:50.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.439+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f038c0acc30 con 0x7f038c070780 2023-11-24T08:08:50.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.440+0000 7f03837fe640 1 -- 172.21.15.64:0/352969410 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f037c017090 con 0x7f038c070780 2023-11-24T08:08:50.442 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:50.443 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/970726691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":970726691},{"type":"v1","addr":"172.21.15.64:6835","nonce":970726691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2338720158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2338720158},{"type":"v1","addr":"172.21.15.91:6836","nonce":2338720158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2942920208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2942920208},{"type":"v1","addr":"172.21.15.91:6837","nonce":2942920208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:23.583381+0000","modified":"2023-11-24T08:08:26.696160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/3674316997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3674316997},{"type":"v1","addr":"172.21.15.64:6834","nonce":3674316997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:08:50.443 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-11-24T08:08:50.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f036c06b470 msgr2=0x7f036c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f036c06b470 0x7f036c06d930 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0374036000 tx=0x7f037403c040 comp rx=0 tx=0).stop 2023-11-24T08:08:50.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 msgr2=0x7f038c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c152e70 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f037c00c400 tx=0x7f037c00c8c0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 shutdown_connections 2023-11-24T08:08:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f036c06b470 0x7f036c06d930 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f038c160970 0x7f038c159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f038c070cd0 0x7f038c158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 --2- 172.21.15.64:0/352969410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c070780 0x7f038c152e70 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 >> 172.21.15.64:0/352969410 conn(0x7f038c0b0d80 msgr2=0x7f038c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 shutdown_connections 2023-11-24T08:08:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.443+0000 7f0391644640 1 -- 172.21.15.64:0/352969410 wait complete. 2023-11-24T08:08:50.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:50.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.672+0000 7fcf58a14640 1 Processor -- start 2023-11-24T08:08:50.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.672+0000 7fcf58a14640 1 -- start start 2023-11-24T08:08:50.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.674+0000 7fcf58a14640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcf54143570 0x7fcf540cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.674+0000 7fcf58a14640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 0x7fcf540cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.674+0000 7fcf58a14640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf54071700 con 0x7fcf5415e6b0 2023-11-24T08:08:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.674+0000 7fcf58a14640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5415b410 con 0x7fcf54143570 2023-11-24T08:08:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.674+0000 7fcf58a14640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5415b130 con 0x7fcf54143950 2023-11-24T08:08:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.674+0000 7fcf52d9d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcf54143570 0x7fcf540cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.674+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 0x7fcf540cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.675+0000 7fcf52d9d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fcf54143570 0x7fcf540cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:51526/0 (socket says 172.21.15.64:51526) 2023-11-24T08:08:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.675+0000 7fcf52d9d640 1 -- 172.21.15.64:0/902444248 learned_addr learned my addr 172.21.15.64:0/902444248 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.675+0000 7fcf52d9d640 1 -- 172.21.15.64:0/902444248 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 msgr2=0x7fcf540cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.675+0000 7fcf52d9d640 1 --2- 172.21.15.64:0/902444248 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 0x7fcf540cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.675+0000 7fcf52d9d640 1 -- 172.21.15.64:0/902444248 >> v1:172.21.15.91:6789/0 conn(0x7fcf5415e6b0 legacy=0x7fcf5415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.675+0000 7fcf52d9d640 1 -- 172.21.15.64:0/902444248 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5415dbd0 con 0x7fcf54143570 2023-11-24T08:08:50.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.677+0000 7fcf52d9d640 1 --2- 172.21.15.64:0/902444248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf54143570 0x7fcf540cf050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009870 tx=0x7fcf4402db00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2561288b85d5c477 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.677+0000 7fcf5259c640 1 -- 172.21.15.64:0/902444248 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4402f8c0 con 0x7fcf54143570 2023-11-24T08:08:50.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.677+0000 7fcf5259c640 1 -- 172.21.15.64:0/902444248 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4402fa20 con 0x7fcf54143570 2023-11-24T08:08:50.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.677+0000 7fcf5259c640 1 -- 172.21.15.64:0/902444248 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4402fdb0 con 0x7fcf54143570 2023-11-24T08:08:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.678+0000 7fcf58a14640 1 -- 172.21.15.64:0/902444248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf54143570 msgr2=0x7fcf540cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.678+0000 7fcf58a14640 1 --2- 172.21.15.64:0/902444248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf54143570 0x7fcf540cf050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009870 tx=0x7fcf4402db00 comp rx=0 tx=0).stop 2023-11-24T08:08:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.678+0000 7fcf58a14640 1 -- 172.21.15.64:0/902444248 shutdown_connections 2023-11-24T08:08:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.678+0000 7fcf58a14640 1 --2- 172.21.15.64:0/902444248 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 0x7fcf540cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.678+0000 7fcf58a14640 1 --2- 172.21.15.64:0/902444248 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf54143570 0x7fcf540cf050 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.678+0000 7fcf58a14640 1 -- 172.21.15.64:0/902444248 >> 172.21.15.64:0/902444248 conn(0x7fcf5406a860 msgr2=0x7fcf5406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.678+0000 7fcf58a14640 1 -- 172.21.15.64:0/902444248 shutdown_connections 2023-11-24T08:08:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.678+0000 7fcf58a14640 1 -- 172.21.15.64:0/902444248 wait complete. 2023-11-24T08:08:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.680+0000 7fcf58a14640 1 Processor -- start 2023-11-24T08:08:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.680+0000 7fcf58a14640 1 -- start start 2023-11-24T08:08:50.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf58a14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54143570 0x7fcf54136860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf58a14640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 0x7fcf540cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf58a14640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5415e6b0 0x7fcf540cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf58a14640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54137c30 con 0x7fcf54143570 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf58a14640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54137e80 con 0x7fcf5415e6b0 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf58a14640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54138160 con 0x7fcf54143950 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf52d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54143570 0x7fcf54136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf52d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54143570 0x7fcf54136860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34906/0 (socket says 172.21.15.64:34906) 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf52d9d640 1 -- 172.21.15.64:0/514241892 learned_addr learned my addr 172.21.15.64:0/514241892 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf4bfff640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 0x7fcf540cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.681+0000 7fcf5359e640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5415e6b0 0x7fcf540cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf52d9d640 1 -- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 msgr2=0x7fcf540cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf52d9d640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 0x7fcf540cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf52d9d640 1 -- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5415e6b0 msgr2=0x7fcf540cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf52d9d640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5415e6b0 0x7fcf540cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf52d9d640 1 -- 172.21.15.64:0/514241892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440094d0 con 0x7fcf54143570 2023-11-24T08:08:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf52d9d640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54143570 0x7fcf54136860 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcf4402db40 tx=0x7fcf440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf4b7fe640 1 -- 172.21.15.64:0/514241892 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf440408f0 con 0x7fcf54143570 2023-11-24T08:08:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf4b7fe640 1 -- 172.21.15.64:0/514241892 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf44040a90 con 0x7fcf54143570 2023-11-24T08:08:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf4b7fe640 1 -- 172.21.15.64:0/514241892 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44040e40 con 0x7fcf54143570 2023-11-24T08:08:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf54138440 con 0x7fcf54143570 2023-11-24T08:08:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.682+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf54138a00 con 0x7fcf54143570 2023-11-24T08:08:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.684+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5404e840 con 0x7fcf54143570 2023-11-24T08:08:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.685+0000 7fcf4b7fe640 1 -- 172.21.15.64:0/514241892 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fcf44037420 con 0x7fcf54143570 2023-11-24T08:08:50.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.685+0000 7fcf4b7fe640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3406b470 0x7fcf3406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.686+0000 7fcf4b7fe640 1 -- 172.21.15.64:0/514241892 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcf44033070 con 0x7fcf54143570 2023-11-24T08:08:50.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.686+0000 7fcf4bfff640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3406b470 0x7fcf3406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.687+0000 7fcf4bfff640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3406b470 0x7fcf3406d930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcf540ce740 tx=0x7fcf3c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.690+0000 7fcf4b7fe640 1 -- 172.21.15.64:0/514241892 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf44078d50 con 0x7fcf54143570 2023-11-24T08:08:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.885+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf540b2920 con 0x7fcf54143570 2023-11-24T08:08:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.889+0000 7fcf4b7fe640 1 -- 172.21.15.64:0/514241892 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16562 (secure 0 0 0) 0x7fcf4407c370 con 0x7fcf54143570 2023-11-24T08:08:50.891 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:50.891 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":89,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:39.037812+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:21.467760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:22.606730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.2a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.891+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3406b470 msgr2=0x7fcf3406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.891+0000 7fcf58a14640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3406b470 0x7fcf3406d930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcf540ce740 tx=0x7fcf3c008040 comp rx=0 tx=0).stop 2023-11-24T08:08:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54143570 msgr2=0x7fcf54136860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54143570 0x7fcf54136860 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcf4402db40 tx=0x7fcf440046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 shutdown_connections 2023-11-24T08:08:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fcf3406b470 0x7fcf3406d930 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fcf5415e6b0 0x7fcf540cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fcf54143950 0x7fcf540cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 --2- 172.21.15.64:0/514241892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf54143570 0x7fcf54136860 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 >> 172.21.15.64:0/514241892 conn(0x7fcf5406a860 msgr2=0x7fcf54141dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 shutdown_connections 2023-11-24T08:08:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:50.892+0000 7fcf58a14640 1 -- 172.21.15.64:0/514241892 wait complete. 2023-11-24T08:08:50.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:03:44.088235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:21.467760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:22.606730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:50.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:51.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.136+0000 7f8b8518a640 1 Processor -- start 2023-11-24T08:08:51.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.136+0000 7f8b8518a640 1 -- start start 2023-11-24T08:08:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.137+0000 7f8b8518a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b80070af0 0x7f8b800d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.137+0000 7f8b8518a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b800cda40 0x7f8b800cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.137+0000 7f8b8518a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800ce3d0 con 0x7f8b8015d340 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.137+0000 7f8b8518a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b800d0d90 con 0x7f8b80070af0 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.137+0000 7f8b8518a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8015d720 con 0x7f8b800cda40 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.138+0000 7f8b7e59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b80070af0 0x7f8b800d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.138+0000 7f8b7e59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b80070af0 0x7f8b800d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34918/0 (socket says 172.21.15.64:34918) 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.138+0000 7f8b7e59c640 1 -- 172.21.15.64:0/2788134563 learned_addr learned my addr 172.21.15.64:0/2788134563 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.138+0000 7f8b7f59e640 1 --2- 172.21.15.64:0/2788134563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b800cda40 0x7f8b800cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.138+0000 7f8b7e59c640 1 -- 172.21.15.64:0/2788134563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b800cda40 msgr2=0x7f8b800cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.138+0000 7f8b7e59c640 1 --2- 172.21.15.64:0/2788134563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b800cda40 0x7f8b800cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.138+0000 7f8b7e59c640 1 -- 172.21.15.64:0/2788134563 >> v1:172.21.15.91:6789/0 conn(0x7f8b8015d340 legacy=0x7f8b800cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.138+0000 7f8b7e59c640 1 -- 172.21.15.64:0/2788134563 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8006d080 con 0x7f8b80070af0 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.139+0000 7f8b7e59c640 1 --2- 172.21.15.64:0/2788134563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b80070af0 0x7f8b800d0850 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009e50 tx=0x7f8b7402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7492d79dd6c9ed7e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b7dd9b640 1 -- 172.21.15.64:0/2788134563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7402c950 con 0x7f8b80070af0 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b7dd9b640 1 -- 172.21.15.64:0/2788134563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7402cab0 con 0x7f8b80070af0 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b7dd9b640 1 -- 172.21.15.64:0/2788134563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7402ce40 con 0x7f8b80070af0 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b8518a640 1 -- 172.21.15.64:0/2788134563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b80070af0 msgr2=0x7f8b800d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2788134563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b80070af0 0x7f8b800d0850 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009e50 tx=0x7f8b7402bdb0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b8518a640 1 -- 172.21.15.64:0/2788134563 shutdown_connections 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2788134563 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b800cda40 0x7f8b800cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2788134563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b80070af0 0x7f8b800d0850 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b8518a640 1 -- 172.21.15.64:0/2788134563 >> 172.21.15.64:0/2788134563 conn(0x7f8b800b2ba0 msgr2=0x7f8b800b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b8518a640 1 -- 172.21.15.64:0/2788134563 shutdown_connections 2023-11-24T08:08:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.140+0000 7f8b8518a640 1 -- 172.21.15.64:0/2788134563 wait complete. 2023-11-24T08:08:51.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.142+0000 7f8b8518a640 1 Processor -- start 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.142+0000 7f8b8518a640 1 -- start start 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.143+0000 7f8b8518a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80070af0 0x7f8b8015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.143+0000 7f8b8518a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b800cda40 0x7f8b800ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.143+0000 7f8b8518a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b8015d340 0x7f8b800b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.143+0000 7f8b8518a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800acc60 con 0x7f8b800cda40 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.143+0000 7f8b8518a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b2280 con 0x7f8b8015d340 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.143+0000 7f8b8518a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b2560 con 0x7f8b80070af0 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.143+0000 7f8b7e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b800cda40 0x7f8b800ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b800cda40 0x7f8b800ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34930/0 (socket says 172.21.15.64:34930) 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7e59c640 1 -- 172.21.15.64:0/2147983646 learned_addr learned my addr 172.21.15.64:0/2147983646 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7ed9d640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80070af0 0x7f8b8015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7e59c640 1 -- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80070af0 msgr2=0x7f8b8015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7e59c640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80070af0 0x7f8b8015a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7e59c640 1 -- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b8015d340 msgr2=0x7f8b800b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7f59e640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b8015d340 0x7f8b800b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7e59c640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b8015d340 0x7f8b800b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7e59c640 1 -- 172.21.15.64:0/2147983646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b74009b30 con 0x7f8b800cda40 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7f59e640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b8015d340 0x7f8b800b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.144+0000 7f8b7e59c640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b800cda40 0x7f8b800ae9b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009e50 tx=0x7f8b74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.145+0000 7f8b677fe640 1 -- 172.21.15.64:0/2147983646 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74009510 con 0x7f8b800cda40 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.145+0000 7f8b677fe640 1 -- 172.21.15.64:0/2147983646 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7403e450 con 0x7f8b800cda40 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.145+0000 7f8b677fe640 1 -- 172.21.15.64:0/2147983646 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7403e800 con 0x7f8b800cda40 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.145+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b800b2840 con 0x7f8b800cda40 2023-11-24T08:08:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.145+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8015ac30 con 0x7f8b800cda40 2023-11-24T08:08:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.146+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b800acda0 con 0x7f8b800cda40 2023-11-24T08:08:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.147+0000 7f8b677fe640 1 -- 172.21.15.64:0/2147983646 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8b7403e960 con 0x7f8b800cda40 2023-11-24T08:08:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.148+0000 7f8b677fe640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6c06b470 0x7f8b6c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.148+0000 7f8b677fe640 1 -- 172.21.15.64:0/2147983646 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8b740acd70 con 0x7f8b800cda40 2023-11-24T08:08:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.149+0000 7f8b7ed9d640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6c06b470 0x7f8b6c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.150+0000 7f8b7ed9d640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6c06b470 0x7f8b6c06d930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8b800ca850 tx=0x7f8b70009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.153+0000 7f8b677fe640 1 -- 172.21.15.64:0/2147983646 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b800acda0 con 0x7f8b800cda40 2023-11-24T08:08:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.351+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b800ce3d0 con 0x7f8b800cda40 2023-11-24T08:08:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.352+0000 7f8b677fe640 1 -- 172.21.15.64:0/2147983646 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b800ce3d0 con 0x7f8b800cda40 2023-11-24T08:08:51.354 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6c06b470 msgr2=0x7f8b6c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6c06b470 0x7f8b6c06d930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8b800ca850 tx=0x7f8b70009040 comp rx=0 tx=0).stop 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b800cda40 msgr2=0x7f8b800ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b800cda40 0x7f8b800ae9b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009e50 tx=0x7f8b74004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 shutdown_connections 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6c06b470 0x7f8b6c06d930 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b8015d340 0x7f8b800b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b800cda40 0x7f8b800ae9b0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 --2- 172.21.15.64:0/2147983646 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b80070af0 0x7f8b8015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 >> 172.21.15.64:0/2147983646 conn(0x7f8b800b2ba0 msgr2=0x7f8b800cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 shutdown_connections 2023-11-24T08:08:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.355+0000 7f8b8518a640 1 -- 172.21.15.64:0/2147983646 wait complete. 2023-11-24T08:08:51.375 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:08:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.591+0000 7f1e7975a640 1 Processor -- start 2023-11-24T08:08:51.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.591+0000 7f1e7975a640 1 -- start start 2023-11-24T08:08:51.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e7975a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 0x7f1e740abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e7975a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e7409cb30 0x7f1e740b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e7975a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1e740adb60 0x7f1e740adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e7975a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7415b730 con 0x7f1e7409cb30 2023-11-24T08:08:51.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e7975a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e740ac740 con 0x7f1e740adb60 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e7975a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740ac460 con 0x7f1e7415b350 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e727fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e7409cb30 0x7f1e740b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e727fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e7409cb30 0x7f1e740b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34944/0 (socket says 172.21.15.64:34944) 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e727fc640 1 -- 172.21.15.64:0/104803015 learned_addr learned my addr 172.21.15.64:0/104803015 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e737fe640 1 --2- 172.21.15.64:0/104803015 >> v2:172.21.15.91:3300/0 conn(0x7f1e740adb60 0x7f1e740adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.593+0000 7f1e72ffd640 1 --2- 172.21.15.64:0/104803015 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 0x7f1e740abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.594+0000 7f1e727fc640 1 -- 172.21.15.64:0/104803015 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 msgr2=0x7f1e740abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.594+0000 7f1e727fc640 1 --2- 172.21.15.64:0/104803015 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 0x7f1e740abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.594+0000 7f1e727fc640 1 -- 172.21.15.64:0/104803015 >> v2:172.21.15.91:3300/0 conn(0x7f1e740adb60 msgr2=0x7f1e740adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.594+0000 7f1e727fc640 1 --2- 172.21.15.64:0/104803015 >> v2:172.21.15.91:3300/0 conn(0x7f1e740adb60 0x7f1e740adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.594+0000 7f1e727fc640 1 -- 172.21.15.64:0/104803015 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740b2a50 con 0x7f1e7409cb30 2023-11-24T08:08:51.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.595+0000 7f1e727fc640 1 --2- 172.21.15.64:0/104803015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e740b3530 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f1e74072730 tx=0x7f1e6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e801e27a077f2f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.595+0000 7f1e71ffb640 1 -- 172.21.15.64:0/104803015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68031870 con 0x7f1e7409cb30 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.595+0000 7f1e71ffb640 1 -- 172.21.15.64:0/104803015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e680319d0 con 0x7f1e7409cb30 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.595+0000 7f1e71ffb640 1 -- 172.21.15.64:0/104803015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68031d60 con 0x7f1e7409cb30 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 -- 172.21.15.64:0/104803015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 msgr2=0x7f1e740b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 --2- 172.21.15.64:0/104803015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e740b3530 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f1e74072730 tx=0x7f1e6802b910 comp rx=0 tx=0).stop 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 -- 172.21.15.64:0/104803015 shutdown_connections 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 --2- 172.21.15.64:0/104803015 >> v2:172.21.15.91:3300/0 conn(0x7f1e740adb60 0x7f1e740adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 --2- 172.21.15.64:0/104803015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e740b3530 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 --2- 172.21.15.64:0/104803015 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 0x7f1e740abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 -- 172.21.15.64:0/104803015 >> 172.21.15.64:0/104803015 conn(0x7f1e740b0ba0 msgr2=0x7f1e740b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:51.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 -- 172.21.15.64:0/104803015 shutdown_connections 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.596+0000 7f1e7975a640 1 -- 172.21.15.64:0/104803015 wait complete. 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.599+0000 7f1e7975a640 1 Processor -- start 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.599+0000 7f1e7975a640 1 -- start start 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.600+0000 7f1e7975a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e7413d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.600+0000 7f1e7975a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e740adb60 0x7f1e7409b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.600+0000 7f1e7975a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 0x7f1e740aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.600+0000 7f1e7975a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74099b30 con 0x7f1e7409cb30 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.600+0000 7f1e7975a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7413f810 con 0x7f1e740adb60 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.600+0000 7f1e7975a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7413faf0 con 0x7f1e7415b350 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.600+0000 7f1e72ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e7413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e72ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e7413d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34954/0 (socket says 172.21.15.64:34954) 2023-11-24T08:08:51.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e72ffd640 1 -- 172.21.15.64:0/248330729 learned_addr learned my addr 172.21.15.64:0/248330729 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e737fe640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 0x7f1e740aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e727fc640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e740adb60 0x7f1e7409b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e72ffd640 1 -- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 msgr2=0x7f1e740aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e72ffd640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 0x7f1e740aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e72ffd640 1 -- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e740adb60 msgr2=0x7f1e7409b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e72ffd640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e740adb60 0x7f1e7409b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e72ffd640 1 -- 172.21.15.64:0/248330729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680094d0 con 0x7f1e7409cb30 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.601+0000 7f1e72ffd640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e7413d140 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400efc0 tx=0x7f1e6400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.602+0000 7f1e5b7fe640 1 -- 172.21.15.64:0/248330729 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64009b80 con 0x7f1e7409cb30 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.602+0000 7f1e5b7fe640 1 -- 172.21.15.64:0/248330729 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6400f040 con 0x7f1e7409cb30 2023-11-24T08:08:51.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.602+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7413fdd0 con 0x7f1e7409cb30 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.602+0000 7f1e5b7fe640 1 -- 172.21.15.64:0/248330729 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e640146a0 con 0x7f1e7409cb30 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.602+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7413dc50 con 0x7f1e7409cb30 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.603+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7409d200 con 0x7f1e7409cb30 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.605+0000 7f1e5b7fe640 1 -- 172.21.15.64:0/248330729 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1e64014800 con 0x7f1e7409cb30 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.606+0000 7f1e5b7fe640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e6006b470 0x7f1e6006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.606+0000 7f1e5b7fe640 1 -- 172.21.15.64:0/248330729 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1e6408ad00 con 0x7f1e7409cb30 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.606+0000 7f1e727fc640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e6006b470 0x7f1e6006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.607+0000 7f1e727fc640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e6006b470 0x7f1e6006d930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1e68036000 tx=0x7f1e6803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.611+0000 7f1e5b7fe640 1 -- 172.21.15.64:0/248330729 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e640553b0 con 0x7f1e7409cb30 2023-11-24T08:08:51.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.805+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e7404e920 con 0x7f1e7409cb30 2023-11-24T08:08:51.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.806+0000 7f1e5b7fe640 1 -- 172.21.15.64:0/248330729 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e640589d0 con 0x7f1e7409cb30 2023-11-24T08:08:51.808 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-11-24T08:08:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.808+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e6006b470 msgr2=0x7f1e6006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e6006b470 0x7f1e6006d930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1e68036000 tx=0x7f1e6803c040 comp rx=0 tx=0).stop 2023-11-24T08:08:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 msgr2=0x7f1e7413d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e7413d140 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400efc0 tx=0x7f1e6400c490 comp rx=0 tx=0).stop 2023-11-24T08:08:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 shutdown_connections 2023-11-24T08:08:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e6006b470 0x7f1e6006d930 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e7415b350 0x7f1e740aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e740adb60 0x7f1e7409b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 --2- 172.21.15.64:0/248330729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e7409cb30 0x7f1e7413d140 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 >> 172.21.15.64:0/248330729 conn(0x7f1e740b0ba0 msgr2=0x7f1e74148ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:51.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 shutdown_connections 2023-11-24T08:08:51.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:51.809+0000 7f1e7975a640 1 -- 172.21.15.64:0/248330729 wait complete. 2023-11-24T08:08:51.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:08:52.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.057+0000 7f2af3fff640 1 Processor -- start 2023-11-24T08:08:52.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.057+0000 7f2af3fff640 1 -- start start 2023-11-24T08:08:52.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.059+0000 7f2af3fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2af40d1d90 0x7f2af40ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.059+0000 7f2af3fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40c6bf0 con 0x7f2af40d0fa0 2023-11-24T08:08:52.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.059+0000 7f2af3fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40d13f0 con 0x7f2af415d3b0 2023-11-24T08:08:52.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.059+0000 7f2af3fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40cedc0 con 0x7f2af40d1d90 2023-11-24T08:08:52.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.059+0000 7f2af37fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2af40d0fa0 0x7f2af40b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33346/0 (socket says 172.21.15.64:33346) 2023-11-24T08:08:52.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.059+0000 7f2af37fe640 1 -- 172.21.15.64:0/2733890180 learned_addr learned my addr 172.21.15.64:0/2733890180 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:52.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.059+0000 7f2af27fc640 1 --2- 172.21.15.64:0/2733890180 >> v2:172.21.15.91:3300/0 conn(0x7f2af40d1d90 0x7f2af40ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556923002 0 0) 0x7f2af40c6bf0 con 0x7f2af40d0fa0 2023-11-24T08:08:52.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2adc003520 con 0x7f2af40d0fa0 2023-11-24T08:08:52.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2499481994 0 0) 0x7f2adc003520 con 0x7f2af40d0fa0 2023-11-24T08:08:52.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 >> v2:172.21.15.91:3300/0 conn(0x7f2af40d1d90 msgr2=0x7f2af40ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:52.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 --2- 172.21.15.64:0/2733890180 >> v2:172.21.15.91:3300/0 conn(0x7f2af40d1d90 0x7f2af40ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 >> v1:172.21.15.91:6789/0 conn(0x7f2af415d3b0 legacy=0x7f2af40d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40c6e40 con 0x7f2af40d0fa0 2023-11-24T08:08:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2aec001670 con 0x7f2af40d0fa0 2023-11-24T08:08:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 >> v1:172.21.15.64:6789/0 conn(0x7f2af40d0fa0 legacy=0x7f2af40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc003a30 0x7f2adc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2adc0047c0 0x7f2adc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2adc0052a0 0x7f2adc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40cedc0 con 0x7f2adc0052a0 2023-11-24T08:08:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc00c170 con 0x7f2adc003a30 2023-11-24T08:08:52.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.060+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc00c330 con 0x7f2adc0047c0 2023-11-24T08:08:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.061+0000 7f2af37fe640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2adc0052a0 0x7f2adc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.061+0000 7f2af2ffd640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc003a30 0x7f2adc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.061+0000 7f2af27fc640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2adc0047c0 0x7f2adc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.061+0000 7f2af37fe640 1 -- 172.21.15.64:0/2733890180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2adc0047c0 msgr2=0x7f2adc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.061+0000 7f2af37fe640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2adc0047c0 0x7f2adc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.061+0000 7f2af37fe640 1 -- 172.21.15.64:0/2733890180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc003a30 msgr2=0x7f2adc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.061+0000 7f2af37fe640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc003a30 0x7f2adc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.061+0000 7f2af37fe640 1 -- 172.21.15.64:0/2733890180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc003520 con 0x7f2adc0052a0 2023-11-24T08:08:52.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.062+0000 7f2af37fe640 1 -- 172.21.15.64:0/2733890180 reap_dead start 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.062+0000 7f2af37fe640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2adc0052a0 0x7f2adc00b980 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00c0e0 tx=0x7f2adc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3299a05f41e42 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec005ad0 con 0x7f2adc0052a0 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec005c70 con 0x7f2adc0052a0 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af1ffb640 1 -- 172.21.15.64:0/2733890180 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec02c810 con 0x7f2adc0052a0 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af3fff640 1 -- 172.21.15.64:0/2733890180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2adc0052a0 msgr2=0x7f2adc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af3fff640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2adc0052a0 0x7f2adc00b980 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00c0e0 tx=0x7f2adc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af3fff640 1 -- 172.21.15.64:0/2733890180 shutdown_connections 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af3fff640 1 --2- 172.21.15.64:0/2733890180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2adc0052a0 0x7f2adc00b980 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af3fff640 1 -- 172.21.15.64:0/2733890180 >> 172.21.15.64:0/2733890180 conn(0x7f2af40d30a0 msgr2=0x7f2af40d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af3fff640 1 -- 172.21.15.64:0/2733890180 shutdown_connections 2023-11-24T08:08:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.063+0000 7f2af3fff640 1 -- 172.21.15.64:0/2733890180 wait complete. 2023-11-24T08:08:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.066+0000 7f2af3fff640 1 Processor -- start 2023-11-24T08:08:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.066+0000 7f2af3fff640 1 -- start start 2023-11-24T08:08:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af3fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc0052a0 0x7f2af40ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af415a8a0 0x7f2af414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af3fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2af414b070 0x7f2af414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af3fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af414beb0 con 0x7f2af415a8a0 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af3fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40b86c0 con 0x7f2adc0052a0 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af3fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af409e740 con 0x7f2af414b070 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af415a8a0 0x7f2af414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af415a8a0 0x7f2af414d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34976/0 (socket says 172.21.15.64:34976) 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.067+0000 7f2af27fc640 1 -- 172.21.15.64:0/4280104221 learned_addr learned my addr 172.21.15.64:0/4280104221 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2af37fe640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2af414b070 0x7f2af414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2af2ffd640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc0052a0 0x7f2af40ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2af27fc640 1 -- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2af414b070 msgr2=0x7f2af414dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2af27fc640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2af414b070 0x7f2af414dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2af27fc640 1 -- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc0052a0 msgr2=0x7f2af40ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2af27fc640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc0052a0 0x7f2af40ccf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2af27fc640 1 -- 172.21.15.64:0/4280104221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af414b830 con 0x7f2af415a8a0 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2af27fc640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af415a8a0 0x7f2af414d720 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009fb0 tx=0x7f2ae80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.068+0000 7f2adb7fe640 1 -- 172.21.15.64:0/4280104221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800e030 con 0x7f2af415a8a0 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.069+0000 7f2adb7fe640 1 -- 172.21.15.64:0/4280104221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8016470 con 0x7f2af415a8a0 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.069+0000 7f2adb7fe640 1 -- 172.21.15.64:0/4280104221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8016820 con 0x7f2af415a8a0 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.069+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af409d2a0 con 0x7f2af415a8a0 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.069+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af409cce0 con 0x7f2af415a8a0 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.070+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af414b690 con 0x7f2af415a8a0 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.071+0000 7f2adb7fe640 1 -- 172.21.15.64:0/4280104221 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2ae8016980 con 0x7f2af415a8a0 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.072+0000 7f2adb7fe640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ae006b470 0x7f2ae006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.072+0000 7f2adb7fe640 1 -- 172.21.15.64:0/4280104221 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2ae800f070 con 0x7f2af415a8a0 2023-11-24T08:08:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.072+0000 7f2af2ffd640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ae006b470 0x7f2ae006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.074+0000 7f2af2ffd640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ae006b470 0x7f2ae006d930 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4007fd0 tx=0x7f2ae4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.076+0000 7f2adb7fe640 1 -- 172.21.15.64:0/4280104221 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2af414b690 con 0x7f2af415a8a0 2023-11-24T08:08:52.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.270+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2af414b690 con 0x7f2af415a8a0 2023-11-24T08:08:52.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.271+0000 7f2adb7fe640 1 -- 172.21.15.64:0/4280104221 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2af414b690 con 0x7f2af415a8a0 2023-11-24T08:08:52.273 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-11-24T08:08:52.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ae006b470 msgr2=0x7f2ae006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ae006b470 0x7f2ae006d930 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4007fd0 tx=0x7f2ae4009040 comp rx=0 tx=0).stop 2023-11-24T08:08:52.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af415a8a0 msgr2=0x7f2af414d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af415a8a0 0x7f2af414d720 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009fb0 tx=0x7f2ae80115a0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 shutdown_connections 2023-11-24T08:08:52.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2ae006b470 0x7f2ae006d930 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2af414b070 0x7f2af414dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af415a8a0 0x7f2af414d720 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 --2- 172.21.15.64:0/4280104221 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2adc0052a0 0x7f2af40ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 >> 172.21.15.64:0/4280104221 conn(0x7f2af40d30a0 msgr2=0x7f2af4160090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:52.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 shutdown_connections 2023-11-24T08:08:52.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.273+0000 7f2af3fff640 1 -- 172.21.15.64:0/4280104221 wait complete. 2023-11-24T08:08:52.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:52.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.522+0000 7fe606ffa640 1 Processor -- start 2023-11-24T08:08:52.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.522+0000 7fe606ffa640 1 -- start start 2023-11-24T08:08:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.524+0000 7fe606ffa640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe6000cf500 0x7fe6000cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.524+0000 7fe606ffa640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe600140900 con 0x7fe6000cf050 2023-11-24T08:08:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.524+0000 7fe606ffa640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60015fa70 con 0x7fe60013dfd0 2023-11-24T08:08:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.524+0000 7fe606ffa640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60015c580 con 0x7fe6000cf500 2023-11-24T08:08:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.524+0000 7fe6057f7640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe6000cf050 0x7fe6000cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33354/0 (socket says 172.21.15.64:33354) 2023-11-24T08:08:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.524+0000 7fe6057f7640 1 -- 172.21.15.64:0/1259392092 learned_addr learned my addr 172.21.15.64:0/1259392092 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.525+0000 7fe6067f9640 1 --2- 172.21.15.64:0/1259392092 >> v2:172.21.15.91:3300/0 conn(0x7fe6000cf500 0x7fe6000cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.525+0000 7fe604ff6640 1 -- 172.21.15.64:0/1259392092 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670442792 0 0) 0x7fe600140900 con 0x7fe6000cf050 2023-11-24T08:08:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.525+0000 7fe604ff6640 1 -- 172.21.15.64:0/1259392092 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5ec003520 con 0x7fe6000cf050 2023-11-24T08:08:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.525+0000 7fe6067f9640 1 -- 172.21.15.64:0/1259392092 >> v1:172.21.15.91:6789/0 conn(0x7fe60013dfd0 legacy=0x7fe60015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.525+0000 7fe604ff6640 1 -- 172.21.15.64:0/1259392092 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464458365 0 0) 0x7fe5ec003520 con 0x7fe6000cf050 2023-11-24T08:08:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.526+0000 7fe6067f9640 1 -- 172.21.15.64:0/1259392092 >> v1:172.21.15.64:6789/0 conn(0x7fe6000cf050 legacy=0x7fe6000cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.526+0000 7fe6067f9640 1 -- 172.21.15.64:0/1259392092 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60015ee60 con 0x7fe6000cf500 2023-11-24T08:08:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.527+0000 7fe6067f9640 1 --2- 172.21.15.64:0/1259392092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf500 0x7fe6000cdec0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009d90 tx=0x7fe5f802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a881b91b175c633 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.527+0000 7fe604ff6640 1 -- 172.21.15.64:0/1259392092 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0001670 con 0x7fe6000cf500 2023-11-24T08:08:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.527+0000 7fe604ff6640 1 -- 172.21.15.64:0/1259392092 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f802c940 con 0x7fe6000cf500 2023-11-24T08:08:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.527+0000 7fe604ff6640 1 -- 172.21.15.64:0/1259392092 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f802ccd0 con 0x7fe6000cf500 2023-11-24T08:08:52.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.528+0000 7fe606ffa640 1 -- 172.21.15.64:0/1259392092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf500 msgr2=0x7fe6000cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.528+0000 7fe606ffa640 1 --2- 172.21.15.64:0/1259392092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf500 0x7fe6000cdec0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009d90 tx=0x7fe5f802be10 comp rx=0 tx=0).stop 2023-11-24T08:08:52.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.528+0000 7fe606ffa640 1 -- 172.21.15.64:0/1259392092 shutdown_connections 2023-11-24T08:08:52.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.528+0000 7fe606ffa640 1 --2- 172.21.15.64:0/1259392092 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf500 0x7fe6000cdec0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.528+0000 7fe606ffa640 1 -- 172.21.15.64:0/1259392092 >> 172.21.15.64:0/1259392092 conn(0x7fe60006a860 msgr2=0x7fe60006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:52.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.528+0000 7fe606ffa640 1 -- 172.21.15.64:0/1259392092 shutdown_connections 2023-11-24T08:08:52.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.528+0000 7fe606ffa640 1 -- 172.21.15.64:0/1259392092 wait complete. 2023-11-24T08:08:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.530+0000 7fe606ffa640 1 Processor -- start 2023-11-24T08:08:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.530+0000 7fe606ffa640 1 -- start start 2023-11-24T08:08:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.531+0000 7fe606ffa640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf050 0x7fe6001f0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.531+0000 7fe606ffa640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6000cf500 0x7fe6000c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.531+0000 7fe606ffa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60013dfd0 0x7fe6000c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.531+0000 7fe606ffa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60014a440 con 0x7fe60013dfd0 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.531+0000 7fe606ffa640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6001f4eb0 con 0x7fe6000cf050 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.531+0000 7fe606ffa640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6001f4bd0 con 0x7fe6000cf500 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60013dfd0 0x7fe6000c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60013dfd0 0x7fe6000c3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34978/0 (socket says 172.21.15.64:34978) 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 -- 172.21.15.64:0/4057635363 learned_addr learned my addr 172.21.15.64:0/4057635363 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe605ff8640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf050 0x7fe6001f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6057f7640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6000cf500 0x7fe6000c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 -- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6000cf500 msgr2=0x7fe6000c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6000cf500 0x7fe6000c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 -- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf050 msgr2=0x7fe6001f0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf050 0x7fe6001f0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 -- 172.21.15.64:0/4057635363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f8009a70 con 0x7fe60013dfd0 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6057f7640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6000cf500 0x7fe6000c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.532+0000 7fe6067f9640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60013dfd0 0x7fe6000c3ff0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009d90 tx=0x7fe5f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.533+0000 7fe5f67fc640 1 -- 172.21.15.64:0/4057635363 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8009450 con 0x7fe60013dfd0 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.533+0000 7fe5f67fc640 1 -- 172.21.15.64:0/4057635363 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8003990 con 0x7fe60013dfd0 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.533+0000 7fe5f67fc640 1 -- 172.21.15.64:0/4057635363 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8003d40 con 0x7fe60013dfd0 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.533+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe600146250 con 0x7fe60013dfd0 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.533+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe600146810 con 0x7fe60013dfd0 2023-11-24T08:08:52.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.534+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6001460c0 con 0x7fe60013dfd0 2023-11-24T08:08:52.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.535+0000 7fe5f67fc640 1 -- 172.21.15.64:0/4057635363 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe5f803b050 con 0x7fe60013dfd0 2023-11-24T08:08:52.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.536+0000 7fe5f67fc640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5e406b470 0x7fe5e406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.536+0000 7fe5f67fc640 1 -- 172.21.15.64:0/4057635363 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe5f80ac4b0 con 0x7fe60013dfd0 2023-11-24T08:08:52.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.537+0000 7fe605ff8640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5e406b470 0x7fe5e406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.538+0000 7fe605ff8640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5e406b470 0x7fe5e406d930 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe60015d740 tx=0x7fe5fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.540+0000 7fe5f67fc640 1 -- 172.21.15.64:0/4057635363 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5f8076c00 con 0x7fe60013dfd0 2023-11-24T08:08:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.775+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe60004e840 con 0x7fe60013dfd0 2023-11-24T08:08:52.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.776+0000 7fe5f67fc640 1 -- 172.21.15.64:0/4057635363 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4912 (secure 0 0 0) 0x7fe5f807a220 con 0x7fe60013dfd0 2023-11-24T08:08:52.778 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:52.778 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/970726691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":970726691},{"type":"v1","addr":"172.21.15.64:6835","nonce":970726691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2338720158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2338720158},{"type":"v1","addr":"172.21.15.91:6836","nonce":2338720158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2942920208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2942920208},{"type":"v1","addr":"172.21.15.91:6837","nonce":2942920208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:23.583381+0000","modified":"2023-11-24T08:08:26.696160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/3674316997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3674316997},{"type":"v1","addr":"172.21.15.64:6834","nonce":3674316997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:08:52.779 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-11-24T08:08:52.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.778+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5e406b470 msgr2=0x7fe5e406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.778+0000 7fe606ffa640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5e406b470 0x7fe5e406d930 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe60015d740 tx=0x7fe5fc009040 comp rx=0 tx=0).stop 2023-11-24T08:08:52.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.778+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60013dfd0 msgr2=0x7fe6000c3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.778+0000 7fe606ffa640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60013dfd0 0x7fe6000c3ff0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009d90 tx=0x7fe5f8004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:52.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.779+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 shutdown_connections 2023-11-24T08:08:52.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.779+0000 7fe606ffa640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe5e406b470 0x7fe5e406d930 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.779+0000 7fe606ffa640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60013dfd0 0x7fe6000c3ff0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.779+0000 7fe606ffa640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe6000cf500 0x7fe6000c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.779+0000 7fe606ffa640 1 --2- 172.21.15.64:0/4057635363 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe6000cf050 0x7fe6001f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.779+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 >> 172.21.15.64:0/4057635363 conn(0x7fe60006a860 msgr2=0x7fe60015db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.779+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 shutdown_connections 2023-11-24T08:08:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:52.779+0000 7fe606ffa640 1 -- 172.21.15.64:0/4057635363 wait complete. 2023-11-24T08:08:52.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:53.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.028+0000 7f77f759e640 1 Processor -- start 2023-11-24T08:08:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.029+0000 7f77f759e640 1 -- start start 2023-11-24T08:08:53.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.030+0000 7f77f759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 0x7f77f80d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.030+0000 7f77f759e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f77f80b1650 0x7f77f80d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.030+0000 7f77f759e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f80cf0c0 con 0x7f77f80d14a0 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.030+0000 7f77f759e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f80b01b0 con 0x7f77f80b1650 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.030+0000 7f77f759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f815d990 con 0x7f77f815d5b0 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.030+0000 7f77effff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f77f80d14a0 0x7f77f80cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33368/0 (socket says 172.21.15.64:33368) 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.030+0000 7f77effff640 1 -- 172.21.15.64:0/2724325452 learned_addr learned my addr 172.21.15.64:0/2724325452 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f6d9d640 1 --2- 172.21.15.64:0/2724325452 >> v2:172.21.15.91:3300/0 conn(0x7f77f80b1650 0x7f77f80d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f659c640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 0x7f77f80d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585951040 0 0) 0x7f77f80cf0c0 con 0x7f77f80d14a0 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e4003520 con 0x7f77f80d14a0 2023-11-24T08:08:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1948693413 0 0) 0x7f77e4003520 con 0x7f77f80d14a0 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 msgr2=0x7f77f80d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 0x7f77f80d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 >> v2:172.21.15.91:3300/0 conn(0x7f77f80b1650 msgr2=0x7f77f80d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 --2- 172.21.15.64:0/2724325452 >> v2:172.21.15.91:3300/0 conn(0x7f77f80b1650 0x7f77f80d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80d1e40 con 0x7f77f80d14a0 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.031+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f77e0001670 con 0x7f77f80d14a0 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 >> v1:172.21.15.64:6789/0 conn(0x7f77f80d14a0 legacy=0x7f77f80cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f5d9b640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f5d9b640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e40047c0 0x7f77e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f5d9b640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 0x7f77e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80b01b0 con 0x7f77e40039e0 2023-11-24T08:08:53.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f815d990 con 0x7f77e40047c0 2023-11-24T08:08:53.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e400b9b0 con 0x7f77f815d5b0 2023-11-24T08:08:53.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f659c640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77effff640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e40047c0 0x7f77e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f6d9d640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 0x7f77e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f659c640 1 -- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 msgr2=0x7f77e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f659c640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 0x7f77e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f659c640 1 -- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e40047c0 msgr2=0x7f77e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f659c640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77e40047c0 0x7f77e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f659c640 1 -- 172.21.15.64:0/2724325452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4003520 con 0x7f77e40039e0 2023-11-24T08:08:53.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.032+0000 7f77f6d9d640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f815d5b0 0x7f77e4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.034+0000 7f77f659c640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77e4003e00 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009ac0 tx=0x7f77e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520118c3c3fecf5f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.034+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e802c6b0 con 0x7f77e40039e0 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.034+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e802c810 con 0x7f77e40039e0 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.034+0000 7f77f5d9b640 1 -- 172.21.15.64:0/2724325452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e802cba0 con 0x7f77e40039e0 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.034+0000 7f77f759e640 1 -- 172.21.15.64:0/2724325452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 msgr2=0x7f77e4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.034+0000 7f77f759e640 1 --2- 172.21.15.64:0/2724325452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77e4003e00 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009ac0 tx=0x7f77e40057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.034+0000 7f77f6d9d640 1 -- 172.21.15.64:0/2724325452 reap_dead start 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.035+0000 7f77f759e640 1 -- 172.21.15.64:0/2724325452 shutdown_connections 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.035+0000 7f77f759e640 1 -- 172.21.15.64:0/2724325452 >> 172.21.15.64:0/2724325452 conn(0x7f77f809c3b0 msgr2=0x7f77f80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.035+0000 7f77f759e640 1 -- 172.21.15.64:0/2724325452 shutdown_connections 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.035+0000 7f77f759e640 1 -- 172.21.15.64:0/2724325452 wait complete. 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.037+0000 7f77f759e640 1 Processor -- start 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.037+0000 7f77f759e640 1 -- start start 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77f8152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f759e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f80b4080 0x7f77f80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f759e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80b2b10 0x7f77f806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8152b40 con 0x7f77e40039e0 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f759e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80b3bf0 con 0x7f77f80b2b10 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f759e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80b2fe0 con 0x7f77f80b4080 2023-11-24T08:08:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77f8152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77f8152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35004/0 (socket says 172.21.15.64:35004) 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f659c640 1 -- 172.21.15.64:0/2457426881 learned_addr learned my addr 172.21.15.64:0/2457426881 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f659c640 1 -- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f80b4080 msgr2=0x7f77f80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f6d9d640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80b2b10 0x7f77f806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77effff640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f80b4080 0x7f77f80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f659c640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f80b4080 0x7f77f80b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f659c640 1 -- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80b2b10 msgr2=0x7f77f806baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.038+0000 7f77f659c640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80b2b10 0x7f77f806baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.039+0000 7f77f659c640 1 -- 172.21.15.64:0/2457426881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80b32c0 con 0x7f77e40039e0 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.039+0000 7f77f659c640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77f8152600 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f77e802c250 tx=0x7f77e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.039+0000 7f77eeffd640 1 -- 172.21.15.64:0/2457426881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8003810 con 0x7f77e40039e0 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.039+0000 7f77eeffd640 1 -- 172.21.15.64:0/2457426881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8003970 con 0x7f77e40039e0 2023-11-24T08:08:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.039+0000 7f77eeffd640 1 -- 172.21.15.64:0/2457426881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8003ce0 con 0x7f77e40039e0 2023-11-24T08:08:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.039+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f80b35a0 con 0x7f77e40039e0 2023-11-24T08:08:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.039+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f815be80 con 0x7f77e40039e0 2023-11-24T08:08:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.040+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f815c010 con 0x7f77e40039e0 2023-11-24T08:08:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.041+0000 7f77eeffd640 1 -- 172.21.15.64:0/2457426881 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f77e8034440 con 0x7f77e40039e0 2023-11-24T08:08:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.042+0000 7f77eeffd640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77d806b470 0x7f77d806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.042+0000 7f77eeffd640 1 -- 172.21.15.64:0/2457426881 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f77e80ac480 con 0x7f77e40039e0 2023-11-24T08:08:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.043+0000 7f77effff640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77d806b470 0x7f77d806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.044+0000 7f77effff640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77d806b470 0x7f77d806d930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f77e0001eb0 tx=0x7f77e0000f50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.046+0000 7f77eeffd640 1 -- 172.21.15.64:0/2457426881 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77e8076c80 con 0x7f77e40039e0 2023-11-24T08:08:53.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.243+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77f8066d20 con 0x7f77e40039e0 2023-11-24T08:08:53.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.246+0000 7f77eeffd640 1 -- 172.21.15.64:0/2457426881 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16562 (secure 0 0 0) 0x7f77e80044e0 con 0x7f77e40039e0 2023-11-24T08:08:53.249 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:53.249 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":89,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:39.037812+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:21.467760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:22.606730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.2a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:53.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77d806b470 msgr2=0x7f77d806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77d806b470 0x7f77d806d930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f77e0001eb0 tx=0x7f77e0000f50 comp rx=0 tx=0).stop 2023-11-24T08:08:53.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 msgr2=0x7f77f8152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77f8152600 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f77e802c250 tx=0x7f77e8004730 comp rx=0 tx=0).stop 2023-11-24T08:08:53.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 shutdown_connections 2023-11-24T08:08:53.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f77d806b470 0x7f77d806d930 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f77f80b2b10 0x7f77f806baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f77f80b4080 0x7f77f80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 --2- 172.21.15.64:0/2457426881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e40039e0 0x7f77f8152600 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 >> 172.21.15.64:0/2457426881 conn(0x7f77f809c3b0 msgr2=0x7f77f80d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.249+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 shutdown_connections 2023-11-24T08:08:53.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.250+0000 7f77f759e640 1 -- 172.21.15.64:0/2457426881 wait complete. 2023-11-24T08:08:53.270 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:08:53.270 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.490+0000 7f60a3fff640 1 Processor -- start 2023-11-24T08:08:53.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.490+0000 7f60a3fff640 1 -- start start 2023-11-24T08:08:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.491+0000 7f60a3fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f60a40cf050 0x7f60a40cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.491+0000 7f60a3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60a40cd7b0 0x7f60a40cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.491+0000 7f60a3fff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a415fa70 con 0x7f60a413dfd0 2023-11-24T08:08:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.492+0000 7f60a3fff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a40cfa40 con 0x7f60a40cd7b0 2023-11-24T08:08:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.492+0000 7f60a3fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a415c580 con 0x7f60a40cf050 2023-11-24T08:08:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.492+0000 7f60a2ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f60a413dfd0 0x7f60a415be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33374/0 (socket says 172.21.15.64:33374) 2023-11-24T08:08:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.492+0000 7f60a2ffd640 1 -- 172.21.15.64:0/3088650789 learned_addr learned my addr 172.21.15.64:0/3088650789 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.492+0000 7f60a37fe640 1 --2- 172.21.15.64:0/3088650789 >> v2:172.21.15.64:3300/0 conn(0x7f60a40cd7b0 0x7f60a40cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.492+0000 7f60a27fc640 1 --2- 172.21.15.64:0/3088650789 >> v2:172.21.15.91:3300/0 conn(0x7f60a40cf050 0x7f60a40cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.492+0000 7f60a1ffb640 1 -- 172.21.15.64:0/3088650789 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848158262 0 0) 0x7f60a415fa70 con 0x7f60a413dfd0 2023-11-24T08:08:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.492+0000 7f60a1ffb640 1 -- 172.21.15.64:0/3088650789 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f608c003520 con 0x7f60a413dfd0 2023-11-24T08:08:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.493+0000 7f60a37fe640 1 -- 172.21.15.64:0/3088650789 >> v2:172.21.15.91:3300/0 conn(0x7f60a40cf050 msgr2=0x7f60a40cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.493+0000 7f60a37fe640 1 --2- 172.21.15.64:0/3088650789 >> v2:172.21.15.91:3300/0 conn(0x7f60a40cf050 0x7f60a40cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.493+0000 7f60a37fe640 1 -- 172.21.15.64:0/3088650789 >> v1:172.21.15.64:6789/0 conn(0x7f60a413dfd0 legacy=0x7f60a415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.493+0000 7f60a37fe640 1 -- 172.21.15.64:0/3088650789 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a4140900 con 0x7f60a40cd7b0 2023-11-24T08:08:53.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.494+0000 7f60a37fe640 1 --2- 172.21.15.64:0/3088650789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a40cd7b0 0x7f60a40cdbb0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f60a406e070 tx=0x7f609c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9120ca1dbae9e56f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.494+0000 7f60a1ffb640 1 -- 172.21.15.64:0/3088650789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c02cc80 con 0x7f60a40cd7b0 2023-11-24T08:08:53.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.494+0000 7f60a1ffb640 1 -- 172.21.15.64:0/3088650789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c02cde0 con 0x7f60a40cd7b0 2023-11-24T08:08:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.494+0000 7f60a1ffb640 1 -- 172.21.15.64:0/3088650789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c034660 con 0x7f60a40cd7b0 2023-11-24T08:08:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.494+0000 7f60a3fff640 1 -- 172.21.15.64:0/3088650789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a40cd7b0 msgr2=0x7f60a40cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.494+0000 7f60a3fff640 1 --2- 172.21.15.64:0/3088650789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a40cd7b0 0x7f60a40cdbb0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f60a406e070 tx=0x7f609c02bed0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.495+0000 7f60a3fff640 1 -- 172.21.15.64:0/3088650789 shutdown_connections 2023-11-24T08:08:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.495+0000 7f60a3fff640 1 --2- 172.21.15.64:0/3088650789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a40cd7b0 0x7f60a40cdbb0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.495+0000 7f60a3fff640 1 --2- 172.21.15.64:0/3088650789 >> v2:172.21.15.91:3300/0 conn(0x7f60a40cf050 0x7f60a40cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.495+0000 7f60a3fff640 1 -- 172.21.15.64:0/3088650789 >> 172.21.15.64:0/3088650789 conn(0x7f60a406a860 msgr2=0x7f60a406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.495+0000 7f60a3fff640 1 -- 172.21.15.64:0/3088650789 shutdown_connections 2023-11-24T08:08:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.495+0000 7f60a3fff640 1 -- 172.21.15.64:0/3088650789 wait complete. 2023-11-24T08:08:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.496+0000 7f60a3fff640 1 Processor -- start 2023-11-24T08:08:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.497+0000 7f60a3fff640 1 -- start start 2023-11-24T08:08:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.497+0000 7f60a3fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60a40cd7b0 0x7f60a40b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a3fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f60a40cf050 0x7f60a413b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a413dfd0 0x7f60a4070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a3fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a413be10 con 0x7f60a413dfd0 2023-11-24T08:08:53.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a3fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a40b1f00 con 0x7f60a40cd7b0 2023-11-24T08:08:53.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a3fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a40b21e0 con 0x7f60a40cf050 2023-11-24T08:08:53.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a413dfd0 0x7f60a4070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a413dfd0 0x7f60a4070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35022/0 (socket says 172.21.15.64:35022) 2023-11-24T08:08:53.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a37fe640 1 -- 172.21.15.64:0/2226203226 learned_addr learned my addr 172.21.15.64:0/2226203226 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a2ffd640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60a40cd7b0 0x7f60a40b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a27fc640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f60a40cf050 0x7f60a413b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a37fe640 1 -- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f60a40cf050 msgr2=0x7f60a413b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a37fe640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f60a40cf050 0x7f60a413b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a37fe640 1 -- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60a40cd7b0 msgr2=0x7f60a40b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a37fe640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60a40cd7b0 0x7f60a40b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a37fe640 1 -- 172.21.15.64:0/2226203226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c009a70 con 0x7f60a413dfd0 2023-11-24T08:08:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.498+0000 7f60a27fc640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f60a40cf050 0x7f60a413b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.499+0000 7f60a37fe640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a413dfd0 0x7f60a4070d60 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f609c009e80 tx=0x7f609c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.499+0000 7f608bfff640 1 -- 172.21.15.64:0/2226203226 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c040650 con 0x7f60a413dfd0 2023-11-24T08:08:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.499+0000 7f608bfff640 1 -- 172.21.15.64:0/2226203226 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c0407b0 con 0x7f60a413dfd0 2023-11-24T08:08:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.499+0000 7f608bfff640 1 -- 172.21.15.64:0/2226203226 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c040b60 con 0x7f60a413dfd0 2023-11-24T08:08:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.499+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a40b24c0 con 0x7f60a413dfd0 2023-11-24T08:08:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.499+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a40b2a80 con 0x7f60a413dfd0 2023-11-24T08:08:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.500+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a40cccd0 con 0x7f60a413dfd0 2023-11-24T08:08:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.501+0000 7f608bfff640 1 -- 172.21.15.64:0/2226203226 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f609c040cc0 con 0x7f60a413dfd0 2023-11-24T08:08:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.502+0000 7f608bfff640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f609006b470 0x7f609006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.502+0000 7f608bfff640 1 -- 172.21.15.64:0/2226203226 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f609c0ace80 con 0x7f60a413dfd0 2023-11-24T08:08:53.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.503+0000 7f60a2ffd640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f609006b470 0x7f609006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.504+0000 7f60a2ffd640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f609006b470 0x7f609006d930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f60940011e0 tx=0x7f6094009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.507+0000 7f608bfff640 1 -- 172.21.15.64:0/2226203226 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f609c0775d0 con 0x7f60a413dfd0 2023-11-24T08:08:53.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.743+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60a40b2c70 con 0x7f60a413dfd0 2023-11-24T08:08:53.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.744+0000 7f608bfff640 1 -- 172.21.15.64:0/2226203226 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f609c009450 con 0x7f60a413dfd0 2023-11-24T08:08:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:53.746 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/970726691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":970726691},{"type":"v1","addr":"172.21.15.64:6835","nonce":970726691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2338720158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2338720158},{"type":"v1","addr":"172.21.15.91:6836","nonce":2338720158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2942920208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2942920208},{"type":"v1","addr":"172.21.15.91:6837","nonce":2942920208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:23.583381+0000","modified":"2023-11-24T08:08:26.696160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/3674316997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3674316997},{"type":"v1","addr":"172.21.15.64:6834","nonce":3674316997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:08:53.746 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-11-24T08:08:53.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f609006b470 msgr2=0x7f609006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f609006b470 0x7f609006d930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f60940011e0 tx=0x7f6094009040 comp rx=0 tx=0).stop 2023-11-24T08:08:53.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a413dfd0 msgr2=0x7f60a4070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a413dfd0 0x7f60a4070d60 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f609c009e80 tx=0x7f609c004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:53.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 shutdown_connections 2023-11-24T08:08:53.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f609006b470 0x7f609006d930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a413dfd0 0x7f60a4070d60 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f60a40cf050 0x7f60a413b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 --2- 172.21.15.64:0/2226203226 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60a40cd7b0 0x7f60a40b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 >> 172.21.15.64:0/2226203226 conn(0x7f60a406a860 msgr2=0x7f60a413f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.747+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 shutdown_connections 2023-11-24T08:08:53.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.748+0000 7f60a3fff640 1 -- 172.21.15.64:0/2226203226 wait complete. 2023-11-24T08:08:53.769 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.995+0000 7f6741947640 1 Processor -- start 2023-11-24T08:08:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.995+0000 7f6741947640 1 -- start start 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6741947640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f673c13dfd0 0x7f673c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6741947640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f673c0cf050 0x7f673c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6741947640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c06c130 0x7f673c06c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6741947640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c0cfa40 con 0x7f673c13dfd0 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6741947640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c15fa70 con 0x7f673c0cf050 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6741947640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c15ee60 con 0x7f673c06c130 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6740945640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f673c13dfd0 0x7f673c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6740945640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f673c13dfd0 0x7f673c15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35026/0 (socket says 172.21.15.64:35026) 2023-11-24T08:08:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.997+0000 7f6740945640 1 -- 172.21.15.64:0/3941120407 learned_addr learned my addr 172.21.15.64:0/3941120407 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f6741146640 1 --2- 172.21.15.64:0/3941120407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c06c130 0x7f673c06c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f673bfff640 1 --2- 172.21.15.64:0/3941120407 >> v2:172.21.15.91:3300/0 conn(0x7f673c0cf050 0x7f673c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f6740945640 1 -- 172.21.15.64:0/3941120407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c06c130 msgr2=0x7f673c06c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f6740945640 1 --2- 172.21.15.64:0/3941120407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c06c130 0x7f673c06c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f6740945640 1 -- 172.21.15.64:0/3941120407 >> v2:172.21.15.91:3300/0 conn(0x7f673c0cf050 msgr2=0x7f673c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f6740945640 1 --2- 172.21.15.64:0/3941120407 >> v2:172.21.15.91:3300/0 conn(0x7f673c0cf050 0x7f673c0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f6740945640 1 -- 172.21.15.64:0/3941120407 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c15f140 con 0x7f673c13dfd0 2023-11-24T08:08:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f6741146640 1 --2- 172.21.15.64:0/3941120407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c06c130 0x7f673c06c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.998+0000 7f673bfff640 1 --2- 172.21.15.64:0/3941120407 >> v2:172.21.15.91:3300/0 conn(0x7f673c0cf050 0x7f673c0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.999+0000 7f6740945640 1 --2- 172.21.15.64:0/3941120407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c13dfd0 0x7f673c15be70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f673c06e070 tx=0x7f673002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce000de1933508 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.999+0000 7f673b7fe640 1 -- 172.21.15.64:0/3941120407 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730031860 con 0x7f673c13dfd0 2023-11-24T08:08:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.999+0000 7f673b7fe640 1 -- 172.21.15.64:0/3941120407 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67300319c0 con 0x7f673c13dfd0 2023-11-24T08:08:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:53.999+0000 7f673b7fe640 1 -- 172.21.15.64:0/3941120407 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730031d50 con 0x7f673c13dfd0 2023-11-24T08:08:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 -- 172.21.15.64:0/3941120407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c13dfd0 msgr2=0x7f673c15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 --2- 172.21.15.64:0/3941120407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c13dfd0 0x7f673c15be70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f673c06e070 tx=0x7f673002b900 comp rx=0 tx=0).stop 2023-11-24T08:08:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 -- 172.21.15.64:0/3941120407 shutdown_connections 2023-11-24T08:08:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 --2- 172.21.15.64:0/3941120407 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c06c130 0x7f673c06c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 --2- 172.21.15.64:0/3941120407 >> v2:172.21.15.91:3300/0 conn(0x7f673c0cf050 0x7f673c0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 --2- 172.21.15.64:0/3941120407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c13dfd0 0x7f673c15be70 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 -- 172.21.15.64:0/3941120407 >> 172.21.15.64:0/3941120407 conn(0x7f673c06a860 msgr2=0x7f673c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 -- 172.21.15.64:0/3941120407 shutdown_connections 2023-11-24T08:08:54.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.000+0000 7f6741947640 1 -- 172.21.15.64:0/3941120407 wait complete. 2023-11-24T08:08:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.002+0000 7f6741947640 1 Processor -- start 2023-11-24T08:08:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.002+0000 7f6741947640 1 -- start start 2023-11-24T08:08:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f6741947640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f673c06c130 0x7f673c1f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f6741947640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c0cf050 0x7f673c1ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f6741947640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c13dfd0 0x7f673c0b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f6741947640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c1f1c90 con 0x7f673c0cf050 2023-11-24T08:08:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f6741947640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c15c1e0 con 0x7f673c06c130 2023-11-24T08:08:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f6741947640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c15c4c0 con 0x7f673c13dfd0 2023-11-24T08:08:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f673bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c0cf050 0x7f673c1ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f673bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c0cf050 0x7f673c1ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35040/0 (socket says 172.21.15.64:35040) 2023-11-24T08:08:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.003+0000 7f673bfff640 1 -- 172.21.15.64:0/1472474515 learned_addr learned my addr 172.21.15.64:0/1472474515 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f6740945640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f673c06c130 0x7f673c1f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f6741146640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c13dfd0 0x7f673c0b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f673bfff640 1 -- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c13dfd0 msgr2=0x7f673c0b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f673bfff640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c13dfd0 0x7f673c0b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f673bfff640 1 -- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f673c06c130 msgr2=0x7f673c1f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f673bfff640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f673c06c130 0x7f673c1f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f673bfff640 1 -- 172.21.15.64:0/1472474515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67300094d0 con 0x7f673c0cf050 2023-11-24T08:08:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f6741146640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c13dfd0 0x7f673c0b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.004+0000 7f673bfff640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c0cf050 0x7f673c1ee0e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c7f0 tx=0x7f672c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.005+0000 7f67397fa640 1 -- 172.21.15.64:0/1472474515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c007a30 con 0x7f673c0cf050 2023-11-24T08:08:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.005+0000 7f67397fa640 1 -- 172.21.15.64:0/1472474515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c007b90 con 0x7f673c0cf050 2023-11-24T08:08:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.005+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c15c7a0 con 0x7f673c0cf050 2023-11-24T08:08:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.005+0000 7f67397fa640 1 -- 172.21.15.64:0/1472474515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00ce40 con 0x7f673c0cf050 2023-11-24T08:08:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.005+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c15cd60 con 0x7f673c0cf050 2023-11-24T08:08:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.006+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c15cf80 con 0x7f673c0cf050 2023-11-24T08:08:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.007+0000 7f67397fa640 1 -- 172.21.15.64:0/1472474515 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f672c0113f0 con 0x7f673c0cf050 2023-11-24T08:08:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.008+0000 7f67397fa640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f672806b470 0x7f672806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.009+0000 7f67397fa640 1 -- 172.21.15.64:0/1472474515 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f672c08a400 con 0x7f673c0cf050 2023-11-24T08:08:54.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.009+0000 7f6740945640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f672806b470 0x7f672806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.010+0000 7f6740945640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f672806b470 0x7f672806d930 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6730036000 tx=0x7f673003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.013+0000 7f67397fa640 1 -- 172.21.15.64:0/1472474515 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f672c054a40 con 0x7f673c0cf050 2023-11-24T08:08:54.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.246+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f673c04e840 con 0x7f673c0cf050 2023-11-24T08:08:54.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.247+0000 7f67397fa640 1 -- 172.21.15.64:0/1472474515 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f672c058060 con 0x7f673c0cf050 2023-11-24T08:08:54.249 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:54.249 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/970726691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":970726691},{"type":"v1","addr":"172.21.15.64:6835","nonce":970726691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2338720158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2338720158},{"type":"v1","addr":"172.21.15.91:6836","nonce":2338720158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2942920208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2942920208},{"type":"v1","addr":"172.21.15.91:6837","nonce":2942920208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:23.583381+0000","modified":"2023-11-24T08:08:26.696160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6834/3674316997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3674316997},{"type":"v1","addr":"172.21.15.64:6834","nonce":3674316997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:08:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-11-24T08:08:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f672806b470 msgr2=0x7f672806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f672806b470 0x7f672806d930 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6730036000 tx=0x7f673003c040 comp rx=0 tx=0).stop 2023-11-24T08:08:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c0cf050 msgr2=0x7f673c1ee0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c0cf050 0x7f673c1ee0e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c7f0 tx=0x7f672c00cbf0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 shutdown_connections 2023-11-24T08:08:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f672806b470 0x7f672806d930 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f673c13dfd0 0x7f673c0b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f673c0cf050 0x7f673c1ee0e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 --2- 172.21.15.64:0/1472474515 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f673c06c130 0x7f673c1f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 >> 172.21.15.64:0/1472474515 conn(0x7f673c06a860 msgr2=0x7f673c0cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 shutdown_connections 2023-11-24T08:08:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.250+0000 7f6741947640 1 -- 172.21.15.64:0/1472474515 wait complete. 2023-11-24T08:08:54.271 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:54.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.488+0000 7f2c92bc1640 1 Processor -- start 2023-11-24T08:08:54.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.488+0000 7f2c92bc1640 1 -- start start 2023-11-24T08:08:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.490+0000 7f2c92bc1640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2c8c0cf050 0x7f2c8c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.490+0000 7f2c92bc1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0cd7b0 0x7f2c8c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.490+0000 7f2c92bc1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8c15fa70 con 0x7f2c8c13dfd0 2023-11-24T08:08:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.490+0000 7f2c92bc1640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8c0cfa40 con 0x7f2c8c0cf050 2023-11-24T08:08:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.490+0000 7f2c92bc1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c15c580 con 0x7f2c8c0cd7b0 2023-11-24T08:08:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.490+0000 7f2c91bbf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2c8c13dfd0 0x7f2c8c15be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33378/0 (socket says 172.21.15.64:33378) 2023-11-24T08:08:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.490+0000 7f2c91bbf640 1 -- 172.21.15.64:0/1716876041 learned_addr learned my addr 172.21.15.64:0/1716876041 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.490+0000 7f2c913be640 1 --2- 172.21.15.64:0/1716876041 >> v2:172.21.15.91:3300/0 conn(0x7f2c8c0cf050 0x7f2c8c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c923c0640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0cd7b0 0x7f2c8c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028299747 0 0) 0x7f2c8c15fa70 con 0x7f2c8c13dfd0 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c78003520 con 0x7f2c8c13dfd0 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1282920857 0 0) 0x7f2c78003520 con 0x7f2c8c13dfd0 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0cd7b0 msgr2=0x7f2c8c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0cd7b0 0x7f2c8c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 >> v2:172.21.15.91:3300/0 conn(0x7f2c8c0cf050 msgr2=0x7f2c8c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 --2- 172.21.15.64:0/1716876041 >> v2:172.21.15.91:3300/0 conn(0x7f2c8c0cf050 0x7f2c8c0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c140900 con 0x7f2c8c13dfd0 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2c88001690 con 0x7f2c8c13dfd0 2023-11-24T08:08:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 >> v1:172.21.15.64:6789/0 conn(0x7f2c8c13dfd0 legacy=0x7f2c8c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c78003a10 0x7f2c78003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0cd7b0 0x7f2c780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78004d00 0x7f2c78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0cfa40 con 0x7f2c78003a10 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.491+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c15c580 con 0x7f2c78004d00 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7800b980 con 0x7f2c8c0cd7b0 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c913be640 1 --2- 172.21.15.64:0/1716876041 >> v2:172.21.15.91:3300/0 conn(0x7f2c8c0cf050 0x7f2c8c0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c91bbf640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c78003a10 0x7f2c78003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c923c0640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78004d00 0x7f2c78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c91bbf640 1 -- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0cd7b0 msgr2=0x7f2c780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c91bbf640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0cd7b0 0x7f2c780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c91bbf640 1 -- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78004d00 msgr2=0x7f2c78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c91bbf640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78004d00 0x7f2c78005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.492+0000 7f2c91bbf640 1 -- 172.21.15.64:0/1716876041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78003520 con 0x7f2c78003a10 2023-11-24T08:08:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.494+0000 7f2c91bbf640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c78003a10 0x7f2c78003e30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2c88007fd0 tx=0x7f2c780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60beb8ab8fcff1b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.494+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88005ad0 con 0x7f2c78003a10 2023-11-24T08:08:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.494+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c88005c50 con 0x7f2c78003a10 2023-11-24T08:08:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.494+0000 7f2c90bbd640 1 -- 172.21.15.64:0/1716876041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8802c850 con 0x7f2c78003a10 2023-11-24T08:08:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.495+0000 7f2c92bc1640 1 -- 172.21.15.64:0/1716876041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c78003a10 msgr2=0x7f2c78003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.495+0000 7f2c92bc1640 1 --2- 172.21.15.64:0/1716876041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c78003a10 0x7f2c78003e30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2c88007fd0 tx=0x7f2c780057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.495+0000 7f2c923c0640 1 -- 172.21.15.64:0/1716876041 reap_dead start 2023-11-24T08:08:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.495+0000 7f2c92bc1640 1 -- 172.21.15.64:0/1716876041 shutdown_connections 2023-11-24T08:08:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.495+0000 7f2c92bc1640 1 -- 172.21.15.64:0/1716876041 >> 172.21.15.64:0/1716876041 conn(0x7f2c8c06a860 msgr2=0x7f2c8c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.495+0000 7f2c92bc1640 1 -- 172.21.15.64:0/1716876041 shutdown_connections 2023-11-24T08:08:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.495+0000 7f2c92bc1640 1 -- 172.21.15.64:0/1716876041 wait complete. 2023-11-24T08:08:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.497+0000 7f2c92bc1640 1 Processor -- start 2023-11-24T08:08:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.497+0000 7f2c92bc1640 1 -- start start 2023-11-24T08:08:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c92bc1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78003a10 0x7f2c8c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c92bc1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0b31c0 0x7f2c8c071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c92bc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c8c154ed0 0x7f2c8c0ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c92bc1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0b2d30 con 0x7f2c8c154ed0 2023-11-24T08:08:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c92bc1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0b2120 con 0x7f2c78003a10 2023-11-24T08:08:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c92bc1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0b2400 con 0x7f2c8c0b31c0 2023-11-24T08:08:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c923c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c8c154ed0 0x7f2c8c0ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c923c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c8c154ed0 0x7f2c8c0ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35048/0 (socket says 172.21.15.64:35048) 2023-11-24T08:08:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.498+0000 7f2c923c0640 1 -- 172.21.15.64:0/3998447295 learned_addr learned my addr 172.21.15.64:0/3998447295 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c913be640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0b31c0 0x7f2c8c071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c91bbf640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78003a10 0x7f2c8c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c923c0640 1 -- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0b31c0 msgr2=0x7f2c8c071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c923c0640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0b31c0 0x7f2c8c071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c923c0640 1 -- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78003a10 msgr2=0x7f2c8c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c923c0640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78003a10 0x7f2c8c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c923c0640 1 -- 172.21.15.64:0/3998447295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0b26e0 con 0x7f2c8c154ed0 2023-11-24T08:08:54.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c923c0640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c8c154ed0 0x7f2c8c0ad5f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0cfd30 tx=0x7f2c8400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c827fc640 1 -- 172.21.15.64:0/3998447295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8400fe90 con 0x7f2c8c154ed0 2023-11-24T08:08:54.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c827fc640 1 -- 172.21.15.64:0/3998447295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c84011040 con 0x7f2c8c154ed0 2023-11-24T08:08:54.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c827fc640 1 -- 172.21.15.64:0/3998447295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84003bd0 con 0x7f2c8c154ed0 2023-11-24T08:08:54.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.499+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c0b29c0 con 0x7f2c8c154ed0 2023-11-24T08:08:54.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.500+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c155350 con 0x7f2c8c154ed0 2023-11-24T08:08:54.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.501+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8c155570 con 0x7f2c8c154ed0 2023-11-24T08:08:54.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.502+0000 7f2c827fc640 1 -- 172.21.15.64:0/3998447295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2c84017400 con 0x7f2c8c154ed0 2023-11-24T08:08:54.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.503+0000 7f2c827fc640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c7006b4a0 0x7f2c7006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.503+0000 7f2c827fc640 1 -- 172.21.15.64:0/3998447295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2c8408a6b0 con 0x7f2c8c154ed0 2023-11-24T08:08:54.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.503+0000 7f2c91bbf640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c7006b4a0 0x7f2c7006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.504+0000 7f2c91bbf640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c7006b4a0 0x7f2c7006d960 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2c88034000 tx=0x7f2c88007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.507+0000 7f2c827fc640 1 -- 172.21.15.64:0/3998447295 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c84058800 con 0x7f2c8c154ed0 2023-11-24T08:08:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.700+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c8c04e840 con 0x7f2c8c154ed0 2023-11-24T08:08:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.702+0000 7f2c827fc640 1 -- 172.21.15.64:0/3998447295 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16562 (secure 0 0 0) 0x7f2c84058390 con 0x7f2c8c154ed0 2023-11-24T08:08:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:08:54.705 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":89,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:39.037812+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:21.467760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:22.606730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.2a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c7006b4a0 msgr2=0x7f2c7006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c7006b4a0 0x7f2c7006d960 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2c88034000 tx=0x7f2c88007000 comp rx=0 tx=0).stop 2023-11-24T08:08:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c8c154ed0 msgr2=0x7f2c8c0ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c8c154ed0 0x7f2c8c0ad5f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0cfd30 tx=0x7f2c8400f4a0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 shutdown_connections 2023-11-24T08:08:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2c7006b4a0 0x7f2c7006d960 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c8c154ed0 0x7f2c8c0ad5f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c8c0b31c0 0x7f2c8c071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 --2- 172.21.15.64:0/3998447295 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c78003a10 0x7f2c8c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 >> 172.21.15.64:0/3998447295 conn(0x7f2c8c06a860 msgr2=0x7f2c8c140bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.705+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 shutdown_connections 2023-11-24T08:08:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.706+0000 7f2c92bc1640 1 -- 172.21.15.64:0/3998447295 wait complete. 2023-11-24T08:08:54.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:08:54.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.945+0000 7fa1b5d16640 1 Processor -- start 2023-11-24T08:08:54.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.946+0000 7fa1b5d16640 1 -- start start 2023-11-24T08:08:54.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.947+0000 7fa1b5d16640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa1b015f460 0x7fa1b006cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.947+0000 7fa1b5d16640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 0x7fa1b006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.947+0000 7fa1b5d16640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b0070bf0 con 0x7fa1b01416f0 2023-11-24T08:08:54.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.947+0000 7fa1b5d16640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b006db30 con 0x7fa1b015f460 2023-11-24T08:08:54.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.947+0000 7fa1b5d16640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b006d5a0 con 0x7fa1b0141e80 2023-11-24T08:08:54.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.947+0000 7fa1affff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa1b015f460 0x7fa1b006cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.947+0000 7fa1b4d14640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa1b01416f0 0x7fa1b015ed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33388/0 (socket says 172.21.15.64:33388) 2023-11-24T08:08:54.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.947+0000 7fa1b4d14640 1 -- 172.21.15.64:0/3712587495 learned_addr learned my addr 172.21.15.64:0/3712587495 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:54.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.948+0000 7fa1b5515640 1 --2- 172.21.15.64:0/3712587495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 0x7fa1b006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.948+0000 7fa1af7fe640 1 -- 172.21.15.64:0/3712587495 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320834837 0 0) 0x7fa1b0070bf0 con 0x7fa1b01416f0 2023-11-24T08:08:54.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.948+0000 7fa1af7fe640 1 -- 172.21.15.64:0/3712587495 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1a0003520 con 0x7fa1b01416f0 2023-11-24T08:08:54.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.948+0000 7fa1affff640 1 -- 172.21.15.64:0/3712587495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 msgr2=0x7fa1b006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.948+0000 7fa1affff640 1 --2- 172.21.15.64:0/3712587495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 0x7fa1b006d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.948+0000 7fa1affff640 1 -- 172.21.15.64:0/3712587495 >> v1:172.21.15.64:6789/0 conn(0x7fa1b01416f0 legacy=0x7fa1b015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.948+0000 7fa1affff640 1 -- 172.21.15.64:0/3712587495 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b006d850 con 0x7fa1b015f460 2023-11-24T08:08:54.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.949+0000 7fa1affff640 1 --2- 172.21.15.64:0/3712587495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b015f460 0x7fa1b006cb20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0097a0 tx=0x7fa19c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcdef0ccadde0a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.949+0000 7fa1af7fe640 1 -- 172.21.15.64:0/3712587495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c031830 con 0x7fa1b015f460 2023-11-24T08:08:54.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1af7fe640 1 -- 172.21.15.64:0/3712587495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c031990 con 0x7fa1b015f460 2023-11-24T08:08:54.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1af7fe640 1 -- 172.21.15.64:0/3712587495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c031d20 con 0x7fa1b015f460 2023-11-24T08:08:54.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1b5d16640 1 -- 172.21.15.64:0/3712587495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b015f460 msgr2=0x7fa1b006cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/3712587495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b015f460 0x7fa1b006cb20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0097a0 tx=0x7fa19c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1b5d16640 1 -- 172.21.15.64:0/3712587495 shutdown_connections 2023-11-24T08:08:54.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/3712587495 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 0x7fa1b006d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/3712587495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b015f460 0x7fa1b006cb20 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1b5d16640 1 -- 172.21.15.64:0/3712587495 >> 172.21.15.64:0/3712587495 conn(0x7fa1b00b0840 msgr2=0x7fa1b00b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1b5d16640 1 -- 172.21.15.64:0/3712587495 shutdown_connections 2023-11-24T08:08:54.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.950+0000 7fa1b5d16640 1 -- 172.21.15.64:0/3712587495 wait complete. 2023-11-24T08:08:54.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.952+0000 7fa1b5d16640 1 Processor -- start 2023-11-24T08:08:54.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.952+0000 7fa1b5d16640 1 -- start start 2023-11-24T08:08:54.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.953+0000 7fa1b5d16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b01416f0 0x7fa1b00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.953+0000 7fa1b5d16640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 0x7fa1b015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.953+0000 7fa1b5d16640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1b015f460 0x7fa1b015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.953+0000 7fa1b5d16640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b00b1f00 con 0x7fa1b01416f0 2023-11-24T08:08:54.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.953+0000 7fa1b5d16640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b00cbb60 con 0x7fa1b015f460 2023-11-24T08:08:54.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.953+0000 7fa1b5d16640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b015cbc0 con 0x7fa1b0141e80 2023-11-24T08:08:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b01416f0 0x7fa1b00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b01416f0 0x7fa1b00cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35066/0 (socket says 172.21.15.64:35066) 2023-11-24T08:08:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 -- 172.21.15.64:0/674964897 learned_addr learned my addr 172.21.15.64:0/674964897 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 -- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 msgr2=0x7fa1b015c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 0x7fa1b015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 -- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1b015f460 msgr2=0x7fa1b015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1b015f460 0x7fa1b015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 -- 172.21.15.64:0/674964897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c009400 con 0x7fa1b01416f0 2023-11-24T08:08:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.954+0000 7fa1b4d14640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b01416f0 0x7fa1b00cb620 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c500 tx=0x7fa1a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.955+0000 7fa1ad7fa640 1 -- 172.21.15.64:0/674964897 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4005ce0 con 0x7fa1b01416f0 2023-11-24T08:08:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.955+0000 7fa1ad7fa640 1 -- 172.21.15.64:0/674964897 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a4005e40 con 0x7fa1b01416f0 2023-11-24T08:08:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.955+0000 7fa1ad7fa640 1 -- 172.21.15.64:0/674964897 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4007330 con 0x7fa1b01416f0 2023-11-24T08:08:54.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.955+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b015cea0 con 0x7fa1b01416f0 2023-11-24T08:08:54.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.955+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b00b1b20 con 0x7fa1b01416f0 2023-11-24T08:08:54.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.956+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b00cca10 con 0x7fa1b01416f0 2023-11-24T08:08:54.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.957+0000 7fa1ad7fa640 1 -- 172.21.15.64:0/674964897 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fa1a4007550 con 0x7fa1b01416f0 2023-11-24T08:08:54.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.958+0000 7fa1ad7fa640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa19406b470 0x7fa19406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.958+0000 7fa1ad7fa640 1 -- 172.21.15.64:0/674964897 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa1a40896c0 con 0x7fa1b01416f0 2023-11-24T08:08:54.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.959+0000 7fa1affff640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa19406b470 0x7fa19406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.960+0000 7fa1affff640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa19406b470 0x7fa19406d930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa19c036000 tx=0x7fa19c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:54.964+0000 7fa1ad7fa640 1 -- 172.21.15.64:0/674964897 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1a4054040 con 0x7fa1b01416f0 2023-11-24T08:08:55.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.202+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa1b004e840 con 0x7fa1b01416f0 2023-11-24T08:08:55.205 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-11-24T08:08:55.203+0000 7fb13dee4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:08:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.207+0000 7fa1ad7fa640 1 -- 172.21.15.64:0/674964897 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa1a4017090 con 0x7fa1b01416f0 2023-11-24T08:08:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:08:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa19406b470 msgr2=0x7fa19406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa19406b470 0x7fa19406d930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa19c036000 tx=0x7fa19c030000 comp rx=0 tx=0).stop 2023-11-24T08:08:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b01416f0 msgr2=0x7fa1b00cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b01416f0 0x7fa1b00cb620 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c500 tx=0x7fa1a400c9c0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 shutdown_connections 2023-11-24T08:08:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa19406b470 0x7fa19406d930 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1b015f460 0x7fa1b015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1b0141e80 0x7fa1b015c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 --2- 172.21.15.64:0/674964897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b01416f0 0x7fa1b00cb620 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.210+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 >> 172.21.15.64:0/674964897 conn(0x7fa1b00b0840 msgr2=0x7fa1b0071080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.211+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 shutdown_connections 2023-11-24T08:08:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.211+0000 7fa1b5d16640 1 -- 172.21.15.64:0/674964897 wait complete. 2023-11-24T08:08:55.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:08:55.242 INFO:tasks.ceph.mds.a.smithi064.stderr: -5114> 2023-11-24T08:08:29.058+0000 7f5c57c47640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:08:55.312 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-11-24T08:08:55.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.439+0000 7fd0ed200640 1 Processor -- start 2023-11-24T08:08:55.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.439+0000 7fd0ed200640 1 -- start start 2023-11-24T08:08:55.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.440+0000 7fd0ed200640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fd0e80b48d0 0x7fd0e80b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.440+0000 7fd0ed200640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e80b23e0 con 0x7fd0e80b3210 2023-11-24T08:08:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.441+0000 7fd0ed200640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e80d41b0 con 0x7fd0e8160bd0 2023-11-24T08:08:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.441+0000 7fd0ed200640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e80cdc70 con 0x7fd0e80b48d0 2023-11-24T08:08:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.441+0000 7fd0e759e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd0e80b3210 0x7fd0e8167290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33404/0 (socket says 172.21.15.64:33404) 2023-11-24T08:08:55.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.441+0000 7fd0e759e640 1 -- 172.21.15.64:0/3415043065 learned_addr learned my addr 172.21.15.64:0/3415043065 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:55.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.441+0000 7fd0e659c640 1 --2- 172.21.15.64:0/3415043065 >> v2:172.21.15.91:3300/0 conn(0x7fd0e80b48d0 0x7fd0e80b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.441+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567840370 0 0) 0x7fd0e80b23e0 con 0x7fd0e80b3210 2023-11-24T08:08:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.441+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d4003520 con 0x7fd0e80b3210 2023-11-24T08:08:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3982788681 0 0) 0x7fd0d4003520 con 0x7fd0e80b3210 2023-11-24T08:08:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 >> v2:172.21.15.91:3300/0 conn(0x7fd0e80b48d0 msgr2=0x7fd0e80b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 --2- 172.21.15.64:0/3415043065 >> v2:172.21.15.91:3300/0 conn(0x7fd0e80b48d0 0x7fd0e80b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 >> v1:172.21.15.91:6789/0 conn(0x7fd0e8160bd0 legacy=0x7fd0e80b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8160fb0 con 0x7fd0e80b3210 2023-11-24T08:08:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fd0e0001670 con 0x7fd0e80b3210 2023-11-24T08:08:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 >> v1:172.21.15.64:6789/0 conn(0x7fd0e80b3210 legacy=0x7fd0e8167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0d4003970 0x7fd0d4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0d40051d0 0x7fd0d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80cdc70 con 0x7fd0d40047c0 2023-11-24T08:08:55.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d400c100 con 0x7fd0d40051d0 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d400c2c0 con 0x7fd0d4003970 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.442+0000 7fd0e659c640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e759e640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0d40051d0 0x7fd0d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e6d9d640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0d4003970 0x7fd0d4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e659c640 1 -- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0d4003970 msgr2=0x7fd0d4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e659c640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0d4003970 0x7fd0d4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e659c640 1 -- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0d40051d0 msgr2=0x7fd0d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e659c640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0d40051d0 0x7fd0d400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e659c640 1 -- 172.21.15.64:0/3415043065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8160fb0 con 0x7fd0d40047c0 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e759e640 1 -- 172.21.15.64:0/3415043065 reap_dead start 2023-11-24T08:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.443+0000 7fd0e6d9d640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0d4003970 0x7fd0d4003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.444+0000 7fd0e659c640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0d4004c40 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80b2a90 tx=0x7fd0d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a81d0ca9771929 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.444+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d002c600 con 0x7fd0d40047c0 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.444+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d002c760 con 0x7fd0d40047c0 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.444+0000 7fd0e5d9b640 1 -- 172.21.15.64:0/3415043065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d002cb10 con 0x7fd0d40047c0 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.445+0000 7fd0ed200640 1 -- 172.21.15.64:0/3415043065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 msgr2=0x7fd0d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.445+0000 7fd0ed200640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0d4004c40 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80b2a90 tx=0x7fd0d40057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.445+0000 7fd0ed200640 1 -- 172.21.15.64:0/3415043065 shutdown_connections 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.445+0000 7fd0ed200640 1 --2- 172.21.15.64:0/3415043065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0d4004c40 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.445+0000 7fd0ed200640 1 -- 172.21.15.64:0/3415043065 >> 172.21.15.64:0/3415043065 conn(0x7fd0e809c5a0 msgr2=0x7fd0e80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.445+0000 7fd0ed200640 1 -- 172.21.15.64:0/3415043065 shutdown_connections 2023-11-24T08:08:55.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.445+0000 7fd0ed200640 1 -- 172.21.15.64:0/3415043065 wait complete. 2023-11-24T08:08:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.447+0000 7fd0ed200640 1 Processor -- start 2023-11-24T08:08:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.447+0000 7fd0ed200640 1 -- start start 2023-11-24T08:08:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0ed200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0e81583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0ed200640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0e80ccd70 0x7fd0e8157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0ed200640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0e80cb690 0x7fd0e80c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0ed200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80cd1f0 con 0x7fd0d40047c0 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0ed200640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80cb200 con 0x7fd0e80ccd70 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0ed200640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80ca310 con 0x7fd0e80cb690 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0e6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0e81583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0e6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0e81583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35094/0 (socket says 172.21.15.64:35094) 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0e6d9d640 1 -- 172.21.15.64:0/2421460166 learned_addr learned my addr 172.21.15.64:0/2421460166 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0e6d9d640 1 -- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0e80cb690 msgr2=0x7fd0e80c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0e6d9d640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0e80cb690 0x7fd0e80c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0e6d9d640 1 -- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0e80ccd70 msgr2=0x7fd0e8157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0e6d9d640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0e80ccd70 0x7fd0e8157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.448+0000 7fd0e6d9d640 1 -- 172.21.15.64:0/2421460166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e80ca5f0 con 0x7fd0d40047c0 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.449+0000 7fd0e6d9d640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0e81583c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800d800 tx=0x7fd0d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.449+0000 7fd0df7fe640 1 -- 172.21.15.64:0/2421460166 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8003da0 con 0x7fd0d40047c0 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.449+0000 7fd0df7fe640 1 -- 172.21.15.64:0/2421460166 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d800b6b0 con 0x7fd0d40047c0 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.449+0000 7fd0df7fe640 1 -- 172.21.15.64:0/2421460166 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d800ba60 con 0x7fd0d40047c0 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.449+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e80ca8d0 con 0x7fd0d40047c0 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.449+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e80cae90 con 0x7fd0d40047c0 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.450+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e80cb020 con 0x7fd0d40047c0 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.451+0000 7fd0df7fe640 1 -- 172.21.15.64:0/2421460166 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd0d800bbc0 con 0x7fd0d40047c0 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.452+0000 7fd0df7fe640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0c806b4a0 0x7fd0c806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.453+0000 7fd0df7fe640 1 -- 172.21.15.64:0/2421460166 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fd0d808a4a0 con 0x7fd0d40047c0 2023-11-24T08:08:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.453+0000 7fd0e659c640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0c806b4a0 0x7fd0c806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.454+0000 7fd0e659c640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0c806b4a0 0x7fd0c806d960 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0036000 tx=0x7fd0d002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.457+0000 7fd0df7fe640 1 -- 172.21.15.64:0/2421460166 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0d8054a50 con 0x7fd0d40047c0 2023-11-24T08:08:55.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:55.689+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd0e80a1210 con 0x7fd0d40047c0 2023-11-24T08:08:56.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.211+0000 7fd0df7fe640 1 -- 172.21.15.64:0/2421460166 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd0d8058070 con 0x7fd0d40047c0 2023-11-24T08:08:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.214+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0c806b4a0 msgr2=0x7fd0c806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.214+0000 7fd0ed200640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0c806b4a0 0x7fd0c806d960 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0036000 tx=0x7fd0d002e000 comp rx=0 tx=0).stop 2023-11-24T08:08:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.214+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 msgr2=0x7fd0e81583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.214+0000 7fd0ed200640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0e81583c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800d800 tx=0x7fd0d800dcc0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.215+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 shutdown_connections 2023-11-24T08:08:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.215+0000 7fd0ed200640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fd0c806b4a0 0x7fd0c806d960 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.215+0000 7fd0ed200640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fd0e80cb690 0x7fd0e80c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.215+0000 7fd0ed200640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fd0e80ccd70 0x7fd0e8157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.215+0000 7fd0ed200640 1 --2- 172.21.15.64:0/2421460166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0d40047c0 0x7fd0e81583c0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.215+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 >> 172.21.15.64:0/2421460166 conn(0x7fd0e809c5a0 msgr2=0x7fd0e80cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.215+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 shutdown_connections 2023-11-24T08:08:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.215+0000 7fd0ed200640 1 -- 172.21.15.64:0/2421460166 wait complete. 2023-11-24T08:08:56.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:08:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.516+0000 7f6f6281a640 1 Processor -- start 2023-11-24T08:08:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.516+0000 7f6f6281a640 1 -- start start 2023-11-24T08:08:56.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.518+0000 7f6f6281a640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f6f5c06dae0 0x7f6f5c13f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.518+0000 7f6f6281a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c140430 con 0x7f6f5c13cf40 2023-11-24T08:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.518+0000 7f6f6281a640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c071220 con 0x7f6f5c13fc40 2023-11-24T08:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.518+0000 7f6f6281a640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c071500 con 0x7f6f5c06dae0 2023-11-24T08:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.518+0000 7f6f61818640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6f5c13cf40 0x7f6f5c06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33412/0 (socket says 172.21.15.64:33412) 2023-11-24T08:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.518+0000 7f6f61818640 1 -- 172.21.15.64:0/3179772454 learned_addr learned my addr 172.21.15.64:0/3179772454 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.518+0000 7f6f61017640 1 --2- 172.21.15.64:0/3179772454 >> v2:172.21.15.91:3300/0 conn(0x7f6f5c06dae0 0x7f6f5c13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.518+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167654987 0 0) 0x7f6f5c140430 con 0x7f6f5c13cf40 2023-11-24T08:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f48003520 con 0x7f6f5c13cf40 2023-11-24T08:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319555339 0 0) 0x7f6f48003520 con 0x7f6f5c13cf40 2023-11-24T08:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 >> v2:172.21.15.91:3300/0 conn(0x7f6f5c06dae0 msgr2=0x7f6f5c13f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 --2- 172.21.15.64:0/3179772454 >> v2:172.21.15.91:3300/0 conn(0x7f6f5c06dae0 0x7f6f5c13f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 >> v1:172.21.15.91:6789/0 conn(0x7f6f5c13fc40 legacy=0x7f6f5c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0717e0 con 0x7f6f5c13cf40 2023-11-24T08:08:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f6f58001670 con 0x7f6f5c13cf40 2023-11-24T08:08:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 >> v1:172.21.15.64:6789/0 conn(0x7f6f5c13cf40 legacy=0x7f6f5c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f480039e0 0x7f6f48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f480047c0 0x7f6f48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48005180 0x7f6f4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c071500 con 0x7f6f48005180 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4800c0b0 con 0x7f6f480047c0 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4800c270 con 0x7f6f480039e0 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.519+0000 7f6f62019640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48005180 0x7f6f4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.520+0000 7f6f61017640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f480047c0 0x7f6f48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.520+0000 7f6f61818640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f480039e0 0x7f6f48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.520+0000 7f6f62019640 1 -- 172.21.15.64:0/3179772454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f480039e0 msgr2=0x7f6f48003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.520+0000 7f6f62019640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f480039e0 0x7f6f48003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.520+0000 7f6f62019640 1 -- 172.21.15.64:0/3179772454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f480047c0 msgr2=0x7f6f48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.520+0000 7f6f62019640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f480047c0 0x7f6f48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.520+0000 7f6f62019640 1 -- 172.21.15.64:0/3179772454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f48003520 con 0x7f6f48005180 2023-11-24T08:08:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.521+0000 7f6f62019640 1 -- 172.21.15.64:0/3179772454 reap_dead start 2023-11-24T08:08:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.521+0000 7f6f62019640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48005180 0x7f6f4800b980 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6f54001e50 tx=0x7f6f480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1506378242342a71 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.521+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54003b70 con 0x7f6f48005180 2023-11-24T08:08:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.521+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f54003cf0 con 0x7f6f48005180 2023-11-24T08:08:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.521+0000 7f6f60816640 1 -- 172.21.15.64:0/3179772454 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54005310 con 0x7f6f48005180 2023-11-24T08:08:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.522+0000 7f6f6281a640 1 -- 172.21.15.64:0/3179772454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48005180 msgr2=0x7f6f4800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.522+0000 7f6f6281a640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48005180 0x7f6f4800b980 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6f54001e50 tx=0x7f6f480057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.522+0000 7f6f6281a640 1 -- 172.21.15.64:0/3179772454 shutdown_connections 2023-11-24T08:08:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.522+0000 7f6f6281a640 1 --2- 172.21.15.64:0/3179772454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f48005180 0x7f6f4800b980 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.522+0000 7f6f6281a640 1 -- 172.21.15.64:0/3179772454 >> 172.21.15.64:0/3179772454 conn(0x7f6f5c0b0aa0 msgr2=0x7f6f5c0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.522+0000 7f6f6281a640 1 -- 172.21.15.64:0/3179772454 shutdown_connections 2023-11-24T08:08:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.522+0000 7f6f6281a640 1 -- 172.21.15.64:0/3179772454 wait complete. 2023-11-24T08:08:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.524+0000 7f6f6281a640 1 Processor -- start 2023-11-24T08:08:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.524+0000 7f6f6281a640 1 -- start start 2023-11-24T08:08:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.525+0000 7f6f6281a640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f48005180 0x7f6f5c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.525+0000 7f6f6281a640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f5c1576f0 0x7f6f5c12f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.525+0000 7f6f6281a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5c153050 0x7f6f5c1534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.525+0000 7f6f6281a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c154c20 con 0x7f6f5c153050 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.525+0000 7f6f6281a640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c13e4d0 con 0x7f6f48005180 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.525+0000 7f6f6281a640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c13e7b0 con 0x7f6f5c1576f0 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.525+0000 7f6f62019640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5c153050 0x7f6f5c1534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f62019640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5c153050 0x7f6f5c1534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35118/0 (socket says 172.21.15.64:35118) 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f62019640 1 -- 172.21.15.64:0/2887818996 learned_addr learned my addr 172.21.15.64:0/2887818996 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f61818640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f48005180 0x7f6f5c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f61017640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f5c1576f0 0x7f6f5c12f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f62019640 1 -- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f5c1576f0 msgr2=0x7f6f5c12f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f62019640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f5c1576f0 0x7f6f5c12f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f62019640 1 -- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f48005180 msgr2=0x7f6f5c1515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f62019640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f48005180 0x7f6f5c1515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f62019640 1 -- 172.21.15.64:0/2887818996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c072160 con 0x7f6f5c153050 2023-11-24T08:08:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.526+0000 7f6f62019640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5c153050 0x7f6f5c1534d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6f54001f70 tx=0x7f6f540014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.527+0000 7f6f527fc640 1 -- 172.21.15.64:0/2887818996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54006d50 con 0x7f6f5c153050 2023-11-24T08:08:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.527+0000 7f6f527fc640 1 -- 172.21.15.64:0/2887818996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f54003b80 con 0x7f6f5c153050 2023-11-24T08:08:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.527+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c13ea90 con 0x7f6f5c153050 2023-11-24T08:08:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.527+0000 7f6f527fc640 1 -- 172.21.15.64:0/2887818996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54039680 con 0x7f6f5c153050 2023-11-24T08:08:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.527+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c153a10 con 0x7f6f5c153050 2023-11-24T08:08:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.528+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c04e840 con 0x7f6f5c153050 2023-11-24T08:08:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.529+0000 7f6f527fc640 1 -- 172.21.15.64:0/2887818996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6f540397e0 con 0x7f6f5c153050 2023-11-24T08:08:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.530+0000 7f6f527fc640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b4a0 0x7f6f4006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.530+0000 7f6f527fc640 1 -- 172.21.15.64:0/2887818996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6f540ac140 con 0x7f6f5c153050 2023-11-24T08:08:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.531+0000 7f6f61818640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b4a0 0x7f6f4006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.532+0000 7f6f61818640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b4a0 0x7f6f4006d960 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6f58001050 tx=0x7f6f58009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:56.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.535+0000 7f6f527fc640 1 -- 172.21.15.64:0/2887818996 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f54076660 con 0x7f6f5c153050 2023-11-24T08:08:56.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:56.731+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6f5c154c20 con 0x7f6f5c153050 2023-11-24T08:08:57.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.215+0000 7f6f527fc640 1 -- 172.21.15.64:0/2887818996 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6f540480d0 con 0x7f6f5c153050 2023-11-24T08:08:57.217 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:08:57.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b4a0 msgr2=0x7f6f4006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b4a0 0x7f6f4006d960 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6f58001050 tx=0x7f6f58009040 comp rx=0 tx=0).stop 2023-11-24T08:08:57.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5c153050 msgr2=0x7f6f5c1534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5c153050 0x7f6f5c1534d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6f54001f70 tx=0x7f6f540014a0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 shutdown_connections 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f6f4006b4a0 0x7f6f4006d960 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f5c153050 0x7f6f5c1534d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6f5c1576f0 0x7f6f5c12f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 --2- 172.21.15.64:0/2887818996 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f6f48005180 0x7f6f5c1515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.219+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 >> 172.21.15.64:0/2887818996 conn(0x7f6f5c0b0aa0 msgr2=0x7f6f5c0ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.220+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 shutdown_connections 2023-11-24T08:08:57.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.220+0000 7f6f6281a640 1 -- 172.21.15.64:0/2887818996 wait complete. 2023-11-24T08:08:57.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:08:57.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.462+0000 7f3202fdb640 1 Processor -- start 2023-11-24T08:08:57.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.462+0000 7f3202fdb640 1 -- start start 2023-11-24T08:08:57.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.464+0000 7f3202fdb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31fc15f270 0x7f31fc0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.464+0000 7f3202fdb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f31fc06d9e0 0x7f31fc0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.464+0000 7f3202fdb640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc15f930 con 0x7f31fc144770 2023-11-24T08:08:57.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.464+0000 7f3202fdb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc0b43b0 con 0x7f31fc15f270 2023-11-24T08:08:57.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.464+0000 7f3202fdb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc0b3b10 con 0x7f31fc06d9e0 2023-11-24T08:08:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.464+0000 7f32017d8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31fc15f270 0x7f31fc0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.465+0000 7f32017d8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31fc15f270 0x7f31fc0cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58286/0 (socket says 172.21.15.64:58286) 2023-11-24T08:08:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.465+0000 7f32017d8640 1 -- 172.21.15.64:0/2025116674 learned_addr learned my addr 172.21.15.64:0/2025116674 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:57.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.465+0000 7f32027da640 1 --2- 172.21.15.64:0/2025116674 >> v2:172.21.15.91:3300/0 conn(0x7f31fc06d9e0 0x7f31fc0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.465+0000 7f32017d8640 1 -- 172.21.15.64:0/2025116674 >> v2:172.21.15.91:3300/0 conn(0x7f31fc06d9e0 msgr2=0x7f31fc0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.465+0000 7f32017d8640 1 --2- 172.21.15.64:0/2025116674 >> v2:172.21.15.91:3300/0 conn(0x7f31fc06d9e0 0x7f31fc0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.465+0000 7f32017d8640 1 -- 172.21.15.64:0/2025116674 >> v1:172.21.15.91:6789/0 conn(0x7f31fc144770 legacy=0x7f31fc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.465+0000 7f32017d8640 1 -- 172.21.15.64:0/2025116674 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0b3df0 con 0x7f31fc15f270 2023-11-24T08:08:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.466+0000 7f32017d8640 1 --2- 172.21.15.64:0/2025116674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc15f270 0x7f31fc0cd4a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f31fc144eb0 tx=0x7f31ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e76af5d8a390c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.466+0000 7f3200fd7640 1 -- 172.21.15.64:0/2025116674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec031870 con 0x7f31fc15f270 2023-11-24T08:08:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.467+0000 7f3200fd7640 1 -- 172.21.15.64:0/2025116674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec0319d0 con 0x7f31fc15f270 2023-11-24T08:08:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.467+0000 7f3200fd7640 1 -- 172.21.15.64:0/2025116674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec031d60 con 0x7f31fc15f270 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.467+0000 7f3202fdb640 1 -- 172.21.15.64:0/2025116674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc15f270 msgr2=0x7f31fc0cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.467+0000 7f3202fdb640 1 --2- 172.21.15.64:0/2025116674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc15f270 0x7f31fc0cd4a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f31fc144eb0 tx=0x7f31ec02b910 comp rx=0 tx=0).stop 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.467+0000 7f3202fdb640 1 -- 172.21.15.64:0/2025116674 shutdown_connections 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.467+0000 7f3202fdb640 1 --2- 172.21.15.64:0/2025116674 >> v2:172.21.15.91:3300/0 conn(0x7f31fc06d9e0 0x7f31fc0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.467+0000 7f3202fdb640 1 --2- 172.21.15.64:0/2025116674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc15f270 0x7f31fc0cd4a0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.467+0000 7f3202fdb640 1 -- 172.21.15.64:0/2025116674 >> 172.21.15.64:0/2025116674 conn(0x7f31fc0cec10 msgr2=0x7f31fc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.468+0000 7f3202fdb640 1 -- 172.21.15.64:0/2025116674 shutdown_connections 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.468+0000 7f3202fdb640 1 -- 172.21.15.64:0/2025116674 wait complete. 2023-11-24T08:08:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.469+0000 7f3202fdb640 1 Processor -- start 2023-11-24T08:08:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.470+0000 7f3202fdb640 1 -- start start 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.470+0000 7f3202fdb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31fc06d9e0 0x7f31fc15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.470+0000 7f3202fdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc144770 0x7f31fc14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f3202fdb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31fc15f270 0x7f31fc13d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f3202fdb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc15cf50 con 0x7f31fc144770 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f3202fdb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc15beb0 con 0x7f31fc06d9e0 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f3202fdb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc15c190 con 0x7f31fc15f270 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32017d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc144770 0x7f31fc14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32017d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc144770 0x7f31fc14d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58298/0 (socket says 172.21.15.64:58298) 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32017d8640 1 -- 172.21.15.64:0/921471124 learned_addr learned my addr 172.21.15.64:0/921471124 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f3201fd9640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31fc06d9e0 0x7f31fc15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32027da640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31fc15f270 0x7f31fc13d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32017d8640 1 -- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31fc15f270 msgr2=0x7f31fc13d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32017d8640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31fc15f270 0x7f31fc13d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32017d8640 1 -- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31fc06d9e0 msgr2=0x7f31fc15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32017d8640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31fc06d9e0 0x7f31fc15b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.471+0000 7f32017d8640 1 -- 172.21.15.64:0/921471124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0094d0 con 0x7f31fc144770 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.472+0000 7f32017d8640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc144770 0x7f31fc14d6e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f31ec000c00 tx=0x7f31ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.472+0000 7f31f27fc640 1 -- 172.21.15.64:0/921471124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec03abe0 con 0x7f31fc144770 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.472+0000 7f31f27fc640 1 -- 172.21.15.64:0/921471124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec03ad40 con 0x7f31fc144770 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.472+0000 7f31f27fc640 1 -- 172.21.15.64:0/921471124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec044610 con 0x7f31fc144770 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.472+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc15c470 con 0x7f31fc144770 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.472+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc15ca30 con 0x7f31fc144770 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.473+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc04e840 con 0x7f31fc144770 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.474+0000 7f31f27fc640 1 -- 172.21.15.64:0/921471124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f31ec04d450 con 0x7f31fc144770 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.475+0000 7f31f27fc640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f31e006b470 0x7f31e006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.475+0000 7f31f27fc640 1 -- 172.21.15.64:0/921471124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f31ec02d040 con 0x7f31fc144770 2023-11-24T08:08:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.476+0000 7f3201fd9640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f31e006b470 0x7f31e006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.477+0000 7f3201fd9640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f31e006b470 0x7f31e006d930 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f31f8007fd0 tx=0x7f31f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.480+0000 7f31f27fc640 1 -- 172.21.15.64:0/921471124 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f31ec076e80 con 0x7f31fc144770 2023-11-24T08:08:57.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:57.675+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f31fc0b2460 con 0x7f31fc144770 2023-11-24T08:08:58.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.228+0000 7f31f27fc640 1 -- 172.21.15.64:0/921471124 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7f31ec0350d0 con 0x7f31fc144770 2023-11-24T08:08:58.231 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-11-24T08:08:58.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f31e006b470 msgr2=0x7f31e006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f31e006b470 0x7f31e006d930 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f31f8007fd0 tx=0x7f31f8009040 comp rx=0 tx=0).stop 2023-11-24T08:08:58.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc144770 msgr2=0x7f31fc14d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc144770 0x7f31fc14d6e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f31ec000c00 tx=0x7f31ec0046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 shutdown_connections 2023-11-24T08:08:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f31e006b470 0x7f31e006d930 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f31fc15f270 0x7f31fc13d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc144770 0x7f31fc14d6e0 secure :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f31ec000c00 tx=0x7f31ec0046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 --2- 172.21.15.64:0/921471124 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f31fc06d9e0 0x7f31fc15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.234+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 >> 172.21.15.64:0/921471124 conn(0x7f31fc0cec10 msgr2=0x7f31fc0ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.235+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 shutdown_connections 2023-11-24T08:08:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.235+0000 7f3202fdb640 1 -- 172.21.15.64:0/921471124 wait complete. 2023-11-24T08:08:58.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:08:58.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.471+0000 7fbf9bfff640 1 Processor -- start 2023-11-24T08:08:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.471+0000 7fbf9bfff640 1 -- start start 2023-11-24T08:08:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.472+0000 7fbf9bfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fbf9c0d1d90 0x7fbf9c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.472+0000 7fbf9bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbf9c0d0fa0 0x7fbf9c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.472+0000 7fbf9bfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0c6bf0 con 0x7fbf9c15d3b0 2023-11-24T08:08:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.472+0000 7fbf9bfff640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c0d13f0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.472+0000 7fbf9bfff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c0cedc0 con 0x7fbf9c0d1d90 2023-11-24T08:08:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.473+0000 7fbf9b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbf9c0d0fa0 0x7fbf9c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.473+0000 7fbf9b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbf9c0d0fa0 0x7fbf9c0b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58312/0 (socket says 172.21.15.64:58312) 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.473+0000 7fbf9b7fe640 1 -- 172.21.15.64:0/3636240603 learned_addr learned my addr 172.21.15.64:0/3636240603 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.473+0000 7fbf9a7fc640 1 --2- 172.21.15.64:0/3636240603 >> v2:172.21.15.91:3300/0 conn(0x7fbf9c0d1d90 0x7fbf9c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.473+0000 7fbf9b7fe640 1 -- 172.21.15.64:0/3636240603 >> v2:172.21.15.91:3300/0 conn(0x7fbf9c0d1d90 msgr2=0x7fbf9c0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.473+0000 7fbf9b7fe640 1 --2- 172.21.15.64:0/3636240603 >> v2:172.21.15.91:3300/0 conn(0x7fbf9c0d1d90 0x7fbf9c0ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.473+0000 7fbf9b7fe640 1 -- 172.21.15.64:0/3636240603 >> v1:172.21.15.91:6789/0 conn(0x7fbf9c15d3b0 legacy=0x7fbf9c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.473+0000 7fbf9b7fe640 1 -- 172.21.15.64:0/3636240603 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0c6e40 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.474+0000 7fbf9b7fe640 1 --2- 172.21.15.64:0/3636240603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0b3ca0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0ca100 tx=0x7fbf9402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f93c7241fb7276 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf99ffb640 1 -- 172.21.15.64:0/3636240603 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf940317e0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf99ffb640 1 -- 172.21.15.64:0/3636240603 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf94031940 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf99ffb640 1 -- 172.21.15.64:0/3636240603 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94031cd0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf9bfff640 1 -- 172.21.15.64:0/3636240603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 msgr2=0x7fbf9c0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/3636240603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0b3ca0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0ca100 tx=0x7fbf9402b880 comp rx=0 tx=0).stop 2023-11-24T08:08:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf9bfff640 1 -- 172.21.15.64:0/3636240603 shutdown_connections 2023-11-24T08:08:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/3636240603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0b3ca0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/3636240603 >> v2:172.21.15.91:3300/0 conn(0x7fbf9c0d1d90 0x7fbf9c0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf9bfff640 1 -- 172.21.15.64:0/3636240603 >> 172.21.15.64:0/3636240603 conn(0x7fbf9c0d30a0 msgr2=0x7fbf9c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf9bfff640 1 -- 172.21.15.64:0/3636240603 shutdown_connections 2023-11-24T08:08:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.475+0000 7fbf9bfff640 1 -- 172.21.15.64:0/3636240603 wait complete. 2023-11-24T08:08:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.477+0000 7fbf9bfff640 1 Processor -- start 2023-11-24T08:08:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.477+0000 7fbf9bfff640 1 -- start start 2023-11-24T08:08:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.478+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.478+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf9c0d1d90 0x7fbf9c15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.478+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf9c15d3b0 0x7fbf9c15eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.478+0000 7fbf9bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0b8cd0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.478+0000 7fbf9bfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c148a00 con 0x7fbf9c0d1d90 2023-11-24T08:08:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.478+0000 7fbf9bfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0b89f0 con 0x7fbf9c15d3b0 2023-11-24T08:08:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58320/0 (socket says 172.21.15.64:58320) 2023-11-24T08:08:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9affd640 1 -- 172.21.15.64:0/304214556 learned_addr learned my addr 172.21.15.64:0/304214556 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9a7fc640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf9c0d1d90 0x7fbf9c15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9affd640 1 -- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf9c15d3b0 msgr2=0x7fbf9c15eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9b7fe640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf9c15d3b0 0x7fbf9c15eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9affd640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf9c15d3b0 0x7fbf9c15eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9affd640 1 -- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf9c0d1d90 msgr2=0x7fbf9c15e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9affd640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf9c0d1d90 0x7fbf9c15e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9affd640 1 -- 172.21.15.64:0/304214556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940094d0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.479+0000 7fbf9b7fe640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf9c15d3b0 0x7fbf9c15eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.480+0000 7fbf9affd640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0c6310 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00ba50 tx=0x7fbf8c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.480+0000 7fbf937fe640 1 -- 172.21.15.64:0/304214556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c00cbe0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.480+0000 7fbf937fe640 1 -- 172.21.15.64:0/304214556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c00cd80 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.480+0000 7fbf937fe640 1 -- 172.21.15.64:0/304214556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c014680 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.480+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c148340 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.480+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c0b14b0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.481+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c1481a0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.482+0000 7fbf937fe640 1 -- 172.21.15.64:0/304214556 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fbf8c0147e0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.483+0000 7fbf937fe640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbf7c06b470 0x7fbf7c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.483+0000 7fbf937fe640 1 -- 172.21.15.64:0/304214556 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbf8c089cf0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.484+0000 7fbf9a7fc640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbf7c06b470 0x7fbf7c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.485+0000 7fbf9a7fc640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbf7c06b470 0x7fbf7c06d930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbf84006fd0 tx=0x7fbf84008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.487+0000 7fbf937fe640 1 -- 172.21.15.64:0/304214556 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf9c1481a0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.682+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbf9c1481a0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.689+0000 7fbf937fe640 1 -- 172.21.15.64:0/304214556 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf9c1481a0 con 0x7fbf9c0d0fa0 2023-11-24T08:08:58.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbf7c06b470 msgr2=0x7fbf7c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbf7c06b470 0x7fbf7c06d930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbf84006fd0 tx=0x7fbf84008040 comp rx=0 tx=0).stop 2023-11-24T08:08:58.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 msgr2=0x7fbf9c0c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0c6310 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00ba50 tx=0x7fbf8c00bf10 comp rx=0 tx=0).stop 2023-11-24T08:08:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 shutdown_connections 2023-11-24T08:08:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbf7c06b470 0x7fbf7c06d930 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbf9c15d3b0 0x7fbf9c15eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbf9c0d1d90 0x7fbf9c15e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 --2- 172.21.15.64:0/304214556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf9c0d0fa0 0x7fbf9c0c6310 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 >> 172.21.15.64:0/304214556 conn(0x7fbf9c0d30a0 msgr2=0x7fbf9c0b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 shutdown_connections 2023-11-24T08:08:58.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.692+0000 7fbf9bfff640 1 -- 172.21.15.64:0/304214556 wait complete. 2023-11-24T08:08:58.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T08:08:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.935+0000 7ff0ff59e640 1 Processor -- start 2023-11-24T08:08:58.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.936+0000 7ff0ff59e640 1 -- start start 2023-11-24T08:08:58.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.937+0000 7ff0ff59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 0x7ff1000d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.937+0000 7ff0ff59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7ff1000d14a0 0x7ff1000d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.937+0000 7ff0ff59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1000cf0c0 con 0x7ff1000cf7a0 2023-11-24T08:08:58.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.937+0000 7ff0ff59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1000b01b0 con 0x7ff1000d14a0 2023-11-24T08:08:58.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.937+0000 7ff0ff59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10015d990 con 0x7ff10015d5b0 2023-11-24T08:08:58.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.937+0000 7ff0fed9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff1000cf7a0 0x7ff1000cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38210/0 (socket says 172.21.15.64:38210) 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.937+0000 7ff0fed9d640 1 -- 172.21.15.64:0/3221582339 learned_addr learned my addr 172.21.15.64:0/3221582339 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fdd9b640 1 --2- 172.21.15.64:0/3221582339 >> v2:172.21.15.91:3300/0 conn(0x7ff1000d14a0 0x7ff1000d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fe59c640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 0x7ff1000d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631171313 0 0) 0x7ff1000cf0c0 con 0x7ff1000cf7a0 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ec003520 con 0x7ff1000cf7a0 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1188557295 0 0) 0x7ff0ec003520 con 0x7ff1000cf7a0 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 msgr2=0x7ff1000d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 0x7ff1000d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 >> v2:172.21.15.91:3300/0 conn(0x7ff1000d14a0 msgr2=0x7ff1000d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 --2- 172.21.15.64:0/3221582339 >> v2:172.21.15.91:3300/0 conn(0x7ff1000d14a0 0x7ff1000d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000d1e40 con 0x7ff1000cf7a0 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7ff0f80019f0 con 0x7ff1000cf7a0 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 >> v1:172.21.15.64:6789/0 conn(0x7ff1000cf7a0 legacy=0x7ff1000cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 0x7ff0ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.938+0000 7ff0fd59a640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0ec003fb0 0x7ff0ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fd59a640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec004d00 0x7ff0ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000b01b0 con 0x7ff0ec003fb0 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10015d990 con 0x7ff0ec004d00 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec00bab0 con 0x7ff10015d5b0 2023-11-24T08:08:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fdd9b640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0ec003fb0 0x7ff0ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fe59c640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 0x7ff0ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fed9d640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec004d00 0x7ff0ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fdd9b640 1 -- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 msgr2=0x7ff0ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fdd9b640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 0x7ff0ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fdd9b640 1 -- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec004d00 msgr2=0x7ff0ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fdd9b640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec004d00 0x7ff0ec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fdd9b640 1 -- 172.21.15.64:0/3221582339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec003520 con 0x7ff0ec003fb0 2023-11-24T08:08:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.939+0000 7ff0fe59c640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff10015d5b0 0x7ff0ec0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.941+0000 7ff0fdd9b640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0ec003fb0 0x7ff0ec0047c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800ba30 tx=0x7ff0ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e12d50314742714 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.941+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8032680 con 0x7ff0ec003fb0 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.941+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e80327e0 con 0x7ff0ec003fb0 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.941+0000 7ff0fd59a640 1 -- 172.21.15.64:0/3221582339 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8032b70 con 0x7ff0ec003fb0 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.941+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3221582339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0ec003fb0 msgr2=0x7ff0ec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.941+0000 7ff0ff59e640 1 --2- 172.21.15.64:0/3221582339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0ec003fb0 0x7ff0ec0047c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800ba30 tx=0x7ff0ec0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.941+0000 7ff0fed9d640 1 -- 172.21.15.64:0/3221582339 reap_dead start 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.942+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3221582339 shutdown_connections 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.942+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3221582339 >> 172.21.15.64:0/3221582339 conn(0x7ff10009c3b0 msgr2=0x7ff1000d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.942+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3221582339 shutdown_connections 2023-11-24T08:08:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.942+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3221582339 wait complete. 2023-11-24T08:08:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.944+0000 7ff0ff59e640 1 Processor -- start 2023-11-24T08:08:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.944+0000 7ff0ff59e640 1 -- start start 2023-11-24T08:08:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0ff59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec003fb0 0x7ff100153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0ff59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1000b89d0 0x7ff1000b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0ff59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1000ccb70 0x7ff1000b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0ff59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000cc6e0 con 0x7ff1000ccb70 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0ff59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000cbad0 con 0x7ff0ec003fb0 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0ff59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000cbdb0 con 0x7ff1000b89d0 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1000ccb70 0x7ff1000b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1000ccb70 0x7ff1000b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58338/0 (socket says 172.21.15.64:58338) 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fed9d640 1 -- 172.21.15.64:0/3659938830 learned_addr learned my addr 172.21.15.64:0/3659938830 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fdd9b640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1000b89d0 0x7ff1000b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fe59c640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec003fb0 0x7ff100153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fed9d640 1 -- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1000b89d0 msgr2=0x7ff1000b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fed9d640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1000b89d0 0x7ff1000b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fed9d640 1 -- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec003fb0 msgr2=0x7ff100153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fed9d640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec003fb0 0x7ff100153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.945+0000 7ff0fed9d640 1 -- 172.21.15.64:0/3659938830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000cc090 con 0x7ff1000ccb70 2023-11-24T08:08:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.946+0000 7ff0fed9d640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1000ccb70 0x7ff1000b0a80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff10008be90 tx=0x7ff0f800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.946+0000 7ff0f6ffd640 1 -- 172.21.15.64:0/3659938830 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800b8d0 con 0x7ff1000ccb70 2023-11-24T08:08:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.946+0000 7ff0f6ffd640 1 -- 172.21.15.64:0/3659938830 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f800ba30 con 0x7ff1000ccb70 2023-11-24T08:08:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.946+0000 7ff0f6ffd640 1 -- 172.21.15.64:0/3659938830 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800bde0 con 0x7ff1000ccb70 2023-11-24T08:08:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.946+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1000cc370 con 0x7ff1000ccb70 2023-11-24T08:08:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.946+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1001f2f80 con 0x7ff1000ccb70 2023-11-24T08:08:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.948+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10004e840 con 0x7ff1000ccb70 2023-11-24T08:08:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.949+0000 7ff0f6ffd640 1 -- 172.21.15.64:0/3659938830 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff0f8007070 con 0x7ff1000ccb70 2023-11-24T08:08:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.950+0000 7ff0f6ffd640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff0e006b4a0 0x7ff0e006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.950+0000 7ff0f6ffd640 1 -- 172.21.15.64:0/3659938830 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff0f808a2a0 con 0x7ff1000ccb70 2023-11-24T08:08:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.951+0000 7ff0fe59c640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff0e006b4a0 0x7ff0e006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.952+0000 7ff0fe59c640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff0e006b4a0 0x7ff0e006d960 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00099c0 tx=0x7ff0f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:58.955+0000 7ff0f6ffd640 1 -- 172.21.15.64:0/3659938830 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0f8054bc0 con 0x7ff1000ccb70 2023-11-24T08:08:59.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.224+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7ff1000678d0 con 0x7ff1000ccb70 2023-11-24T08:08:59.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.231+0000 7ff0f6ffd640 1 -- 172.21.15.64:0/3659938830 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v298) v1 ==== 164+0+0 (secure 0 0 0) 0x7ff0f80581e0 con 0x7ff1000ccb70 2023-11-24T08:08:59.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff0e006b4a0 msgr2=0x7ff0e006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff0e006b4a0 0x7ff0e006d960 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00099c0 tx=0x7ff0f0008040 comp rx=0 tx=0).stop 2023-11-24T08:08:59.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1000ccb70 msgr2=0x7ff1000b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1000ccb70 0x7ff1000b0a80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff10008be90 tx=0x7ff0f800b0d0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 shutdown_connections 2023-11-24T08:08:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7ff0e006b4a0 0x7ff0e006d960 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1000ccb70 0x7ff1000b0a80 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7ff1000b89d0 0x7ff1000b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 --2- 172.21.15.64:0/3659938830 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7ff0ec003fb0 0x7ff100153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.234+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 >> 172.21.15.64:0/3659938830 conn(0x7ff10009c3b0 msgr2=0x7ff100160160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.235+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 shutdown_connections 2023-11-24T08:08:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:08:59.235+0000 7ff0ff59e640 1 -- 172.21.15.64:0/3659938830 wait complete. 2023-11-24T08:08:59.257 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ERROR 2023-11-24T08:08:59.258 INFO:tasks.cephfs_test_runner: 2023-11-24T08:08:59.258 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T08:08:59.258 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:08:59.258 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:08:59.258 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T08:08:59.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs/test_volumes.py", line 4530, in test_subvolume_inherited_snapshot_ls 2023-11-24T08:08:59.258 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T08:08:59.258 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner: 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner:Ran 10 tests in 261.098s 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner: 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=5) 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner: 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs/test_volumes.py", line 4530, in test_subvolume_inherited_snapshot_ls 2023-11-24T08:08:59.259 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T08:08:59.260 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T08:08:59.260 INFO:tasks.cephfs_test_runner: 2023-11-24T08:08:59.260 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:08:59.364 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1b15498a0d304d0d88e9d9b56b9c10e6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:08:59.365 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T08:08:59.373 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-24T08:08:59.392 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-24T08:08:59.392 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:08:59.392 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:59.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:08:59.457 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-11-24T08:08:59.458 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:59.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054305,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249416k,nr_inodes=812354,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498836k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:08:59.507 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:08:59.507 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:59.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:08:59.567 DEBUG:teuthology.orchestra.run.smithi091:> set -ex 2023-11-24T08:08:59.567 DEBUG:teuthology.orchestra.run.smithi091:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:08:59.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:08:59.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:59.621 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-11-24T08:08:59.686 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:08:59.686 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:08:59.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:59.687 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:08:59.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:59.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:08:59.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:08:59.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-11-24T08:09:00.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:00.010 DEBUG:teuthology.orchestra.run.smithi091:> ip netns list 2023-11-24T08:09:00.071 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:09:00.071 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:09:00.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:00.072 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:09:00.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:00.167 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:09:00.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:00.252 DEBUG:teuthology.orchestra.run.smithi091:> sudo ip link delete ceph-brx 2023-11-24T08:09:00.375 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T08:09:00.384 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T08:09:00.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:09:00.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.614+0000 7f1636d61640 1 Processor -- start 2023-11-24T08:09:00.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.614+0000 7f1636d61640 1 -- start start 2023-11-24T08:09:00.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.615+0000 7f1636d61640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f163006d520 0x7f163006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.615+0000 7f1636d61640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16300ad720 0x7f163015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.616+0000 7f1636d61640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163015c360 con 0x7f16300abff0 2023-11-24T08:09:00.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.616+0000 7f1636d61640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16300b41a0 con 0x7f163006d520 2023-11-24T08:09:00.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.616+0000 7f1636d61640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163006de40 con 0x7f16300ad720 2023-11-24T08:09:00.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.616+0000 7f16352fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f16300abff0 0x7f163015f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38216/0 (socket says 172.21.15.64:38216) 2023-11-24T08:09:00.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.616+0000 7f16352fe640 1 -- 172.21.15.64:0/2963613635 learned_addr learned my addr 172.21.15.64:0/2963613635 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:00.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.616+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623132944 0 0) 0x7f163015c360 con 0x7f16300abff0 2023-11-24T08:09:00.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.616+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1618003520 con 0x7f16300abff0 2023-11-24T08:09:00.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f1634afd640 1 --2- 172.21.15.64:0/2963613635 >> v2:172.21.15.91:3300/0 conn(0x7f163006d520 0x7f163006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162ffff640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16300ad720 0x7f163015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2067042430 0 0) 0x7f1618003520 con 0x7f16300abff0 2023-11-24T08:09:00.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16300ad720 msgr2=0x7f163015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:00.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16300ad720 0x7f163015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 >> v2:172.21.15.91:3300/0 conn(0x7f163006d520 msgr2=0x7f163006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:00.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 --2- 172.21.15.64:0/2963613635 >> v2:172.21.15.91:3300/0 conn(0x7f163006d520 0x7f163006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163015ba30 con 0x7f16300abff0 2023-11-24T08:09:00.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f1628001670 con 0x7f16300abff0 2023-11-24T08:09:00.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 >> v1:172.21.15.64:6789/0 conn(0x7f16300abff0 legacy=0x7f163015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618003970 0x7f1618003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16180047c0 0x7f1618004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1618005180 0x7f161800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163006de40 con 0x7f1618005180 2023-11-24T08:09:00.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161800c050 con 0x7f16180047c0 2023-11-24T08:09:00.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161800c210 con 0x7f1618003970 2023-11-24T08:09:00.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f162ffff640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f16300ad720 0x7f163015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:00.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.617+0000 7f16352fe640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1618005180 0x7f161800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.618+0000 7f1634afd640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618003970 0x7f1618003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.618+0000 7f162ffff640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16180047c0 0x7f1618004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.618+0000 7f16352fe640 1 -- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618003970 msgr2=0x7f1618003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:00.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.618+0000 7f16352fe640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618003970 0x7f1618003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.618+0000 7f16352fe640 1 -- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16180047c0 msgr2=0x7f1618004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:00.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.618+0000 7f16352fe640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16180047c0 0x7f1618004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.618+0000 7f16352fe640 1 -- 172.21.15.64:0/2963613635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618003520 con 0x7f1618005180 2023-11-24T08:09:00.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.618+0000 7f162ffff640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f16180047c0 0x7f1618004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:00.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.619+0000 7f16352fe640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1618005180 0x7f161800b980 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f162800c160 tx=0x7f16180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae8715db936499d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.619+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628005ad0 con 0x7f1618005180 2023-11-24T08:09:00.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.619+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1628005c30 con 0x7f1618005180 2023-11-24T08:09:00.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.619+0000 7f162f7fe640 1 -- 172.21.15.64:0/2963613635 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162802c7e0 con 0x7f1618005180 2023-11-24T08:09:00.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.620+0000 7f1636d61640 1 -- 172.21.15.64:0/2963613635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1618005180 msgr2=0x7f161800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.620+0000 7f1636d61640 1 --2- 172.21.15.64:0/2963613635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1618005180 0x7f161800b980 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f162800c160 tx=0x7f16180057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.620+0000 7f16352fe640 1 -- 172.21.15.64:0/2963613635 reap_dead start 2023-11-24T08:09:00.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.620+0000 7f1636d61640 1 -- 172.21.15.64:0/2963613635 shutdown_connections 2023-11-24T08:09:00.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.620+0000 7f1636d61640 1 -- 172.21.15.64:0/2963613635 >> 172.21.15.64:0/2963613635 conn(0x7f16300b0d80 msgr2=0x7f16300b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:00.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.620+0000 7f1636d61640 1 -- 172.21.15.64:0/2963613635 shutdown_connections 2023-11-24T08:09:00.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.620+0000 7f1636d61640 1 -- 172.21.15.64:0/2963613635 wait complete. 2023-11-24T08:09:00.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.622+0000 7f1636d61640 1 Processor -- start 2023-11-24T08:09:00.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.622+0000 7f1636d61640 1 -- start start 2023-11-24T08:09:00.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f1636d61640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618005180 0x7f163014d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f1636d61640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1630156c90 0x7f1630157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f1636d61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1630070800 0x7f1630070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f1636d61640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630157650 con 0x7f1630070800 2023-11-24T08:09:00.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f1636d61640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630072220 con 0x7f1630156c90 2023-11-24T08:09:00.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f1636d61640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163014d850 con 0x7f1618005180 2023-11-24T08:09:00.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f16352fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1630070800 0x7f1630070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f16352fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1630070800 0x7f1630070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58346/0 (socket says 172.21.15.64:58346) 2023-11-24T08:09:00.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.623+0000 7f16352fe640 1 -- 172.21.15.64:0/4588462 learned_addr learned my addr 172.21.15.64:0/4588462 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:00.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f1634afd640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618005180 0x7f163014d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f162ffff640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1630156c90 0x7f1630157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f16352fe640 1 -- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618005180 msgr2=0x7f163014d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f16352fe640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618005180 0x7f163014d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f16352fe640 1 -- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1630156c90 msgr2=0x7f1630157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f16352fe640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1630156c90 0x7f1630157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f16352fe640 1 -- 172.21.15.64:0/4588462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1630071610 con 0x7f1630070800 2023-11-24T08:09:00.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f1634afd640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618005180 0x7f163014d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:00.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f162ffff640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1630156c90 0x7f1630157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:00.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.624+0000 7f16352fe640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1630070800 0x7f1630070c80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f162800c280 tx=0x7f1628001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.625+0000 7f162d7fa640 1 -- 172.21.15.64:0/4588462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162803c040 con 0x7f1630070800 2023-11-24T08:09:00.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.625+0000 7f162d7fa640 1 -- 172.21.15.64:0/4588462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162802cbf0 con 0x7f1630070800 2023-11-24T08:09:00.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.625+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16300718f0 con 0x7f1630070800 2023-11-24T08:09:00.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.625+0000 7f162d7fa640 1 -- 172.21.15.64:0/4588462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628007070 con 0x7f1630070800 2023-11-24T08:09:00.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.625+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1630071eb0 con 0x7f1630070800 2023-11-24T08:09:00.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.626+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1630074770 con 0x7f1630070800 2023-11-24T08:09:00.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.627+0000 7f162d7fa640 1 -- 172.21.15.64:0/4588462 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1628032070 con 0x7f1630070800 2023-11-24T08:09:00.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.628+0000 7f162d7fa640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161c06b470 0x7f161c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.628+0000 7f162d7fa640 1 -- 172.21.15.64:0/4588462 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f16280aacd0 con 0x7f1630070800 2023-11-24T08:09:00.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.629+0000 7f1634afd640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161c06b470 0x7f161c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.630+0000 7f1634afd640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161c06b470 0x7f161c06d930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f16301f6590 tx=0x7f162000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.632+0000 7f162d7fa640 1 -- 172.21.15.64:0/4588462 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16280758d0 con 0x7f1630070800 2023-11-24T08:09:00.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.827+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f163015b6e0 con 0x7f161c06b470 2023-11-24T08:09:00.834 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:09:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.833+0000 7f162d7fa640 1 -- 172.21.15.64:0/4588462 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48773 (secure 0 0 0) 0x7f1620003b10 con 0x7f161c06b470 2023-11-24T08:09:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:09:00.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161c06b470 msgr2=0x7f161c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161c06b470 0x7f161c06d930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f16301f6590 tx=0x7f162000b040 comp rx=0 tx=0).stop 2023-11-24T08:09:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1630070800 msgr2=0x7f1630070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1630070800 0x7f1630070c80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f162800c280 tx=0x7f1628001930 comp rx=0 tx=0).stop 2023-11-24T08:09:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 shutdown_connections 2023-11-24T08:09:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f161c06b470 0x7f161c06d930 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1630070800 0x7f1630070c80 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1630156c90 0x7f1630157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 --2- 172.21.15.64:0/4588462 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1618005180 0x7f163014d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.835+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 >> 172.21.15.64:0/4588462 conn(0x7f16300b0d80 msgr2=0x7f16300af690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:00.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.836+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 shutdown_connections 2023-11-24T08:09:00.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:00.836+0000 7f1636d61640 1 -- 172.21.15.64:0/4588462 wait complete. 2023-11-24T08:09:00.856 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-24T08:09:00.125355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259428,"kb_used_data":9980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715292,"statfs":{"total":773094113280,"available":772828459008,"internally_reserved":0,"allocated":10219520,"data_stored":5062765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-35583,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002114"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":227,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-24T08:08:57.223161+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:57.223161+0000","last_peered":"2023-11-24T08:08:57.223161+0000","last_clean":"2023-11-24T08:08:57.223161+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:57.223161+0000","last_undegraded":"2023-11-24T08:08:57.223161+0000","last_fullsized":"2023-11-24T08:08:57.223161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1073622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.022,"15min":1.01},"min":{"1min":0.63700000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.05,"15min":1.0800000000000001},"min":{"1min":0.72099999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.72}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.27,"15min":1.298},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2909999999999999,"15min":1.367},"min":{"1min":0.89100000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0939999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.865}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1699999999999999,"15min":1.1539999999999999},"min":{"1min":0.77300000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.6840000000000002,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.266,"15min":1.302},"min":{"1min":0.85299999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.25,"15min":1.3100000000000001},"min":{"1min":0.78200000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":4.3570000000000002,"5min":4.3570000000000002,"15min":4.3570000000000002},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.365,"15min":1.4870000000000001},"min":{"1min":0.75900000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.2480000000000002,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.405}]},{"osd":4,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.4910000000000001},"min":{"1min":0.755,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.327,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":2.012},{"interface":"front","average":{"1min":1.45,"5min":1.3360000000000001,"15min":1.369},"min":{"1min":0.95499999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.3049999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.587},"min":{"1min":0.91100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.589},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.5289999999999999,"15min":1.637},"min":{"1min":0.96199999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":2.1160000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.583,"15min":1.7190000000000001},"min":{"1min":1.016,"5min":0.755,"15min":0.755},"max":{"1min":2.5329999999999999,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":2.2040000000000002},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.486,"15min":1.625},"min":{"1min":1.1950000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9410000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.6799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2090000000000001,"15min":1.3069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.8460000000000001,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.246,"5min":1.236,"15min":1.3260000000000001},"min":{"1min":1.0369999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.4530000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.038}]},{"osd":1,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1339999999999999,"15min":1.1519999999999999},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.4710000000000001,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1699999999999999,"15min":1.284},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.532,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1579999999999999,"15min":1.272},"min":{"1min":0.83799999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5329999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.256,"15min":1.4530000000000001},"min":{"1min":0.755,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.788,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.622}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3420000000000001,"15min":1.4279999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.8839999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.472,"5min":1.4419999999999999,"15min":1.4890000000000001},"min":{"1min":1.153,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.754,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.98599999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4399999999999999,"15min":1.6950000000000001},"min":{"1min":0.88400000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.925,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":0.752},{"interface":"front","average":{"1min":1.488,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.117,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9830000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4750000000000001,"15min":1.744},"min":{"1min":0.90400000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.98,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.613,"15min":1.885},"min":{"1min":0.96999999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.036,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.1299999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5660000000000001,"15min":1.665},"min":{"1min":1.2629999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1589999999999998,"5min":2.98,"15min":2.98},"last":1.099},{"interface":"front","average":{"1min":1.532,"5min":1.5529999999999999,"15min":1.629},"min":{"1min":0.97399999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0830000000000002,"5min":2.29,"15min":2.29},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.425,"5min":1.764,"15min":1.764},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0780000000000001,"15min":1.0209999999999999},"min":{"1min":0.71899999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.6890000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.133,"15min":1.1679999999999999},"min":{"1min":0.68799999999999994,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.542,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.1339999999999999,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6319999999999999,"5min":1.635,"15min":1.635},"last":1.5629999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.90500000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7050000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.258,"15min":1.226},"min":{"1min":0.86899999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.577,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.9630000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.3320000000000001,"15min":1.4099999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5649999999999999,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2709999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7589999999999999,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.861}]},{"osd":4,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.298,"15min":1.302},"min":{"1min":0.96699999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4279999999999999,"15min":1.4179999999999999},"min":{"1min":0.95399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9350000000000001,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.456,"15min":1.472},"min":{"1min":0.82499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8919999999999999,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4339999999999999,"15min":1.4159999999999999},"min":{"1min":0.98899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7070000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.3759999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5329999999999999,"15min":1.532},"min":{"1min":1.2330000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.849,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.786},{"interface":"front","average":{"1min":1.534,"5min":1.5649999999999999,"15min":1.5389999999999999},"min":{"1min":1.1779999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9790000000000001,"5min":2.859,"15min":2.859},"last":2.02}]}]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":483389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0289999999999999,"15min":1.038},"min":{"1min":0.66400000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.3979999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.135,"15min":1.129},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.651,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.80100000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.454,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.119},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.175,"15min":1.226},"min":{"1min":0.86199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5509999999999999,"5min":1.903,"15min":1.903},"last":1.0649999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1759999999999999,"15min":1.196},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.649,"5min":1.823,"15min":1.823},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.1970000000000001,"15min":1.2430000000000001},"min":{"1min":0.77100000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6379999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.262,"15min":1.29},"min":{"1min":0.98399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5940000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.3},{"interface":"front","average":{"1min":1.262,"5min":1.2749999999999999,"15min":1.3029999999999999},"min":{"1min":0.83599999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.645,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3420000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3080000000000001,"15min":1.3049999999999999},"min":{"1min":0.91800000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7529999999999999,"5min":1.944,"15min":1.944},"last":1.496},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3640000000000001,"15min":1.3660000000000001},"min":{"1min":0.80700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.792,"5min":2.012,"15min":2.012},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.96699999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.7709999999999999,"5min":2.105,"15min":2.105},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.4990000000000001,"15min":1.5109999999999999},"min":{"1min":0.94499999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8640000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.542}]},{"osd":7,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.4390000000000001},"min":{"1min":1.018,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.8919999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.663},{"interface":"front","average":{"1min":1.544,"5min":1.5369999999999999,"15min":1.5529999999999999},"min":{"1min":1.101,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8500000000000001,"5min":2.722,"15min":2.722},"last":1.5960000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":0.90600000000000003,"15min":0.84699999999999998},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.377},"max":{"1min":6.1849999999999996,"5min":6.1849999999999996,"15min":6.1849999999999996},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91300000000000003,"15min":0.94499999999999995},"min":{"1min":0.60699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0800000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":1.1120000000000001},"min":{"1min":0.57399999999999995,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3420000000000001,"5min":3.246,"15min":3.246},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.40400000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.651,"5min":1.718,"15min":1.718},"last":1.254}]},{"osd":2,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0920000000000001,"15min":1.1659999999999999},"min":{"1min":0.64000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6020000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.1839999999999999,"15min":1.1779999999999999},"min":{"1min":0.71099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.016,"5min":6.016,"15min":6.016},"last":1.1519999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2230000000000001,"15min":1.2569999999999999},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5469999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.496},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3220000000000001,"15min":1.379},"min":{"1min":1.0369999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8540000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.6419999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3700000000000001,"15min":1.4419999999999999},"min":{"1min":0.97299999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.772,"5min":2.395,"15min":2.395},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.722,"5min":1.405,"15min":1.333},"min":{"1min":1.093,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":5.9349999999999996,"5min":5.9349999999999996,"15min":5.9349999999999996},"last":1.3740000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.8149999999999999,"5min":1.498,"15min":1.458},"min":{"1min":1.208,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.3859999999999999,"15min":1.4139999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.837,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.7490000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4259999999999999,"15min":1.4450000000000001},"min":{"1min":0.871,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9419999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.887},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.4930000000000001,"15min":1.569},"min":{"1min":1.1950000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.712}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82099999999999995,"15min":0.83799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3560000000000001,"5min":1.524,"15min":1.524},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93700000000000006,"15min":1.0029999999999999},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3640000000000001,"5min":2.133,"15min":2.133},"last":0.628}]},{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.042,"15min":1.1919999999999999},"min":{"1min":0.71299999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.492,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.048},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.516,"5min":1.998,"15min":1.998},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.262,"15min":1.3180000000000001},"min":{"1min":0.64600000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9410000000000001,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.133,"15min":1.1899999999999999},"min":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7789999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.214,"15min":1.3100000000000001},"min":{"1min":0.66500000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.774,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.3799999999999999,"15min":1.4830000000000001},"min":{"1min":0.996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.869,"5min":3.161,"15min":3.161},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.288,"15min":1.3740000000000001},"min":{"1min":0.88500000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.518,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2909999999999999,"15min":1.3899999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.966,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.494}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4399999999999999,"15min":1.5680000000000001},"min":{"1min":0.95399999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.944,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.397,"5min":1.4059999999999999,"15min":1.538},"min":{"1min":0.98599999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.867,"5min":2.702,"15min":2.702},"last":2.8780000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3979999999999999,"15min":1.4970000000000001},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9970000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.427},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5049999999999999,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0019999999999998,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.875}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83699999999999997,"15min":0.878},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4179999999999999,"5min":1.732,"15min":1.732},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90500000000000003,"15min":1.0249999999999999},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2410000000000001,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":0.753}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91000000000000003,"15min":1.006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5920000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.097,"15min":1.202},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.629,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1699999999999999,"15min":1.248},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.738,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0700000000000001,"15min":1.196},"min":{"1min":0.46200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.5800000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.90000000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.165,"15min":1.2290000000000001},"min":{"1min":0.76200000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.427,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.107},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3380000000000001,"15min":1.427},"min":{"1min":0.78100000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9670000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.1759999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.345,"15min":1.4299999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.921,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.698},{"interface":"front","average":{"1min":1.286,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.8100000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.002}]},{"osd":6,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3799999999999999,"15min":1.4410000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.972,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.373},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3560000000000001,"15min":1.454},"min":{"1min":0.80000000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.2609999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4339999999999999,"15min":1.532},"min":{"1min":0.94999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8540000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.071},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4950000000000001,"15min":1.571},"min":{"1min":0.89600000000000002,"5min":0.872,"15min":0.872},"max":{"1min":2.0779999999999998,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.5309999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1072738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.86099999999999999,"15min":0.875},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.423,"5min":3.423,"15min":3.423},"last":1.179},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97699999999999998,"15min":0.94699999999999995},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2370000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.4510000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.55600000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.673,"5min":3.899,"15min":3.899},"last":1.768},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.97699999999999998,"15min":0.95299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5429999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.319}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0349999999999999,"15min":1.0289999999999999},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.464,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.2490000000000001,"15min":1.2190000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7430000000000001,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":2.0179999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2849999999999999,"15min":1.264},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.673,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.105},{"interface":"front","average":{"1min":1.341,"5min":1.4550000000000001,"15min":1.4670000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8149999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.881}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":0.64600000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9610000000000001,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.4350000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0430000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.6359999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.351,"15min":1.3999999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7310000000000001,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":0.73499999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1070000000000002,"5min":3.879,"15min":3.879},"last":1.522}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4930000000000001,"15min":1.524},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8859999999999999,"5min":3.823,"15min":3.823},"last":2.1259999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.464,"15min":1.478},"min":{"1min":0.77600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8240000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.732}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:09:00.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:09:01.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.077+0000 7fa16ed1d640 1 Processor -- start 2023-11-24T08:09:01.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.078+0000 7fa16ed1d640 1 -- start start 2023-11-24T08:09:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.079+0000 7fa16ed1d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 0x7fa16806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.079+0000 7fa16ed1d640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa1680ad720 0x7fa16815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.079+0000 7fa16ed1d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16815c360 con 0x7fa1680abff0 2023-11-24T08:09:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.079+0000 7fa16ed1d640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1680b41a0 con 0x7fa1680ad720 2023-11-24T08:09:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.079+0000 7fa16ed1d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16806de40 con 0x7fa16806d520 2023-11-24T08:09:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.080+0000 7fa16d2ba640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa1680abff0 0x7fa16815f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38232/0 (socket says 172.21.15.64:38232) 2023-11-24T08:09:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.080+0000 7fa16d2ba640 1 -- 172.21.15.64:0/3633322666 learned_addr learned my addr 172.21.15.64:0/3633322666 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.080+0000 7fa16cab9640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 0x7fa16806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.080+0000 7fa167fff640 1 --2- 172.21.15.64:0/3633322666 >> v2:172.21.15.91:3300/0 conn(0x7fa1680ad720 0x7fa16815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.080+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926381578 0 0) 0x7fa16815c360 con 0x7fa1680abff0 2023-11-24T08:09:01.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa158003520 con 0x7fa1680abff0 2023-11-24T08:09:01.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3537516156 0 0) 0x7fa158003520 con 0x7fa1680abff0 2023-11-24T08:09:01.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 msgr2=0x7fa16806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 0x7fa16806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 >> v2:172.21.15.91:3300/0 conn(0x7fa1680ad720 msgr2=0x7fa16815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 --2- 172.21.15.64:0/3633322666 >> v2:172.21.15.91:3300/0 conn(0x7fa1680ad720 0x7fa16815ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16815ba30 con 0x7fa1680abff0 2023-11-24T08:09:01.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fa1600019f0 con 0x7fa1680abff0 2023-11-24T08:09:01.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa16cab9640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 0x7fa16806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:09:01.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 >> v1:172.21.15.64:6789/0 conn(0x7fa1680abff0 legacy=0x7fa16815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa158003970 0x7fa158003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1580047c0 0x7fa158004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 0x7fa158005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16806de40 con 0x7fa1580047c0 2023-11-24T08:09:01.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158004600 con 0x7fa158003970 2023-11-24T08:09:01.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15800b9d0 con 0x7fa16806d520 2023-11-24T08:09:01.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.081+0000 7fa167fff640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1580047c0 0x7fa158004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.082+0000 7fa16cab9640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa158003970 0x7fa158003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.082+0000 7fa16d2ba640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 0x7fa158005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.082+0000 7fa167fff640 1 -- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 msgr2=0x7fa158005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.082+0000 7fa167fff640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa16806d520 0x7fa158005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.082+0000 7fa167fff640 1 -- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa158003970 msgr2=0x7fa158003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.082+0000 7fa167fff640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa158003970 0x7fa158003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.082+0000 7fa167fff640 1 -- 172.21.15.64:0/3633322666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c009400 con 0x7fa1580047c0 2023-11-24T08:09:01.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.083+0000 7fa167fff640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1580047c0 0x7fa158004c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa154009a90 tx=0x7fa1580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5aca1027a74632 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.084+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15402c6b0 con 0x7fa1580047c0 2023-11-24T08:09:01.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.084+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15402c810 con 0x7fa1580047c0 2023-11-24T08:09:01.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.084+0000 7fa1677fe640 1 -- 172.21.15.64:0/3633322666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15402cba0 con 0x7fa1580047c0 2023-11-24T08:09:01.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.084+0000 7fa16ed1d640 1 -- 172.21.15.64:0/3633322666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1580047c0 msgr2=0x7fa158004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.084+0000 7fa16ed1d640 1 --2- 172.21.15.64:0/3633322666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1580047c0 0x7fa158004c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa154009a90 tx=0x7fa1580057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.084+0000 7fa16d2ba640 1 -- 172.21.15.64:0/3633322666 reap_dead start 2023-11-24T08:09:01.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.084+0000 7fa16ed1d640 1 -- 172.21.15.64:0/3633322666 shutdown_connections 2023-11-24T08:09:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.084+0000 7fa16ed1d640 1 -- 172.21.15.64:0/3633322666 >> 172.21.15.64:0/3633322666 conn(0x7fa1680b0d80 msgr2=0x7fa1680b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.085+0000 7fa16ed1d640 1 -- 172.21.15.64:0/3633322666 shutdown_connections 2023-11-24T08:09:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.085+0000 7fa16ed1d640 1 -- 172.21.15.64:0/3633322666 wait complete. 2023-11-24T08:09:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.086+0000 7fa16ed1d640 1 Processor -- start 2023-11-24T08:09:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.087+0000 7fa16ed1d640 1 -- start start 2023-11-24T08:09:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16ed1d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1580047c0 0x7fa16814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16ed1d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1680711f0 0x7fa168071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16ed1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1680b21b0 0x7fa1680b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16ed1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168071bb0 con 0x7fa1680b21b0 2023-11-24T08:09:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16ed1d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680b2f20 con 0x7fa1680711f0 2023-11-24T08:09:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16ed1d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680b3200 con 0x7fa1580047c0 2023-11-24T08:09:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16d2ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1680b21b0 0x7fa1680b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16d2ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1680b21b0 0x7fa1680b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58362/0 (socket says 172.21.15.64:58362) 2023-11-24T08:09:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16d2ba640 1 -- 172.21.15.64:0/745965521 learned_addr learned my addr 172.21.15.64:0/745965521 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:01.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16cab9640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1580047c0 0x7fa16814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa167fff640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1680711f0 0x7fa168071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16d2ba640 1 -- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1580047c0 msgr2=0x7fa16814f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.088+0000 7fa16d2ba640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1580047c0 0x7fa16814f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa16d2ba640 1 -- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1680711f0 msgr2=0x7fa168071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa16d2ba640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1680711f0 0x7fa168071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa16d2ba640 1 -- 172.21.15.64:0/745965521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1680b34e0 con 0x7fa1680b21b0 2023-11-24T08:09:01.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa167fff640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1680711f0 0x7fa168071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:01.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa16d2ba640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1680b21b0 0x7fa1680b2660 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa16806fda0 tx=0x7fa16000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa1657fa640 1 -- 172.21.15.64:0/745965521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16000ec10 con 0x7fa1680b21b0 2023-11-24T08:09:01.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa1657fa640 1 -- 172.21.15.64:0/745965521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16000ed70 con 0x7fa1680b21b0 2023-11-24T08:09:01.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1680b37c0 con 0x7fa1680b21b0 2023-11-24T08:09:01.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa1657fa640 1 -- 172.21.15.64:0/745965521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16000eee0 con 0x7fa1680b21b0 2023-11-24T08:09:01.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.089+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16814f970 con 0x7fa1680b21b0 2023-11-24T08:09:01.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.090+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168074770 con 0x7fa1680b21b0 2023-11-24T08:09:01.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.091+0000 7fa1657fa640 1 -- 172.21.15.64:0/745965521 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fa160013660 con 0x7fa1680b21b0 2023-11-24T08:09:01.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.092+0000 7fa1657fa640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa14c06b4a0 0x7fa14c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.092+0000 7fa1657fa640 1 -- 172.21.15.64:0/745965521 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa16008a2d0 con 0x7fa1680b21b0 2023-11-24T08:09:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.093+0000 7fa16cab9640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa14c06b4a0 0x7fa14c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.094+0000 7fa16cab9640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa14c06b4a0 0x7fa14c06d960 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa1680cb500 tx=0x7fa15c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.097+0000 7fa1657fa640 1 -- 172.21.15.64:0/745965521 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa160054d40 con 0x7fa1680b21b0 2023-11-24T08:09:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.287+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa16815b6e0 con 0x7fa14c06b4a0 2023-11-24T08:09:01.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.294+0000 7fa1657fa640 1 -- 172.21.15.64:0/745965521 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48773 (secure 0 0 0) 0x7fa15c013a20 con 0x7fa14c06b4a0 2023-11-24T08:09:01.295 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:09:01.296 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:09:01.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa14c06b4a0 msgr2=0x7fa14c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa14c06b4a0 0x7fa14c06d960 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa1680cb500 tx=0x7fa15c008040 comp rx=0 tx=0).stop 2023-11-24T08:09:01.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1680b21b0 msgr2=0x7fa1680b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1680b21b0 0x7fa1680b2660 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa16806fda0 tx=0x7fa16000e530 comp rx=0 tx=0).stop 2023-11-24T08:09:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 shutdown_connections 2023-11-24T08:09:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa14c06b4a0 0x7fa14c06d960 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1680b21b0 0x7fa1680b2660 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa1680711f0 0x7fa168071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 --2- 172.21.15.64:0/745965521 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa1580047c0 0x7fa16814f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 >> 172.21.15.64:0/745965521 conn(0x7fa1680b0d80 msgr2=0x7fa16815ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:01.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.296+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 shutdown_connections 2023-11-24T08:09:01.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.297+0000 7fa16ed1d640 1 -- 172.21.15.64:0/745965521 wait complete. 2023-11-24T08:09:01.316 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-24T08:09:00.125355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259428,"kb_used_data":9980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715292,"statfs":{"total":773094113280,"available":772828459008,"internally_reserved":0,"allocated":10219520,"data_stored":5062765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-35583,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002114"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":227,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-24T08:08:57.223161+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:57.223161+0000","last_peered":"2023-11-24T08:08:57.223161+0000","last_clean":"2023-11-24T08:08:57.223161+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:57.223161+0000","last_undegraded":"2023-11-24T08:08:57.223161+0000","last_fullsized":"2023-11-24T08:08:57.223161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1073622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.022,"15min":1.01},"min":{"1min":0.63700000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.05,"15min":1.0800000000000001},"min":{"1min":0.72099999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.72}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.27,"15min":1.298},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2909999999999999,"15min":1.367},"min":{"1min":0.89100000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0939999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.865}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1699999999999999,"15min":1.1539999999999999},"min":{"1min":0.77300000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.6840000000000002,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.266,"15min":1.302},"min":{"1min":0.85299999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.25,"15min":1.3100000000000001},"min":{"1min":0.78200000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":4.3570000000000002,"5min":4.3570000000000002,"15min":4.3570000000000002},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.365,"15min":1.4870000000000001},"min":{"1min":0.75900000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.2480000000000002,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.405}]},{"osd":4,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.4910000000000001},"min":{"1min":0.755,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.327,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":2.012},{"interface":"front","average":{"1min":1.45,"5min":1.3360000000000001,"15min":1.369},"min":{"1min":0.95499999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.3049999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.587},"min":{"1min":0.91100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.589},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.5289999999999999,"15min":1.637},"min":{"1min":0.96199999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":2.1160000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.583,"15min":1.7190000000000001},"min":{"1min":1.016,"5min":0.755,"15min":0.755},"max":{"1min":2.5329999999999999,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":2.2040000000000002},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.486,"15min":1.625},"min":{"1min":1.1950000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9410000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.6799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2090000000000001,"15min":1.3069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.8460000000000001,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.246,"5min":1.236,"15min":1.3260000000000001},"min":{"1min":1.0369999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.4530000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.038}]},{"osd":1,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1339999999999999,"15min":1.1519999999999999},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.4710000000000001,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1699999999999999,"15min":1.284},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.532,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1579999999999999,"15min":1.272},"min":{"1min":0.83799999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5329999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.256,"15min":1.4530000000000001},"min":{"1min":0.755,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.788,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.622}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3420000000000001,"15min":1.4279999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.8839999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.472,"5min":1.4419999999999999,"15min":1.4890000000000001},"min":{"1min":1.153,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.754,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.98599999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4399999999999999,"15min":1.6950000000000001},"min":{"1min":0.88400000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.925,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":0.752},{"interface":"front","average":{"1min":1.488,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.117,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9830000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4750000000000001,"15min":1.744},"min":{"1min":0.90400000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.98,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.613,"15min":1.885},"min":{"1min":0.96999999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.036,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.1299999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5660000000000001,"15min":1.665},"min":{"1min":1.2629999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1589999999999998,"5min":2.98,"15min":2.98},"last":1.099},{"interface":"front","average":{"1min":1.532,"5min":1.5529999999999999,"15min":1.629},"min":{"1min":0.97399999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0830000000000002,"5min":2.29,"15min":2.29},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.425,"5min":1.764,"15min":1.764},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0780000000000001,"15min":1.0209999999999999},"min":{"1min":0.71899999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.6890000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.133,"15min":1.1679999999999999},"min":{"1min":0.68799999999999994,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.542,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.1339999999999999,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6319999999999999,"5min":1.635,"15min":1.635},"last":1.5629999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.90500000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7050000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.258,"15min":1.226},"min":{"1min":0.86899999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.577,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.9630000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.3320000000000001,"15min":1.4099999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5649999999999999,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2709999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7589999999999999,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.861}]},{"osd":4,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.298,"15min":1.302},"min":{"1min":0.96699999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4279999999999999,"15min":1.4179999999999999},"min":{"1min":0.95399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9350000000000001,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.456,"15min":1.472},"min":{"1min":0.82499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8919999999999999,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4339999999999999,"15min":1.4159999999999999},"min":{"1min":0.98899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7070000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.3759999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5329999999999999,"15min":1.532},"min":{"1min":1.2330000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.849,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.786},{"interface":"front","average":{"1min":1.534,"5min":1.5649999999999999,"15min":1.5389999999999999},"min":{"1min":1.1779999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9790000000000001,"5min":2.859,"15min":2.859},"last":2.02}]}]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":483389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0289999999999999,"15min":1.038},"min":{"1min":0.66400000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.3979999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.135,"15min":1.129},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.651,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.80100000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.454,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.119},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.175,"15min":1.226},"min":{"1min":0.86199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5509999999999999,"5min":1.903,"15min":1.903},"last":1.0649999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1759999999999999,"15min":1.196},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.649,"5min":1.823,"15min":1.823},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.1970000000000001,"15min":1.2430000000000001},"min":{"1min":0.77100000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6379999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.262,"15min":1.29},"min":{"1min":0.98399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5940000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.3},{"interface":"front","average":{"1min":1.262,"5min":1.2749999999999999,"15min":1.3029999999999999},"min":{"1min":0.83599999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.645,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3420000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3080000000000001,"15min":1.3049999999999999},"min":{"1min":0.91800000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7529999999999999,"5min":1.944,"15min":1.944},"last":1.496},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3640000000000001,"15min":1.3660000000000001},"min":{"1min":0.80700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.792,"5min":2.012,"15min":2.012},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.96699999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.7709999999999999,"5min":2.105,"15min":2.105},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.4990000000000001,"15min":1.5109999999999999},"min":{"1min":0.94499999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8640000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.542}]},{"osd":7,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.4390000000000001},"min":{"1min":1.018,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.8919999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.663},{"interface":"front","average":{"1min":1.544,"5min":1.5369999999999999,"15min":1.5529999999999999},"min":{"1min":1.101,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8500000000000001,"5min":2.722,"15min":2.722},"last":1.5960000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":0.90600000000000003,"15min":0.84699999999999998},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.377},"max":{"1min":6.1849999999999996,"5min":6.1849999999999996,"15min":6.1849999999999996},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91300000000000003,"15min":0.94499999999999995},"min":{"1min":0.60699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0800000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":1.1120000000000001},"min":{"1min":0.57399999999999995,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3420000000000001,"5min":3.246,"15min":3.246},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.40400000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.651,"5min":1.718,"15min":1.718},"last":1.254}]},{"osd":2,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0920000000000001,"15min":1.1659999999999999},"min":{"1min":0.64000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6020000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.1839999999999999,"15min":1.1779999999999999},"min":{"1min":0.71099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.016,"5min":6.016,"15min":6.016},"last":1.1519999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2230000000000001,"15min":1.2569999999999999},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5469999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.496},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3220000000000001,"15min":1.379},"min":{"1min":1.0369999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8540000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.6419999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3700000000000001,"15min":1.4419999999999999},"min":{"1min":0.97299999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.772,"5min":2.395,"15min":2.395},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.722,"5min":1.405,"15min":1.333},"min":{"1min":1.093,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":5.9349999999999996,"5min":5.9349999999999996,"15min":5.9349999999999996},"last":1.3740000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.8149999999999999,"5min":1.498,"15min":1.458},"min":{"1min":1.208,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.3859999999999999,"15min":1.4139999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.837,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.7490000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4259999999999999,"15min":1.4450000000000001},"min":{"1min":0.871,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9419999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.887},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.4930000000000001,"15min":1.569},"min":{"1min":1.1950000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.712}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82099999999999995,"15min":0.83799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3560000000000001,"5min":1.524,"15min":1.524},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93700000000000006,"15min":1.0029999999999999},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3640000000000001,"5min":2.133,"15min":2.133},"last":0.628}]},{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.042,"15min":1.1919999999999999},"min":{"1min":0.71299999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.492,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.048},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.516,"5min":1.998,"15min":1.998},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.262,"15min":1.3180000000000001},"min":{"1min":0.64600000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9410000000000001,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.133,"15min":1.1899999999999999},"min":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7789999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.214,"15min":1.3100000000000001},"min":{"1min":0.66500000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.774,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.3799999999999999,"15min":1.4830000000000001},"min":{"1min":0.996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.869,"5min":3.161,"15min":3.161},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.288,"15min":1.3740000000000001},"min":{"1min":0.88500000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.518,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2909999999999999,"15min":1.3899999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.966,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.494}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4399999999999999,"15min":1.5680000000000001},"min":{"1min":0.95399999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.944,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.397,"5min":1.4059999999999999,"15min":1.538},"min":{"1min":0.98599999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.867,"5min":2.702,"15min":2.702},"last":2.8780000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3979999999999999,"15min":1.4970000000000001},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9970000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.427},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5049999999999999,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0019999999999998,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.875}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83699999999999997,"15min":0.878},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4179999999999999,"5min":1.732,"15min":1.732},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90500000000000003,"15min":1.0249999999999999},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2410000000000001,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":0.753}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91000000000000003,"15min":1.006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5920000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.097,"15min":1.202},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.629,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1699999999999999,"15min":1.248},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.738,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0700000000000001,"15min":1.196},"min":{"1min":0.46200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.5800000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.90000000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.165,"15min":1.2290000000000001},"min":{"1min":0.76200000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.427,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.107},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3380000000000001,"15min":1.427},"min":{"1min":0.78100000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9670000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.1759999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.345,"15min":1.4299999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.921,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.698},{"interface":"front","average":{"1min":1.286,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.8100000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.002}]},{"osd":6,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3799999999999999,"15min":1.4410000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.972,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.373},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3560000000000001,"15min":1.454},"min":{"1min":0.80000000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.2609999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4339999999999999,"15min":1.532},"min":{"1min":0.94999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8540000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.071},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4950000000000001,"15min":1.571},"min":{"1min":0.89600000000000002,"5min":0.872,"15min":0.872},"max":{"1min":2.0779999999999998,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.5309999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1072738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.86099999999999999,"15min":0.875},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.423,"5min":3.423,"15min":3.423},"last":1.179},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97699999999999998,"15min":0.94699999999999995},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2370000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.4510000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.55600000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.673,"5min":3.899,"15min":3.899},"last":1.768},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.97699999999999998,"15min":0.95299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5429999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.319}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0349999999999999,"15min":1.0289999999999999},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.464,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.2490000000000001,"15min":1.2190000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7430000000000001,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":2.0179999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2849999999999999,"15min":1.264},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.673,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.105},{"interface":"front","average":{"1min":1.341,"5min":1.4550000000000001,"15min":1.4670000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8149999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.881}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":0.64600000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9610000000000001,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.4350000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0430000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.6359999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.351,"15min":1.3999999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7310000000000001,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":0.73499999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1070000000000002,"5min":3.879,"15min":3.879},"last":1.522}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4930000000000001,"15min":1.524},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8859999999999999,"5min":3.823,"15min":3.823},"last":2.1259999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.464,"15min":1.478},"min":{"1min":0.77600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8240000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.732}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:09:01.318 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T08:09:01.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:09:01.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.541+0000 7f3e04f3c640 1 Processor -- start 2023-11-24T08:09:01.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.541+0000 7f3e04f3c640 1 -- start start 2023-11-24T08:09:01.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3e04f3c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 0x7f3e000cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3e04f3c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3e000cd7b0 0x7f3e000cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3e04f3c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0015fa70 con 0x7f3e0013dfd0 2023-11-24T08:09:01.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3e04f3c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0015c580 con 0x7f3e000cd7b0 2023-11-24T08:09:01.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3e04f3c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e000cfa10 con 0x7f3e000cf050 2023-11-24T08:09:01.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3dfffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3e000cd7b0 0x7f3e000cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3dfffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3e000cd7b0 0x7f3e000cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58376/0 (socket says 172.21.15.64:58376) 2023-11-24T08:09:01.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3dfffff640 1 -- 172.21.15.64:0/2018237081 learned_addr learned my addr 172.21.15.64:0/2018237081 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:01.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3dfeffd640 1 --2- 172.21.15.64:0/2018237081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 0x7f3e000cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3dfffff640 1 -- 172.21.15.64:0/2018237081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 msgr2=0x7f3e000cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3dfffff640 1 --2- 172.21.15.64:0/2018237081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 0x7f3e000cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3dfffff640 1 -- 172.21.15.64:0/2018237081 >> v1:172.21.15.91:6789/0 conn(0x7f3e0013dfd0 legacy=0x7f3e0015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.543+0000 7f3dfffff640 1 -- 172.21.15.64:0/2018237081 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e00140900 con 0x7f3e000cd7b0 2023-11-24T08:09:01.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.545+0000 7f3dfffff640 1 --2- 172.21.15.64:0/2018237081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e000cd7b0 0x7f3e000cdc00 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009870 tx=0x7f3df802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319e468b55da7d05 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.545+0000 7f3dfe7fc640 1 -- 172.21.15.64:0/2018237081 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8031640 con 0x7f3e000cd7b0 2023-11-24T08:09:01.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.545+0000 7f3dfe7fc640 1 -- 172.21.15.64:0/2018237081 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df80317a0 con 0x7f3e000cd7b0 2023-11-24T08:09:01.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.545+0000 7f3dfe7fc640 1 -- 172.21.15.64:0/2018237081 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8031b30 con 0x7f3e000cd7b0 2023-11-24T08:09:01.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.546+0000 7f3e04f3c640 1 -- 172.21.15.64:0/2018237081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e000cd7b0 msgr2=0x7f3e000cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.546+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/2018237081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e000cd7b0 0x7f3e000cdc00 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009870 tx=0x7f3df802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.546+0000 7f3e04f3c640 1 -- 172.21.15.64:0/2018237081 shutdown_connections 2023-11-24T08:09:01.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.546+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/2018237081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e000cd7b0 0x7f3e000cdc00 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.546+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/2018237081 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 0x7f3e000cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.546+0000 7f3e04f3c640 1 -- 172.21.15.64:0/2018237081 >> 172.21.15.64:0/2018237081 conn(0x7f3e0006a860 msgr2=0x7f3e0006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:01.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.546+0000 7f3e04f3c640 1 -- 172.21.15.64:0/2018237081 shutdown_connections 2023-11-24T08:09:01.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.546+0000 7f3e04f3c640 1 -- 172.21.15.64:0/2018237081 wait complete. 2023-11-24T08:09:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.548+0000 7f3e04f3c640 1 Processor -- start 2023-11-24T08:09:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.548+0000 7f3e04f3c640 1 -- start start 2023-11-24T08:09:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3e04f3c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e000cd7b0 0x7f3e000b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3e04f3c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 0x7f3e00155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3e04f3c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e0013dfd0 0x7f3e000c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3e04f3c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e000c8e80 con 0x7f3e0013dfd0 2023-11-24T08:09:01.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3e04f3c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e000b2300 con 0x7f3e000cd7b0 2023-11-24T08:09:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3e04f3c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e000b25e0 con 0x7f3e000cf050 2023-11-24T08:09:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e0013dfd0 0x7f3e000c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e0013dfd0 0x7f3e000c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58392/0 (socket says 172.21.15.64:58392) 2023-11-24T08:09:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.549+0000 7f3dfffff640 1 -- 172.21.15.64:0/1912478031 learned_addr learned my addr 172.21.15.64:0/1912478031 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dfeffd640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 0x7f3e00155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dff7fe640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e000cd7b0 0x7f3e000b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dfffff640 1 -- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 msgr2=0x7f3e00155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dfffff640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 0x7f3e00155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dfffff640 1 -- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e000cd7b0 msgr2=0x7f3e000b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dfffff640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e000cd7b0 0x7f3e000b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dfffff640 1 -- 172.21.15.64:0/1912478031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df80094d0 con 0x7f3e0013dfd0 2023-11-24T08:09:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dff7fe640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e000cd7b0 0x7f3e000b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3dfffff640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e0013dfd0 0x7f3e000c8940 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3df802b930 tx=0x7f3df80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3df7fff640 1 -- 172.21.15.64:0/1912478031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df803ac60 con 0x7f3e0013dfd0 2023-11-24T08:09:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3df7fff640 1 -- 172.21.15.64:0/1912478031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df803adc0 con 0x7f3e0013dfd0 2023-11-24T08:09:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3df7fff640 1 -- 172.21.15.64:0/1912478031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8044630 con 0x7f3e0013dfd0 2023-11-24T08:09:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.550+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e000b28c0 con 0x7f3e0013dfd0 2023-11-24T08:09:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.551+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e000b2e80 con 0x7f3e0013dfd0 2023-11-24T08:09:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.552+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e000b3070 con 0x7f3e0013dfd0 2023-11-24T08:09:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.553+0000 7f3df7fff640 1 -- 172.21.15.64:0/1912478031 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f3df802f050 con 0x7f3e0013dfd0 2023-11-24T08:09:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.554+0000 7f3df7fff640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3de006b470 0x7f3de006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.554+0000 7f3df7fff640 1 -- 172.21.15.64:0/1912478031 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3df80abe90 con 0x7f3e0013dfd0 2023-11-24T08:09:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.554+0000 7f3dff7fe640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3de006b470 0x7f3de006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.555+0000 7f3dff7fe640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3de006b470 0x7f3de006d930 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3df000a830 tx=0x7f3df0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.558+0000 7f3df7fff640 1 -- 172.21.15.64:0/1912478031 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3df80767b0 con 0x7f3e0013dfd0 2023-11-24T08:09:01.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.750+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e0015f420 con 0x7f3de006b470 2023-11-24T08:09:01.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.756+0000 7f3df7fff640 1 -- 172.21.15.64:0/1912478031 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48773 (secure 0 0 0) 0x7f3df000adb0 con 0x7f3de006b470 2023-11-24T08:09:01.758 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:09:01.758 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:09:01.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3de006b470 msgr2=0x7f3de006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3de006b470 0x7f3de006d930 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3df000a830 tx=0x7f3df0008040 comp rx=0 tx=0).stop 2023-11-24T08:09:01.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e0013dfd0 msgr2=0x7f3e000c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e0013dfd0 0x7f3e000c8940 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3df802b930 tx=0x7f3df80046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 shutdown_connections 2023-11-24T08:09:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f3de006b470 0x7f3de006d930 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e0013dfd0 0x7f3e000c8940 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f3e000cf050 0x7f3e00155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 --2- 172.21.15.64:0/1912478031 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f3e000cd7b0 0x7f3e000b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 >> 172.21.15.64:0/1912478031 conn(0x7f3e0006a860 msgr2=0x7f3e000719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:01.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 shutdown_connections 2023-11-24T08:09:01.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:01.759+0000 7f3e04f3c640 1 -- 172.21.15.64:0/1912478031 wait complete. 2023-11-24T08:09:01.780 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-24T08:09:00.125355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259428,"kb_used_data":9980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715292,"statfs":{"total":773094113280,"available":772828459008,"internally_reserved":0,"allocated":10219520,"data_stored":5062765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-35583,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002114"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":227,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-24T08:08:57.223161+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:57.223161+0000","last_peered":"2023-11-24T08:08:57.223161+0000","last_clean":"2023-11-24T08:08:57.223161+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:57.223161+0000","last_undegraded":"2023-11-24T08:08:57.223161+0000","last_fullsized":"2023-11-24T08:08:57.223161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1073622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.022,"15min":1.01},"min":{"1min":0.63700000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.05,"15min":1.0800000000000001},"min":{"1min":0.72099999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.72}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.27,"15min":1.298},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2909999999999999,"15min":1.367},"min":{"1min":0.89100000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0939999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.865}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1699999999999999,"15min":1.1539999999999999},"min":{"1min":0.77300000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.6840000000000002,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.266,"15min":1.302},"min":{"1min":0.85299999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.25,"15min":1.3100000000000001},"min":{"1min":0.78200000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":4.3570000000000002,"5min":4.3570000000000002,"15min":4.3570000000000002},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.365,"15min":1.4870000000000001},"min":{"1min":0.75900000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.2480000000000002,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.405}]},{"osd":4,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.4910000000000001},"min":{"1min":0.755,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.327,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":2.012},{"interface":"front","average":{"1min":1.45,"5min":1.3360000000000001,"15min":1.369},"min":{"1min":0.95499999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.3049999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.587},"min":{"1min":0.91100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.589},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.5289999999999999,"15min":1.637},"min":{"1min":0.96199999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":2.1160000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.583,"15min":1.7190000000000001},"min":{"1min":1.016,"5min":0.755,"15min":0.755},"max":{"1min":2.5329999999999999,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":2.2040000000000002},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.486,"15min":1.625},"min":{"1min":1.1950000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9410000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.6799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2090000000000001,"15min":1.3069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.8460000000000001,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.246,"5min":1.236,"15min":1.3260000000000001},"min":{"1min":1.0369999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.4530000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.038}]},{"osd":1,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1339999999999999,"15min":1.1519999999999999},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.4710000000000001,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1699999999999999,"15min":1.284},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.532,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1579999999999999,"15min":1.272},"min":{"1min":0.83799999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5329999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.256,"15min":1.4530000000000001},"min":{"1min":0.755,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.788,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.622}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3420000000000001,"15min":1.4279999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.8839999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.472,"5min":1.4419999999999999,"15min":1.4890000000000001},"min":{"1min":1.153,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.754,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.98599999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4399999999999999,"15min":1.6950000000000001},"min":{"1min":0.88400000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.925,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":0.752},{"interface":"front","average":{"1min":1.488,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.117,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9830000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4750000000000001,"15min":1.744},"min":{"1min":0.90400000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.98,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.613,"15min":1.885},"min":{"1min":0.96999999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.036,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.1299999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5660000000000001,"15min":1.665},"min":{"1min":1.2629999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1589999999999998,"5min":2.98,"15min":2.98},"last":1.099},{"interface":"front","average":{"1min":1.532,"5min":1.5529999999999999,"15min":1.629},"min":{"1min":0.97399999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0830000000000002,"5min":2.29,"15min":2.29},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.425,"5min":1.764,"15min":1.764},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0780000000000001,"15min":1.0209999999999999},"min":{"1min":0.71899999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.6890000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.133,"15min":1.1679999999999999},"min":{"1min":0.68799999999999994,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.542,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.1339999999999999,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6319999999999999,"5min":1.635,"15min":1.635},"last":1.5629999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.90500000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7050000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.258,"15min":1.226},"min":{"1min":0.86899999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.577,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.9630000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.3320000000000001,"15min":1.4099999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5649999999999999,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2709999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7589999999999999,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.861}]},{"osd":4,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.298,"15min":1.302},"min":{"1min":0.96699999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4279999999999999,"15min":1.4179999999999999},"min":{"1min":0.95399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9350000000000001,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.456,"15min":1.472},"min":{"1min":0.82499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8919999999999999,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4339999999999999,"15min":1.4159999999999999},"min":{"1min":0.98899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7070000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.3759999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5329999999999999,"15min":1.532},"min":{"1min":1.2330000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.849,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.786},{"interface":"front","average":{"1min":1.534,"5min":1.5649999999999999,"15min":1.5389999999999999},"min":{"1min":1.1779999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9790000000000001,"5min":2.859,"15min":2.859},"last":2.02}]}]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":483389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0289999999999999,"15min":1.038},"min":{"1min":0.66400000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.3979999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.135,"15min":1.129},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.651,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.80100000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.454,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.119},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.175,"15min":1.226},"min":{"1min":0.86199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5509999999999999,"5min":1.903,"15min":1.903},"last":1.0649999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1759999999999999,"15min":1.196},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.649,"5min":1.823,"15min":1.823},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.1970000000000001,"15min":1.2430000000000001},"min":{"1min":0.77100000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6379999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.262,"15min":1.29},"min":{"1min":0.98399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5940000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.3},{"interface":"front","average":{"1min":1.262,"5min":1.2749999999999999,"15min":1.3029999999999999},"min":{"1min":0.83599999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.645,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3420000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3080000000000001,"15min":1.3049999999999999},"min":{"1min":0.91800000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7529999999999999,"5min":1.944,"15min":1.944},"last":1.496},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3640000000000001,"15min":1.3660000000000001},"min":{"1min":0.80700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.792,"5min":2.012,"15min":2.012},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.96699999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.7709999999999999,"5min":2.105,"15min":2.105},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.4990000000000001,"15min":1.5109999999999999},"min":{"1min":0.94499999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8640000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.542}]},{"osd":7,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.4390000000000001},"min":{"1min":1.018,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.8919999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.663},{"interface":"front","average":{"1min":1.544,"5min":1.5369999999999999,"15min":1.5529999999999999},"min":{"1min":1.101,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8500000000000001,"5min":2.722,"15min":2.722},"last":1.5960000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":0.90600000000000003,"15min":0.84699999999999998},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.377},"max":{"1min":6.1849999999999996,"5min":6.1849999999999996,"15min":6.1849999999999996},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91300000000000003,"15min":0.94499999999999995},"min":{"1min":0.60699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0800000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":1.1120000000000001},"min":{"1min":0.57399999999999995,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3420000000000001,"5min":3.246,"15min":3.246},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.40400000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.651,"5min":1.718,"15min":1.718},"last":1.254}]},{"osd":2,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0920000000000001,"15min":1.1659999999999999},"min":{"1min":0.64000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6020000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.1839999999999999,"15min":1.1779999999999999},"min":{"1min":0.71099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.016,"5min":6.016,"15min":6.016},"last":1.1519999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2230000000000001,"15min":1.2569999999999999},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5469999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.496},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3220000000000001,"15min":1.379},"min":{"1min":1.0369999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8540000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.6419999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3700000000000001,"15min":1.4419999999999999},"min":{"1min":0.97299999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.772,"5min":2.395,"15min":2.395},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.722,"5min":1.405,"15min":1.333},"min":{"1min":1.093,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":5.9349999999999996,"5min":5.9349999999999996,"15min":5.9349999999999996},"last":1.3740000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.8149999999999999,"5min":1.498,"15min":1.458},"min":{"1min":1.208,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.3859999999999999,"15min":1.4139999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.837,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.7490000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4259999999999999,"15min":1.4450000000000001},"min":{"1min":0.871,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9419999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.887},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.4930000000000001,"15min":1.569},"min":{"1min":1.1950000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.712}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82099999999999995,"15min":0.83799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3560000000000001,"5min":1.524,"15min":1.524},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93700000000000006,"15min":1.0029999999999999},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3640000000000001,"5min":2.133,"15min":2.133},"last":0.628}]},{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.042,"15min":1.1919999999999999},"min":{"1min":0.71299999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.492,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.048},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.516,"5min":1.998,"15min":1.998},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.262,"15min":1.3180000000000001},"min":{"1min":0.64600000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9410000000000001,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.133,"15min":1.1899999999999999},"min":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7789999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.214,"15min":1.3100000000000001},"min":{"1min":0.66500000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.774,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.3799999999999999,"15min":1.4830000000000001},"min":{"1min":0.996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.869,"5min":3.161,"15min":3.161},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.288,"15min":1.3740000000000001},"min":{"1min":0.88500000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.518,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2909999999999999,"15min":1.3899999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.966,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.494}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4399999999999999,"15min":1.5680000000000001},"min":{"1min":0.95399999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.944,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.397,"5min":1.4059999999999999,"15min":1.538},"min":{"1min":0.98599999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.867,"5min":2.702,"15min":2.702},"last":2.8780000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3979999999999999,"15min":1.4970000000000001},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9970000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.427},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5049999999999999,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0019999999999998,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.875}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83699999999999997,"15min":0.878},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4179999999999999,"5min":1.732,"15min":1.732},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90500000000000003,"15min":1.0249999999999999},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2410000000000001,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":0.753}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91000000000000003,"15min":1.006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5920000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.097,"15min":1.202},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.629,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1699999999999999,"15min":1.248},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.738,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0700000000000001,"15min":1.196},"min":{"1min":0.46200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.5800000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.90000000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.165,"15min":1.2290000000000001},"min":{"1min":0.76200000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.427,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.107},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3380000000000001,"15min":1.427},"min":{"1min":0.78100000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9670000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.1759999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.345,"15min":1.4299999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.921,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.698},{"interface":"front","average":{"1min":1.286,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.8100000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.002}]},{"osd":6,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3799999999999999,"15min":1.4410000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.972,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.373},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3560000000000001,"15min":1.454},"min":{"1min":0.80000000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.2609999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4339999999999999,"15min":1.532},"min":{"1min":0.94999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8540000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.071},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4950000000000001,"15min":1.571},"min":{"1min":0.89600000000000002,"5min":0.872,"15min":0.872},"max":{"1min":2.0779999999999998,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.5309999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1072738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.86099999999999999,"15min":0.875},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.423,"5min":3.423,"15min":3.423},"last":1.179},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97699999999999998,"15min":0.94699999999999995},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2370000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.4510000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.55600000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.673,"5min":3.899,"15min":3.899},"last":1.768},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.97699999999999998,"15min":0.95299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5429999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.319}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0349999999999999,"15min":1.0289999999999999},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.464,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.2490000000000001,"15min":1.2190000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7430000000000001,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":2.0179999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2849999999999999,"15min":1.264},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.673,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.105},{"interface":"front","average":{"1min":1.341,"5min":1.4550000000000001,"15min":1.4670000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8149999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.881}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":0.64600000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9610000000000001,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.4350000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0430000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.6359999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.351,"15min":1.3999999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7310000000000001,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":0.73499999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1070000000000002,"5min":3.879,"15min":3.879},"last":1.522}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4930000000000001,"15min":1.524},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8859999999999999,"5min":3.823,"15min":3.823},"last":2.1259999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.464,"15min":1.478},"min":{"1min":0.77600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8240000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.732}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:09:01.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.006+0000 7f1ce64c9640 1 Processor -- start 2023-11-24T08:09:02.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.007+0000 7f1ce64c9640 1 -- start start 2023-11-24T08:09:02.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.008+0000 7f1ce64c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ce006d520 0x7f1ce006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.008+0000 7f1ce64c9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce00ad720 0x7f1ce015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.008+0000 7f1ce64c9640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1ce00abff0 0x7f1ce015f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.008+0000 7f1ce64c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce00b41a0 con 0x7f1ce006d520 2023-11-24T08:09:02.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.008+0000 7f1ce64c9640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce015c360 con 0x7f1ce00abff0 2023-11-24T08:09:02.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.008+0000 7f1ce64c9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce006de40 con 0x7f1ce00ad720 2023-11-24T08:09:02.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ce006d520 0x7f1ce006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ce006d520 0x7f1ce006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58396/0 (socket says 172.21.15.64:58396) 2023-11-24T08:09:02.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdffff640 1 -- 172.21.15.64:0/1225174010 learned_addr learned my addr 172.21.15.64:0/1225174010 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:02.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdf7fe640 1 --2- 172.21.15.64:0/1225174010 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce00ad720 0x7f1ce015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1ce4a66640 1 --2- 172.21.15.64:0/1225174010 >> v2:172.21.15.91:3300/0 conn(0x7f1ce00abff0 0x7f1ce015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdffff640 1 -- 172.21.15.64:0/1225174010 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce00ad720 msgr2=0x7f1ce015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:02.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdffff640 1 --2- 172.21.15.64:0/1225174010 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce00ad720 0x7f1ce015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdffff640 1 -- 172.21.15.64:0/1225174010 >> v2:172.21.15.91:3300/0 conn(0x7f1ce00abff0 msgr2=0x7f1ce015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:02.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdffff640 1 --2- 172.21.15.64:0/1225174010 >> v2:172.21.15.91:3300/0 conn(0x7f1ce00abff0 0x7f1ce015f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.009+0000 7f1cdffff640 1 -- 172.21.15.64:0/1225174010 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce015ba30 con 0x7f1ce006d520 2023-11-24T08:09:02.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.011+0000 7f1cdffff640 1 --2- 172.21.15.64:0/1225174010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce006d520 0x7f1ce006d900 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00aca20 tx=0x7f1cd002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34737a09c0d51dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.011+0000 7f1cdeffd640 1 -- 172.21.15.64:0/1225174010 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0031870 con 0x7f1ce006d520 2023-11-24T08:09:02.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.011+0000 7f1cdeffd640 1 -- 172.21.15.64:0/1225174010 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd00319d0 con 0x7f1ce006d520 2023-11-24T08:09:02.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.011+0000 7f1cdeffd640 1 -- 172.21.15.64:0/1225174010 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0031d60 con 0x7f1ce006d520 2023-11-24T08:09:02.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1225174010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce006d520 msgr2=0x7f1ce006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1225174010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce006d520 0x7f1ce006d900 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00aca20 tx=0x7f1cd002b910 comp rx=0 tx=0).stop 2023-11-24T08:09:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1225174010 shutdown_connections 2023-11-24T08:09:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1225174010 >> v2:172.21.15.91:3300/0 conn(0x7f1ce00abff0 0x7f1ce015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1225174010 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce00ad720 0x7f1ce015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1225174010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce006d520 0x7f1ce006d900 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1225174010 >> 172.21.15.64:0/1225174010 conn(0x7f1ce00b0d80 msgr2=0x7f1ce00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:02.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1225174010 shutdown_connections 2023-11-24T08:09:02.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.012+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1225174010 wait complete. 2023-11-24T08:09:02.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.015+0000 7f1ce64c9640 1 Processor -- start 2023-11-24T08:09:02.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.015+0000 7f1ce64c9640 1 -- start start 2023-11-24T08:09:02.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.016+0000 7f1ce64c9640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce006d520 0x7f1ce0157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.016+0000 7f1ce64c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00abff0 0x7f1ce009a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.016+0000 7f1ce64c9640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ce00ad720 0x7f1ce00c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.016+0000 7f1ce64c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00cc670 con 0x7f1ce00abff0 2023-11-24T08:09:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.016+0000 7f1ce64c9640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0158500 con 0x7f1ce00ad720 2023-11-24T08:09:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.016+0000 7f1ce64c9640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00cbd10 con 0x7f1ce006d520 2023-11-24T08:09:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00abff0 0x7f1ce009a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00abff0 0x7f1ce009a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58410/0 (socket says 172.21.15.64:58410) 2023-11-24T08:09:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdf7fe640 1 -- 172.21.15.64:0/1702963662 learned_addr learned my addr 172.21.15.64:0/1702963662 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdffff640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce006d520 0x7f1ce0157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1ce4a66640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ce00ad720 0x7f1ce00c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdf7fe640 1 -- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce006d520 msgr2=0x7f1ce0157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdf7fe640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce006d520 0x7f1ce0157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdf7fe640 1 -- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ce00ad720 msgr2=0x7f1ce00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdf7fe640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ce00ad720 0x7f1ce00c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1cdf7fe640 1 -- 172.21.15.64:0/1702963662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd00094d0 con 0x7f1ce00abff0 2023-11-24T08:09:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.017+0000 7f1ce4a66640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ce00ad720 0x7f1ce00c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.018+0000 7f1cdf7fe640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00abff0 0x7f1ce009a360 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800d5c0 tx=0x7f1cc800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.018+0000 7f1cdcff9640 1 -- 172.21.15.64:0/1702963662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80125f0 con 0x7f1ce00abff0 2023-11-24T08:09:02.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.018+0000 7f1cdcff9640 1 -- 172.21.15.64:0/1702963662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc8012750 con 0x7f1ce00abff0 2023-11-24T08:09:02.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.018+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce009a8a0 con 0x7f1ce00abff0 2023-11-24T08:09:02.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.018+0000 7f1cdcff9640 1 -- 172.21.15.64:0/1702963662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8012b00 con 0x7f1ce00abff0 2023-11-24T08:09:02.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.018+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce00cc290 con 0x7f1ce00abff0 2023-11-24T08:09:02.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.019+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce00cc8e0 con 0x7f1ce00abff0 2023-11-24T08:09:02.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.020+0000 7f1cdcff9640 1 -- 172.21.15.64:0/1702963662 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1cc8012c60 con 0x7f1ce00abff0 2023-11-24T08:09:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.021+0000 7f1cdcff9640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cc006b470 0x7f1cc006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.022+0000 7f1cdcff9640 1 -- 172.21.15.64:0/1702963662 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1cc808a2c0 con 0x7f1ce00abff0 2023-11-24T08:09:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.022+0000 7f1cdffff640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cc006b470 0x7f1cc006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.023+0000 7f1cdffff640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cc006b470 0x7f1cc006d930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0036000 tx=0x7f1cd003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.026+0000 7f1cdcff9640 1 -- 172.21.15.64:0/1702963662 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cc8054d80 con 0x7f1ce00abff0 2023-11-24T08:09:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.220+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ce00ae6d0 con 0x7f1ce00abff0 2023-11-24T08:09:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.221+0000 7f1cdcff9640 1 -- 172.21.15.64:0/1702963662 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+12939 (secure 0 0 0) 0x7f1cc80583a0 con 0x7f1ce00abff0 2023-11-24T08:09:02.222 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:09:02.222 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":92,"fsid":"2a12aa8f-b2e5-4b01-9e3c-9a766fd6b918","created":"2023-11-24T08:03:30.920627+0000","modified":"2023-11-24T08:08:58.216705+0000","last_up_change":"2023-11-24T08:03:43.478487+0000","last_in_change":"2023-11-24T08:03:34.403042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:03:44.088235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75bdc55f-4ba8-4fce-8181-d53970482ee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6801","nonce":1432182325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6803","nonce":1432182325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6807","nonce":1432182325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1432182325},{"type":"v1","addr":"172.21.15.64:6805","nonce":1432182325}]},"public_addr":"172.21.15.64:6801/1432182325","cluster_addr":"172.21.15.64:6803/1432182325","heartbeat_back_addr":"172.21.15.64:6807/1432182325","heartbeat_front_addr":"172.21.15.64:6805/1432182325","state":["exists","up"]},{"osd":1,"uuid":"5288e629-bdb6-4be4-9165-11a1d6d7d39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6809","nonce":2214220706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6811","nonce":2214220706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6815","nonce":2214220706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2214220706},{"type":"v1","addr":"172.21.15.64:6813","nonce":2214220706}]},"public_addr":"172.21.15.64:6809/2214220706","cluster_addr":"172.21.15.64:6811/2214220706","heartbeat_back_addr":"172.21.15.64:6815/2214220706","heartbeat_front_addr":"172.21.15.64:6813/2214220706","state":["exists","up"]},{"osd":2,"uuid":"5ab7bfce-76ed-4cea-892b-fd522573d0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6817","nonce":492491626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6819","nonce":492491626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6823","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6826","nonce":492491626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":492491626},{"type":"v1","addr":"172.21.15.64:6821","nonce":492491626}]},"public_addr":"172.21.15.64:6817/492491626","cluster_addr":"172.21.15.64:6819/492491626","heartbeat_back_addr":"172.21.15.64:6826/492491626","heartbeat_front_addr":"172.21.15.64:6821/492491626","state":["exists","up"]},{"osd":3,"uuid":"814b901b-da27-4b2b-95a8-19ddad51b97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6824","nonce":3251355202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6825","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6827","nonce":3251355202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6831","nonce":3251355202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3251355202},{"type":"v1","addr":"172.21.15.64:6829","nonce":3251355202}]},"public_addr":"172.21.15.64:6824/3251355202","cluster_addr":"172.21.15.64:6827/3251355202","heartbeat_back_addr":"172.21.15.64:6831/3251355202","heartbeat_front_addr":"172.21.15.64:6829/3251355202","state":["exists","up"]},{"osd":4,"uuid":"7833546d-f061-414c-a385-3875be27241e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6821","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6823","nonce":3295720288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6825","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6827","nonce":3295720288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6831","nonce":3295720288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":3295720288},{"type":"v1","addr":"172.21.15.91:6829","nonce":3295720288}]},"public_addr":"172.21.15.91:6823/3295720288","cluster_addr":"172.21.15.91:6827/3295720288","heartbeat_back_addr":"172.21.15.91:6831/3295720288","heartbeat_front_addr":"172.21.15.91:6829/3295720288","state":["exists","up"]},{"osd":5,"uuid":"b45f456d-e76a-47b6-a1ee-d7e726c68288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6817","nonce":3653724610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6818","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6819","nonce":3653724610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6826","nonce":3653724610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":3653724610},{"type":"v1","addr":"172.21.15.91:6822","nonce":3653724610}]},"public_addr":"172.21.15.91:6817/3653724610","cluster_addr":"172.21.15.91:6819/3653724610","heartbeat_back_addr":"172.21.15.91:6826/3653724610","heartbeat_front_addr":"172.21.15.91:6822/3653724610","state":["exists","up"]},{"osd":6,"uuid":"cb020221-0c2e-4aef-9cea-26ea555234a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6801","nonce":3533292378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6802","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6803","nonce":3533292378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6807","nonce":3533292378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":3533292378},{"type":"v1","addr":"172.21.15.91:6805","nonce":3533292378}]},"public_addr":"172.21.15.91:6801/3533292378","cluster_addr":"172.21.15.91:6803/3533292378","heartbeat_back_addr":"172.21.15.91:6807/3533292378","heartbeat_front_addr":"172.21.15.91:6805/3533292378","state":["exists","up"]},{"osd":7,"uuid":"5a7445aa-1172-4b2e-9d8e-7a2dd6a95208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6809","nonce":4224688629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6810","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6811","nonce":4224688629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6815","nonce":4224688629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":4224688629},{"type":"v1","addr":"172.21.15.91:6813","nonce":4224688629}]},"public_addr":"172.21.15.91:6809/4224688629","cluster_addr":"172.21.15.91:6811/4224688629","heartbeat_back_addr":"172.21.15.91:6815/4224688629","heartbeat_front_addr":"172.21.15.91:6813/4224688629","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.904462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.768014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.798689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.763425+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.833543+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.755385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.921475+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:03:39.777691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6834/3674316997":"2023-11-25T08:08:55.204209+0000","172.21.15.64:6832/3674316997":"2023-11-25T08:08:55.204209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.223+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cc006b470 msgr2=0x7f1cc006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.223+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cc006b470 0x7f1cc006d930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0036000 tx=0x7f1cd003c040 comp rx=0 tx=0).stop 2023-11-24T08:09:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.223+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00abff0 msgr2=0x7f1ce009a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.223+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00abff0 0x7f1ce009a360 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800d5c0 tx=0x7f1cc800da80 comp rx=0 tx=0).stop 2023-11-24T08:09:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.224+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 shutdown_connections 2023-11-24T08:09:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.224+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1cc006b470 0x7f1cc006d930 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.224+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ce00ad720 0x7f1ce00c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.224+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00abff0 0x7f1ce009a360 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.224+0000 7f1ce64c9640 1 --2- 172.21.15.64:0/1702963662 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ce006d520 0x7f1ce0157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.224+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 >> 172.21.15.64:0/1702963662 conn(0x7f1ce00b0d80 msgr2=0x7f1ce00afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.224+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 shutdown_connections 2023-11-24T08:09:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:02.224+0000 7f1ce64c9640 1 -- 172.21.15.64:0/1702963662 wait complete. 2023-11-24T08:09:03.246 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T08:09:03.247 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:09:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.488+0000 7f0b8c3ce640 1 Processor -- start 2023-11-24T08:09:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.488+0000 7f0b8c3ce640 1 -- start start 2023-11-24T08:09:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.490+0000 7f0b8c3ce640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f0b8415b350 0x7f0b840abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.490+0000 7f0b8c3ce640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b8409cb30 0x7f0b840b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.490+0000 7f0b8c3ce640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8415b730 con 0x7f0b840adb60 2023-11-24T08:09:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.490+0000 7f0b8c3ce640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b840ac740 con 0x7f0b8415b350 2023-11-24T08:09:03.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.490+0000 7f0b8c3ce640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840ac460 con 0x7f0b8409cb30 2023-11-24T08:09:03.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.491+0000 7f0b8a96b640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0b840adb60 0x7f0b840adfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38236/0 (socket says 172.21.15.64:38236) 2023-11-24T08:09:03.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.491+0000 7f0b8a96b640 1 -- 172.21.15.64:0/4261872987 learned_addr learned my addr 172.21.15.64:0/4261872987 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:03.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.491+0000 7f0b89969640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b8409cb30 0x7f0b840b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.491+0000 7f0b8a16a640 1 --2- 172.21.15.64:0/4261872987 >> v2:172.21.15.91:3300/0 conn(0x7f0b8415b350 0x7f0b840abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.491+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425347907 0 0) 0x7f0b7c001110 con 0x7f0b840adb60 2023-11-24T08:09:03.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.491+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b70003520 con 0x7f0b840adb60 2023-11-24T08:09:03.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.491+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534869194 0 0) 0x7f0b70003520 con 0x7f0b840adb60 2023-11-24T08:09:03.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b8409cb30 msgr2=0x7f0b840b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b8409cb30 0x7f0b840b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 >> v2:172.21.15.91:3300/0 conn(0x7f0b8415b350 msgr2=0x7f0b840abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 --2- 172.21.15.64:0/4261872987 >> v2:172.21.15.91:3300/0 conn(0x7f0b8415b350 0x7f0b840abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b840b2a50 con 0x7f0b840adb60 2023-11-24T08:09:03.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f0b7c001800 con 0x7f0b840adb60 2023-11-24T08:09:03.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 >> v1:172.21.15.64:6789/0 conn(0x7f0b840adb60 legacy=0x7f0b840adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b70003970 0x7f0b70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b700047c0 0x7f0b70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b8409cb30 0x7f0b70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840ac740 con 0x7f0b70003970 2023-11-24T08:09:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840ac460 con 0x7f0b700047c0 2023-11-24T08:09:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7000b980 con 0x7f0b8409cb30 2023-11-24T08:09:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b8a16a640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b70003970 0x7f0b70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.492+0000 7f0b89969640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b700047c0 0x7f0b70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.493+0000 7f0b8a96b640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b8409cb30 0x7f0b70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.493+0000 7f0b8a16a640 1 -- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b8409cb30 msgr2=0x7f0b70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.493+0000 7f0b8a16a640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b8409cb30 0x7f0b70005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.493+0000 7f0b8a16a640 1 -- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b700047c0 msgr2=0x7f0b70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.493+0000 7f0b8a16a640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b700047c0 0x7f0b70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.493+0000 7f0b8a16a640 1 -- 172.21.15.64:0/4261872987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70003520 con 0x7f0b70003970 2023-11-24T08:09:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.493+0000 7f0b89969640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b700047c0 0x7f0b70004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:03.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.494+0000 7f0b8a16a640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b70003970 0x7f0b70003d90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009fc0 tx=0x7f0b700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654f1127bea3dc14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.494+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8002cc60 con 0x7f0b70003970 2023-11-24T08:09:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.494+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8000f040 con 0x7f0b70003970 2023-11-24T08:09:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.494+0000 7f0b89168640 1 -- 172.21.15.64:0/4261872987 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80034440 con 0x7f0b70003970 2023-11-24T08:09:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.495+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/4261872987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b70003970 msgr2=0x7f0b70003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.495+0000 7f0b8c3ce640 1 --2- 172.21.15.64:0/4261872987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b70003970 0x7f0b70003d90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009fc0 tx=0x7f0b700057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.495+0000 7f0b8a96b640 1 -- 172.21.15.64:0/4261872987 reap_dead start 2023-11-24T08:09:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.495+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/4261872987 shutdown_connections 2023-11-24T08:09:03.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.495+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/4261872987 >> 172.21.15.64:0/4261872987 conn(0x7f0b840b0ba0 msgr2=0x7f0b840b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.495+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/4261872987 shutdown_connections 2023-11-24T08:09:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.495+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/4261872987 wait complete. 2023-11-24T08:09:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.497+0000 7f0b8c3ce640 1 Processor -- start 2023-11-24T08:09:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.497+0000 7f0b8c3ce640 1 -- start start 2023-11-24T08:09:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8c3ce640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b70003970 0x7f0b84146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8c3ce640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b84071730 0x7f0b841f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8c3ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b84071bb0 0x7f0b841f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8c3ce640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b841f7ad0 con 0x7f0b84071bb0 2023-11-24T08:09:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8c3ce640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b841f33b0 con 0x7f0b70003970 2023-11-24T08:09:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8c3ce640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b84147290 con 0x7f0b84071730 2023-11-24T08:09:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8a96b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b84071bb0 0x7f0b841f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8a96b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b84071bb0 0x7f0b841f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58432/0 (socket says 172.21.15.64:58432) 2023-11-24T08:09:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b8a96b640 1 -- 172.21.15.64:0/444972359 learned_addr learned my addr 172.21.15.64:0/444972359 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.498+0000 7f0b89969640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b84071730 0x7f0b841f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b8a16a640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b70003970 0x7f0b84146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b8a96b640 1 -- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b84071730 msgr2=0x7f0b841f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b8a96b640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f0b84071730 0x7f0b841f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b8a96b640 1 -- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b70003970 msgr2=0x7f0b84146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b8a96b640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f0b70003970 0x7f0b84146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b8a96b640 1 -- 172.21.15.64:0/444972359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b841f6f50 con 0x7f0b84071bb0 2023-11-24T08:09:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b8a96b640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b84071bb0 0x7f0b841f2e70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d540 tx=0x7f0b7c00da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0074e0 con 0x7f0b84071bb0 2023-11-24T08:09:03.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c007660 con 0x7f0b84071bb0 2023-11-24T08:09:03.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.499+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004a20 con 0x7f0b84071bb0 2023-11-24T08:09:03.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.500+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b841f7230 con 0x7f0b84071bb0 2023-11-24T08:09:03.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.500+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b841f77f0 con 0x7f0b84071bb0 2023-11-24T08:09:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.501+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b840b0a30 con 0x7f0b84071bb0 2023-11-24T08:09:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.502+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f0b7c004b80 con 0x7f0b84071bb0 2023-11-24T08:09:03.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.503+0000 7f0b7affd640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b6806b470 0x7f0b6806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.503+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0b7c089a50 con 0x7f0b84071bb0 2023-11-24T08:09:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.503+0000 7f0b7affd640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f0b680712d0 0x7f0b680736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.503+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 --> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b68073da0 con 0x7f0b680712d0 2023-11-24T08:09:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.503+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b7c08e2a0 con 0x7f0b84071bb0 2023-11-24T08:09:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.503+0000 7f0b89969640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f0b680712d0 0x7f0b680736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.504+0000 7f0b8a16a640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b6806b470 0x7f0b6806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.504+0000 7f0b89969640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f0b680712d0 0x7f0b680736f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.505+0000 7f0b8a16a640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b6806b470 0x7f0b6806d930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0b80036000 tx=0x7f0b8002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.505+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 <== osd.0 v2:172.21.15.64:6800/1432182325 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0b68073da0 con 0x7f0b680712d0 2023-11-24T08:09:03.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.520+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 --> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0b8406e720 con 0x7f0b680712d0 2023-11-24T08:09:03.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.520+0000 7f0b7affd640 1 -- 172.21.15.64:0/444972359 <== osd.0 v2:172.21.15.64:6800/1432182325 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0b740097d0 con 0x7f0b680712d0 2023-11-24T08:09:03.521 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:09:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:09:03.522 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:09:03.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f0b680712d0 msgr2=0x7f0b680736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.64:6800/1432182325,v1:172.21.15.64:6801/1432182325] conn(0x7f0b680712d0 0x7f0b680736f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b6806b470 msgr2=0x7f0b6806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f0b6806b470 0x7f0b6806d930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0b80036000 tx=0x7f0b8002e000 comp rx=0 tx=0).stop 2023-11-24T08:09:03.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b84071bb0 msgr2=0x7f0b841f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 --2- 172.21.15.64:0/444972359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b84071bb0 0x7f0b841f2e70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d540 tx=0x7f0b7c00da00 comp rx=0 tx=0).stop 2023-11-24T08:09:03.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8a96b640 1 -- 172.21.15.64:0/444972359 reap_dead start 2023-11-24T08:09:03.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 shutdown_connections 2023-11-24T08:09:03.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 >> 172.21.15.64:0/444972359 conn(0x7f0b840b0ba0 msgr2=0x7f0b840aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:03.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 shutdown_connections 2023-11-24T08:09:03.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.521+0000 7f0b8c3ce640 1 -- 172.21.15.64:0/444972359 wait complete. 2023-11-24T08:09:03.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T08:09:03.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.763+0000 7f960e389640 1 Processor -- start 2023-11-24T08:09:03.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.763+0000 7f960e389640 1 -- start start 2023-11-24T08:09:03.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.764+0000 7f960e389640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96080719c0 0x7f96080ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.764+0000 7f960e389640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080ad510 con 0x7f9608161930 2023-11-24T08:09:03.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.764+0000 7f960e389640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080acc50 con 0x7f960806d520 2023-11-24T08:09:03.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.764+0000 7f960e389640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9608071e60 con 0x7f96080719c0 2023-11-24T08:09:03.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.765+0000 7f9607fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9608161930 0x7f96080ad9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38238/0 (socket says 172.21.15.64:38238) 2023-11-24T08:09:03.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.765+0000 7f9607fff640 1 -- 172.21.15.64:0/1163174056 learned_addr learned my addr 172.21.15.64:0/1163174056 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:03.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.765+0000 7f95fffff640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96080719c0 0x7f96080ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.765+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033848135 0 0) 0x7f96080ad510 con 0x7f9608161930 2023-11-24T08:09:03.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.765+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95f4003520 con 0x7f9608161930 2023-11-24T08:09:03.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.765+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4146971190 0 0) 0x7f95f4003520 con 0x7f9608161930 2023-11-24T08:09:03.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96080719c0 msgr2=0x7f96080ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96080719c0 0x7f96080ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 >> v1:172.21.15.91:6789/0 conn(0x7f960806d520 legacy=0x7f96080b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96080acec0 con 0x7f9608161930 2023-11-24T08:09:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f95f8001670 con 0x7f9608161930 2023-11-24T08:09:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 >> v1:172.21.15.64:6789/0 conn(0x7f9608161930 legacy=0x7f96080ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96080719c0 0x7f95f40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95f4003fd0 0x7f95f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95f4004d00 0x7f95f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9608071e60 con 0x7f95f4003fd0 2023-11-24T08:09:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f400bb40 con 0x7f95f4004d00 2023-11-24T08:09:03.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f400bd00 con 0x7f96080719c0 2023-11-24T08:09:03.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f95fffff640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95f4003fd0 0x7f95f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f9607fff640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96080719c0 0x7f95f40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f960c926640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95f4004d00 0x7f95f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f95fffff640 1 -- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96080719c0 msgr2=0x7f95f40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f95fffff640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f96080719c0 0x7f95f40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f95fffff640 1 -- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95f4004d00 msgr2=0x7f95f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f95fffff640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95f4004d00 0x7f95f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.766+0000 7f95fffff640 1 -- 172.21.15.64:0/1163174056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4003520 con 0x7f95f4003fd0 2023-11-24T08:09:03.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.767+0000 7f960c926640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f95f4004d00 0x7f95f4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:03.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.768+0000 7f95fffff640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95f4003fd0 0x7f95f40047c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f95f000ba30 tx=0x7f95f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c79ac215e4341d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.768+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0032640 con 0x7f95f4003fd0 2023-11-24T08:09:03.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.768+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00327a0 con 0x7f95f4003fd0 2023-11-24T08:09:03.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.768+0000 7f96077fe640 1 -- 172.21.15.64:0/1163174056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0032b30 con 0x7f95f4003fd0 2023-11-24T08:09:03.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.769+0000 7f960e389640 1 -- 172.21.15.64:0/1163174056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95f4003fd0 msgr2=0x7f95f40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.769+0000 7f960e389640 1 --2- 172.21.15.64:0/1163174056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95f4003fd0 0x7f95f40047c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f95f000ba30 tx=0x7f95f40057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.769+0000 7f960c926640 1 -- 172.21.15.64:0/1163174056 reap_dead start 2023-11-24T08:09:03.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.769+0000 7f960e389640 1 -- 172.21.15.64:0/1163174056 shutdown_connections 2023-11-24T08:09:03.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.769+0000 7f960e389640 1 -- 172.21.15.64:0/1163174056 >> 172.21.15.64:0/1163174056 conn(0x7f96080b0d80 msgr2=0x7f96080b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:03.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.769+0000 7f960e389640 1 -- 172.21.15.64:0/1163174056 shutdown_connections 2023-11-24T08:09:03.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.769+0000 7f960e389640 1 -- 172.21.15.64:0/1163174056 wait complete. 2023-11-24T08:09:03.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.771+0000 7f960e389640 1 Processor -- start 2023-11-24T08:09:03.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.771+0000 7f960e389640 1 -- start start 2023-11-24T08:09:03.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960e389640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95f4003fd0 0x7f96081532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960e389640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f960809c860 0x7f960809ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960e389640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080af3f0 0x7f96080af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960e389640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080cccb0 con 0x7f96080af3f0 2023-11-24T08:09:03.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960e389640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080afdb0 con 0x7f960809c860 2023-11-24T08:09:03.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960e389640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960809a690 con 0x7f95f4003fd0 2023-11-24T08:09:03.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960c926640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080af3f0 0x7f96080af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960c926640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080af3f0 0x7f96080af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58446/0 (socket says 172.21.15.64:58446) 2023-11-24T08:09:03.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.772+0000 7f960c926640 1 -- 172.21.15.64:0/1807264209 learned_addr learned my addr 172.21.15.64:0/1807264209 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:03.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f95fffff640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f960809c860 0x7f960809ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f9607fff640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95f4003fd0 0x7f96081532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f960c926640 1 -- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95f4003fd0 msgr2=0x7f96081532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f960c926640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95f4003fd0 0x7f96081532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f960c926640 1 -- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f960809c860 msgr2=0x7f960809ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f960c926640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f960809c860 0x7f960809ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f960c926640 1 -- 172.21.15.64:0/1807264209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608099df0 con 0x7f96080af3f0 2023-11-24T08:09:03.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f960c926640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080af3f0 0x7f96080af870 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f960806c200 tx=0x7f960000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f96057fa640 1 -- 172.21.15.64:0/1807264209 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600006d50 con 0x7f96080af3f0 2023-11-24T08:09:03.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f96057fa640 1 -- 172.21.15.64:0/1807264209 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600003970 con 0x7f96080af3f0 2023-11-24T08:09:03.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f96057fa640 1 -- 172.21.15.64:0/1807264209 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600003d60 con 0x7f96080af3f0 2023-11-24T08:09:03.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.773+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960809a0d0 con 0x7f96080af3f0 2023-11-24T08:09:03.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.774+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9608153820 con 0x7f96080af3f0 2023-11-24T08:09:03.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.775+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9608074770 con 0x7f96080af3f0 2023-11-24T08:09:03.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.776+0000 7f96057fa640 1 -- 172.21.15.64:0/1807264209 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f9600005070 con 0x7f96080af3f0 2023-11-24T08:09:03.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.777+0000 7f96057fa640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95e806b470 0x7f95e806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.777+0000 7f96057fa640 1 -- 172.21.15.64:0/1807264209 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9600089f50 con 0x7f96080af3f0 2023-11-24T08:09:03.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.777+0000 7f9607fff640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95e806b470 0x7f95e806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.778+0000 7f9607fff640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95e806b470 0x7f95e806d930 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f95f8001050 tx=0x7f95f8001d40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.781+0000 7f96057fa640 1 -- 172.21.15.64:0/1807264209 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96000547e0 con 0x7f96080af3f0 2023-11-24T08:09:03.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.973+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9608070c90 con 0x7f95e806b470 2023-11-24T08:09:03.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.974+0000 7f96057fa640 1 -- 172.21.15.64:0/1807264209 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f95f8005e60 con 0x7f95e806b470 2023-11-24T08:09:03.975 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T08:09:03.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95e806b470 msgr2=0x7f95e806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95e806b470 0x7f95e806d930 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f95f8001050 tx=0x7f95f8001d40 comp rx=0 tx=0).stop 2023-11-24T08:09:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080af3f0 msgr2=0x7f96080af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080af3f0 0x7f96080af870 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f960806c200 tx=0x7f960000dc00 comp rx=0 tx=0).stop 2023-11-24T08:09:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 shutdown_connections 2023-11-24T08:09:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f95e806b470 0x7f95e806d930 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080af3f0 0x7f96080af870 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f960809c860 0x7f960809ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 --2- 172.21.15.64:0/1807264209 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f95f4003fd0 0x7f96081532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.976+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 >> 172.21.15.64:0/1807264209 conn(0x7f96080b0d80 msgr2=0x7f9608165540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:03.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.977+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 shutdown_connections 2023-11-24T08:09:03.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:03.977+0000 7f960e389640 1 -- 172.21.15.64:0/1807264209 wait complete. 2023-11-24T08:09:03.998 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T08:09:03.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:09:04.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.233+0000 7fa3f5f99640 1 Processor -- start 2023-11-24T08:09:04.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.233+0000 7fa3f5f99640 1 -- start start 2023-11-24T08:09:04.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.234+0000 7fa3f5f99640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3f015d3b0 0x7fa3f00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.234+0000 7fa3f5f99640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fa3f00d0fa0 0x7fa3f00d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.234+0000 7fa3f5f99640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f00c6bf0 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.234+0000 7fa3f5f99640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f00d1430 con 0x7fa3f015d3b0 2023-11-24T08:09:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.234+0000 7fa3f5f99640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f00d1fa0 con 0x7fa3f00d0fa0 2023-11-24T08:09:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.235+0000 7fa3f4f97640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3f015d3b0 0x7fa3f00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.235+0000 7fa3f4f97640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3f015d3b0 0x7fa3f00ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58450/0 (socket says 172.21.15.64:58450) 2023-11-24T08:09:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.235+0000 7fa3f4f97640 1 -- 172.21.15.64:0/716441769 learned_addr learned my addr 172.21.15.64:0/716441769 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.235+0000 7fa3e7fff640 1 --2- 172.21.15.64:0/716441769 >> v2:172.21.15.91:3300/0 conn(0x7fa3f00d0fa0 0x7fa3f00d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.235+0000 7fa3f4f97640 1 -- 172.21.15.64:0/716441769 >> v2:172.21.15.91:3300/0 conn(0x7fa3f00d0fa0 msgr2=0x7fa3f00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.235+0000 7fa3f4f97640 1 --2- 172.21.15.64:0/716441769 >> v2:172.21.15.91:3300/0 conn(0x7fa3f00d0fa0 0x7fa3f00d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.235+0000 7fa3f4f97640 1 -- 172.21.15.64:0/716441769 >> v1:172.21.15.91:6789/0 conn(0x7fa3f00d1bc0 legacy=0x7fa3f00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.235+0000 7fa3f4f97640 1 -- 172.21.15.64:0/716441769 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00cedc0 con 0x7fa3f015d3b0 2023-11-24T08:09:04.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.236+0000 7fa3f4f97640 1 --2- 172.21.15.64:0/716441769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f015d3b0 0x7fa3f00ce6c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa3f008c4a0 tx=0x7fa3ec02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae40f03199d9c59 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.237+0000 7fa3e77fe640 1 -- 172.21.15.64:0/716441769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec02c9f0 con 0x7fa3f015d3b0 2023-11-24T08:09:04.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.237+0000 7fa3e77fe640 1 -- 172.21.15.64:0/716441769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec02cb50 con 0x7fa3f015d3b0 2023-11-24T08:09:04.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.237+0000 7fa3e77fe640 1 -- 172.21.15.64:0/716441769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0343f0 con 0x7fa3f015d3b0 2023-11-24T08:09:04.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.237+0000 7fa3f5f99640 1 -- 172.21.15.64:0/716441769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f015d3b0 msgr2=0x7fa3f00ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.237+0000 7fa3f5f99640 1 --2- 172.21.15.64:0/716441769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f015d3b0 0x7fa3f00ce6c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa3f008c4a0 tx=0x7fa3ec02be50 comp rx=0 tx=0).stop 2023-11-24T08:09:04.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.238+0000 7fa3f5f99640 1 -- 172.21.15.64:0/716441769 shutdown_connections 2023-11-24T08:09:04.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.238+0000 7fa3f5f99640 1 --2- 172.21.15.64:0/716441769 >> v2:172.21.15.91:3300/0 conn(0x7fa3f00d0fa0 0x7fa3f00d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.238+0000 7fa3f5f99640 1 --2- 172.21.15.64:0/716441769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f015d3b0 0x7fa3f00ce6c0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.238+0000 7fa3f5f99640 1 -- 172.21.15.64:0/716441769 >> 172.21.15.64:0/716441769 conn(0x7fa3f00d30a0 msgr2=0x7fa3f00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.238+0000 7fa3f5f99640 1 -- 172.21.15.64:0/716441769 shutdown_connections 2023-11-24T08:09:04.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.238+0000 7fa3f5f99640 1 -- 172.21.15.64:0/716441769 wait complete. 2023-11-24T08:09:04.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.240+0000 7fa3f5f99640 1 Processor -- start 2023-11-24T08:09:04.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.240+0000 7fa3f5f99640 1 -- start start 2023-11-24T08:09:04.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3f5f99640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3f00d0fa0 0x7fa3f014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3f5f99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f00d1bc0 0x7fa3f00b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3f5f99640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3f015d3b0 0x7fa3f0157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3f5f99640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00b85d0 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3f5f99640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f014e900 con 0x7fa3f00d0fa0 2023-11-24T08:09:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3f5f99640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00b8010 con 0x7fa3f015d3b0 2023-11-24T08:09:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f00d1bc0 0x7fa3f00b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f00d1bc0 0x7fa3f00b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58458/0 (socket says 172.21.15.64:58458) 2023-11-24T08:09:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3e7fff640 1 -- 172.21.15.64:0/928590415 learned_addr learned my addr 172.21.15.64:0/928590415 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3f4f97640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3f00d0fa0 0x7fa3f014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3e7fff640 1 -- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3f015d3b0 msgr2=0x7fa3f0157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.241+0000 7fa3f5798640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3f015d3b0 0x7fa3f0157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3e7fff640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3f015d3b0 0x7fa3f0157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3e7fff640 1 -- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3f00d0fa0 msgr2=0x7fa3f014e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3e7fff640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fa3f00d0fa0 0x7fa3f014e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3e7fff640 1 -- 172.21.15.64:0/928590415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec009b30 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3f5798640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fa3f015d3b0 0x7fa3f0157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3e7fff640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f00d1bc0 0x7fa3f00b76c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d5f0 tx=0x7fa3e000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00125f0 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0012750 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0012b00 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f00b82f0 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.242+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f00cafe0 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.243+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3f00c7400 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.244+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fa3e0012c60 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.245+0000 7fa3e57fa640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3dc06b4c0 0x7fa3dc06d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.245+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa3e008a360 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.246+0000 7fa3e57fa640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fa3dc071320 0x7fa3dc073740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.246+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 --> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3dc073df0 con 0x7fa3dc071320 2023-11-24T08:09:04.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.246+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3e008c7c0 con 0x7fa3f00d1bc0 2023-11-24T08:09:04.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.246+0000 7fa3f5798640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fa3dc071320 0x7fa3dc073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.246+0000 7fa3f4f97640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3dc06b4c0 0x7fa3dc06d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.247+0000 7fa3f5798640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fa3dc071320 0x7fa3dc073740 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.247+0000 7fa3f4f97640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3dc06b4c0 0x7fa3dc06d980 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec035000 tx=0x7fa3ec030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.247+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 <== osd.1 v2:172.21.15.64:6808/2214220706 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fa3dc073df0 con 0x7fa3dc071320 2023-11-24T08:09:04.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.262+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 --> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa3f00c7120 con 0x7fa3dc071320 2023-11-24T08:09:04.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.263+0000 7fa3e57fa640 1 -- 172.21.15.64:0/928590415 <== osd.1 v2:172.21.15.64:6808/2214220706 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa3e800a440 con 0x7fa3dc071320 2023-11-24T08:09:04.264 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:09:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:09:04.264 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:09:04.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.263+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fa3dc071320 msgr2=0x7fa3dc073740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.263+0000 7fa3f5f99640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.64:6808/2214220706,v1:172.21.15.64:6809/2214220706] conn(0x7fa3dc071320 0x7fa3dc073740 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3dc06b4c0 msgr2=0x7fa3dc06d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5f99640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fa3dc06b4c0 0x7fa3dc06d980 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec035000 tx=0x7fa3ec030000 comp rx=0 tx=0).stop 2023-11-24T08:09:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f00d1bc0 msgr2=0x7fa3f00b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5f99640 1 --2- 172.21.15.64:0/928590415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f00d1bc0 0x7fa3f00b76c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d5f0 tx=0x7fa3e000dab0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5798640 1 -- 172.21.15.64:0/928590415 reap_dead start 2023-11-24T08:09:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 shutdown_connections 2023-11-24T08:09:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 >> 172.21.15.64:0/928590415 conn(0x7fa3f00d30a0 msgr2=0x7fa3f00c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 shutdown_connections 2023-11-24T08:09:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.264+0000 7fa3f5f99640 1 -- 172.21.15.64:0/928590415 wait complete. 2023-11-24T08:09:04.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T08:09:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.492+0000 7f1e27c5c640 1 Processor -- start 2023-11-24T08:09:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.492+0000 7f1e27c5c640 1 -- start start 2023-11-24T08:09:04.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.493+0000 7f1e27c5c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e200abff0 0x7f1e200b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.493+0000 7f1e27c5c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2015c360 con 0x7f1e2006d520 2023-11-24T08:09:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.493+0000 7f1e27c5c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2015ba30 con 0x7f1e200ad720 2023-11-24T08:09:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.493+0000 7f1e27c5c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e200b41a0 con 0x7f1e200abff0 2023-11-24T08:09:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.494+0000 7f1e259f8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1e2006d520 0x7f1e2006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38242/0 (socket says 172.21.15.64:38242) 2023-11-24T08:09:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.494+0000 7f1e259f8640 1 -- 172.21.15.64:0/619351394 learned_addr learned my addr 172.21.15.64:0/619351394 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.494+0000 7f1e261f9640 1 --2- 172.21.15.64:0/619351394 >> v2:172.21.15.64:3300/0 conn(0x7f1e200abff0 0x7f1e200b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.494+0000 7f1e249f6640 1 -- 172.21.15.64:0/619351394 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632402657 0 0) 0x7f1e2015c360 con 0x7f1e2006d520 2023-11-24T08:09:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.494+0000 7f1e249f6640 1 -- 172.21.15.64:0/619351394 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c003520 con 0x7f1e2006d520 2023-11-24T08:09:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.495+0000 7f1e261f9640 1 -- 172.21.15.64:0/619351394 >> v1:172.21.15.91:6789/0 conn(0x7f1e200ad720 legacy=0x7f1e200b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.495+0000 7f1e261f9640 1 -- 172.21.15.64:0/619351394 >> v1:172.21.15.64:6789/0 conn(0x7f1e2006d520 legacy=0x7f1e2006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.495+0000 7f1e261f9640 1 -- 172.21.15.64:0/619351394 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2015bd10 con 0x7f1e200abff0 2023-11-24T08:09:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.496+0000 7f1e261f9640 1 --2- 172.21.15.64:0/619351394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200b2790 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009d90 tx=0x7f1e1802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e541fdd609fb9f28 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.496+0000 7f1e249f6640 1 -- 172.21.15.64:0/619351394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1802cc60 con 0x7f1e200abff0 2023-11-24T08:09:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.496+0000 7f1e249f6640 1 -- 172.21.15.64:0/619351394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1802cdc0 con 0x7f1e200abff0 2023-11-24T08:09:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.496+0000 7f1e249f6640 1 -- 172.21.15.64:0/619351394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18034660 con 0x7f1e200abff0 2023-11-24T08:09:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.497+0000 7f1e27c5c640 1 -- 172.21.15.64:0/619351394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 msgr2=0x7f1e200b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.497+0000 7f1e27c5c640 1 --2- 172.21.15.64:0/619351394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200b2790 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009d90 tx=0x7f1e1802beb0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.497+0000 7f1e27c5c640 1 -- 172.21.15.64:0/619351394 shutdown_connections 2023-11-24T08:09:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.497+0000 7f1e27c5c640 1 --2- 172.21.15.64:0/619351394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200b2790 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.497+0000 7f1e27c5c640 1 -- 172.21.15.64:0/619351394 >> 172.21.15.64:0/619351394 conn(0x7f1e200b0d80 msgr2=0x7f1e200b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.497+0000 7f1e27c5c640 1 -- 172.21.15.64:0/619351394 shutdown_connections 2023-11-24T08:09:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.497+0000 7f1e27c5c640 1 -- 172.21.15.64:0/619351394 wait complete. 2023-11-24T08:09:04.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.499+0000 7f1e27c5c640 1 Processor -- start 2023-11-24T08:09:04.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.499+0000 7f1e27c5c640 1 -- start start 2023-11-24T08:09:04.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e27c5c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e2006d520 0x7f1e201f0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e27c5c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e27c5c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e200ad720 0x7f1e200af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e27c5c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201f39c0 con 0x7f1e200abff0 2023-11-24T08:09:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e27c5c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201f2d40 con 0x7f1e2006d520 2023-11-24T08:09:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e27c5c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201f3020 con 0x7f1e200ad720 2023-11-24T08:09:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e251f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e251f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58488/0 (socket says 172.21.15.64:58488) 2023-11-24T08:09:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e251f7640 1 -- 172.21.15.64:0/1701760066 learned_addr learned my addr 172.21.15.64:0/1701760066 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e261f9640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e200ad720 0x7f1e200af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e251f7640 1 -- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e200ad720 msgr2=0x7f1e200af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e251f7640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e200ad720 0x7f1e200af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e259f8640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e2006d520 0x7f1e201f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.500+0000 7f1e251f7640 1 -- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e2006d520 msgr2=0x7f1e201f0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.501+0000 7f1e251f7640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e2006d520 0x7f1e201f0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.501+0000 7f1e251f7640 1 -- 172.21.15.64:0/1701760066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e18009a70 con 0x7f1e200abff0 2023-11-24T08:09:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.501+0000 7f1e251f7640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200af470 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c400 tx=0x7f1e1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.501+0000 7f1e167fc640 1 -- 172.21.15.64:0/1701760066 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10011790 con 0x7f1e200abff0 2023-11-24T08:09:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.501+0000 7f1e167fc640 1 -- 172.21.15.64:0/1701760066 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10011930 con 0x7f1e200abff0 2023-11-24T08:09:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.501+0000 7f1e167fc640 1 -- 172.21.15.64:0/1701760066 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10011ce0 con 0x7f1e200abff0 2023-11-24T08:09:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.501+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e201f3300 con 0x7f1e200abff0 2023-11-24T08:09:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.501+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201f05c0 con 0x7f1e200abff0 2023-11-24T08:09:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.502+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e20074770 con 0x7f1e200abff0 2023-11-24T08:09:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.504+0000 7f1e167fc640 1 -- 172.21.15.64:0/1701760066 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1e1001c460 con 0x7f1e200abff0 2023-11-24T08:09:04.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.504+0000 7f1e167fc640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e0406b470 0x7f1e0406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.505+0000 7f1e167fc640 1 -- 172.21.15.64:0/1701760066 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e10089dd0 con 0x7f1e200abff0 2023-11-24T08:09:04.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.505+0000 7f1e259f8640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e0406b470 0x7f1e0406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.506+0000 7f1e259f8640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e0406b470 0x7f1e0406d930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0011e0 tx=0x7f1e1c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.509+0000 7f1e167fc640 1 -- 172.21.15.64:0/1701760066 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e10054890 con 0x7f1e200abff0 2023-11-24T08:09:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.703+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e200b2e80 con 0x7f1e0406b470 2023-11-24T08:09:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.705+0000 7f1e167fc640 1 -- 172.21.15.64:0/1701760066 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1e200b2e80 con 0x7f1e0406b470 2023-11-24T08:09:04.706 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T08:09:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.707+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e0406b470 msgr2=0x7f1e0406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.707+0000 7f1e27c5c640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e0406b470 0x7f1e0406d930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0011e0 tx=0x7f1e1c009040 comp rx=0 tx=0).stop 2023-11-24T08:09:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.707+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 msgr2=0x7f1e200af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.707+0000 7f1e27c5c640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200af470 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c400 tx=0x7f1e1000c8c0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.707+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 shutdown_connections 2023-11-24T08:09:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.707+0000 7f1e27c5c640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e0406b470 0x7f1e0406d930 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.707+0000 7f1e27c5c640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e200ad720 0x7f1e200af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.708+0000 7f1e27c5c640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e200abff0 0x7f1e200af470 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.708+0000 7f1e27c5c640 1 --2- 172.21.15.64:0/1701760066 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e2006d520 0x7f1e201f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.708+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 >> 172.21.15.64:0/1701760066 conn(0x7f1e200b0d80 msgr2=0x7f1e200aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.708+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 shutdown_connections 2023-11-24T08:09:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.708+0000 7f1e27c5c640 1 -- 172.21.15.64:0/1701760066 wait complete. 2023-11-24T08:09:04.728 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T08:09:04.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:09:04.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.956+0000 7fc69df6c640 1 Processor -- start 2023-11-24T08:09:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.957+0000 7fc69df6c640 1 -- start start 2023-11-24T08:09:04.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.958+0000 7fc69df6c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980d1d90 0x7fc6980ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.958+0000 7fc69df6c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980c6bf0 con 0x7fc69815d3b0 2023-11-24T08:09:04.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.958+0000 7fc69df6c640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980d13d0 con 0x7fc6980d0fa0 2023-11-24T08:09:04.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.958+0000 7fc69df6c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980cedc0 con 0x7fc6980d1d90 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.959+0000 7fc69cf6a640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc69815d3b0 0x7fc6980d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38248/0 (socket says 172.21.15.64:38248) 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.959+0000 7fc69cf6a640 1 -- 172.21.15.64:0/855715784 learned_addr learned my addr 172.21.15.64:0/855715784 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.959+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930295172 0 0) 0x7fc6980c6bf0 con 0x7fc69815d3b0 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.959+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc684003520 con 0x7fc69815d3b0 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.959+0000 7fc697fff640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980d1d90 0x7fc6980ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.959+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3779250128 0 0) 0x7fc684003520 con 0x7fc69815d3b0 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980d1d90 msgr2=0x7fc6980ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980d1d90 0x7fc6980ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 >> v1:172.21.15.91:6789/0 conn(0x7fc6980d0fa0 legacy=0x7fc6980b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980c6e40 con 0x7fc69815d3b0 2023-11-24T08:09:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fc6880019f0 con 0x7fc69815d3b0 2023-11-24T08:09:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 >> v1:172.21.15.64:6789/0 conn(0x7fc69815d3b0 legacy=0x7fc6980d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc684003a10 0x7fc684003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc684004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980d1d90 0x7fc684005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980cedc0 con 0x7fc6840047c0 2023-11-24T08:09:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68400bb70 con 0x7fc684003a10 2023-11-24T08:09:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68400bd30 con 0x7fc6980d1d90 2023-11-24T08:09:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc697fff640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc684004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc69cf6a640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc684003a10 0x7fc684003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.960+0000 7fc69d76b640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980d1d90 0x7fc684005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.961+0000 7fc697fff640 1 -- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980d1d90 msgr2=0x7fc684005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.961+0000 7fc697fff640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980d1d90 0x7fc684005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.961+0000 7fc697fff640 1 -- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc684003a10 msgr2=0x7fc684003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.961+0000 7fc697fff640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc684003a10 0x7fc684003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.961+0000 7fc697fff640 1 -- 172.21.15.64:0/855715784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc684003520 con 0x7fc6840047c0 2023-11-24T08:09:04.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.961+0000 7fc69cf6a640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc684003a10 0x7fc684003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.962+0000 7fc697fff640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc684004c40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc680009a90 tx=0x7fc6840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d913cafa609a4b11 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.962+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68002c680 con 0x7fc6840047c0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.962+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68002c7e0 con 0x7fc6840047c0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.962+0000 7fc6977fe640 1 -- 172.21.15.64:0/855715784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68002cb70 con 0x7fc6840047c0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.963+0000 7fc69df6c640 1 -- 172.21.15.64:0/855715784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 msgr2=0x7fc684004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.963+0000 7fc69df6c640 1 --2- 172.21.15.64:0/855715784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc684004c40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc680009a90 tx=0x7fc6840057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.963+0000 7fc69d76b640 1 -- 172.21.15.64:0/855715784 reap_dead start 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.963+0000 7fc69df6c640 1 -- 172.21.15.64:0/855715784 shutdown_connections 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.963+0000 7fc69df6c640 1 -- 172.21.15.64:0/855715784 >> 172.21.15.64:0/855715784 conn(0x7fc6980d30a0 msgr2=0x7fc6980d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.963+0000 7fc69df6c640 1 -- 172.21.15.64:0/855715784 shutdown_connections 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.963+0000 7fc69df6c640 1 -- 172.21.15.64:0/855715784 wait complete. 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.965+0000 7fc69df6c640 1 Processor -- start 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.965+0000 7fc69df6c640 1 -- start start 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69df6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc6980b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69df6c640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc698151030 0x7fc6980b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69df6c640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980b71f0 0x7fc6980b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69df6c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980cd2d0 con 0x7fc6840047c0 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69df6c640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69814b940 con 0x7fc698151030 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69df6c640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69814b1b0 con 0x7fc6980b71f0 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69cf6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc6980b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69cf6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc6980b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58500/0 (socket says 172.21.15.64:58500) 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69cf6a640 1 -- 172.21.15.64:0/2406560893 learned_addr learned my addr 172.21.15.64:0/2406560893 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69cf6a640 1 -- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980b71f0 msgr2=0x7fc6980b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69cf6a640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc6980b71f0 0x7fc6980b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69cf6a640 1 -- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc698151030 msgr2=0x7fc6980b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc697fff640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc698151030 0x7fc6980b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69cf6a640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc698151030 0x7fc6980b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.966+0000 7fc69cf6a640 1 -- 172.21.15.64:0/2406560893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980cca90 con 0x7fc6840047c0 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.967+0000 7fc69cf6a640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc6980b8590 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc688001ac0 tx=0x7fc68800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.967+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688004d60 con 0x7fc6840047c0 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.967+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6880070d0 con 0x7fc6840047c0 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.967+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69814b610 con 0x7fc6840047c0 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.967+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688007440 con 0x7fc6840047c0 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.967+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6980ccff0 con 0x7fc6840047c0 2023-11-24T08:09:04.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.968+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc69815d790 con 0x7fc6840047c0 2023-11-24T08:09:04.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.969+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fc6880075a0 con 0x7fc6840047c0 2023-11-24T08:09:04.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.970+0000 7fc6957fa640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc67806b470 0x7fc67806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.970+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc688089fb0 con 0x7fc6840047c0 2023-11-24T08:09:04.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.970+0000 7fc6957fa640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7fc6780712d0 0x7fc6780736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.970+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 --> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc678073da0 con 0x7fc6780712d0 2023-11-24T08:09:04.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.970+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc68808f4f0 con 0x7fc6840047c0 2023-11-24T08:09:04.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.971+0000 7fc69d76b640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7fc6780712d0 0x7fc6780736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.971+0000 7fc697fff640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc67806b470 0x7fc67806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.971+0000 7fc69d76b640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7fc6780712d0 0x7fc6780736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.972+0000 7fc697fff640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc67806b470 0x7fc67806d930 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc680036000 tx=0x7fc680030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.972+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 <== osd.2 v2:172.21.15.64:6816/492491626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc678073da0 con 0x7fc6780712d0 2023-11-24T08:09:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.987+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 --> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc6980c7400 con 0x7fc6780712d0 2023-11-24T08:09:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc6957fa640 1 -- 172.21.15.64:0/2406560893 <== osd.2 v2:172.21.15.64:6816/492491626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc6900050e0 con 0x7fc6780712d0 2023-11-24T08:09:04.989 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:09:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:09:04.989 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:09:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7fc6780712d0 msgr2=0x7fc6780736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69df6c640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.64:6816/492491626,v1:172.21.15.64:6817/492491626] conn(0x7fc6780712d0 0x7fc6780736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc67806b470 msgr2=0x7fc67806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69df6c640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc67806b470 0x7fc67806d930 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc680036000 tx=0x7fc680030000 comp rx=0 tx=0).stop 2023-11-24T08:09:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 msgr2=0x7fc6980b8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69df6c640 1 --2- 172.21.15.64:0/2406560893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6840047c0 0x7fc6980b8590 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc688001ac0 tx=0x7fc68800b560 comp rx=0 tx=0).stop 2023-11-24T08:09:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69d76b640 1 -- 172.21.15.64:0/2406560893 reap_dead start 2023-11-24T08:09:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 shutdown_connections 2023-11-24T08:09:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.988+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 >> 172.21.15.64:0/2406560893 conn(0x7fc6980d30a0 msgr2=0x7fc6980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.989+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 shutdown_connections 2023-11-24T08:09:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:04.989+0000 7fc69df6c640 1 -- 172.21.15.64:0/2406560893 wait complete. 2023-11-24T08:09:05.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T08:09:05.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.228+0000 7f9529639640 1 Processor -- start 2023-11-24T08:09:05.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.228+0000 7f9529639640 1 -- start start 2023-11-24T08:09:05.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.230+0000 7f9529639640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9524141e80 0x7f9524071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.230+0000 7f9529639640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952406cae0 con 0x7f9524071650 2023-11-24T08:09:05.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.230+0000 7f9529639640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9524071e40 con 0x7f952415ed20 2023-11-24T08:09:05.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.230+0000 7f9529639640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95241422e0 con 0x7f9524141e80 2023-11-24T08:09:05.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.231+0000 7f9528e38640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9524071650 0x7f952406cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38260/0 (socket says 172.21.15.64:38260) 2023-11-24T08:09:05.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.231+0000 7f9528e38640 1 -- 172.21.15.64:0/3292008926 learned_addr learned my addr 172.21.15.64:0/3292008926 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:05.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.231+0000 7f95237fe640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9524141e80 0x7f9524071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.231+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501079993 0 0) 0x7f952406cae0 con 0x7f9524071650 2023-11-24T08:09:05.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.231+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f950c003520 con 0x7f9524071650 2023-11-24T08:09:05.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.231+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519223723 0 0) 0x7f950c003520 con 0x7f9524071650 2023-11-24T08:09:05.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9524141e80 msgr2=0x7f9524071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:05.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9524141e80 0x7f9524071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 >> v1:172.21.15.91:6789/0 conn(0x7f952415ed20 legacy=0x7f952415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952406c1b0 con 0x7f9524071650 2023-11-24T08:09:05.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f951c0019f0 con 0x7f9524071650 2023-11-24T08:09:05.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 >> v1:172.21.15.64:6789/0 conn(0x7f9524071650 legacy=0x7f952406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9524141e80 0x7f950c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f950c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950c004d00 0x7f950c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c00bb10 con 0x7f950c003fe0 2023-11-24T08:09:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c00bc80 con 0x7f950c004d00 2023-11-24T08:09:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950c00be40 con 0x7f9524141e80 2023-11-24T08:09:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f95237fe640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f950c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.232+0000 7f9523fff640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9524141e80 0x7f950c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.233+0000 7f95237fe640 1 -- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9524141e80 msgr2=0x7f950c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.233+0000 7f95237fe640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f9524141e80 0x7f950c003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.233+0000 7f95237fe640 1 -- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950c004d00 msgr2=0x7f950c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:05.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.233+0000 7f95237fe640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f950c004d00 0x7f950c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.233+0000 7f95237fe640 1 -- 172.21.15.64:0/3292008926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c003520 con 0x7f950c003fe0 2023-11-24T08:09:05.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.234+0000 7f95237fe640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f950c0047c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9518009ac0 tx=0x7f950c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4ea49c8af8b09f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.234+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951802c6b0 con 0x7f950c003fe0 2023-11-24T08:09:05.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.234+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951802c810 con 0x7f950c003fe0 2023-11-24T08:09:05.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.234+0000 7f9522ffd640 1 -- 172.21.15.64:0/3292008926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951802cba0 con 0x7f950c003fe0 2023-11-24T08:09:05.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.234+0000 7f9529639640 1 -- 172.21.15.64:0/3292008926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 msgr2=0x7f950c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.234+0000 7f9529639640 1 --2- 172.21.15.64:0/3292008926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f950c0047c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9518009ac0 tx=0x7f950c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.234+0000 7f9528e38640 1 -- 172.21.15.64:0/3292008926 reap_dead start 2023-11-24T08:09:05.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.235+0000 7f9529639640 1 -- 172.21.15.64:0/3292008926 shutdown_connections 2023-11-24T08:09:05.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.235+0000 7f9529639640 1 -- 172.21.15.64:0/3292008926 >> 172.21.15.64:0/3292008926 conn(0x7f95240b0840 msgr2=0x7f95240b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:05.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.235+0000 7f9529639640 1 -- 172.21.15.64:0/3292008926 shutdown_connections 2023-11-24T08:09:05.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.235+0000 7f9529639640 1 -- 172.21.15.64:0/3292008926 wait complete. 2023-11-24T08:09:05.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.236+0000 7f9529639640 1 Processor -- start 2023-11-24T08:09:05.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.237+0000 7f9529639640 1 -- start start 2023-11-24T08:09:05.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.237+0000 7f9529639640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f95241586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9529639640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f952415d700 0x7f952415db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9529639640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9524137b10 0x7f9524137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9529639640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9524158c10 con 0x7f950c003fe0 2023-11-24T08:09:05.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9529639640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9524138500 con 0x7f9524137b10 2023-11-24T08:09:05.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9529639640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952415e210 con 0x7f952415d700 2023-11-24T08:09:05.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9523fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f95241586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9523fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f95241586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58520/0 (socket says 172.21.15.64:58520) 2023-11-24T08:09:05.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9523fff640 1 -- 172.21.15.64:0/331384051 learned_addr learned my addr 172.21.15.64:0/331384051 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:05.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9528e38640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9524137b10 0x7f9524137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f95237fe640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f952415d700 0x7f952415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9523fff640 1 -- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f952415d700 msgr2=0x7f952415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9523fff640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f952415d700 0x7f952415db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9523fff640 1 -- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9524137b10 msgr2=0x7f9524137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9523fff640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9524137b10 0x7f9524137fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f9523fff640 1 -- 172.21.15.64:0/331384051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952415e4f0 con 0x7f950c003fe0 2023-11-24T08:09:05.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.238+0000 7f95237fe640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f952415d700 0x7f952415db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:05.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.239+0000 7f9523fff640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f95241586d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f951400e550 tx=0x7f951400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.239+0000 7f9520ff9640 1 -- 172.21.15.64:0/331384051 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140052e0 con 0x7f950c003fe0 2023-11-24T08:09:05.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.239+0000 7f9520ff9640 1 -- 172.21.15.64:0/331384051 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514005440 con 0x7f950c003fe0 2023-11-24T08:09:05.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.239+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9524142de0 con 0x7f950c003fe0 2023-11-24T08:09:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.239+0000 7f9520ff9640 1 -- 172.21.15.64:0/331384051 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514003810 con 0x7f950c003fe0 2023-11-24T08:09:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.239+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95241433a0 con 0x7f950c003fe0 2023-11-24T08:09:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.240+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9524143560 con 0x7f950c003fe0 2023-11-24T08:09:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.241+0000 7f9520ff9640 1 -- 172.21.15.64:0/331384051 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f9514003970 con 0x7f950c003fe0 2023-11-24T08:09:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.243+0000 7f9520ff9640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f951006b470 0x7f951006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.243+0000 7f9520ff9640 1 -- 172.21.15.64:0/331384051 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9514089c70 con 0x7f950c003fe0 2023-11-24T08:09:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.243+0000 7f95237fe640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f951006b470 0x7f951006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.244+0000 7f95237fe640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f951006b470 0x7f951006d930 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9518036000 tx=0x7f9518030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.247+0000 7f9520ff9640 1 -- 172.21.15.64:0/331384051 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95140546a0 con 0x7f950c003fe0 2023-11-24T08:09:05.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.441+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f952406c770 con 0x7f951006b470 2023-11-24T08:09:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.442+0000 7f9520ff9640 1 -- 172.21.15.64:0/331384051 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f95180041c0 con 0x7f951006b470 2023-11-24T08:09:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T08:09:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f951006b470 msgr2=0x7f951006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f951006b470 0x7f951006d930 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9518036000 tx=0x7f9518030000 comp rx=0 tx=0).stop 2023-11-24T08:09:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 msgr2=0x7f95241586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f95241586d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f951400e550 tx=0x7f951400ea10 comp rx=0 tx=0).stop 2023-11-24T08:09:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 shutdown_connections 2023-11-24T08:09:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f951006b470 0x7f951006d930 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f9524137b10 0x7f9524137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f952415d700 0x7f952415db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 --2- 172.21.15.64:0/331384051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950c003fe0 0x7f95241586d0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 >> 172.21.15.64:0/331384051 conn(0x7f95240b0840 msgr2=0x7f95240b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 shutdown_connections 2023-11-24T08:09:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.445+0000 7f9529639640 1 -- 172.21.15.64:0/331384051 wait complete. 2023-11-24T08:09:05.464 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T08:09:05.464 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:09:05.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.694+0000 7f64a4a90640 1 Processor -- start 2023-11-24T08:09:05.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.694+0000 7f64a4a90640 1 -- start start 2023-11-24T08:09:05.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.696+0000 7f64a4a90640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f64a00b29d0 0x7f64a0162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.696+0000 7f64a4a90640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64a0162570 0x7f64a00717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.696+0000 7f64a4a90640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a0144d70 con 0x7f64a0146820 2023-11-24T08:09:05.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.696+0000 7f64a4a90640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a0147010 con 0x7f64a0162570 2023-11-24T08:09:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.696+0000 7f64a4a90640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a01472f0 con 0x7f64a00b29d0 2023-11-24T08:09:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.696+0000 7f649e59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f64a0146820 0x7f64a00b22c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38264/0 (socket says 172.21.15.64:38264) 2023-11-24T08:09:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.696+0000 7f649e59c640 1 -- 172.21.15.64:0/2412972288 learned_addr learned my addr 172.21.15.64:0/2412972288 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.696+0000 7f649ed9d640 1 --2- 172.21.15.64:0/2412972288 >> v2:172.21.15.64:3300/0 conn(0x7f64a0162570 0x7f64a00717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.697+0000 7f649dd9b640 1 --2- 172.21.15.64:0/2412972288 >> v2:172.21.15.91:3300/0 conn(0x7f64a00b29d0 0x7f64a0162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.697+0000 7f649d59a640 1 -- 172.21.15.64:0/2412972288 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577674378 0 0) 0x7f64a0144d70 con 0x7f64a0146820 2023-11-24T08:09:05.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.697+0000 7f649d59a640 1 -- 172.21.15.64:0/2412972288 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6488003520 con 0x7f64a0146820 2023-11-24T08:09:05.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.697+0000 7f649ed9d640 1 -- 172.21.15.64:0/2412972288 >> v2:172.21.15.91:3300/0 conn(0x7f64a00b29d0 msgr2=0x7f64a0162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:05.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.697+0000 7f649ed9d640 1 --2- 172.21.15.64:0/2412972288 >> v2:172.21.15.91:3300/0 conn(0x7f64a00b29d0 0x7f64a0162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.697+0000 7f649ed9d640 1 -- 172.21.15.64:0/2412972288 >> v1:172.21.15.64:6789/0 conn(0x7f64a0146820 legacy=0x7f64a00b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.697+0000 7f649ed9d640 1 -- 172.21.15.64:0/2412972288 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a01475d0 con 0x7f64a0162570 2023-11-24T08:09:05.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.697+0000 7f649dd9b640 1 --2- 172.21.15.64:0/2412972288 >> v2:172.21.15.91:3300/0 conn(0x7f64a00b29d0 0x7f64a0162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:05.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.698+0000 7f649ed9d640 1 --2- 172.21.15.64:0/2412972288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a0162570 0x7f64a00717b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f64a006d610 tx=0x7f649802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c70a8ad3ef970c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f649d59a640 1 -- 172.21.15.64:0/2412972288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498031830 con 0x7f64a0162570 2023-11-24T08:09:05.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f649d59a640 1 -- 172.21.15.64:0/2412972288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6498031990 con 0x7f64a0162570 2023-11-24T08:09:05.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f649d59a640 1 -- 172.21.15.64:0/2412972288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498031d00 con 0x7f64a0162570 2023-11-24T08:09:05.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f64a4a90640 1 -- 172.21.15.64:0/2412972288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a0162570 msgr2=0x7f64a00717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f64a4a90640 1 --2- 172.21.15.64:0/2412972288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a0162570 0x7f64a00717b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f64a006d610 tx=0x7f649802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f64a4a90640 1 -- 172.21.15.64:0/2412972288 shutdown_connections 2023-11-24T08:09:05.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f64a4a90640 1 --2- 172.21.15.64:0/2412972288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a0162570 0x7f64a00717b0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f64a4a90640 1 --2- 172.21.15.64:0/2412972288 >> v2:172.21.15.91:3300/0 conn(0x7f64a00b29d0 0x7f64a0162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f64a4a90640 1 -- 172.21.15.64:0/2412972288 >> 172.21.15.64:0/2412972288 conn(0x7f64a00661f0 msgr2=0x7f64a00665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:05.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f64a4a90640 1 -- 172.21.15.64:0/2412972288 shutdown_connections 2023-11-24T08:09:05.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.699+0000 7f64a4a90640 1 -- 172.21.15.64:0/2412972288 wait complete. 2023-11-24T08:09:05.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.701+0000 7f64a4a90640 1 Processor -- start 2023-11-24T08:09:05.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.701+0000 7f64a4a90640 1 -- start start 2023-11-24T08:09:05.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.702+0000 7f64a4a90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a00b29d0 0x7f64a00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.702+0000 7f64a4a90640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64a0146820 0x7f64a015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.702+0000 7f64a4a90640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f64a0162570 0x7f64a015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.702+0000 7f64a4a90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0134940 con 0x7f64a00b29d0 2023-11-24T08:09:05.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.702+0000 7f64a4a90640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01edb20 con 0x7f64a0146820 2023-11-24T08:09:05.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.702+0000 7f64a4a90640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01ede00 con 0x7f64a0162570 2023-11-24T08:09:05.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.702+0000 7f649e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a00b29d0 0x7f64a00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a00b29d0 0x7f64a00b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58536/0 (socket says 172.21.15.64:58536) 2023-11-24T08:09:05.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649e59c640 1 -- 172.21.15.64:0/1788186557 learned_addr learned my addr 172.21.15.64:0/1788186557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:05.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649dd9b640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64a0146820 0x7f64a015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649ed9d640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f64a0162570 0x7f64a015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649e59c640 1 -- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f64a0162570 msgr2=0x7f64a015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649e59c640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f64a0162570 0x7f64a015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649e59c640 1 -- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64a0146820 msgr2=0x7f64a015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649e59c640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f64a0146820 0x7f64a015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.703+0000 7f649e59c640 1 -- 172.21.15.64:0/1788186557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6498009400 con 0x7f64a00b29d0 2023-11-24T08:09:05.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.704+0000 7f649e59c640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a00b29d0 0x7f64a00b42d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f649000cb40 tx=0x7f649000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.704+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000aca0 con 0x7f64a00b29d0 2023-11-24T08:09:05.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.704+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649000ae00 con 0x7f64a00b29d0 2023-11-24T08:09:05.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.704+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490012620 con 0x7f64a00b29d0 2023-11-24T08:09:05.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.704+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a015c330 con 0x7f64a00b29d0 2023-11-24T08:09:05.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.704+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a015c8f0 con 0x7f64a00b29d0 2023-11-24T08:09:05.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.705+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64a0146c00 con 0x7f64a00b29d0 2023-11-24T08:09:05.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.706+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6490007070 con 0x7f64a00b29d0 2023-11-24T08:09:05.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.707+0000 7f648effd640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f648006b470 0x7f648006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.707+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6490089e80 con 0x7f64a00b29d0 2023-11-24T08:09:05.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.707+0000 7f648effd640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f64800712d0 0x7f64800736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.707+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 --> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6480073da0 con 0x7f64800712d0 2023-11-24T08:09:05.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.707+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f649008c2e0 con 0x7f64a00b29d0 2023-11-24T08:09:05.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.708+0000 7f649ed9d640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f64800712d0 0x7f64800736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.708+0000 7f649dd9b640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f648006b470 0x7f648006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.708+0000 7f649ed9d640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f64800712d0 0x7f64800736f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.709+0000 7f649dd9b640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f648006b470 0x7f648006d930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f64a00b55f0 tx=0x7f6494008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.709+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 <== osd.3 v2:172.21.15.64:6822/3251355202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6480073da0 con 0x7f64800712d0 2023-11-24T08:09:05.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.724+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 --> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f64a006e800 con 0x7f64800712d0 2023-11-24T08:09:05.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f648effd640 1 -- 172.21.15.64:0/1788186557 <== osd.3 v2:172.21.15.64:6822/3251355202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f64a006e800 con 0x7f64800712d0 2023-11-24T08:09:05.727 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:09:05.727 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:09:05.727 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:09:05.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f64800712d0 msgr2=0x7f64800736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.64:6822/3251355202,v1:172.21.15.64:6824/3251355202] conn(0x7f64800712d0 0x7f64800736f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f648006b470 msgr2=0x7f648006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f648006b470 0x7f648006d930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f64a00b55f0 tx=0x7f6494008040 comp rx=0 tx=0).stop 2023-11-24T08:09:05.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a00b29d0 msgr2=0x7f64a00b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 --2- 172.21.15.64:0/1788186557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a00b29d0 0x7f64a00b42d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f649000cb40 tx=0x7f649000a060 comp rx=0 tx=0).stop 2023-11-24T08:09:05.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f649ed9d640 1 -- 172.21.15.64:0/1788186557 reap_dead start 2023-11-24T08:09:05.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 shutdown_connections 2023-11-24T08:09:05.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 >> 172.21.15.64:0/1788186557 conn(0x7f64a00661f0 msgr2=0x7f64a0072200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:05.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 shutdown_connections 2023-11-24T08:09:05.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.725+0000 7f64a4a90640 1 -- 172.21.15.64:0/1788186557 wait complete. 2023-11-24T08:09:05.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T08:09:05.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.967+0000 7f1d1d362640 1 Processor -- start 2023-11-24T08:09:05.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.967+0000 7f1d1d362640 1 -- start start 2023-11-24T08:09:05.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.969+0000 7f1d1d362640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d18141e80 0x7f1d18071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.969+0000 7f1d1d362640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.969+0000 7f1d1d362640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1806cae0 con 0x7f1d1815ed20 2023-11-24T08:09:05.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.969+0000 7f1d1d362640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d181422e0 con 0x7f1d18141e80 2023-11-24T08:09:05.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.969+0000 7f1d1d362640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d18071e40 con 0x7f1d18071650 2023-11-24T08:09:05.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.970+0000 7f1d177fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d18141e80 0x7f1d18071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.970+0000 7f1d177fe640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1d18141e80 0x7f1d18071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3300/0 says I am v2:172.21.15.64:33518/0 (socket says 172.21.15.64:33518) 2023-11-24T08:09:05.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.970+0000 7f1d177fe640 1 -- 172.21.15.64:0/60739192 learned_addr learned my addr 172.21.15.64:0/60739192 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:05.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.970+0000 7f1d1cb61640 1 --2- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.971+0000 7f1d177fe640 1 -- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 msgr2=0x7f1d1806cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:05.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.971+0000 7f1d177fe640 1 --2- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1806cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.971+0000 7f1d177fe640 1 -- 172.21.15.64:0/60739192 >> v1:172.21.15.91:6789/0 conn(0x7f1d1815ed20 legacy=0x7f1d1815f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.971+0000 7f1d177fe640 1 -- 172.21.15.64:0/60739192 --> v2:172.21.15.91:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1806c1b0 con 0x7f1d18141e80 2023-11-24T08:09:05.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.971+0000 7f1d1cb61640 1 --2- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1806cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:09:05.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.972+0000 7f1d177fe640 1 --2- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d18141e80 0x7f1d18071110 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009870 tx=0x7f1d0c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3413db23371f8c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.973+0000 7f1d16ffd640 1 -- 172.21.15.64:0/60739192 <== mon.1 v2:172.21.15.91:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c030840 con 0x7f1d18141e80 2023-11-24T08:09:05.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.973+0000 7f1d16ffd640 1 -- 172.21.15.64:0/60739192 <== mon.1 v2:172.21.15.91:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c0309a0 con 0x7f1d18141e80 2023-11-24T08:09:05.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.973+0000 7f1d16ffd640 1 -- 172.21.15.64:0/60739192 <== mon.1 v2:172.21.15.91:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c030d30 con 0x7f1d18141e80 2023-11-24T08:09:05.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.974+0000 7f1d1d362640 1 -- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d18141e80 msgr2=0x7f1d18071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.974+0000 7f1d1d362640 1 --2- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d18141e80 0x7f1d18071110 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009870 tx=0x7f1d0c02b980 comp rx=0 tx=0).stop 2023-11-24T08:09:05.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.974+0000 7f1d1d362640 1 -- 172.21.15.64:0/60739192 shutdown_connections 2023-11-24T08:09:05.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.974+0000 7f1d1d362640 1 --2- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1806cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.974+0000 7f1d1d362640 1 --2- 172.21.15.64:0/60739192 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d18141e80 0x7f1d18071110 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.974+0000 7f1d1d362640 1 -- 172.21.15.64:0/60739192 >> 172.21.15.64:0/60739192 conn(0x7f1d180b0840 msgr2=0x7f1d180b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.974+0000 7f1d1d362640 1 -- 172.21.15.64:0/60739192 shutdown_connections 2023-11-24T08:09:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.974+0000 7f1d1d362640 1 -- 172.21.15.64:0/60739192 wait complete. 2023-11-24T08:09:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.976+0000 7f1d1d362640 1 Processor -- start 2023-11-24T08:09:05.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.976+0000 7f1d1d362640 1 -- start start 2023-11-24T08:09:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d1d362640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d1d362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d18141e80 0x7f1d18140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d1d362640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d1815ed20 0x7f1d180c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d1d362640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180c4800 con 0x7f1d18141e80 2023-11-24T08:09:05.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d1d362640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815d660 con 0x7f1d1815ed20 2023-11-24T08:09:05.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d1d362640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815d940 con 0x7f1d18071650 2023-11-24T08:09:05.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d18141e80 0x7f1d18140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d18141e80 0x7f1d18140490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58552/0 (socket says 172.21.15.64:58552) 2023-11-24T08:09:05.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.977+0000 7f1d177fe640 1 -- 172.21.15.64:0/2068025414 learned_addr learned my addr 172.21.15.64:0/2068025414 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d17fff640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d1cb61640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d1815ed20 0x7f1d180c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d177fe640 1 -- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 msgr2=0x7f1d1815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d177fe640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d177fe640 1 -- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d1815ed20 msgr2=0x7f1d180c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d177fe640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d1815ed20 0x7f1d180c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d177fe640 1 -- 172.21.15.64:0/2068025414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d10009a70 con 0x7f1d18141e80 2023-11-24T08:09:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d1cb61640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d1815ed20 0x7f1d180c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.978+0000 7f1d177fe640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d18141e80 0x7f1d18140490 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.979+0000 7f1d14ff9640 1 -- 172.21.15.64:0/2068025414 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c039c90 con 0x7f1d18141e80 2023-11-24T08:09:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.979+0000 7f1d14ff9640 1 -- 172.21.15.64:0/2068025414 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c039e10 con 0x7f1d18141e80 2023-11-24T08:09:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.979+0000 7f1d14ff9640 1 -- 172.21.15.64:0/2068025414 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c043640 con 0x7f1d18141e80 2023-11-24T08:09:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.979+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0c0094d0 con 0x7f1d18141e80 2023-11-24T08:09:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.979+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1815df00 con 0x7f1d18141e80 2023-11-24T08:09:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.980+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d18067d10 con 0x7f1d18141e80 2023-11-24T08:09:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.981+0000 7f1d14ff9640 1 -- 172.21.15.64:0/2068025414 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1d0c0437a0 con 0x7f1d18141e80 2023-11-24T08:09:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.982+0000 7f1d14ff9640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d0406b470 0x7f1d0406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.982+0000 7f1d14ff9640 1 -- 172.21.15.64:0/2068025414 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1d0c03e070 con 0x7f1d18141e80 2023-11-24T08:09:05.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.983+0000 7f1d17fff640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d0406b470 0x7f1d0406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.984+0000 7f1d17fff640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d0406b470 0x7f1d0406d930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1d18144dd0 tx=0x7f1d08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:05.987+0000 7f1d14ff9640 1 -- 172.21.15.64:0/2068025414 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d0c077900 con 0x7f1d18141e80 2023-11-24T08:09:06.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.182+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d18071ae0 con 0x7f1d0406b470 2023-11-24T08:09:06.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.184+0000 7f1d14ff9640 1 -- 172.21.15.64:0/2068025414 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1d08001c70 con 0x7f1d0406b470 2023-11-24T08:09:06.186 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T08:09:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d0406b470 msgr2=0x7f1d0406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d0406b470 0x7f1d0406d930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1d18144dd0 tx=0x7f1d08009040 comp rx=0 tx=0).stop 2023-11-24T08:09:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d18141e80 msgr2=0x7f1d18140490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d18141e80 0x7f1d18140490 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c004730 comp rx=0 tx=0).stop 2023-11-24T08:09:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 shutdown_connections 2023-11-24T08:09:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1d0406b470 0x7f1d0406d930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1d1815ed20 0x7f1d180c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d18141e80 0x7f1d18140490 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 --2- 172.21.15.64:0/2068025414 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1d18071650 0x7f1d1815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.186+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 >> 172.21.15.64:0/2068025414 conn(0x7f1d180b0840 msgr2=0x7f1d180afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.187+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 shutdown_connections 2023-11-24T08:09:06.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.187+0000 7f1d1d362640 1 -- 172.21.15.64:0/2068025414 wait complete. 2023-11-24T08:09:06.208 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T08:09:06.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:09:06.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.439+0000 7f8b77fff640 1 Processor -- start 2023-11-24T08:09:06.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.439+0000 7f8b77fff640 1 -- start start 2023-11-24T08:09:06.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.440+0000 7f8b77fff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8b7815f270 0x7f8b780cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.440+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 0x7f8b780cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.440+0000 7f8b77fff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7815f930 con 0x7f8b78144770 2023-11-24T08:09:06.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.440+0000 7f8b77fff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b780b43b0 con 0x7f8b7815f270 2023-11-24T08:09:06.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.440+0000 7f8b77fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780b3b10 con 0x7f8b7806d9e0 2023-11-24T08:09:06.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.441+0000 7f8b777fe640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 0x7f8b780cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.441+0000 7f8b767fc640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f8b7815f270 0x7f8b780cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.441+0000 7f8b76ffd640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f8b78144770 0x7f8b7815eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:57290/0 (socket says 172.21.15.64:57290) 2023-11-24T08:09:06.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.441+0000 7f8b76ffd640 1 -- 172.21.15.64:0/1783819531 learned_addr learned my addr 172.21.15.64:0/1783819531 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:06.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.442+0000 7f8b75ffb640 1 -- 172.21.15.64:0/1783819531 <== mon.1 v1:172.21.15.91:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995952648 0 0) 0x7f8b7815f930 con 0x7f8b78144770 2023-11-24T08:09:06.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.442+0000 7f8b777fe640 1 -- 172.21.15.64:0/1783819531 >> v2:172.21.15.91:3300/0 conn(0x7f8b7815f270 msgr2=0x7f8b780cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:06.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.442+0000 7f8b777fe640 1 --2- 172.21.15.64:0/1783819531 >> v2:172.21.15.91:3300/0 conn(0x7f8b7815f270 0x7f8b780cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.442+0000 7f8b777fe640 1 -- 172.21.15.64:0/1783819531 >> v1:172.21.15.91:6789/0 conn(0x7f8b78144770 legacy=0x7f8b7815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.442+0000 7f8b777fe640 1 -- 172.21.15.64:0/1783819531 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780b3df0 con 0x7f8b7806d9e0 2023-11-24T08:09:06.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.444+0000 7f8b777fe640 1 --2- 172.21.15.64:0/1783819531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 0x7f8b780cd9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009d90 tx=0x7f8b7002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9b44fe43340357 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:06.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.444+0000 7f8b75ffb640 1 -- 172.21.15.64:0/1783819531 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7002cc60 con 0x7f8b7806d9e0 2023-11-24T08:09:06.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.444+0000 7f8b75ffb640 1 -- 172.21.15.64:0/1783819531 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7002cdc0 con 0x7f8b7806d9e0 2023-11-24T08:09:06.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.444+0000 7f8b75ffb640 1 -- 172.21.15.64:0/1783819531 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70034660 con 0x7f8b7806d9e0 2023-11-24T08:09:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.445+0000 7f8b77fff640 1 -- 172.21.15.64:0/1783819531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 msgr2=0x7f8b780cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.445+0000 7f8b77fff640 1 --2- 172.21.15.64:0/1783819531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 0x7f8b780cd9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009d90 tx=0x7f8b7002beb0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.445+0000 7f8b77fff640 1 -- 172.21.15.64:0/1783819531 shutdown_connections 2023-11-24T08:09:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.445+0000 7f8b77fff640 1 --2- 172.21.15.64:0/1783819531 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 0x7f8b780cd9e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.445+0000 7f8b77fff640 1 --2- 172.21.15.64:0/1783819531 >> v2:172.21.15.91:3300/0 conn(0x7f8b7815f270 0x7f8b780cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.445+0000 7f8b77fff640 1 -- 172.21.15.64:0/1783819531 >> 172.21.15.64:0/1783819531 conn(0x7f8b780cec10 msgr2=0x7f8b780cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.445+0000 7f8b77fff640 1 -- 172.21.15.64:0/1783819531 shutdown_connections 2023-11-24T08:09:06.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.445+0000 7f8b77fff640 1 -- 172.21.15.64:0/1783819531 wait complete. 2023-11-24T08:09:06.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.447+0000 7f8b77fff640 1 Processor -- start 2023-11-24T08:09:06.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.447+0000 7f8b77fff640 1 -- start start 2023-11-24T08:09:06.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 0x7f8b7815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b78144770 0x7f8b7813d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b7815f270 0x7f8b78140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b77fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7815d550 con 0x7f8b78144770 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b77fff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7815d830 con 0x7f8b7815f270 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b77fff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7815db10 con 0x7f8b7806d9e0 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b767fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b78144770 0x7f8b7813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b767fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b78144770 0x7f8b7813d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58554/0 (socket says 172.21.15.64:58554) 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b767fc640 1 -- 172.21.15.64:0/2035465356 learned_addr learned my addr 172.21.15.64:0/2035465356 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:06.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b76ffd640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 0x7f8b7815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b777fe640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b7815f270 0x7f8b78140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b767fc640 1 -- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 msgr2=0x7f8b7815a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b767fc640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f8b7806d9e0 0x7f8b7815a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b767fc640 1 -- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b7815f270 msgr2=0x7f8b78140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.448+0000 7f8b767fc640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f8b7815f270 0x7f8b78140c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.449+0000 7f8b767fc640 1 -- 172.21.15.64:0/2035465356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70009a70 con 0x7f8b78144770 2023-11-24T08:09:06.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.449+0000 7f8b767fc640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b78144770 0x7f8b7813d010 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00e870 tx=0x7f8b6c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:06.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.449+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0094f0 con 0x7f8b78144770 2023-11-24T08:09:06.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.449+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c00c940 con 0x7f8b78144770 2023-11-24T08:09:06.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.449+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00ccf0 con 0x7f8b78144770 2023-11-24T08:09:06.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.449+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7815ddf0 con 0x7f8b78144770 2023-11-24T08:09:06.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.449+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7815adf0 con 0x7f8b78144770 2023-11-24T08:09:06.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.450+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b78144b50 con 0x7f8b78144770 2023-11-24T08:09:06.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.452+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8b6c013470 con 0x7f8b78144770 2023-11-24T08:09:06.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.453+0000 7f8b5ffff640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6406b470 0x7f8b6406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.453+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8b6c089fb0 con 0x7f8b78144770 2023-11-24T08:09:06.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.453+0000 7f8b5ffff640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f8b640712d0 0x7f8b640736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.453+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 --> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b64073da0 con 0x7f8b640712d0 2023-11-24T08:09:06.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.453+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b6c08f490 con 0x7f8b78144770 2023-11-24T08:09:06.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.453+0000 7f8b76ffd640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6406b470 0x7f8b6406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.453+0000 7f8b777fe640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f8b640712d0 0x7f8b640736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.454+0000 7f8b76ffd640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6406b470 0x7f8b6406d930 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8b68000f60 tx=0x7f8b68007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:06.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.454+0000 7f8b777fe640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f8b640712d0 0x7f8b640736f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:06.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.456+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 <== osd.4 v2:172.21.15.91:6821/3295720288 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8b64073da0 con 0x7f8b640712d0 2023-11-24T08:09:06.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.471+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 --> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8b7806c4b0 con 0x7f8b640712d0 2023-11-24T08:09:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.472+0000 7f8b5ffff640 1 -- 172.21.15.64:0/2035465356 <== osd.4 v2:172.21.15.91:6821/3295720288 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8b70034440 con 0x7f8b640712d0 2023-11-24T08:09:06.474 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:09:06.474 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:09:06.475 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:09:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f8b640712d0 msgr2=0x7f8b640736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6821/3295720288,v1:172.21.15.91:6823/3295720288] conn(0x7f8b640712d0 0x7f8b640736f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6406b470 msgr2=0x7f8b6406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f8b6406b470 0x7f8b6406d930 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8b68000f60 tx=0x7f8b68007b50 comp rx=0 tx=0).stop 2023-11-24T08:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b78144770 msgr2=0x7f8b7813d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 --2- 172.21.15.64:0/2035465356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b78144770 0x7f8b7813d010 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00e870 tx=0x7f8b6c00ed30 comp rx=0 tx=0).stop 2023-11-24T08:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b777fe640 1 -- 172.21.15.64:0/2035465356 reap_dead start 2023-11-24T08:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 shutdown_connections 2023-11-24T08:09:06.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 >> 172.21.15.64:0/2035465356 conn(0x7f8b780cec10 msgr2=0x7f8b780b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:06.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 shutdown_connections 2023-11-24T08:09:06.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.473+0000 7f8b77fff640 1 -- 172.21.15.64:0/2035465356 wait complete. 2023-11-24T08:09:06.494 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T08:09:06.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.717+0000 7f2c11d1d640 1 Processor -- start 2023-11-24T08:09:06.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.717+0000 7f2c11d1d640 1 -- start start 2023-11-24T08:09:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.719+0000 7f2c11d1d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c0719c0 0x7f2c0c0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.719+0000 7f2c11d1d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c0ad510 con 0x7f2c0c06d520 2023-11-24T08:09:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.719+0000 7f2c11d1d640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c0acc50 con 0x7f2c0c161930 2023-11-24T08:09:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.719+0000 7f2c11d1d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c071e60 con 0x7f2c0c0719c0 2023-11-24T08:09:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.719+0000 7f2c0bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2c0c06d520 0x7f2c0c0b2080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38274/0 (socket says 172.21.15.64:38274) 2023-11-24T08:09:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.719+0000 7f2c0bfff640 1 -- 172.21.15.64:0/446857386 learned_addr learned my addr 172.21.15.64:0/446857386 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.719+0000 7f2c03fff640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c0719c0 0x7f2c0c0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604441904 0 0) 0x7f2c0c0ad510 con 0x7f2c0c06d520 2023-11-24T08:09:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bf8003520 con 0x7f2c0c06d520 2023-11-24T08:09:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1851109774 0 0) 0x7f2bf8003520 con 0x7f2c0c06d520 2023-11-24T08:09:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c0719c0 msgr2=0x7f2c0c0ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c0719c0 0x7f2c0c0ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 >> v1:172.21.15.91:6789/0 conn(0x7f2c0c161930 legacy=0x7f2c0c0ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0acec0 con 0x7f2c0c06d520 2023-11-24T08:09:06.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2c040019f0 con 0x7f2c0c06d520 2023-11-24T08:09:06.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 >> v1:172.21.15.64:6789/0 conn(0x7f2c0c06d520 legacy=0x7f2c0c0b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c0719c0 0x7f2bf80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2bf80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bf8004d00 0x7f2bf8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c071e60 con 0x7f2bf8003f70 2023-11-24T08:09:06.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf800bac0 con 0x7f2bf8004d00 2023-11-24T08:09:06.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf800bc80 con 0x7f2c0c0719c0 2023-11-24T08:09:06.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.720+0000 7f2c03fff640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2bf80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.721+0000 7f2c0b7fe640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c0719c0 0x7f2bf80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.721+0000 7f2c03fff640 1 -- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c0719c0 msgr2=0x7f2bf80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.721+0000 7f2c03fff640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c0719c0 0x7f2bf80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.721+0000 7f2c03fff640 1 -- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bf8004d00 msgr2=0x7f2bf8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.721+0000 7f2c03fff640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2bf8004d00 0x7f2bf8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.721+0000 7f2c03fff640 1 -- 172.21.15.64:0/446857386 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf8003520 con 0x7f2bf8003f70 2023-11-24T08:09:06.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.722+0000 7f2c03fff640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2bf80047c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009ac0 tx=0x7f2bf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614906987d7d9d9e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:06.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.722+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf402c600 con 0x7f2bf8003f70 2023-11-24T08:09:06.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf402c760 con 0x7f2bf8003f70 2023-11-24T08:09:06.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c0affd640 1 -- 172.21.15.64:0/446857386 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf402caf0 con 0x7f2bf8003f70 2023-11-24T08:09:06.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c11d1d640 1 -- 172.21.15.64:0/446857386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 msgr2=0x7f2bf80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c11d1d640 1 --2- 172.21.15.64:0/446857386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2bf80047c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009ac0 tx=0x7f2bf80057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c0bfff640 1 -- 172.21.15.64:0/446857386 reap_dead start 2023-11-24T08:09:06.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c11d1d640 1 -- 172.21.15.64:0/446857386 shutdown_connections 2023-11-24T08:09:06.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c11d1d640 1 -- 172.21.15.64:0/446857386 >> 172.21.15.64:0/446857386 conn(0x7f2c0c0b0d80 msgr2=0x7f2c0c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:06.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c11d1d640 1 -- 172.21.15.64:0/446857386 shutdown_connections 2023-11-24T08:09:06.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.723+0000 7f2c11d1d640 1 -- 172.21.15.64:0/446857386 wait complete. 2023-11-24T08:09:06.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.725+0000 7f2c11d1d640 1 Processor -- start 2023-11-24T08:09:06.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.725+0000 7f2c11d1d640 1 -- start start 2023-11-24T08:09:06.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c11d1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2c0c1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c11d1d640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c0c0af190 0x7f2c0c0af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c11d1d640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c151ca0 0x7f2c0c152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c11d1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0afb50 con 0x7f2bf8003f70 2023-11-24T08:09:06.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c11d1d640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c151810 con 0x7f2c0c0af190 2023-11-24T08:09:06.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c11d1d640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c150c00 con 0x7f2c0c151ca0 2023-11-24T08:09:06.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2c0c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2c0c1502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58576/0 (socket says 172.21.15.64:58576) 2023-11-24T08:09:06.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.726+0000 7f2c0b7fe640 1 -- 172.21.15.64:0/3993975825 learned_addr learned my addr 172.21.15.64:0/3993975825 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:06.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.727+0000 7f2c0bfff640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c151ca0 0x7f2c0c152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.727+0000 7f2c03fff640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c0c0af190 0x7f2c0c0af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.727+0000 7f2c0b7fe640 1 -- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c151ca0 msgr2=0x7f2c0c152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.727+0000 7f2c0b7fe640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c151ca0 0x7f2c0c152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.727+0000 7f2c0b7fe640 1 -- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c0c0af190 msgr2=0x7f2c0c0af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.727+0000 7f2c0b7fe640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c0c0af190 0x7f2c0c0af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.727+0000 7f2c0b7fe640 1 -- 172.21.15.64:0/3993975825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c150ee0 con 0x7f2bf8003f70 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.727+0000 7f2c0b7fe640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2c0c1502b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00e550 tx=0x7f2bfc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.728+0000 7f2c08ff9640 1 -- 172.21.15.64:0/3993975825 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0052e0 con 0x7f2bf8003f70 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.728+0000 7f2c08ff9640 1 -- 172.21.15.64:0/3993975825 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc005440 con 0x7f2bf8003f70 2023-11-24T08:09:06.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.728+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c1511c0 con 0x7f2bf8003f70 2023-11-24T08:09:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.728+0000 7f2c08ff9640 1 -- 172.21.15.64:0/3993975825 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc003810 con 0x7f2bf8003f70 2023-11-24T08:09:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.728+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c1507f0 con 0x7f2bf8003f70 2023-11-24T08:09:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.729+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c074770 con 0x7f2bf8003f70 2023-11-24T08:09:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.730+0000 7f2c08ff9640 1 -- 172.21.15.64:0/3993975825 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2bfc003970 con 0x7f2bf8003f70 2023-11-24T08:09:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.731+0000 7f2c08ff9640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2bec06b470 0x7f2bec06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.731+0000 7f2c08ff9640 1 -- 172.21.15.64:0/3993975825 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2bfc089c70 con 0x7f2bf8003f70 2023-11-24T08:09:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.732+0000 7f2c03fff640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2bec06b470 0x7f2bec06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.733+0000 7f2c03fff640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2bec06b470 0x7f2bec06d930 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4036000 tx=0x7f2bf4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:06.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.735+0000 7f2c08ff9640 1 -- 172.21.15.64:0/3993975825 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bfc0546a0 con 0x7f2bf8003f70 2023-11-24T08:09:06.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.932+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c0c06e0e0 con 0x7f2bec06b470 2023-11-24T08:09:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.934+0000 7f2c08ff9640 1 -- 172.21.15.64:0/3993975825 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2c0c06e0e0 con 0x7f2bec06b470 2023-11-24T08:09:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T08:09:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.936+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2bec06b470 msgr2=0x7f2bec06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.936+0000 7f2c11d1d640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2bec06b470 0x7f2bec06d930 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4036000 tx=0x7f2bf4030000 comp rx=0 tx=0).stop 2023-11-24T08:09:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.936+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 msgr2=0x7f2c0c1502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.936+0000 7f2c11d1d640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2c0c1502b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00e550 tx=0x7f2bfc00ea10 comp rx=0 tx=0).stop 2023-11-24T08:09:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.936+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 shutdown_connections 2023-11-24T08:09:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.936+0000 7f2c11d1d640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2bec06b470 0x7f2bec06d930 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.937+0000 7f2c11d1d640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2c0c151ca0 0x7f2c0c152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.937+0000 7f2c11d1d640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2c0c0af190 0x7f2c0c0af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.937+0000 7f2c11d1d640 1 --2- 172.21.15.64:0/3993975825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8003f70 0x7f2c0c1502b0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.937+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 >> 172.21.15.64:0/3993975825 conn(0x7f2c0c0b0d80 msgr2=0x7f2c0c162070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.937+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 shutdown_connections 2023-11-24T08:09:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:06.937+0000 7f2c11d1d640 1 -- 172.21.15.64:0/3993975825 wait complete. 2023-11-24T08:09:06.958 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T08:09:06.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:09:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.192+0000 7f2a4e5b3640 1 Processor -- start 2023-11-24T08:09:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.192+0000 7f2a4e5b3640 1 -- start start 2023-11-24T08:09:07.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.194+0000 7f2a4e5b3640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f2a4815efa0 0x7f2a48161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.194+0000 7f2a4e5b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a480b3080 0x7f2a4806c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.194+0000 7f2a4e5b3640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4806c9a0 con 0x7f2a4806da40 2023-11-24T08:09:07.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.194+0000 7f2a4e5b3640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4806cc80 con 0x7f2a4815efa0 2023-11-24T08:09:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.194+0000 7f2a4e5b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4806cf60 con 0x7f2a480b3080 2023-11-24T08:09:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.194+0000 7f2a4cdb0640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2a4806da40 0x7f2a480b2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57372/0 (socket says 172.21.15.64:57372) 2023-11-24T08:09:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.194+0000 7f2a4cdb0640 1 -- 172.21.15.64:0/3791484759 learned_addr learned my addr 172.21.15.64:0/3791484759 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a4ddb2640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a480b3080 0x7f2a4806c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a4d5b1640 1 --2- 172.21.15.64:0/3791484759 >> v2:172.21.15.91:3300/0 conn(0x7f2a4815efa0 0x7f2a48161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150566550 0 0) 0x7f2a4806c9a0 con 0x7f2a4806da40 2023-11-24T08:09:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a34003520 con 0x7f2a4806da40 2023-11-24T08:09:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208800300 0 0) 0x7f2a34003520 con 0x7f2a4806da40 2023-11-24T08:09:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a480b3080 msgr2=0x7f2a4806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a480b3080 0x7f2a4806c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 >> v2:172.21.15.91:3300/0 conn(0x7f2a4815efa0 msgr2=0x7f2a48161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 --2- 172.21.15.64:0/3791484759 >> v2:172.21.15.91:3300/0 conn(0x7f2a4815efa0 0x7f2a48161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4806d240 con 0x7f2a4806da40 2023-11-24T08:09:07.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f2a38001670 con 0x7f2a4806da40 2023-11-24T08:09:07.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 >> v1:172.21.15.64:6789/0 conn(0x7f2a4806da40 legacy=0x7f2a480b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a340039e0 0x7f2a34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.195+0000 7f2a3ffff640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a340047c0 0x7f2a34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a3ffff640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a480b3080 0x7f2a34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4806cc80 con 0x7f2a340047c0 2023-11-24T08:09:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4806cf60 con 0x7f2a340039e0 2023-11-24T08:09:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3400bb10 con 0x7f2a480b3080 2023-11-24T08:09:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a4cdb0640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a340047c0 0x7f2a34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a4d5b1640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a340039e0 0x7f2a34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a4cdb0640 1 -- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a480b3080 msgr2=0x7f2a34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a4cdb0640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a480b3080 0x7f2a34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a4cdb0640 1 -- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a340039e0 msgr2=0x7f2a34003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a4cdb0640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a340039e0 0x7f2a34003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.196+0000 7f2a4cdb0640 1 -- 172.21.15.64:0/3791484759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34003520 con 0x7f2a340047c0 2023-11-24T08:09:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.197+0000 7f2a4cdb0640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a340047c0 0x7f2a34004c40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001eb0 tx=0x7f2a340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3868eccf7f835fa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38005ad0 con 0x7f2a340047c0 2023-11-24T08:09:07.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a38005c30 con 0x7f2a340047c0 2023-11-24T08:09:07.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a3ffff640 1 -- 172.21.15.64:0/3791484759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38007070 con 0x7f2a340047c0 2023-11-24T08:09:07.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3791484759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a340047c0 msgr2=0x7f2a34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a4e5b3640 1 --2- 172.21.15.64:0/3791484759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a340047c0 0x7f2a34004c40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001eb0 tx=0x7f2a340057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a4ddb2640 1 -- 172.21.15.64:0/3791484759 reap_dead start 2023-11-24T08:09:07.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3791484759 shutdown_connections 2023-11-24T08:09:07.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3791484759 >> 172.21.15.64:0/3791484759 conn(0x7f2a480cec10 msgr2=0x7f2a480cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:07.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.198+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3791484759 shutdown_connections 2023-11-24T08:09:07.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.199+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3791484759 wait complete. 2023-11-24T08:09:07.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.200+0000 7f2a4e5b3640 1 Processor -- start 2023-11-24T08:09:07.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.201+0000 7f2a4e5b3640 1 -- start start 2023-11-24T08:09:07.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.201+0000 7f2a4e5b3640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a340047c0 0x7f2a48159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.201+0000 7f2a4e5b3640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a4814ce30 0x7f2a48158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.201+0000 7f2a4e5b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a480b0970 0x7f2a480b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.201+0000 7f2a4e5b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480b1360 con 0x7f2a480b0970 2023-11-24T08:09:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.201+0000 7f2a4e5b3640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48157f20 con 0x7f2a340047c0 2023-11-24T08:09:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.201+0000 7f2a4e5b3640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48158200 con 0x7f2a4814ce30 2023-11-24T08:09:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4ddb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a480b0970 0x7f2a480b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4ddb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a480b0970 0x7f2a480b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41326/0 (socket says 172.21.15.64:41326) 2023-11-24T08:09:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4ddb2640 1 -- 172.21.15.64:0/3934335548 learned_addr learned my addr 172.21.15.64:0/3934335548 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4d5b1640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a340047c0 0x7f2a48159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4cdb0640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a4814ce30 0x7f2a48158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4ddb2640 1 -- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a4814ce30 msgr2=0x7f2a48158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4ddb2640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a4814ce30 0x7f2a48158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4ddb2640 1 -- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a340047c0 msgr2=0x7f2a48159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4ddb2640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f2a340047c0 0x7f2a48159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4ddb2640 1 -- 172.21.15.64:0/3934335548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a481584e0 con 0x7f2a480b0970 2023-11-24T08:09:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.202+0000 7f2a4cdb0640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f2a4814ce30 0x7f2a48158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:07.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.203+0000 7f2a4ddb2640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a480b0970 0x7f2a480b0e20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2a4806e660 tx=0x7f2a4000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.203+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40010040 con 0x7f2a480b0970 2023-11-24T08:09:07.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.203+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4000bce0 con 0x7f2a480b0970 2023-11-24T08:09:07.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.203+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40014640 con 0x7f2a480b0970 2023-11-24T08:09:07.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.203+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a481587c0 con 0x7f2a480b0970 2023-11-24T08:09:07.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.203+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a48159500 con 0x7f2a480b0970 2023-11-24T08:09:07.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.204+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a48158370 con 0x7f2a480b0970 2023-11-24T08:09:07.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.205+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2a400147a0 con 0x7f2a480b0970 2023-11-24T08:09:07.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.206+0000 7f2a3dffb640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a2c06b4a0 0x7f2a2c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.206+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a4008a6b0 con 0x7f2a480b0970 2023-11-24T08:09:07.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.207+0000 7f2a3dffb640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7f2a2c071300 0x7f2a2c073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.207+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 --> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a2c073dd0 con 0x7f2a2c071300 2023-11-24T08:09:07.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.207+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a4008cb70 con 0x7f2a480b0970 2023-11-24T08:09:07.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.207+0000 7f2a4d5b1640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a2c06b4a0 0x7f2a2c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.207+0000 7f2a4cdb0640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7f2a2c071300 0x7f2a2c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.208+0000 7f2a4d5b1640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a2c06b4a0 0x7f2a2c06d960 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2a480b1eb0 tx=0x7f2a44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.208+0000 7f2a4cdb0640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7f2a2c071300 0x7f2a2c073720 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.211+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 <== osd.5 v2:172.21.15.91:6816/3653724610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2a2c073dd0 con 0x7f2a2c071300 2023-11-24T08:09:07.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.226+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 --> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2a480b4d20 con 0x7f2a2c071300 2023-11-24T08:09:07.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.227+0000 7f2a3dffb640 1 -- 172.21.15.64:0/3934335548 <== osd.5 v2:172.21.15.91:6816/3653724610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2a38001550 con 0x7f2a2c071300 2023-11-24T08:09:07.229 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:09:07.229 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:09:07.229 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:09:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.227+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7f2a2c071300 msgr2=0x7f2a2c073720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.227+0000 7f2a4e5b3640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6816/3653724610,v1:172.21.15.91:6817/3653724610] conn(0x7f2a2c071300 0x7f2a2c073720 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a2c06b4a0 msgr2=0x7f2a2c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4e5b3640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f2a2c06b4a0 0x7f2a2c06d960 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2a480b1eb0 tx=0x7f2a44008040 comp rx=0 tx=0).stop 2023-11-24T08:09:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a480b0970 msgr2=0x7f2a480b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4e5b3640 1 --2- 172.21.15.64:0/3934335548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a480b0970 0x7f2a480b0e20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2a4806e660 tx=0x7f2a4000b480 comp rx=0 tx=0).stop 2023-11-24T08:09:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4ddb2640 1 -- 172.21.15.64:0/3934335548 reap_dead start 2023-11-24T08:09:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 shutdown_connections 2023-11-24T08:09:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 >> 172.21.15.64:0/3934335548 conn(0x7f2a480cec10 msgr2=0x7f2a4815f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:07.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 shutdown_connections 2023-11-24T08:09:07.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.228+0000 7f2a4e5b3640 1 -- 172.21.15.64:0/3934335548 wait complete. 2023-11-24T08:09:07.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T08:09:07.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.473+0000 7fbd1c8ba640 1 Processor -- start 2023-11-24T08:09:07.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.474+0000 7fbd1c8ba640 1 -- start start 2023-11-24T08:09:07.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.475+0000 7fbd1c8ba640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd1815ed20 0x7fbd18141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.476+0000 7fbd1c8ba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd18071820 0x7fbd1806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.476+0000 7fbd1c8ba640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1806cae0 con 0x7fbd1815f100 2023-11-24T08:09:07.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.476+0000 7fbd1c8ba640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd18071e40 con 0x7fbd18071820 2023-11-24T08:09:07.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.476+0000 7fbd1c8ba640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1806c1b0 con 0x7fbd1815ed20 2023-11-24T08:09:07.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.476+0000 7fbd1759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd18071820 0x7fbd1806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.476+0000 7fbd1759e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd18071820 0x7fbd1806cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41328/0 (socket says 172.21.15.64:41328) 2023-11-24T08:09:07.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.476+0000 7fbd1759e640 1 -- 172.21.15.64:0/577065271 learned_addr learned my addr 172.21.15.64:0/577065271 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:07.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.476+0000 7fbd16d9d640 1 --2- 172.21.15.64:0/577065271 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd1815ed20 0x7fbd18141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.477+0000 7fbd1759e640 1 -- 172.21.15.64:0/577065271 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd1815ed20 msgr2=0x7fbd18141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.477+0000 7fbd1759e640 1 --2- 172.21.15.64:0/577065271 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd1815ed20 0x7fbd18141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.477+0000 7fbd1759e640 1 -- 172.21.15.64:0/577065271 >> v1:172.21.15.91:6789/0 conn(0x7fbd1815f100 legacy=0x7fbd18071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.477+0000 7fbd1759e640 1 -- 172.21.15.64:0/577065271 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1806c490 con 0x7fbd18071820 2023-11-24T08:09:07.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.478+0000 7fbd1759e640 1 --2- 172.21.15.64:0/577065271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd18071820 0x7fbd1806cf70 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009e50 tx=0x7fbd1002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a08fb3b5f28d087 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1659c640 1 -- 172.21.15.64:0/577065271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1002cb00 con 0x7fbd18071820 2023-11-24T08:09:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1659c640 1 -- 172.21.15.64:0/577065271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1002cc60 con 0x7fbd18071820 2023-11-24T08:09:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1659c640 1 -- 172.21.15.64:0/577065271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100343f0 con 0x7fbd18071820 2023-11-24T08:09:07.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/577065271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd18071820 msgr2=0x7fbd1806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/577065271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd18071820 0x7fbd1806cf70 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009e50 tx=0x7fbd1002bf60 comp rx=0 tx=0).stop 2023-11-24T08:09:07.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/577065271 shutdown_connections 2023-11-24T08:09:07.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/577065271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd18071820 0x7fbd1806cf70 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/577065271 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd1815ed20 0x7fbd18141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/577065271 >> 172.21.15.64:0/577065271 conn(0x7fbd180b0840 msgr2=0x7fbd180b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:07.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/577065271 shutdown_connections 2023-11-24T08:09:07.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.479+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/577065271 wait complete. 2023-11-24T08:09:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.481+0000 7fbd1c8ba640 1 Processor -- start 2023-11-24T08:09:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.481+0000 7fbd1c8ba640 1 -- start start 2023-11-24T08:09:07.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.482+0000 7fbd1c8ba640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd18071820 0x7fbd1815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.482+0000 7fbd1c8ba640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd1815ed20 0x7fbd180b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.482+0000 7fbd1c8ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd1815f100 0x7fbd180ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.482+0000 7fbd1c8ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180ac870 con 0x7fbd1815f100 2023-11-24T08:09:07.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.482+0000 7fbd1c8ba640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180aca80 con 0x7fbd1815ed20 2023-11-24T08:09:07.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.482+0000 7fbd1c8ba640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180acd60 con 0x7fbd18071820 2023-11-24T08:09:07.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd1759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd1815f100 0x7fbd180ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd1759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd1815f100 0x7fbd180ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41330/0 (socket says 172.21.15.64:41330) 2023-11-24T08:09:07.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd1759e640 1 -- 172.21.15.64:0/522896291 learned_addr learned my addr 172.21.15.64:0/522896291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:07.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd16d9d640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd18071820 0x7fbd1815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd1759e640 1 -- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd18071820 msgr2=0x7fbd1815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd1759e640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd18071820 0x7fbd1815d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd1759e640 1 -- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd1815ed20 msgr2=0x7fbd180b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd0ffff640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd1815ed20 0x7fbd180b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd1759e640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd1815ed20 0x7fbd180b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd1759e640 1 -- 172.21.15.64:0/522896291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd10009b30 con 0x7fbd1815f100 2023-11-24T08:09:07.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.483+0000 7fbd0ffff640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd1815ed20 0x7fbd180b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:07.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.484+0000 7fbd1759e640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd1815f100 0x7fbd180ad840 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009e50 tx=0x7fbd10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.484+0000 7fbd0f7fe640 1 -- 172.21.15.64:0/522896291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10009510 con 0x7fbd1815f100 2023-11-24T08:09:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.484+0000 7fbd0f7fe640 1 -- 172.21.15.64:0/522896291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1003e450 con 0x7fbd1815f100 2023-11-24T08:09:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.484+0000 7fbd0f7fe640 1 -- 172.21.15.64:0/522896291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1003e800 con 0x7fbd1815f100 2023-11-24T08:09:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.484+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd180ad040 con 0x7fbd1815f100 2023-11-24T08:09:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.484+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1815d8e0 con 0x7fbd1815f100 2023-11-24T08:09:07.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.486+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd180ad560 con 0x7fbd1815f100 2023-11-24T08:09:07.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.487+0000 7fbd0f7fe640 1 -- 172.21.15.64:0/522896291 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fbd1003e960 con 0x7fbd1815f100 2023-11-24T08:09:07.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.488+0000 7fbd0f7fe640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbcf806b470 0x7fbcf806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.488+0000 7fbd0f7fe640 1 -- 172.21.15.64:0/522896291 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbd100acc00 con 0x7fbd1815f100 2023-11-24T08:09:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.489+0000 7fbd16d9d640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbcf806b470 0x7fbcf806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.490+0000 7fbd16d9d640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbcf806b470 0x7fbcf806d930 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbd08006fd0 tx=0x7fbd08008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.493+0000 7fbd0f7fe640 1 -- 172.21.15.64:0/522896291 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd180ad560 con 0x7fbd1815f100 2023-11-24T08:09:07.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.687+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd1815f560 con 0x7fbcf806b470 2023-11-24T08:09:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.688+0000 7fbd0f7fe640 1 -- 172.21.15.64:0/522896291 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbd0800b200 con 0x7fbcf806b470 2023-11-24T08:09:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T08:09:07.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbcf806b470 msgr2=0x7fbcf806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbcf806b470 0x7fbcf806d930 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbd08006fd0 tx=0x7fbd08008040 comp rx=0 tx=0).stop 2023-11-24T08:09:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd1815f100 msgr2=0x7fbd180ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd1815f100 0x7fbd180ad840 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009e50 tx=0x7fbd10004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 shutdown_connections 2023-11-24T08:09:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fbcf806b470 0x7fbcf806d930 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd1815f100 0x7fbd180ad840 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fbd1815ed20 0x7fbd180b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 --2- 172.21.15.64:0/522896291 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fbd18071820 0x7fbd1815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 >> 172.21.15.64:0/522896291 conn(0x7fbd180b0840 msgr2=0x7fbd18141ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:07.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.690+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 shutdown_connections 2023-11-24T08:09:07.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.691+0000 7fbd1c8ba640 1 -- 172.21.15.64:0/522896291 wait complete. 2023-11-24T08:09:07.711 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T08:09:07.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:09:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.939+0000 7fc85e91b640 1 Processor -- start 2023-11-24T08:09:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.939+0000 7fc85e91b640 1 -- start start 2023-11-24T08:09:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.941+0000 7fc85e91b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc85806d9e0 0x7fc858165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.941+0000 7fc85e91b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85815f930 con 0x7fc858144770 2023-11-24T08:09:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.941+0000 7fc85e91b640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8580b3b10 con 0x7fc85815f270 2023-11-24T08:09:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.941+0000 7fc85e91b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580b43b0 con 0x7fc85806d9e0 2023-11-24T08:09:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.941+0000 7fc85d919640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc858144770 0x7fc85815eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57384/0 (socket says 172.21.15.64:57384) 2023-11-24T08:09:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.941+0000 7fc85d919640 1 -- 172.21.15.64:0/2850996273 learned_addr learned my addr 172.21.15.64:0/2850996273 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:07.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85e11a640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc85806d9e0 0x7fc858165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368001254 0 0) 0x7fc85815f930 con 0x7fc858144770 2023-11-24T08:09:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc844003520 con 0x7fc858144770 2023-11-24T08:09:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2033934845 0 0) 0x7fc844003520 con 0x7fc858144770 2023-11-24T08:09:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc85806d9e0 msgr2=0x7fc858165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc85806d9e0 0x7fc858165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 >> v1:172.21.15.91:6789/0 conn(0x7fc85815f270 legacy=0x7fc8580cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8580b3df0 con 0x7fc858144770 2023-11-24T08:09:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fc854001670 con 0x7fc858144770 2023-11-24T08:09:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 >> v1:172.21.15.64:6789/0 conn(0x7fc858144770 legacy=0x7fc85815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc85806d9e0 0x7fc844003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc844003f90 0x7fc8440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc844004d00 0x7fc844005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580b43b0 con 0x7fc844003f90 2023-11-24T08:09:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84400bae0 con 0x7fc844004d00 2023-11-24T08:09:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.942+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84400bca0 con 0x7fc85806d9e0 2023-11-24T08:09:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.943+0000 7fc85d118640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc844003f90 0x7fc8440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.943+0000 7fc85d919640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc85806d9e0 0x7fc844003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.943+0000 7fc85e11a640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc844004d00 0x7fc844005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.943+0000 7fc85d118640 1 -- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc85806d9e0 msgr2=0x7fc844003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.943+0000 7fc85d118640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc85806d9e0 0x7fc844003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.943+0000 7fc85d118640 1 -- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc844004d00 msgr2=0x7fc844005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.943+0000 7fc85d118640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc844004d00 0x7fc844005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.943+0000 7fc85d118640 1 -- 172.21.15.64:0/2850996273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc844003520 con 0x7fc844003f90 2023-11-24T08:09:07.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.944+0000 7fc85d118640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc844003f90 0x7fc8440047c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc848001750 tx=0x7fc8440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac88adb8d06aa0a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.944+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8480052d0 con 0x7fc844003f90 2023-11-24T08:09:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84802e050 con 0x7fc844003f90 2023-11-24T08:09:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85c917640 1 -- 172.21.15.64:0/2850996273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848003830 con 0x7fc844003f90 2023-11-24T08:09:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85e91b640 1 -- 172.21.15.64:0/2850996273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc844003f90 msgr2=0x7fc8440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85e91b640 1 --2- 172.21.15.64:0/2850996273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc844003f90 0x7fc8440047c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc848001750 tx=0x7fc8440057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85e11a640 1 -- 172.21.15.64:0/2850996273 reap_dead start 2023-11-24T08:09:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85e91b640 1 -- 172.21.15.64:0/2850996273 shutdown_connections 2023-11-24T08:09:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85e91b640 1 -- 172.21.15.64:0/2850996273 >> 172.21.15.64:0/2850996273 conn(0x7fc8580cec10 msgr2=0x7fc8580cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85e91b640 1 -- 172.21.15.64:0/2850996273 shutdown_connections 2023-11-24T08:09:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.945+0000 7fc85e91b640 1 -- 172.21.15.64:0/2850996273 wait complete. 2023-11-24T08:09:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.947+0000 7fc85e91b640 1 Processor -- start 2023-11-24T08:09:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.947+0000 7fc85e91b640 1 -- start start 2023-11-24T08:09:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.948+0000 7fc85e91b640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc844003f90 0x7fc85815d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.948+0000 7fc85e91b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc85813a800 0x7fc8580c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.948+0000 7fc85e91b640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8580c4a00 0x7fc8581f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.948+0000 7fc85e91b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580c4e80 con 0x7fc85813a800 2023-11-24T08:09:07.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.948+0000 7fc85e91b640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8581f7330 con 0x7fc844003f90 2023-11-24T08:09:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.948+0000 7fc85e91b640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc858139a40 con 0x7fc8580c4a00 2023-11-24T08:09:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d118640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc85813a800 0x7fc8580c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d118640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc85813a800 0x7fc8580c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41358/0 (socket says 172.21.15.64:41358) 2023-11-24T08:09:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d118640 1 -- 172.21.15.64:0/1843677859 learned_addr learned my addr 172.21.15.64:0/1843677859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85e11a640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8580c4a00 0x7fc8581f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d919640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc844003f90 0x7fc85815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d118640 1 -- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8580c4a00 msgr2=0x7fc8581f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d118640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8580c4a00 0x7fc8581f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d118640 1 -- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc844003f90 msgr2=0x7fc85815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d118640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fc844003f90 0x7fc85815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85d118640 1 -- 172.21.15.64:0/1843677859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858139d20 con 0x7fc85813a800 2023-11-24T08:09:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.949+0000 7fc85e11a640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fc8580c4a00 0x7fc8581f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.950+0000 7fc85d118640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc85813a800 0x7fc8580c44c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc848008fd0 tx=0x7fc848001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.950+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848007a30 con 0x7fc85813a800 2023-11-24T08:09:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.950+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8480069e0 con 0x7fc85813a800 2023-11-24T08:09:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.950+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848006d90 con 0x7fc85813a800 2023-11-24T08:09:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.950+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85813a000 con 0x7fc85813a800 2023-11-24T08:09:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.950+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85815de90 con 0x7fc85813a800 2023-11-24T08:09:07.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.951+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc85815f100 con 0x7fc85813a800 2023-11-24T08:09:07.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.952+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fc8480035f0 con 0x7fc85813a800 2023-11-24T08:09:07.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.953+0000 7fc84e7fc640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc83c06b4a0 0x7fc83c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.953+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc8480abe10 con 0x7fc85813a800 2023-11-24T08:09:07.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.953+0000 7fc84e7fc640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fc83c071300 0x7fc83c073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:07.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.953+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 --> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc83c073dd0 con 0x7fc83c071300 2023-11-24T08:09:07.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.953+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8480ae250 con 0x7fc85813a800 2023-11-24T08:09:07.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.954+0000 7fc85d919640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc83c06b4a0 0x7fc83c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.954+0000 7fc85e11a640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fc83c071300 0x7fc83c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:07.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.955+0000 7fc85d919640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc83c06b4a0 0x7fc83c06d960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc854001050 tx=0x7fc854001130 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.955+0000 7fc85e11a640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fc83c071300 0x7fc83c073720 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:07.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.957+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 <== osd.6 v2:172.21.15.91:6800/3533292378 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc83c073dd0 con 0x7fc83c071300 2023-11-24T08:09:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.972+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 --> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc858144b50 con 0x7fc83c071300 2023-11-24T08:09:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.973+0000 7fc84e7fc640 1 -- 172.21.15.64:0/1843677859 <== osd.6 v2:172.21.15.91:6800/3533292378 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc850009ce0 con 0x7fc83c071300 2023-11-24T08:09:07.975 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:09:07.975 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:09:07.975 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:09:07.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.973+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fc83c071300 msgr2=0x7fc83c073720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.973+0000 7fc85e91b640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6800/3533292378,v1:172.21.15.91:6801/3533292378] conn(0x7fc83c071300 0x7fc83c073720 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:07.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.973+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc83c06b4a0 msgr2=0x7fc83c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.973+0000 7fc85e91b640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fc83c06b4a0 0x7fc83c06d960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc854001050 tx=0x7fc854001130 comp rx=0 tx=0).stop 2023-11-24T08:09:07.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.973+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc85813a800 msgr2=0x7fc8580c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:07.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.973+0000 7fc85e91b640 1 --2- 172.21.15.64:0/1843677859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc85813a800 0x7fc8580c44c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc848008fd0 tx=0x7fc848001120 comp rx=0 tx=0).stop 2023-11-24T08:09:07.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.973+0000 7fc85e11a640 1 -- 172.21.15.64:0/1843677859 reap_dead start 2023-11-24T08:09:07.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.974+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 shutdown_connections 2023-11-24T08:09:07.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.974+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 >> 172.21.15.64:0/1843677859 conn(0x7fc8580cec10 msgr2=0x7fc858143860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:07.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.974+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 shutdown_connections 2023-11-24T08:09:07.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:07.974+0000 7fc85e91b640 1 -- 172.21.15.64:0/1843677859 wait complete. 2023-11-24T08:09:07.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T08:09:08.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.211+0000 7fb98ba56640 1 Processor -- start 2023-11-24T08:09:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.211+0000 7fb98ba56640 1 -- start start 2023-11-24T08:09:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.212+0000 7fb98ba56640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fb98415a100 0x7fb9840b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.212+0000 7fb98ba56640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98413da30 con 0x7fb9840abff0 2023-11-24T08:09:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.212+0000 7fb98ba56640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98406e340 con 0x7fb9840ae7e0 2023-11-24T08:09:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.212+0000 7fb98ba56640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9840ac840 con 0x7fb98415a100 2023-11-24T08:09:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.212+0000 7fb988ff1640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb9840abff0 0x7fb98406d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57390/0 (socket says 172.21.15.64:57390) 2023-11-24T08:09:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.212+0000 7fb988ff1640 1 -- 172.21.15.64:0/4126032199 learned_addr learned my addr 172.21.15.64:0/4126032199 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:08.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204217109 0 0) 0x7fb98413da30 con 0x7fb9840abff0 2023-11-24T08:09:08.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb970003520 con 0x7fb9840abff0 2023-11-24T08:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb9897f2640 1 --2- 172.21.15.64:0/4126032199 >> v2:172.21.15.91:3300/0 conn(0x7fb98415a100 0x7fb9840b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 26809608 0 0) 0x7fb970003520 con 0x7fb9840abff0 2023-11-24T08:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 >> v2:172.21.15.91:3300/0 conn(0x7fb98415a100 msgr2=0x7fb9840b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 --2- 172.21.15.64:0/4126032199 >> v2:172.21.15.91:3300/0 conn(0x7fb98415a100 0x7fb9840b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 >> v1:172.21.15.91:6789/0 conn(0x7fb9840ae7e0 legacy=0x7fb98406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9840ac560 con 0x7fb9840abff0 2023-11-24T08:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7fb974001670 con 0x7fb9840abff0 2023-11-24T08:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 >> v1:172.21.15.64:6789/0 conn(0x7fb9840abff0 legacy=0x7fb98406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9700039c0 0x7fb970003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb970004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9700051d0 0x7fb97000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840ac840 con 0x7fb9700047c0 2023-11-24T08:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97000e330 con 0x7fb9700039c0 2023-11-24T08:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.213+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97000e4f0 con 0x7fb9700051d0 2023-11-24T08:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.214+0000 7fb988ff1640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb970004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.214+0000 7fb9897f2640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9700039c0 0x7fb970003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.214+0000 7fb988ff1640 1 -- 172.21.15.64:0/4126032199 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9700051d0 msgr2=0x7fb97000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.214+0000 7fb988ff1640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb9700051d0 0x7fb97000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.214+0000 7fb988ff1640 1 -- 172.21.15.64:0/4126032199 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9700039c0 msgr2=0x7fb970003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.214+0000 7fb988ff1640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb9700039c0 0x7fb970003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.214+0000 7fb988ff1640 1 -- 172.21.15.64:0/4126032199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970003520 con 0x7fb9700047c0 2023-11-24T08:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.214+0000 7fb989ff3640 1 -- 172.21.15.64:0/4126032199 reap_dead start 2023-11-24T08:09:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.215+0000 7fb988ff1640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb970004c40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c0e0 tx=0x7fb9700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c398e356c68a53a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.215+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974005ad0 con 0x7fb9700047c0 2023-11-24T08:09:08.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.215+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb974005c50 con 0x7fb9700047c0 2023-11-24T08:09:08.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.215+0000 7fb97bfff640 1 -- 172.21.15.64:0/4126032199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97402c810 con 0x7fb9700047c0 2023-11-24T08:09:08.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.216+0000 7fb98ba56640 1 -- 172.21.15.64:0/4126032199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 msgr2=0x7fb970004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.216+0000 7fb98ba56640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb970004c40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c0e0 tx=0x7fb9700057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.216+0000 7fb98ba56640 1 -- 172.21.15.64:0/4126032199 shutdown_connections 2023-11-24T08:09:08.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.216+0000 7fb98ba56640 1 --2- 172.21.15.64:0/4126032199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb970004c40 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.216+0000 7fb98ba56640 1 -- 172.21.15.64:0/4126032199 >> 172.21.15.64:0/4126032199 conn(0x7fb9840b0d80 msgr2=0x7fb9840b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:08.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.216+0000 7fb98ba56640 1 -- 172.21.15.64:0/4126032199 shutdown_connections 2023-11-24T08:09:08.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.216+0000 7fb98ba56640 1 -- 172.21.15.64:0/4126032199 wait complete. 2023-11-24T08:09:08.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.218+0000 7fb98ba56640 1 Processor -- start 2023-11-24T08:09:08.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.218+0000 7fb98ba56640 1 -- start start 2023-11-24T08:09:08.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb98ba56640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb9840cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb98ba56640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb98414e390 0x7fb9841f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb98ba56640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb98414e7f0 0x7fb98415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb98ba56640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98415c8f0 con 0x7fb9700047c0 2023-11-24T08:09:08.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb98ba56640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98415bd00 con 0x7fb98414e7f0 2023-11-24T08:09:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb98ba56640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840cc8f0 con 0x7fb98414e390 2023-11-24T08:09:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb9897f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb9840cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb9897f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb9840cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41364/0 (socket says 172.21.15.64:41364) 2023-11-24T08:09:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.219+0000 7fb9897f2640 1 -- 172.21.15.64:0/1719743370 learned_addr learned my addr 172.21.15.64:0/1719743370 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb988ff1640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb98414e390 0x7fb9841f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb9897f2640 1 -- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb98414e390 msgr2=0x7fb9841f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb9897f2640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb98414e390 0x7fb9841f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb9897f2640 1 -- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb98414e7f0 msgr2=0x7fb98415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb9897f2640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb98414e7f0 0x7fb98415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb9897f2640 1 -- 172.21.15.64:0/1719743370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98415bf50 con 0x7fb9700047c0 2023-11-24T08:09:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb9897f2640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb9840cc3b0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb98000e7e0 tx=0x7fb98000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb979ffb640 1 -- 172.21.15.64:0/1719743370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000c6d0 con 0x7fb9700047c0 2023-11-24T08:09:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb979ffb640 1 -- 172.21.15.64:0/1719743370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98000c830 con 0x7fb9700047c0 2023-11-24T08:09:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb979ffb640 1 -- 172.21.15.64:0/1719743370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000cba0 con 0x7fb9700047c0 2023-11-24T08:09:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.220+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98415c230 con 0x7fb9700047c0 2023-11-24T08:09:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.221+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9841f3570 con 0x7fb9700047c0 2023-11-24T08:09:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.222+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98406e180 con 0x7fb9700047c0 2023-11-24T08:09:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.223+0000 7fb979ffb640 1 -- 172.21.15.64:0/1719743370 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fb98000cd00 con 0x7fb9700047c0 2023-11-24T08:09:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.224+0000 7fb979ffb640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb96806b4a0 0x7fb96806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.224+0000 7fb979ffb640 1 -- 172.21.15.64:0/1719743370 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb980089dc0 con 0x7fb9700047c0 2023-11-24T08:09:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.224+0000 7fb988ff1640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb96806b4a0 0x7fb96806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.226+0000 7fb988ff1640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb96806b4a0 0x7fb96806d960 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb974034000 tx=0x7fb974009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.228+0000 7fb979ffb640 1 -- 172.21.15.64:0/1719743370 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb980054720 con 0x7fb9700047c0 2023-11-24T08:09:08.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.424+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fb984070c10 con 0x7fb96806b4a0 2023-11-24T08:09:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.425+0000 7fb979ffb640 1 -- 172.21.15.64:0/1719743370 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb974001530 con 0x7fb96806b4a0 2023-11-24T08:09:08.427 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T08:09:08.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb96806b4a0 msgr2=0x7fb96806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb96806b4a0 0x7fb96806d960 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb974034000 tx=0x7fb974009030 comp rx=0 tx=0).stop 2023-11-24T08:09:08.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 msgr2=0x7fb9840cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb9840cc3b0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb98000e7e0 tx=0x7fb98000eca0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 shutdown_connections 2023-11-24T08:09:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fb96806b4a0 0x7fb96806d960 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fb98414e7f0 0x7fb98415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fb98414e390 0x7fb9841f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 --2- 172.21.15.64:0/1719743370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9700047c0 0x7fb9840cc3b0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 >> 172.21.15.64:0/1719743370 conn(0x7fb9840b0d80 msgr2=0x7fb98406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:08.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 shutdown_connections 2023-11-24T08:09:08.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.428+0000 7fb98ba56640 1 -- 172.21.15.64:0/1719743370 wait complete. 2023-11-24T08:09:08.450 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T08:09:08.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:09:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.682+0000 7f1e892e1640 1 Processor -- start 2023-11-24T08:09:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.682+0000 7f1e892e1640 1 -- start start 2023-11-24T08:09:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.683+0000 7f1e892e1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e8415d3b0 0x7f1e840ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.683+0000 7f1e892e1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 0x7f1e840b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.683+0000 7f1e892e1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e840c6bf0 con 0x7f1e840d0fa0 2023-11-24T08:09:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.683+0000 7f1e892e1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e840d1430 con 0x7f1e8415d3b0 2023-11-24T08:09:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.684+0000 7f1e892e1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840cedc0 con 0x7f1e840d1d90 2023-11-24T08:09:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.684+0000 7f1e83fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e8415d3b0 0x7f1e840ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.684+0000 7f1e83fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e8415d3b0 0x7f1e840ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41378/0 (socket says 172.21.15.64:41378) 2023-11-24T08:09:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.684+0000 7f1e837fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1e840d0fa0 0x7f1e840d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57396/0 (socket says 172.21.15.64:57396) 2023-11-24T08:09:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.684+0000 7f1e83fff640 1 -- 172.21.15.64:0/734845161 learned_addr learned my addr 172.21.15.64:0/734845161 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.684+0000 7f1e88ae0640 1 --2- 172.21.15.64:0/734845161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 0x7f1e840b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.685+0000 7f1e83fff640 1 -- 172.21.15.64:0/734845161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 msgr2=0x7f1e840b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.685+0000 7f1e83fff640 1 --2- 172.21.15.64:0/734845161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 0x7f1e840b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.685+0000 7f1e83fff640 1 -- 172.21.15.64:0/734845161 >> v1:172.21.15.64:6789/0 conn(0x7f1e840d0fa0 legacy=0x7f1e840d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.685+0000 7f1e83fff640 1 -- 172.21.15.64:0/734845161 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840c6e40 con 0x7f1e8415d3b0 2023-11-24T08:09:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.685+0000 7f1e82ffd640 1 -- 172.21.15.64:0/734845161 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617163348 0 0) 0x7f1e840c6bf0 con 0x7f1e840d0fa0 2023-11-24T08:09:08.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.686+0000 7f1e83fff640 1 --2- 172.21.15.64:0/734845161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 0x7f1e840ce6c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009e50 tx=0x7f1e7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b856f2a4474f1b54 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.686+0000 7f1e82ffd640 1 -- 172.21.15.64:0/734845161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7402ca70 con 0x7f1e8415d3b0 2023-11-24T08:09:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.686+0000 7f1e82ffd640 1 -- 172.21.15.64:0/734845161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7402cbd0 con 0x7f1e8415d3b0 2023-11-24T08:09:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.686+0000 7f1e82ffd640 1 -- 172.21.15.64:0/734845161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e740343f0 con 0x7f1e8415d3b0 2023-11-24T08:09:08.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.687+0000 7f1e892e1640 1 -- 172.21.15.64:0/734845161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 msgr2=0x7f1e840ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.687+0000 7f1e892e1640 1 --2- 172.21.15.64:0/734845161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 0x7f1e840ce6c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009e50 tx=0x7f1e7402bed0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.687+0000 7f1e892e1640 1 -- 172.21.15.64:0/734845161 shutdown_connections 2023-11-24T08:09:08.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.687+0000 7f1e892e1640 1 --2- 172.21.15.64:0/734845161 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 0x7f1e840b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.687+0000 7f1e892e1640 1 --2- 172.21.15.64:0/734845161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 0x7f1e840ce6c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.687+0000 7f1e892e1640 1 -- 172.21.15.64:0/734845161 >> 172.21.15.64:0/734845161 conn(0x7f1e840d30a0 msgr2=0x7f1e840d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.687+0000 7f1e892e1640 1 -- 172.21.15.64:0/734845161 shutdown_connections 2023-11-24T08:09:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.687+0000 7f1e892e1640 1 -- 172.21.15.64:0/734845161 wait complete. 2023-11-24T08:09:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.689+0000 7f1e892e1640 1 Processor -- start 2023-11-24T08:09:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.689+0000 7f1e892e1640 1 -- start start 2023-11-24T08:09:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e892e1640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e840d0fa0 0x7f1e84152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e892e1640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 0x7f1e8409cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e892e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 0x7f1e8409d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e892e1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840ccac0 con 0x7f1e8415d3b0 2023-11-24T08:09:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e892e1640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840cbbd0 con 0x7f1e840d0fa0 2023-11-24T08:09:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e892e1640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840cbeb0 con 0x7f1e840d1d90 2023-11-24T08:09:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e88ae0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 0x7f1e8409d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e88ae0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 0x7f1e8409d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41386/0 (socket says 172.21.15.64:41386) 2023-11-24T08:09:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e88ae0640 1 -- 172.21.15.64:0/2329617852 learned_addr learned my addr 172.21.15.64:0/2329617852 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.690+0000 7f1e837fe640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 0x7f1e8409cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e88ae0640 1 -- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 msgr2=0x7f1e8409cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e88ae0640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 0x7f1e8409cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e88ae0640 1 -- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e840d0fa0 msgr2=0x7f1e84152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e88ae0640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1e840d0fa0 0x7f1e84152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e88ae0640 1 -- 172.21.15.64:0/2329617852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74009b30 con 0x7f1e8415d3b0 2023-11-24T08:09:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e837fe640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1e840d1d90 0x7f1e8409cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e88ae0640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 0x7f1e8409d320 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00d5d0 tx=0x7f1e7c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0125f0 con 0x7f1e8415d3b0 2023-11-24T08:09:08.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c012750 con 0x7f1e8415d3b0 2023-11-24T08:09:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c012b00 con 0x7f1e8415d3b0 2023-11-24T08:09:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e840cc190 con 0x7f1e8415d3b0 2023-11-24T08:09:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.691+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e840cc750 con 0x7f1e8415d3b0 2023-11-24T08:09:08.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.692+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e8415d790 con 0x7f1e8415d3b0 2023-11-24T08:09:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.694+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1e7c012c60 con 0x7f1e8415d3b0 2023-11-24T08:09:08.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.694+0000 7f1e80ff9640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e7006b470 0x7f1e7006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.695+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e7c08a3a0 con 0x7f1e8415d3b0 2023-11-24T08:09:08.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.695+0000 7f1e80ff9640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f1e700712d0 0x7f1e700736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.695+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 --> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e70073da0 con 0x7f1e700712d0 2023-11-24T08:09:08.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.695+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e7c08c860 con 0x7f1e8415d3b0 2023-11-24T08:09:08.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.695+0000 7f1e83fff640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e7006b470 0x7f1e7006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.695+0000 7f1e837fe640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f1e700712d0 0x7f1e700736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.696+0000 7f1e83fff640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e7006b470 0x7f1e7006d930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1e74035000 tx=0x7f1e7402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.697+0000 7f1e837fe640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f1e700712d0 0x7f1e700736f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.699+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 <== osd.7 v2:172.21.15.91:6808/4224688629 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1e70073da0 con 0x7f1e700712d0 2023-11-24T08:09:08.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.713+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 --> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1e840c7400 con 0x7f1e700712d0 2023-11-24T08:09:08.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.714+0000 7f1e80ff9640 1 -- 172.21.15.64:0/2329617852 <== osd.7 v2:172.21.15.91:6808/4224688629 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1e780018d0 con 0x7f1e700712d0 2023-11-24T08:09:08.717 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-11-24T08:09:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:09:08.717 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-11-24T08:09:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f1e700712d0 msgr2=0x7f1e700736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6808/4224688629,v1:172.21.15.91:6809/4224688629] conn(0x7f1e700712d0 0x7f1e700736f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e7006b470 msgr2=0x7f1e7006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1e7006b470 0x7f1e7006d930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1e74035000 tx=0x7f1e7402e000 comp rx=0 tx=0).stop 2023-11-24T08:09:08.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 msgr2=0x7f1e8409d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 --2- 172.21.15.64:0/2329617852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8415d3b0 0x7f1e8409d320 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00d5d0 tx=0x7f1e7c00da90 comp rx=0 tx=0).stop 2023-11-24T08:09:08.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e88ae0640 1 -- 172.21.15.64:0/2329617852 reap_dead start 2023-11-24T08:09:08.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 shutdown_connections 2023-11-24T08:09:08.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 >> 172.21.15.64:0/2329617852 conn(0x7f1e840d30a0 msgr2=0x7f1e8415c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:08.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 shutdown_connections 2023-11-24T08:09:08.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.715+0000 7f1e892e1640 1 -- 172.21.15.64:0/2329617852 wait complete. 2023-11-24T08:09:08.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T08:09:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.959+0000 7f6026d06640 1 Processor -- start 2023-11-24T08:09:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.959+0000 7f6026d06640 1 -- start start 2023-11-24T08:09:08.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.961+0000 7f6026d06640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f602015d540 0x7f6020070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.961+0000 7f6026d06640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f60200cad30 0x7f6020160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.961+0000 7f6026d06640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602006dfa0 con 0x7f6020161460 2023-11-24T08:09:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.961+0000 7f6026d06640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6020161d80 con 0x7f60200cad30 2023-11-24T08:09:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.961+0000 7f6026d06640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6020161890 con 0x7f602015d540 2023-11-24T08:09:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.962+0000 7f6025d04640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f602015d540 0x7f6020070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.962+0000 7f6025d04640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f602015d540 0x7f6020070fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.91:3301/0 says I am v2:172.21.15.64:45940/0 (socket says 172.21.15.64:45940) 2023-11-24T08:09:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.962+0000 7f6026505640 1 --1- >> v1:172.21.15.91:6789/0 conn(0x7f6020161460 0x7f60200b1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.91:6789/0 says I am v1:172.21.15.64:40582/0 (socket says 172.21.15.64:40582) 2023-11-24T08:09:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.962+0000 7f6025d04640 1 -- 172.21.15.64:0/2407344926 learned_addr learned my addr 172.21.15.64:0/2407344926 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.962+0000 7f6025503640 1 --2- 172.21.15.64:0/2407344926 >> v2:172.21.15.91:3300/0 conn(0x7f60200cad30 0x7f6020160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.963+0000 7f6025d04640 1 -- 172.21.15.64:0/2407344926 >> v2:172.21.15.91:3300/0 conn(0x7f60200cad30 msgr2=0x7f6020160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.963+0000 7f6025d04640 1 --2- 172.21.15.64:0/2407344926 >> v2:172.21.15.91:3300/0 conn(0x7f60200cad30 0x7f6020160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.963+0000 7f6025d04640 1 -- 172.21.15.64:0/2407344926 >> v1:172.21.15.91:6789/0 conn(0x7f6020161460 legacy=0x7f60200b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.963+0000 7f6025d04640 1 -- 172.21.15.64:0/2407344926 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60200cb190 con 0x7f602015d540 2023-11-24T08:09:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.963+0000 7f6025503640 1 --2- 172.21.15.64:0/2407344926 >> v2:172.21.15.91:3300/0 conn(0x7f60200cad30 0x7f6020160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:09:08.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.965+0000 7f6025d04640 1 --2- 172.21.15.64:0/2407344926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f602015d540 0x7f6020070fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f601c009870 tx=0x7f601c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ea4a0bed9f3e4e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.965+0000 7f6024d02640 1 -- 172.21.15.64:0/2407344926 <== mon.2 v2:172.21.15.91:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0317c0 con 0x7f602015d540 2023-11-24T08:09:08.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6024d02640 1 -- 172.21.15.64:0/2407344926 <== mon.2 v2:172.21.15.91:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c031920 con 0x7f602015d540 2023-11-24T08:09:08.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6024d02640 1 -- 172.21.15.64:0/2407344926 <== mon.2 v2:172.21.15.91:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c031cb0 con 0x7f602015d540 2023-11-24T08:09:08.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6026d06640 1 -- 172.21.15.64:0/2407344926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f602015d540 msgr2=0x7f6020070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6026d06640 1 --2- 172.21.15.64:0/2407344926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f602015d540 0x7f6020070fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f601c009870 tx=0x7f601c02b860 comp rx=0 tx=0).stop 2023-11-24T08:09:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6026d06640 1 -- 172.21.15.64:0/2407344926 shutdown_connections 2023-11-24T08:09:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6026d06640 1 --2- 172.21.15.64:0/2407344926 >> v2:172.21.15.91:3300/0 conn(0x7f60200cad30 0x7f6020160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6026d06640 1 --2- 172.21.15.64:0/2407344926 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f602015d540 0x7f6020070fd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6026d06640 1 -- 172.21.15.64:0/2407344926 >> 172.21.15.64:0/2407344926 conn(0x7f60200b0840 msgr2=0x7f60200b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:08.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.966+0000 7f6026d06640 1 -- 172.21.15.64:0/2407344926 shutdown_connections 2023-11-24T08:09:08.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.967+0000 7f6026d06640 1 -- 172.21.15.64:0/2407344926 wait complete. 2023-11-24T08:09:08.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.968+0000 7f6026d06640 1 Processor -- start 2023-11-24T08:09:08.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.968+0000 7f6026d06640 1 -- start start 2023-11-24T08:09:08.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.969+0000 7f6026d06640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60200cad30 0x7f602014f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.969+0000 7f6026d06640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6020161460 0x7f60200c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.969+0000 7f6026d06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60200c51d0 0x7f602015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.969+0000 7f6026d06640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60200cd360 con 0x7f60200c51d0 2023-11-24T08:09:08.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.969+0000 7f6026d06640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60200cd080 con 0x7f60200cad30 2023-11-24T08:09:08.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.969+0000 7f6026d06640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602015dad0 con 0x7f6020161460 2023-11-24T08:09:08.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6026505640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60200c51d0 0x7f602015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6026505640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60200c51d0 0x7f602015e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41388/0 (socket says 172.21.15.64:41388) 2023-11-24T08:09:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6026505640 1 -- 172.21.15.64:0/1860437918 learned_addr learned my addr 172.21.15.64:0/1860437918 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6025d04640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60200cad30 0x7f602014f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6025503640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6020161460 0x7f60200c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6026505640 1 -- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6020161460 msgr2=0x7f60200c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6026505640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6020161460 0x7f60200c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6026505640 1 -- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60200cad30 msgr2=0x7f602014f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6026505640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60200cad30 0x7f602014f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.970+0000 7f6026505640 1 -- 172.21.15.64:0/1860437918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6010009400 con 0x7f60200c51d0 2023-11-24T08:09:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.971+0000 7f6026505640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60200c51d0 0x7f602015e5b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f601800c730 tx=0x7f601800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.971+0000 7f60167fc640 1 -- 172.21.15.64:0/1860437918 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018004d60 con 0x7f60200c51d0 2023-11-24T08:09:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.971+0000 7f60167fc640 1 -- 172.21.15.64:0/1860437918 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601800a640 con 0x7f60200c51d0 2023-11-24T08:09:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.971+0000 7f60167fc640 1 -- 172.21.15.64:0/1860437918 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601800a9f0 con 0x7f60200c51d0 2023-11-24T08:09:08.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.971+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c0094d0 con 0x7f60200c51d0 2023-11-24T08:09:08.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.971+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602015e090 con 0x7f60200c51d0 2023-11-24T08:09:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.972+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602015e280 con 0x7f60200c51d0 2023-11-24T08:09:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.973+0000 7f60167fc640 1 -- 172.21.15.64:0/1860437918 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f601800ab50 con 0x7f60200c51d0 2023-11-24T08:09:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.974+0000 7f60167fc640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f600406b470 0x7f600406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:08.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.974+0000 7f60167fc640 1 -- 172.21.15.64:0/1860437918 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f60180890e0 con 0x7f60200c51d0 2023-11-24T08:09:08.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.975+0000 7f6025d04640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f600406b470 0x7f600406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:08.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.976+0000 7f6025d04640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f600406b470 0x7f600406d930 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f601c036000 tx=0x7f601c03d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:08.979+0000 7f60167fc640 1 -- 172.21.15.64:0/1860437918 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6018057630 con 0x7f60200c51d0 2023-11-24T08:09:09.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.180+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f60200718e0 con 0x7f600406b470 2023-11-24T08:09:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.181+0000 7f60167fc640 1 -- 172.21.15.64:0/1860437918 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f601c031b10 con 0x7f600406b470 2023-11-24T08:09:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T08:09:09.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f600406b470 msgr2=0x7f600406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f600406b470 0x7f600406d930 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f601c036000 tx=0x7f601c03d040 comp rx=0 tx=0).stop 2023-11-24T08:09:09.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60200c51d0 msgr2=0x7f602015e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60200c51d0 0x7f602015e5b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f601800c730 tx=0x7f601800cb30 comp rx=0 tx=0).stop 2023-11-24T08:09:09.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 shutdown_connections 2023-11-24T08:09:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f600406b470 0x7f600406d930 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60200c51d0 0x7f602015e5b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f6020161460 0x7f60200c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 --2- 172.21.15.64:0/1860437918 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f60200cad30 0x7f602014f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.183+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 >> 172.21.15.64:0/1860437918 conn(0x7f60200b0840 msgr2=0x7f602015c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.184+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 shutdown_connections 2023-11-24T08:09:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.184+0000 7f6026d06640 1 -- 172.21.15.64:0/1860437918 wait complete. 2023-11-24T08:09:09.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:09:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.425+0000 7f777eddb640 1 Processor -- start 2023-11-24T08:09:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.425+0000 7f777eddb640 1 -- start start 2023-11-24T08:09:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.427+0000 7f777eddb640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f7778143570 0x7f77780cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.427+0000 7f777eddb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f777815e6b0 0x7f77780cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.427+0000 7f777eddb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7778071700 con 0x7f7778143950 2023-11-24T08:09:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.427+0000 7f777eddb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777815b410 con 0x7f777815e6b0 2023-11-24T08:09:09.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.427+0000 7f777eddb640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777815b130 con 0x7f7778143570 2023-11-24T08:09:09.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.427+0000 7f777d5d8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7778143950 0x7f777815a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57404/0 (socket says 172.21.15.64:57404) 2023-11-24T08:09:09.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.428+0000 7f777d5d8640 1 -- 172.21.15.64:0/3673998032 learned_addr learned my addr 172.21.15.64:0/3673998032 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:09.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.428+0000 7f777e5da640 1 --2- 172.21.15.64:0/3673998032 >> v2:172.21.15.64:3300/0 conn(0x7f777815e6b0 0x7f77780cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.428+0000 7f777ddd9640 1 --2- 172.21.15.64:0/3673998032 >> v2:172.21.15.91:3300/0 conn(0x7f7778143570 0x7f77780cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.428+0000 7f777e5da640 1 -- 172.21.15.64:0/3673998032 >> v2:172.21.15.91:3300/0 conn(0x7f7778143570 msgr2=0x7f77780cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:09.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.428+0000 7f777e5da640 1 --2- 172.21.15.64:0/3673998032 >> v2:172.21.15.91:3300/0 conn(0x7f7778143570 0x7f77780cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.428+0000 7f777e5da640 1 -- 172.21.15.64:0/3673998032 >> v1:172.21.15.64:6789/0 conn(0x7f7778143950 legacy=0x7f777815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.428+0000 7f777e5da640 1 -- 172.21.15.64:0/3673998032 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777815dbd0 con 0x7f777815e6b0 2023-11-24T08:09:09.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.430+0000 7f777e5da640 1 --2- 172.21.15.64:0/3673998032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777815e6b0 0x7f77780cf590 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f7778070c70 tx=0x7f777002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7509095d74694dd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:09.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.430+0000 7f777cdd7640 1 -- 172.21.15.64:0/3673998032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777002cbe0 con 0x7f777815e6b0 2023-11-24T08:09:09.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.430+0000 7f777cdd7640 1 -- 172.21.15.64:0/3673998032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777002cd40 con 0x7f777815e6b0 2023-11-24T08:09:09.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.430+0000 7f777cdd7640 1 -- 172.21.15.64:0/3673998032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770034660 con 0x7f777815e6b0 2023-11-24T08:09:09.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.431+0000 7f777eddb640 1 -- 172.21.15.64:0/3673998032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777815e6b0 msgr2=0x7f77780cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.431+0000 7f777eddb640 1 --2- 172.21.15.64:0/3673998032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777815e6b0 0x7f77780cf590 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f7778070c70 tx=0x7f777002be30 comp rx=0 tx=0).stop 2023-11-24T08:09:09.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.431+0000 7f777eddb640 1 -- 172.21.15.64:0/3673998032 shutdown_connections 2023-11-24T08:09:09.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.431+0000 7f777eddb640 1 --2- 172.21.15.64:0/3673998032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777815e6b0 0x7f77780cf590 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.431+0000 7f777eddb640 1 --2- 172.21.15.64:0/3673998032 >> v2:172.21.15.91:3300/0 conn(0x7f7778143570 0x7f77780cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.431+0000 7f777eddb640 1 -- 172.21.15.64:0/3673998032 >> 172.21.15.64:0/3673998032 conn(0x7f777806a860 msgr2=0x7f777806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:09.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.431+0000 7f777eddb640 1 -- 172.21.15.64:0/3673998032 shutdown_connections 2023-11-24T08:09:09.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.431+0000 7f777eddb640 1 -- 172.21.15.64:0/3673998032 wait complete. 2023-11-24T08:09:09.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.433+0000 7f777eddb640 1 Processor -- start 2023-11-24T08:09:09.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.433+0000 7f777eddb640 1 -- start start 2023-11-24T08:09:09.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777eddb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7778143570 0x7f77780c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777eddb640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7778143950 0x7f77781f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777eddb640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f777815e6b0 0x7f77781e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777eddb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777812eb10 con 0x7f7778143570 2023-11-24T08:09:09.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777eddb640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77780c8400 con 0x7f777815e6b0 2023-11-24T08:09:09.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777eddb640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777812df00 con 0x7f7778143950 2023-11-24T08:09:09.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777ddd9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7778143570 0x7f77780c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777ddd9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7778143570 0x7f77780c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41412/0 (socket says 172.21.15.64:41412) 2023-11-24T08:09:09.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777ddd9640 1 -- 172.21.15.64:0/1339155431 learned_addr learned my addr 172.21.15.64:0/1339155431 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:09.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777d5d8640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7778143950 0x7f77781f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777ddd9640 1 -- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7778143950 msgr2=0x7f77781f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777ddd9640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7778143950 0x7f77781f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777ddd9640 1 -- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f777815e6b0 msgr2=0x7f77781e42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777e5da640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f777815e6b0 0x7f77781e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777ddd9640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f777815e6b0 0x7f77781e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.434+0000 7f777ddd9640 1 -- 172.21.15.64:0/1339155431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770009a70 con 0x7f7778143570 2023-11-24T08:09:09.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.435+0000 7f777ddd9640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7778143570 0x7f77780c7ec0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f777400d620 tx=0x7f777400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:09.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.435+0000 7f776e7fc640 1 -- 172.21.15.64:0/1339155431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777400b620 con 0x7f7778143570 2023-11-24T08:09:09.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.435+0000 7f776e7fc640 1 -- 172.21.15.64:0/1339155431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777400b780 con 0x7f7778143570 2023-11-24T08:09:09.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.435+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777812e1e0 con 0x7f7778143570 2023-11-24T08:09:09.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.435+0000 7f776e7fc640 1 -- 172.21.15.64:0/1339155431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777400bb30 con 0x7f7778143570 2023-11-24T08:09:09.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.435+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777812e7a0 con 0x7f7778143570 2023-11-24T08:09:09.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.436+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777804e840 con 0x7f7778143570 2023-11-24T08:09:09.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.437+0000 7f776e7fc640 1 -- 172.21.15.64:0/1339155431 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f777400bc90 con 0x7f7778143570 2023-11-24T08:09:09.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.438+0000 7f776e7fc640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f775c06b470 0x7f775c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.438+0000 7f776e7fc640 1 -- 172.21.15.64:0/1339155431 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f777408a350 con 0x7f7778143570 2023-11-24T08:09:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.439+0000 7f777d5d8640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f775c06b470 0x7f775c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.440+0000 7f777d5d8640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f775c06b470 0x7f775c06d930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f777806c870 tx=0x7f7768009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.443+0000 7f776e7fc640 1 -- 172.21.15.64:0/1339155431 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7774054d80 con 0x7f7778143570 2023-11-24T08:09:09.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.633+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f777815e190 con 0x7f775c06b470 2023-11-24T08:09:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.640+0000 7f776e7fc640 1 -- 172.21.15.64:0/1339155431 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48646 (secure 0 0 0) 0x7f7768001350 con 0x7f775c06b470 2023-11-24T08:09:09.642 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:09:09.643 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:09:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f775c06b470 msgr2=0x7f775c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f775c06b470 0x7f775c06d930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f777806c870 tx=0x7f7768009040 comp rx=0 tx=0).stop 2023-11-24T08:09:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7778143570 msgr2=0x7f77780c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7778143570 0x7f77780c7ec0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f777400d620 tx=0x7f777400dae0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 shutdown_connections 2023-11-24T08:09:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f775c06b470 0x7f775c06d930 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f777815e6b0 0x7f77781e42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f7778143950 0x7f77781f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 --2- 172.21.15.64:0/1339155431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7778143570 0x7f77780c7ec0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.642+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 >> 172.21.15.64:0/1339155431 conn(0x7f777806a860 msgr2=0x7f777806ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.643+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 shutdown_connections 2023-11-24T08:09:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:09.643+0000 7f777eddb640 1 -- 172.21.15.64:0/1339155431 wait complete. 2023-11-24T08:09:09.665 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2023-11-24T08:09:08.127718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259520,"kb_used_data":10052,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715200,"statfs":{"total":773094113280,"available":772828364800,"internally_reserved":0,"allocated":10293248,"data_stored":5090405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-135,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002903"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":227,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-24T08:08:57.223161+0000","last_change":"2023-11-24T08:03:45.904956+0000","last_active":"2023-11-24T08:08:57.223161+0000","last_peered":"2023-11-24T08:08:57.223161+0000","last_clean":"2023-11-24T08:08:57.223161+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:08:57.223161+0000","last_undegraded":"2023-11-24T08:08:57.223161+0000","last_fullsized":"2023-11-24T08:08:57.223161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:03:44.482777+0000","last_clean_scrub_stamp":"2023-11-24T08:03:44.482777+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:36:17.501406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1732608,"data_stored":1079204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.022,"15min":1.01},"min":{"1min":0.63700000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.875},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.05,"15min":1.0800000000000001},"min":{"1min":0.72099999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.27,"15min":1.298},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.04},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2909999999999999,"15min":1.367},"min":{"1min":0.89100000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0939999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1699999999999999,"15min":1.1539999999999999},"min":{"1min":0.77300000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.6840000000000002,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":1.212},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.266,"15min":1.302},"min":{"1min":0.85299999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.1439999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.25,"15min":1.3100000000000001},"min":{"1min":0.78200000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":4.3570000000000002,"5min":4.3570000000000002,"15min":4.3570000000000002},"last":1.325},{"interface":"front","average":{"1min":1.395,"5min":1.365,"15min":1.4870000000000001},"min":{"1min":0.75900000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.2480000000000002,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.304}]},{"osd":4,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.4910000000000001},"min":{"1min":0.755,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.327,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.997},{"interface":"front","average":{"1min":1.45,"5min":1.3360000000000001,"15min":1.369},"min":{"1min":0.95499999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.587},"min":{"1min":0.91100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.5289999999999999,"15min":1.637},"min":{"1min":0.96199999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":1.091}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.583,"15min":1.7190000000000001},"min":{"1min":1.016,"5min":0.755,"15min":0.755},"max":{"1min":2.5329999999999999,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.486,"15min":1.625},"min":{"1min":1.1950000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9410000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.4419999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2090000000000001,"15min":1.3069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.8460000000000001,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.236,"15min":1.3260000000000001},"min":{"1min":1.0369999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.4530000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.3799999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1339999999999999,"15min":1.1519999999999999},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.4710000000000001,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1699999999999999,"15min":1.284},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.532,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.6559999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1579999999999999,"15min":1.272},"min":{"1min":0.83799999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5329999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.256,"15min":1.4530000000000001},"min":{"1min":0.755,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.788,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.25}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3420000000000001,"15min":1.4279999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.8839999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.4419999999999999,"15min":1.4890000000000001},"min":{"1min":1.153,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.754,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.111}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4399999999999999,"15min":1.6950000000000001},"min":{"1min":0.88400000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.925,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.321},{"interface":"front","average":{"1min":1.488,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.117,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9830000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.2689999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4750000000000001,"15min":1.744},"min":{"1min":0.90400000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.98,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.165},{"interface":"front","average":{"1min":1.518,"5min":1.613,"15min":1.885},"min":{"1min":0.96999999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.036,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.429}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5660000000000001,"15min":1.665},"min":{"1min":1.2629999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1589999999999998,"5min":2.98,"15min":2.98},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.5529999999999999,"15min":1.629},"min":{"1min":0.97399999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0830000000000002,"5min":2.29,"15min":2.29},"last":1.353}]}]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.425,"5min":1.764,"15min":1.764},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0780000000000001,"15min":1.0209999999999999},"min":{"1min":0.71899999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.389}]},{"osd":1,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.133,"15min":1.1679999999999999},"min":{"1min":0.68799999999999994,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.542,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.1339999999999999,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6319999999999999,"5min":1.635,"15min":1.635},"last":1.036}]},{"osd":2,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.90500000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7050000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.528},{"interface":"front","average":{"1min":1.238,"5min":1.258,"15min":1.226},"min":{"1min":0.86899999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.577,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.337}]},{"osd":3,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.3320000000000001,"15min":1.4099999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5649999999999999,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.629},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2709999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7589999999999999,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.2889999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.298,"15min":1.302},"min":{"1min":0.96699999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4279999999999999,"15min":1.4179999999999999},"min":{"1min":0.95399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9350000000000001,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.7669999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.456,"15min":1.472},"min":{"1min":0.82499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8919999999999999,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.821},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4339999999999999,"15min":1.4159999999999999},"min":{"1min":0.98899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7070000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.2430000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5329999999999999,"15min":1.532},"min":{"1min":1.2330000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.849,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.869},{"interface":"front","average":{"1min":1.534,"5min":1.5649999999999999,"15min":1.5389999999999999},"min":{"1min":1.1779999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9790000000000001,"5min":2.859,"15min":2.859},"last":1.909}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0289999999999999,"15min":1.038},"min":{"1min":0.66400000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.3979999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.135,"15min":1.129},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.651,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.0700000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.454,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.129},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.175,"15min":1.226},"min":{"1min":0.86199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5509999999999999,"5min":1.903,"15min":1.903},"last":1.3480000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1759999999999999,"15min":1.196},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.649,"5min":1.823,"15min":1.823},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.1970000000000001,"15min":1.2430000000000001},"min":{"1min":0.77100000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6379999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.262,"15min":1.29},"min":{"1min":0.98399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5940000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.55},{"interface":"front","average":{"1min":1.262,"5min":1.2749999999999999,"15min":1.3029999999999999},"min":{"1min":0.83599999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.645,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.97199999999999998}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3080000000000001,"15min":1.3049999999999999},"min":{"1min":0.91800000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7529999999999999,"5min":1.944,"15min":1.944},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3640000000000001,"15min":1.3660000000000001},"min":{"1min":0.80700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.792,"5min":2.012,"15min":2.012},"last":1.4399999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.96699999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.7709999999999999,"5min":2.105,"15min":2.105},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.4990000000000001,"15min":1.5109999999999999},"min":{"1min":0.94499999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8640000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.615}]},{"osd":7,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.4390000000000001},"min":{"1min":1.018,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.8919999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.474},{"interface":"front","average":{"1min":1.544,"5min":1.5369999999999999,"15min":1.5529999999999999},"min":{"1min":1.101,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8500000000000001,"5min":2.722,"15min":2.722},"last":1.3089999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":0.90600000000000003,"15min":0.84699999999999998},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.377},"max":{"1min":6.1849999999999996,"5min":6.1849999999999996,"15min":6.1849999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91300000000000003,"15min":0.94499999999999995},"min":{"1min":0.60699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":1.1120000000000001},"min":{"1min":0.57399999999999995,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3420000000000001,"5min":3.246,"15min":3.246},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.097,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.40400000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.651,"5min":1.718,"15min":1.718},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0920000000000001,"15min":1.1659999999999999},"min":{"1min":0.64000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6020000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.1839999999999999,"15min":1.1779999999999999},"min":{"1min":0.71099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.016,"5min":6.016,"15min":6.016},"last":0.98599999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2230000000000001,"15min":1.2569999999999999},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5469999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3220000000000001,"15min":1.379},"min":{"1min":1.0369999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8540000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0129999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3700000000000001,"15min":1.4419999999999999},"min":{"1min":0.97299999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.772,"5min":2.395,"15min":2.395},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.722,"5min":1.405,"15min":1.333},"min":{"1min":1.093,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":5.9349999999999996,"5min":5.9349999999999996,"15min":5.9349999999999996},"last":1.327}]},{"osd":6,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.8149999999999999,"5min":1.498,"15min":1.458},"min":{"1min":1.208,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.3859999999999999,"15min":1.4139999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.837,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.871}]},{"osd":7,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4259999999999999,"15min":1.4450000000000001},"min":{"1min":0.871,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9419999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.046},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.4930000000000001,"15min":1.569},"min":{"1min":1.1950000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.2989999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82099999999999995,"15min":0.83799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3560000000000001,"5min":1.524,"15min":1.524},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93700000000000006,"15min":1.0029999999999999},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3640000000000001,"5min":2.133,"15min":2.133},"last":0.628}]},{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.042,"15min":1.1919999999999999},"min":{"1min":0.71299999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.492,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.048},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.516,"5min":1.998,"15min":1.998},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.262,"15min":1.3180000000000001},"min":{"1min":0.64600000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9410000000000001,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.133,"15min":1.1899999999999999},"min":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7789999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.214,"15min":1.3100000000000001},"min":{"1min":0.66500000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.774,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.3799999999999999,"15min":1.4830000000000001},"min":{"1min":0.996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.869,"5min":3.161,"15min":3.161},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.288,"15min":1.3740000000000001},"min":{"1min":0.88500000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.518,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2909999999999999,"15min":1.3899999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.966,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.494}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4399999999999999,"15min":1.5680000000000001},"min":{"1min":0.95399999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.944,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.397,"5min":1.4059999999999999,"15min":1.538},"min":{"1min":0.98599999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.867,"5min":2.702,"15min":2.702},"last":2.8780000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3979999999999999,"15min":1.4970000000000001},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9970000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.427},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5049999999999999,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0019999999999998,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.875}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83699999999999997,"15min":0.878},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4179999999999999,"5min":1.732,"15min":1.732},"last":1.371},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90500000000000003,"15min":1.0249999999999999},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2410000000000001,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.0549999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91000000000000003,"15min":1.006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5920000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.224},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.097,"15min":1.202},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.629,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.7709999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1699999999999999,"15min":1.248},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.738,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.992},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0700000000000001,"15min":1.196},"min":{"1min":0.46200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.5800000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.7390000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.165,"15min":1.2290000000000001},"min":{"1min":0.76200000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.427,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.466},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3380000000000001,"15min":1.427},"min":{"1min":0.78100000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9670000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.6479999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.345,"15min":1.4299999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.921,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.8720000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.8100000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.7010000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3799999999999999,"15min":1.4410000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.972,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3560000000000001,"15min":1.454},"min":{"1min":0.80000000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.5820000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4339999999999999,"15min":1.532},"min":{"1min":0.94999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8540000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":2.3220000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4950000000000001,"15min":1.571},"min":{"1min":0.89600000000000002,"5min":0.872,"15min":0.872},"max":{"1min":2.0779999999999998,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.9510000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1728512,"data_stored":1078185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.86099999999999999,"15min":0.875},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.423,"5min":3.423,"15min":3.423},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97699999999999998,"15min":0.94699999999999995},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2370000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.72899999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.55600000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.673,"5min":3.899,"15min":3.899},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.97699999999999998,"15min":0.95299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5429999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0349999999999999,"15min":1.0289999999999999},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.464,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.2490000000000001,"15min":1.2190000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7430000000000001,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.1160000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2849999999999999,"15min":1.264},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.673,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.4550000000000001,"15min":1.4670000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8149999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.5389999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":0.64600000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9610000000000001,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.345},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.4350000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0430000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.5649999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.351,"15min":1.3999999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7310000000000001,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.367},{"interface":"front","average":{"1min":1.444,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":0.73499999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1070000000000002,"5min":3.879,"15min":3.879},"last":1.403}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4930000000000001,"15min":1.524},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8859999999999999,"5min":3.823,"15min":3.823},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.464,"15min":1.478},"min":{"1min":0.77600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8240000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.593}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:09:09.668 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T08:03:44.482777+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=8, tm_min=3, tm_sec=44, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=8, tm_min=9, tm_sec=2, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T08:09:09.668 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T08:09:16.215 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:16.225 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:29.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:09:29.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.904+0000 7f071f59e640 1 Processor -- start 2023-11-24T08:09:29.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.904+0000 7f071f59e640 1 -- start start 2023-11-24T08:09:29.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.906+0000 7f071f59e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07200cf050 0x7f07200cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.906+0000 7f071f59e640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f07200cd7b0 0x7f07200cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.906+0000 7f071f59e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072015fa70 con 0x7f072013dfd0 2023-11-24T08:09:29.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.906+0000 7f071f59e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07200cfa40 con 0x7f07200cf050 2023-11-24T08:09:29.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.906+0000 7f071f59e640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072015c580 con 0x7f07200cd7b0 2023-11-24T08:09:29.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.906+0000 7f0717fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07200cf050 0x7f07200cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.906+0000 7f071e59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f072013dfd0 0x7f072015be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60592/0 (socket says 172.21.15.64:60592) 2023-11-24T08:09:29.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.906+0000 7f071e59c640 1 -- 172.21.15.64:0/4096065145 learned_addr learned my addr 172.21.15.64:0/4096065145 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:29.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.907+0000 7f071ed9d640 1 --2- 172.21.15.64:0/4096065145 >> v2:172.21.15.91:3300/0 conn(0x7f07200cd7b0 0x7f07200cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.907+0000 7f0717fff640 1 -- 172.21.15.64:0/4096065145 >> v2:172.21.15.91:3300/0 conn(0x7f07200cd7b0 msgr2=0x7f07200cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:29.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.907+0000 7f0717fff640 1 --2- 172.21.15.64:0/4096065145 >> v2:172.21.15.91:3300/0 conn(0x7f07200cd7b0 0x7f07200cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.907+0000 7f0717fff640 1 -- 172.21.15.64:0/4096065145 >> v1:172.21.15.64:6789/0 conn(0x7f072013dfd0 legacy=0x7f072015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.907+0000 7f0717fff640 1 -- 172.21.15.64:0/4096065145 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720140900 con 0x7f07200cf050 2023-11-24T08:09:29.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.907+0000 7f071dd9b640 1 -- 172.21.15.64:0/4096065145 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144223204 0 0) 0x7f072015fa70 con 0x7f072013dfd0 2023-11-24T08:09:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.908+0000 7f0717fff640 1 --2- 172.21.15.64:0/4096065145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07200cf500 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f07201416f0 tx=0x7f070802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e710d60a687e883 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.909+0000 7f071dd9b640 1 -- 172.21.15.64:0/4096065145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708031830 con 0x7f07200cf050 2023-11-24T08:09:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.909+0000 7f071dd9b640 1 -- 172.21.15.64:0/4096065145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0708031990 con 0x7f07200cf050 2023-11-24T08:09:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.909+0000 7f071dd9b640 1 -- 172.21.15.64:0/4096065145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708031d20 con 0x7f07200cf050 2023-11-24T08:09:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.909+0000 7f071f59e640 1 -- 172.21.15.64:0/4096065145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 msgr2=0x7f07200cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.909+0000 7f071f59e640 1 --2- 172.21.15.64:0/4096065145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07200cf500 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f07201416f0 tx=0x7f070802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.910+0000 7f071f59e640 1 -- 172.21.15.64:0/4096065145 shutdown_connections 2023-11-24T08:09:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.910+0000 7f071f59e640 1 --2- 172.21.15.64:0/4096065145 >> v2:172.21.15.91:3300/0 conn(0x7f07200cd7b0 0x7f07200cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.910+0000 7f071f59e640 1 --2- 172.21.15.64:0/4096065145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07200cf500 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.910+0000 7f071f59e640 1 -- 172.21.15.64:0/4096065145 >> 172.21.15.64:0/4096065145 conn(0x7f072006a860 msgr2=0x7f072006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:29.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.910+0000 7f071f59e640 1 -- 172.21.15.64:0/4096065145 shutdown_connections 2023-11-24T08:09:29.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.910+0000 7f071f59e640 1 -- 172.21.15.64:0/4096065145 wait complete. 2023-11-24T08:09:29.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.911+0000 7f071f59e640 1 Processor -- start 2023-11-24T08:09:29.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.912+0000 7f071f59e640 1 -- start start 2023-11-24T08:09:29.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f071f59e640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07200cd7b0 0x7f07200acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f071f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07201313e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f071f59e640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072013dfd0 0x7f0720131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f071f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07201f2e10 con 0x7f07200cf050 2023-11-24T08:09:29.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f071f59e640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07200ad520 con 0x7f072013dfd0 2023-11-24T08:09:29.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f071f59e640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07200ad800 con 0x7f07200cd7b0 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f0717fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07201313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f0717fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07201313e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40072/0 (socket says 172.21.15.64:40072) 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f0717fff640 1 -- 172.21.15.64:0/1234218816 learned_addr learned my addr 172.21.15.64:0/1234218816 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f0717fff640 1 -- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07200cd7b0 msgr2=0x7f07200acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f071ed9d640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072013dfd0 0x7f0720131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f071e59c640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07200cd7b0 0x7f07200acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f0717fff640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07200cd7b0 0x7f07200acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f0717fff640 1 -- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072013dfd0 msgr2=0x7f0720131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f0717fff640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072013dfd0 0x7f0720131920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.913+0000 7f0717fff640 1 -- 172.21.15.64:0/1234218816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708009400 con 0x7f07200cf050 2023-11-24T08:09:29.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.914+0000 7f0717fff640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07201313e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0708009890 tx=0x7f07080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.914+0000 7f0716ffd640 1 -- 172.21.15.64:0/1234218816 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070803ba70 con 0x7f07200cf050 2023-11-24T08:09:29.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.914+0000 7f0716ffd640 1 -- 172.21.15.64:0/1234218816 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070803bbf0 con 0x7f07200cf050 2023-11-24T08:09:29.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.914+0000 7f0716ffd640 1 -- 172.21.15.64:0/1234218816 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708045400 con 0x7f07200cf050 2023-11-24T08:09:29.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.914+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07200adae0 con 0x7f07200cf050 2023-11-24T08:09:29.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.914+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0720132140 con 0x7f07200cf050 2023-11-24T08:09:29.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.915+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07200ae000 con 0x7f07200cf050 2023-11-24T08:09:29.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.916+0000 7f0716ffd640 1 -- 172.21.15.64:0/1234218816 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f070804e440 con 0x7f07200cf050 2023-11-24T08:09:29.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.918+0000 7f0716ffd640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006b470 0x7f070006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.918+0000 7f0716ffd640 1 -- 172.21.15.64:0/1234218816 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f07080ac170 con 0x7f07200cf050 2023-11-24T08:09:29.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.918+0000 7f071e59c640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006b470 0x7f070006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.919+0000 7f071e59c640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006b470 0x7f070006d930 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0710000f60 tx=0x7f0710009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:29.923+0000 7f0716ffd640 1 -- 172.21.15.64:0/1234218816 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07200ae000 con 0x7f07200cf050 2023-11-24T08:09:30.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.113+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 --> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f072015f420 con 0x7f070006b470 2023-11-24T08:09:30.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.119+0000 7f0716ffd640 1 -- 172.21.15.64:0/1234218816 <== mgr.4113 v2:172.21.15.91:6832/401100545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48576 (secure 0 0 0) 0x7f07100070d0 con 0x7f070006b470 2023-11-24T08:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:09:30.122 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-11-24T08:09:30.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006b470 msgr2=0x7f070006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006b470 0x7f070006d930 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0710000f60 tx=0x7f0710009040 comp rx=0 tx=0).stop 2023-11-24T08:09:30.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 msgr2=0x7f07201313e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07201313e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0708009890 tx=0x7f07080046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 shutdown_connections 2023-11-24T08:09:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f070006b470 0x7f070006d930 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f072013dfd0 0x7f0720131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07200cf050 0x7f07201313e0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 --2- 172.21.15.64:0/1234218816 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f07200cd7b0 0x7f07200acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 >> 172.21.15.64:0/1234218816 conn(0x7f072006a860 msgr2=0x7f072015e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 shutdown_connections 2023-11-24T08:09:30.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.122+0000 7f071f59e640 1 -- 172.21.15.64:0/1234218816 wait complete. 2023-11-24T08:09:30.144 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2023-11-24T08:09:28.133705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259504,"kb_used_data":10052,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715216,"statfs":{"total":773094113280,"available":772828381184,"internally_reserved":0,"allocated":10293248,"data_stored":5090405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003764"},"pg_stats":[{"pgid":"1.0","version":"92'119","reported_seq":238,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-24T08:09:10.404644+0000","last_change":"2023-11-24T08:09:10.404703+0000","last_active":"2023-11-24T08:09:10.404644+0000","last_peered":"2023-11-24T08:09:10.404644+0000","last_clean":"2023-11-24T08:09:10.404644+0000","last_became_active":"2023-11-24T08:03:45.902047+0000","last_became_peered":"2023-11-24T08:03:45.902047+0000","last_unstale":"2023-11-24T08:09:10.404644+0000","last_undegraded":"2023-11-24T08:09:10.404644+0000","last_fullsized":"2023-11-24T08:09:10.404644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'119","last_scrub_stamp":"2023-11-24T08:09:10.404462+0000","last_deep_scrub":"92'119","last_deep_scrub_stamp":"2023-11-24T08:09:10.404462+0000","last_clean_scrub_stamp":"2023-11-24T08:09:10.404462+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:32.562414+0000","scrub_duration":0.012505613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1732608,"data_stored":1079204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.022,"15min":1.01},"min":{"1min":0.63700000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.05,"15min":1.0800000000000001},"min":{"1min":0.72099999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.3009999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.27,"15min":1.298},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2909999999999999,"15min":1.367},"min":{"1min":0.89100000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0939999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.369}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1699999999999999,"15min":1.1539999999999999},"min":{"1min":0.77300000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.6840000000000002,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":1.458},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.266,"15min":1.302},"min":{"1min":0.85299999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.0509999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.25,"15min":1.3100000000000001},"min":{"1min":0.78200000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":4.3570000000000002,"5min":4.3570000000000002,"15min":4.3570000000000002},"last":1.224},{"interface":"front","average":{"1min":1.395,"5min":1.365,"15min":1.4870000000000001},"min":{"1min":0.75900000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.2480000000000002,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.508}]},{"osd":4,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.4910000000000001},"min":{"1min":0.755,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.327,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.3360000000000001,"15min":1.369},"min":{"1min":0.95499999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.121}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.587},"min":{"1min":0.91100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.25},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.5289999999999999,"15min":1.637},"min":{"1min":0.96199999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":1.1799999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.583,"15min":1.7190000000000001},"min":{"1min":1.016,"5min":0.755,"15min":0.755},"max":{"1min":2.5329999999999999,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.486,"15min":1.625},"min":{"1min":1.1950000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9410000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.343}]}]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2090000000000001,"15min":1.3069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.8460000000000001,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.246,"5min":1.236,"15min":1.3260000000000001},"min":{"1min":1.0369999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.4530000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.659}]},{"osd":1,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1339999999999999,"15min":1.1519999999999999},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.4710000000000001,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1699999999999999,"15min":1.284},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.532,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.087}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1579999999999999,"15min":1.272},"min":{"1min":0.83799999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5329999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.256,"15min":1.4530000000000001},"min":{"1min":0.755,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.788,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.0329999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3420000000000001,"15min":1.4279999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.8839999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.792},{"interface":"front","average":{"1min":1.472,"5min":1.4419999999999999,"15min":1.4890000000000001},"min":{"1min":1.153,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.754,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.863}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4399999999999999,"15min":1.6950000000000001},"min":{"1min":0.88400000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.925,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.718},{"interface":"front","average":{"1min":1.488,"5min":1.431,"15min":1.4690000000000001},"min":{"1min":1.117,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9830000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.8109999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4750000000000001,"15min":1.744},"min":{"1min":0.90400000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.98,"5min":9.8569999999999993,"15min":9.8569999999999993},"last":1.329},{"interface":"front","average":{"1min":1.518,"5min":1.613,"15min":1.885},"min":{"1min":0.96999999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.036,"5min":9.9450000000000003,"15min":9.9450000000000003},"last":1.6060000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5660000000000001,"15min":1.665},"min":{"1min":1.2629999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1589999999999998,"5min":2.98,"15min":2.98},"last":1.7529999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.5529999999999999,"15min":1.629},"min":{"1min":0.97399999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0830000000000002,"5min":2.29,"15min":2.29},"last":1.581}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.425,"5min":1.764,"15min":1.764},"last":1.071},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0780000000000001,"15min":1.0209999999999999},"min":{"1min":0.71899999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.84899999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.133,"15min":1.1679999999999999},"min":{"1min":0.68799999999999994,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.542,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.1339999999999999,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6319999999999999,"5min":1.635,"15min":1.635},"last":1.2}]},{"osd":2,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.90500000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7050000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.01},{"interface":"front","average":{"1min":1.238,"5min":1.258,"15min":1.226},"min":{"1min":0.86899999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.577,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.6120000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.3320000000000001,"15min":1.4099999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5649999999999999,"5min":4.7400000000000002,"15min":4.7400000000000002},"last":1.349},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2709999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7589999999999999,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.4299999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.298,"15min":1.302},"min":{"1min":0.96699999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4279999999999999,"15min":1.4179999999999999},"min":{"1min":0.95399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.9350000000000001,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.153}]},{"osd":6,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.456,"15min":1.472},"min":{"1min":0.82499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8919999999999999,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4339999999999999,"15min":1.4159999999999999},"min":{"1min":0.98899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7070000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":1.2589999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:52 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5329999999999999,"15min":1.532},"min":{"1min":1.2330000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.849,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.407},{"interface":"front","average":{"1min":1.534,"5min":1.5649999999999999,"15min":1.5389999999999999},"min":{"1min":1.1779999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.9790000000000001,"5min":2.859,"15min":2.859},"last":1.7030000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0289999999999999,"15min":1.038},"min":{"1min":0.66400000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.3979999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.135,"15min":1.129},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.651,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.4079999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.454,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.026},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.175,"15min":1.226},"min":{"1min":0.86199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5509999999999999,"5min":1.903,"15min":1.903},"last":1.2589999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1759999999999999,"15min":1.196},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.649,"5min":1.823,"15min":1.823},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.1970000000000001,"15min":1.2430000000000001},"min":{"1min":0.77100000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6379999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.1910000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.262,"15min":1.29},"min":{"1min":0.98399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5940000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.468},{"interface":"front","average":{"1min":1.262,"5min":1.2749999999999999,"15min":1.3029999999999999},"min":{"1min":0.83599999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.645,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3080000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3080000000000001,"15min":1.3049999999999999},"min":{"1min":0.91800000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7529999999999999,"5min":1.944,"15min":1.944},"last":1.653},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3640000000000001,"15min":1.3660000000000001},"min":{"1min":0.80700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.792,"5min":2.012,"15min":2.012},"last":1.367}]},{"osd":6,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.96699999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.7709999999999999,"5min":2.105,"15min":2.105},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.4990000000000001,"15min":1.5109999999999999},"min":{"1min":0.94499999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8640000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.726}]},{"osd":7,"last update":"Fri Nov 24 08:08:53 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.4390000000000001},"min":{"1min":1.018,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.8919999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.5369999999999999,"15min":1.5529999999999999},"min":{"1min":1.101,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8500000000000001,"5min":2.722,"15min":2.722},"last":1.514}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":0.90600000000000003,"15min":0.84699999999999998},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.377},"max":{"1min":6.1849999999999996,"5min":6.1849999999999996,"15min":6.1849999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91300000000000003,"15min":0.94499999999999995},"min":{"1min":0.60699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.99099999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":1.1120000000000001},"min":{"1min":0.57399999999999995,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3420000000000001,"5min":3.246,"15min":3.246},"last":0.746},{"interface":"front","average":{"1min":1.097,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.40400000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.651,"5min":1.718,"15min":1.718},"last":1.1579999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0920000000000001,"15min":1.1659999999999999},"min":{"1min":0.64000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6020000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.1839999999999999,"15min":1.1779999999999999},"min":{"1min":0.71099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.016,"5min":6.016,"15min":6.016},"last":1.4019999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2230000000000001,"15min":1.2569999999999999},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5469999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3220000000000001,"15min":1.379},"min":{"1min":1.0369999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8540000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.28}]},{"osd":5,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3700000000000001,"15min":1.4419999999999999},"min":{"1min":0.97299999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.772,"5min":2.395,"15min":2.395},"last":1.097},{"interface":"front","average":{"1min":1.722,"5min":1.405,"15min":1.333},"min":{"1min":1.093,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":5.9349999999999996,"5min":5.9349999999999996,"15min":5.9349999999999996},"last":1.238}]},{"osd":6,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.8149999999999999,"5min":1.498,"15min":1.458},"min":{"1min":1.208,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":1.508},{"interface":"front","average":{"1min":1.389,"5min":1.3859999999999999,"15min":1.4139999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.837,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.3169999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:08:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4259999999999999,"15min":1.4450000000000001},"min":{"1min":0.871,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9419999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.4930000000000001,"15min":1.569},"min":{"1min":1.1950000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9430000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3859999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82099999999999995,"15min":0.83799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3560000000000001,"5min":1.524,"15min":1.524},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93700000000000006,"15min":1.0029999999999999},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3640000000000001,"5min":2.133,"15min":2.133},"last":0.74399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.042,"15min":1.1919999999999999},"min":{"1min":0.71299999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.492,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.048},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.516,"5min":1.998,"15min":1.998},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.262,"15min":1.3180000000000001},"min":{"1min":0.64600000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9410000000000001,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.133,"15min":1.1899999999999999},"min":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.7789999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.2829999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.214,"15min":1.3100000000000001},"min":{"1min":0.66500000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.774,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.677},{"interface":"front","average":{"1min":1.383,"5min":1.3799999999999999,"15min":1.4830000000000001},"min":{"1min":0.996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.869,"5min":3.161,"15min":3.161},"last":1.583}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.288,"15min":1.3740000000000001},"min":{"1min":0.88500000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.518,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2909999999999999,"15min":1.3899999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.966,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.454}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4399999999999999,"15min":1.5680000000000001},"min":{"1min":0.95399999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.944,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.4059999999999999,"15min":1.538},"min":{"1min":0.98599999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.867,"5min":2.702,"15min":2.702},"last":1.4910000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3979999999999999,"15min":1.4970000000000001},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9970000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":1.54},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5049999999999999,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0019999999999998,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.748}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1138688,"data_stored":488836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83699999999999997,"15min":0.878},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4179999999999999,"5min":1.732,"15min":1.732},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90500000000000003,"15min":1.0249999999999999},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2410000000000001,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.1040000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91000000000000003,"15min":1.006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5920000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.097,"15min":1.202},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.629,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.377}]},{"osd":3,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1699999999999999,"15min":1.248},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.738,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.798},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0700000000000001,"15min":1.196},"min":{"1min":0.46200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.5800000000000001,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.8540000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.165,"15min":1.2290000000000001},"min":{"1min":0.76200000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.427,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3380000000000001,"15min":1.427},"min":{"1min":0.78100000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9670000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.554}]},{"osd":5,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.345,"15min":1.4299999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.921,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.7629999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.8100000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.5009999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3799999999999999,"15min":1.4410000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.972,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3560000000000001,"15min":1.454},"min":{"1min":0.80000000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.208}]},{"osd":7,"last update":"Fri Nov 24 08:08:54 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4339999999999999,"15min":1.532},"min":{"1min":0.94999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8540000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4950000000000001,"15min":1.571},"min":{"1min":0.89600000000000002,"5min":0.872,"15min":0.872},"max":{"1min":2.0779999999999998,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.599}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1728512,"data_stored":1078185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.86099999999999999,"15min":0.875},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.423,"5min":3.423,"15min":3.423},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97699999999999998,"15min":0.94699999999999995},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2370000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.2330000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.55600000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.673,"5min":3.899,"15min":3.899},"last":1.3380000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.97699999999999998,"15min":0.95299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5429999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.89000000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0349999999999999,"15min":1.0289999999999999},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.464,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.2490000000000001,"15min":1.2190000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7430000000000001,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.5509999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2849999999999999,"15min":1.264},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.673,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.4550000000000001,"15min":1.4670000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8149999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.464}]},{"osd":5,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":0.64600000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9610000000000001,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.4350000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0430000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.202}]},{"osd":6,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.351,"15min":1.3999999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7310000000000001,"5min":3.9729999999999999,"15min":3.9729999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":0.73499999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1070000000000002,"5min":3.879,"15min":3.879},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:08:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4930000000000001,"15min":1.524},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8859999999999999,"5min":3.823,"15min":3.823},"last":1.413},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.464,"15min":1.478},"min":{"1min":0.77600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8240000000000001,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.427}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:09:30.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T08:09:30.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.363+0000 7fe2dbab4640 1 Processor -- start 2023-11-24T08:09:30.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.363+0000 7fe2dbab4640 1 -- start start 2023-11-24T08:09:30.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.364+0000 7fe2dbab4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40acd70 0x7fe2d406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.364+0000 7fe2dbab4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe2d40ae490 0x7fe2d40ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.364+0000 7fe2dbab4640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7fe2d40ad690 0x7fe2d4070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.364+0000 7fe2dbab4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d415a5b0 con 0x7fe2d40ae490 2023-11-24T08:09:30.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.364+0000 7fe2dbab4640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d406d7a0 con 0x7fe2d40ad690 2023-11-24T08:09:30.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.364+0000 7fe2dbab4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d406ce70 con 0x7fe2d40acd70 2023-11-24T08:09:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d904f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe2d40ae490 0x7fe2d40ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d904f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe2d40ae490 0x7fe2d40ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40078/0 (socket says 172.21.15.64:40078) 2023-11-24T08:09:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d904f640 1 -- 172.21.15.64:0/2323689931 learned_addr learned my addr 172.21.15.64:0/2323689931 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2da051640 1 --2- 172.21.15.64:0/2323689931 >> v2:172.21.15.91:3300/0 conn(0x7fe2d40ad690 0x7fe2d4070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d9850640 1 --2- 172.21.15.64:0/2323689931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40acd70 0x7fe2d406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d904f640 1 -- 172.21.15.64:0/2323689931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40acd70 msgr2=0x7fe2d406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d904f640 1 --2- 172.21.15.64:0/2323689931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40acd70 0x7fe2d406dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d904f640 1 -- 172.21.15.64:0/2323689931 >> v2:172.21.15.91:3300/0 conn(0x7fe2d40ad690 msgr2=0x7fe2d4070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d904f640 1 --2- 172.21.15.64:0/2323689931 >> v2:172.21.15.91:3300/0 conn(0x7fe2d40ad690 0x7fe2d4070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2d904f640 1 -- 172.21.15.64:0/2323689931 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d406d150 con 0x7fe2d40ae490 2023-11-24T08:09:30.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.365+0000 7fe2da051640 1 --2- 172.21.15.64:0/2323689931 >> v2:172.21.15.91:3300/0 conn(0x7fe2d40ad690 0x7fe2d4070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:30.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.366+0000 7fe2d904f640 1 --2- 172.21.15.64:0/2323689931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d40ad150 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4072660 tx=0x7fe2c402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bdeb5446560d01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2d884e640 1 -- 172.21.15.64:0/2323689931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4031860 con 0x7fe2d40ae490 2023-11-24T08:09:30.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2d884e640 1 -- 172.21.15.64:0/2323689931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c40319c0 con 0x7fe2d40ae490 2023-11-24T08:09:30.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2d884e640 1 -- 172.21.15.64:0/2323689931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4031d50 con 0x7fe2d40ae490 2023-11-24T08:09:30.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2dbab4640 1 -- 172.21.15.64:0/2323689931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 msgr2=0x7fe2d40ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/2323689931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d40ad150 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4072660 tx=0x7fe2c402b900 comp rx=0 tx=0).stop 2023-11-24T08:09:30.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2dbab4640 1 -- 172.21.15.64:0/2323689931 shutdown_connections 2023-11-24T08:09:30.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/2323689931 >> v2:172.21.15.91:3300/0 conn(0x7fe2d40ad690 0x7fe2d4070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/2323689931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d40ad150 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.367+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/2323689931 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40acd70 0x7fe2d406dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.368+0000 7fe2dbab4640 1 -- 172.21.15.64:0/2323689931 >> 172.21.15.64:0/2323689931 conn(0x7fe2d40b0ba0 msgr2=0x7fe2d40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:30.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.368+0000 7fe2dbab4640 1 -- 172.21.15.64:0/2323689931 shutdown_connections 2023-11-24T08:09:30.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.368+0000 7fe2dbab4640 1 -- 172.21.15.64:0/2323689931 wait complete. 2023-11-24T08:09:30.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.369+0000 7fe2dbab4640 1 Processor -- start 2023-11-24T08:09:30.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.370+0000 7fe2dbab4640 1 -- start start 2023-11-24T08:09:30.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.370+0000 7fe2dbab4640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2d40acd70 0x7fe2d41e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.370+0000 7fe2dbab4640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40ad690 0x7fe2d40af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.370+0000 7fe2dbab4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d4154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.370+0000 7fe2dbab4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d41f2f90 con 0x7fe2d40ae490 2023-11-24T08:09:30.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.370+0000 7fe2dbab4640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d41f2380 con 0x7fe2d40acd70 2023-11-24T08:09:30.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.370+0000 7fe2dbab4640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d41f2660 con 0x7fe2d40ad690 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2da051640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d4154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2da051640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d4154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40092/0 (socket says 172.21.15.64:40092) 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2da051640 1 -- 172.21.15.64:0/941792506 learned_addr learned my addr 172.21.15.64:0/941792506 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2d9850640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2d40acd70 0x7fe2d41e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2d904f640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40ad690 0x7fe2d40af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2da051640 1 -- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40ad690 msgr2=0x7fe2d40af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2da051640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40ad690 0x7fe2d40af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2da051640 1 -- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2d40acd70 msgr2=0x7fe2d41e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2da051640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2d40acd70 0x7fe2d41e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2da051640 1 -- 172.21.15.64:0/941792506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c40094d0 con 0x7fe2d40ae490 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2d9850640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2d40acd70 0x7fe2d41e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.371+0000 7fe2d904f640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40ad690 0x7fe2d40af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.372+0000 7fe2da051640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d4154be0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc007dc0 tx=0x7fe2cc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.372+0000 7fe2ca7fc640 1 -- 172.21.15.64:0/941792506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc003990 con 0x7fe2d40ae490 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.372+0000 7fe2ca7fc640 1 -- 172.21.15.64:0/941792506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc00f040 con 0x7fe2d40ae490 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.372+0000 7fe2ca7fc640 1 -- 172.21.15.64:0/941792506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc003d30 con 0x7fe2d40ae490 2023-11-24T08:09:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.372+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d41f2940 con 0x7fe2d40ae490 2023-11-24T08:09:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.372+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d4155f30 con 0x7fe2d40ae490 2023-11-24T08:09:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.373+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d41f3100 con 0x7fe2d40ae490 2023-11-24T08:09:30.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.374+0000 7fe2ca7fc640 1 -- 172.21.15.64:0/941792506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe2cc004030 con 0x7fe2d40ae490 2023-11-24T08:09:30.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.375+0000 7fe2ca7fc640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe2b806b4a0 0x7fe2b806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.375+0000 7fe2ca7fc640 1 -- 172.21.15.64:0/941792506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe2cc08a220 con 0x7fe2d40ae490 2023-11-24T08:09:30.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.376+0000 7fe2d9850640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe2b806b4a0 0x7fe2b806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.377+0000 7fe2d9850640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe2b806b4a0 0x7fe2b806d960 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0006fd0 tx=0x7fe2d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.380+0000 7fe2ca7fc640 1 -- 172.21.15.64:0/941792506 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2cc054b40 con 0x7fe2d40ae490 2023-11-24T08:09:30.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.572+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe2d409d200 con 0x7fe2d40ae490 2023-11-24T08:09:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.579+0000 7fe2ca7fc640 1 -- 172.21.15.64:0/941792506 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v20)=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe2cc003e90 con 0x7fe2d40ae490 2023-11-24T08:09:30.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.580+0000 7fe2ca7fc640 1 -- 172.21.15.64:0/941792506 <== mon.0 v2:172.21.15.64:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe2cc087e90 con 0x7fe2d40ae490 2023-11-24T08:09:30.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe2b806b4a0 msgr2=0x7fe2b806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe2b806b4a0 0x7fe2b806d960 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0006fd0 tx=0x7fe2d0008040 comp rx=0 tx=0).stop 2023-11-24T08:09:30.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 msgr2=0x7fe2d4154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d4154be0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc007dc0 tx=0x7fe2cc00d4a0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 shutdown_connections 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7fe2b806b4a0 0x7fe2b806d960 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2d40ae490 0x7fe2d4154be0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7fe2d40ad690 0x7fe2d40af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 --2- 172.21.15.64:0/941792506 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7fe2d40acd70 0x7fe2d41e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 >> 172.21.15.64:0/941792506 conn(0x7fe2d40b0ba0 msgr2=0x7fe2d40cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 shutdown_connections 2023-11-24T08:09:30.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.582+0000 7fe2dbab4640 1 -- 172.21.15.64:0/941792506 wait complete. 2023-11-24T08:09:30.603 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:09:30.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:09:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.828+0000 7f1afbfff640 1 Processor -- start 2023-11-24T08:09:30.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.828+0000 7f1afbfff640 1 -- start start 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.830+0000 7f1afbfff640 1 --2- >> v2:172.21.15.91:3300/0 conn(0x7f1afc143950 0x7f1afc164e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.830+0000 7f1afbfff640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1afc071700 con 0x7f1afc143570 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.830+0000 7f1afbfff640 1 -- --> v1:172.21.15.91:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1afc15b410 con 0x7f1afc0cf760 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.830+0000 7f1afbfff640 1 -- --> v2:172.21.15.91:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1afc15b130 con 0x7f1afc143950 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.830+0000 7f1afaffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1afc143570 0x7f1afc0cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60594/0 (socket says 172.21.15.64:60594) 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.830+0000 7f1afaffd640 1 -- 172.21.15.64:0/2683566836 learned_addr learned my addr 172.21.15.64:0/2683566836 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afb7fe640 1 --2- 172.21.15.64:0/2683566836 >> v2:172.21.15.91:3300/0 conn(0x7f1afc143950 0x7f1afc164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484575216 0 0) 0x7f1aec001110 con 0x7f1afc143570 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae8003520 con 0x7f1afc143570 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3995003814 0 0) 0x7f1ae8003520 con 0x7f1afc143570 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 >> v2:172.21.15.91:3300/0 conn(0x7f1afc143950 msgr2=0x7f1afc164e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 --2- 172.21.15.64:0/2683566836 >> v2:172.21.15.91:3300/0 conn(0x7f1afc143950 0x7f1afc164e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 >> v1:172.21.15.91:6789/0 conn(0x7f1afc0cf760 legacy=0x7f1afc15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1afc15dbd0 con 0x7f1afc143570 2023-11-24T08:09:30.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1077443254 0 0) 0x7f1aec001800 con 0x7f1afc143570 2023-11-24T08:09:30.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.831+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 >> v1:172.21.15.64:6789/0 conn(0x7f1afc143570 legacy=0x7f1afc0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afa7fc640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80039e0 0x7f1ae8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afa7fc640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ae80047c0 0x7f1ae8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afa7fc640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae80051f0 0x7f1ae800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc15b130 con 0x7f1ae80051f0 2023-11-24T08:09:30.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae800c150 con 0x7f1ae80047c0 2023-11-24T08:09:30.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae800c310 con 0x7f1ae80039e0 2023-11-24T08:09:30.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afb7fe640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae80051f0 0x7f1ae800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afaffd640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80039e0 0x7f1ae8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1af3fff640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ae80047c0 0x7f1ae8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afb7fe640 1 -- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80039e0 msgr2=0x7f1ae8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afb7fe640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80039e0 0x7f1ae8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afb7fe640 1 -- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ae80047c0 msgr2=0x7f1ae8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afb7fe640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ae80047c0 0x7f1ae8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1afb7fe640 1 -- 172.21.15.64:0/2683566836 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae8003520 con 0x7f1ae80051f0 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.832+0000 7f1af3fff640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1ae80047c0 0x7f1ae8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.833+0000 7f1afb7fe640 1 -- 172.21.15.64:0/2683566836 reap_dead start 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.834+0000 7f1afb7fe640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae80051f0 0x7f1ae800b980 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009f90 tx=0x7f1ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcef4dac522aec9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.834+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af402cb30 con 0x7f1ae80051f0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.834+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1af400f040 con 0x7f1ae80051f0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.834+0000 7f1afa7fc640 1 -- 172.21.15.64:0/2683566836 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4034440 con 0x7f1ae80051f0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.834+0000 7f1afbfff640 1 -- 172.21.15.64:0/2683566836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae80051f0 msgr2=0x7f1ae800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.834+0000 7f1afbfff640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae80051f0 0x7f1ae800b980 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009f90 tx=0x7f1ae80057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.835+0000 7f1afbfff640 1 -- 172.21.15.64:0/2683566836 shutdown_connections 2023-11-24T08:09:30.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.835+0000 7f1afbfff640 1 --2- 172.21.15.64:0/2683566836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae80051f0 0x7f1ae800b980 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.835+0000 7f1afbfff640 1 -- 172.21.15.64:0/2683566836 >> 172.21.15.64:0/2683566836 conn(0x7f1afc06a860 msgr2=0x7f1afc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:30.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.835+0000 7f1afbfff640 1 -- 172.21.15.64:0/2683566836 shutdown_connections 2023-11-24T08:09:30.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.835+0000 7f1afbfff640 1 -- 172.21.15.64:0/2683566836 wait complete. 2023-11-24T08:09:30.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.837+0000 7f1afbfff640 1 Processor -- start 2023-11-24T08:09:30.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.837+0000 7f1afbfff640 1 -- start start 2023-11-24T08:09:30.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1afbfff640 1 --2- >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80051f0 0x7f1afc14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1afbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1afc0c9650 0x7f1afc1f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1afbfff640 1 --2- >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1afc06d2e0 0x7f1afc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1afbfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc159c00 con 0x7f1afc0c9650 2023-11-24T08:09:30.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1afbfff640 1 -- --> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc06dca0 con 0x7f1afc06d2e0 2023-11-24T08:09:30.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1afbfff640 1 -- --> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc14cb70 con 0x7f1ae80051f0 2023-11-24T08:09:30.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1af3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1afc0c9650 0x7f1afc1f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1af3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1afc0c9650 0x7f1afc1f7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40112/0 (socket says 172.21.15.64:40112) 2023-11-24T08:09:30.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1af3fff640 1 -- 172.21.15.64:0/2655310377 learned_addr learned my addr 172.21.15.64:0/2655310377 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-11-24T08:09:30.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1afaffd640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80051f0 0x7f1afc14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1af3fff640 1 -- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80051f0 msgr2=0x7f1afc14c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1af3fff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80051f0 0x7f1afc14c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1af3fff640 1 -- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1afc06d2e0 msgr2=0x7f1afc06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.838+0000 7f1afb7fe640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1afc06d2e0 0x7f1afc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.839+0000 7f1af3fff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1afc06d2e0 0x7f1afc06d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.839+0000 7f1af3fff640 1 -- 172.21.15.64:0/2655310377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1afc13ef20 con 0x7f1afc0c9650 2023-11-24T08:09:30.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.839+0000 7f1af3fff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1afc0c9650 0x7f1afc1f7290 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400d570 tx=0x7f1ae400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.839+0000 7f1af8ff9640 1 -- 172.21.15.64:0/2655310377 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4005480 con 0x7f1afc0c9650 2023-11-24T08:09:30.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.839+0000 7f1af8ff9640 1 -- 172.21.15.64:0/2655310377 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1ae4003610 con 0x7f1afc0c9650 2023-11-24T08:09:30.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.839+0000 7f1af8ff9640 1 -- 172.21.15.64:0/2655310377 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40039e0 con 0x7f1afc0c9650 2023-11-24T08:09:30.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.839+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1afc1f77d0 con 0x7f1afc0c9650 2023-11-24T08:09:30.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.839+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc0c9e40 con 0x7f1afc0c9650 2023-11-24T08:09:30.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.840+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1afc04e840 con 0x7f1afc0c9650 2023-11-24T08:09:30.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.842+0000 7f1af8ff9640 1 -- 172.21.15.64:0/2655310377 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1ae4003b40 con 0x7f1afc0c9650 2023-11-24T08:09:30.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.842+0000 7f1af8ff9640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1adc06b470 0x7f1adc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.843+0000 7f1af8ff9640 1 -- 172.21.15.64:0/2655310377 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1ae4089df0 con 0x7f1afc0c9650 2023-11-24T08:09:30.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.843+0000 7f1afaffd640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1adc06b470 0x7f1adc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.844+0000 7f1afaffd640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1adc06b470 0x7f1adc06d930 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1aec001e20 tx=0x7f1aec001310 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:30.847+0000 7f1af8ff9640 1 -- 172.21.15.64:0/2655310377 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ae4054710 con 0x7f1afc0c9650 2023-11-24T08:09:31.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.079+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1afc0b3910 con 0x7f1afc0c9650 2023-11-24T08:09:31.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.080+0000 7f1af8ff9640 1 -- 172.21.15.64:0/2655310377 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f1ae4057d30 con 0x7f1afc0c9650 2023-11-24T08:09:31.082 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-11-24T08:09:31.082 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/970726691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":970726691},{"type":"v1","addr":"172.21.15.64:6835","nonce":970726691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6836/2338720158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6834","nonce":2338720158},{"type":"v1","addr":"172.21.15.91:6836","nonce":2338720158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.91:6837/2942920208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6835","nonce":2942920208},{"type":"v1","addr":"172.21.15.91:6837","nonce":2942920208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/229748565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":229748565},{"type":"v1","addr":"172.21.15.64:6834","nonce":229748565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2023-11-24T08:09:31.083 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 56 2023-11-24T08:09:31.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.082+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1adc06b470 msgr2=0x7f1adc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.082+0000 7f1afbfff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1adc06b470 0x7f1adc06d930 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1aec001e20 tx=0x7f1aec001310 comp rx=0 tx=0).stop 2023-11-24T08:09:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.082+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1afc0c9650 msgr2=0x7f1afc1f7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.082+0000 7f1afbfff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1afc0c9650 0x7f1afc1f7290 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400d570 tx=0x7f1ae400da30 comp rx=0 tx=0).stop 2023-11-24T08:09:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.083+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 shutdown_connections 2023-11-24T08:09:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.083+0000 7f1afbfff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:6832/401100545,v1:172.21.15.91:6833/401100545] conn(0x7f1adc06b470 0x7f1adc06d930 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.083+0000 7f1afbfff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:3300/0,v1:172.21.15.91:6789/0] conn(0x7f1afc06d2e0 0x7f1afc06d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.083+0000 7f1afbfff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1afc0c9650 0x7f1afc1f7290 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.083+0000 7f1afbfff640 1 --2- 172.21.15.64:0/2655310377 >> [v2:172.21.15.91:3301/0,v1:172.21.15.91:6790/0] conn(0x7f1ae80051f0 0x7f1afc14c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.083+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 >> 172.21.15.64:0/2655310377 conn(0x7f1afc06a860 msgr2=0x7f1afc15bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:31.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.083+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 shutdown_connections 2023-11-24T08:09:31.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-11-24T08:09:31.083+0000 7f1afbfff640 1 -- 172.21.15.64:0/2655310377 wait complete. 2023-11-24T08:09:31.105 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T08:09:31.105 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T08:09:31.105 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:09:31.106 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.125 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:09:31.126 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:09:31.126 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.193 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:09:31.194 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:09:31.194 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.213 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:09:31.213 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:09:31.213 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.276 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:09:31.276 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T08:09:31.277 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T08:09:31.277 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.338 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T08:09:31.339 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T08:09:31.339 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.387 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T08:09:31.388 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T08:09:31.388 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.454 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T08:09:31.455 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T08:09:31.455 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.522 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T08:09:31.523 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T08:09:31.523 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.576 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T08:09:31.576 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T08:09:31.576 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.632 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T08:09:31.632 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T08:09:31.632 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.692 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T08:09:31.692 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T08:09:31.692 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.755 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T08:09:31.755 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T08:09:31.755 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T08:09:31.755 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.830 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T08:09:31.830 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T08:09:31.831 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.911 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T08:09:31.911 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T08:09:31.911 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T08:09:31.911 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.964 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T08:09:31.965 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T08:09:31.965 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:31.994 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T08:09:31.994 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T08:09:31.994 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:32.073 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T08:09:32.074 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T08:09:32.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T08:09:32.190 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:09:32.190 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T08:09:32.448 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:09:32.449 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T08:09:32.613 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:09:32.614 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T08:09:32.783 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:09:32.783 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T08:09:32.950 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:09:32.951 DEBUG:teuthology.orchestra.run.smithi091:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T08:09:33.211 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:09:33.212 DEBUG:teuthology.orchestra.run.smithi091:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T08:09:33.337 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:09:33.337 DEBUG:teuthology.orchestra.run.smithi091:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T08:09:33.490 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:09:33.490 DEBUG:teuthology.orchestra.run.smithi091:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T08:09:33.655 INFO:tasks.ceph:Archiving mon data... 2023-11-24T08:09:33.657 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/data/mon.a.tgz 2023-11-24T08:09:33.657 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-11-24T08:09:33.720 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.vQN8nKKRaY 2023-11-24T08:09:33.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vQN8nKKRaY 2023-11-24T08:09:33.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.vQN8nKKRaY 2023-11-24T08:09:34.150 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.vQN8nKKRaY is 748KB 2023-11-24T08:09:34.273 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.vQN8nKKRaY 2023-11-24T08:09:34.337 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/data/mon.b.tgz 2023-11-24T08:09:34.337 DEBUG:teuthology.orchestra.run.smithi091:> mktemp 2023-11-24T08:09:34.398 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.cZ7cG19Urh 2023-11-24T08:09:34.399 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.cZ7cG19Urh 2023-11-24T08:09:34.704 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.cZ7cG19Urh 2023-11-24T08:09:34.857 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.cZ7cG19Urh is 1013KB 2023-11-24T08:09:34.978 DEBUG:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.cZ7cG19Urh 2023-11-24T08:09:35.037 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/data/mon.c.tgz 2023-11-24T08:09:35.037 DEBUG:teuthology.orchestra.run.smithi091:> mktemp 2023-11-24T08:09:35.090 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.WgcYroT6jg 2023-11-24T08:09:35.091 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WgcYroT6jg 2023-11-24T08:09:35.394 DEBUG:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.WgcYroT6jg 2023-11-24T08:09:35.546 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.WgcYroT6jg is 1013KB 2023-11-24T08:09:35.669 DEBUG:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.WgcYroT6jg 2023-11-24T08:09:35.729 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T08:09:35.729 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T08:09:35.738 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T08:09:35.841 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T08:09:35.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T08:09:35.847 DEBUG:teuthology.orchestra.run.smithi091:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T08:09:35.954 INFO:teuthology.orchestra.run.smithi091.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T08:09:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T08:09:35.968 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T08:09:35.970 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/remote/smithi064/crash 2023-11-24T08:09:35.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T08:09:36.058 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/remote/smithi091/crash 2023-11-24T08:09:36.059 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T08:09:36.158 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T08:09:36.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:36.165 DEBUG:teuthology.orchestra.run.smithi091:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:36.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:36.420 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:36.520 INFO:tasks.ceph:Compressing logs... 2023-11-24T08:09:36.521 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T08:09:36.530 DEBUG:teuthology.orchestra.run.smithi091:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T08:09:36.619 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117969.log 2023-11-24T08:09:36.621 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101043.log 2023-11-24T08:09:36.621 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94579.log 2023-11-24T08:09:36.622 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97848.log 2023-11-24T08:09:36.622 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117969.log.gz 2023-11-24T08:09:36.623 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T08:09:36.624 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93504.log 2023-11-24T08:09:36.624 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101043.log.gz 2023-11-24T08:09:36.625 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112737.log 2023-11-24T08:09:36.625 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97848.log.gz 2023-11-24T08:09:36.625 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.1.94579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88559.log 2023-11-24T08:09:36.625 INFO:teuthology.orchestra.run.smithi091.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.1.94579.log.gz 2023-11-24T08:09:36.625 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88319.log 2023-11-24T08:09:36.627 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.admin.88559.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.88559.log.gz 2023-11-24T08:09:36.627 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97915.log 2023-11-24T08:09:36.627 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94853.log 2023-11-24T08:09:36.627 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.93504.log.gz 2023-11-24T08:09:36.628 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99401.log 2023-11-24T08:09:36.628 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112737.log.gz 2023-11-24T08:09:36.629 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94853.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94853.log.gz 2023-11-24T08:09:36.629 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100083.log 2023-11-24T08:09:36.630 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.admin.88319.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88319.log.gzgzip 2023-11-24T08:09:36.630 INFO:teuthology.orchestra.run.smithi091.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.88479.log 2023-11-24T08:09:36.630 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.1.97915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88359.log 2023-11-24T08:09:36.630 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.admin.88479.log: 89.1% -- replaced with /var/log/ceph/ceph-client.1.97915.log.gz 2023-11-24T08:09:36.630 INFO:teuthology.orchestra.run.smithi091.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88479.log.gz 2023-11-24T08:09:36.630 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88399.log 2023-11-24T08:09:36.632 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.admin.88359.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T08:09:36.634 INFO:teuthology.orchestra.run.smithi091.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88359.log.gz 2023-11-24T08:09:36.636 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99401.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99401.log.gz 2023-11-24T08:09:36.637 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95192.log 2023-11-24T08:09:36.637 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107405.log 2023-11-24T08:09:36.637 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100083.log.gz 2023-11-24T08:09:36.637 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95192.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95192.log.gz 2023-11-24T08:09:36.637 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T08:09:36.637 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110477.log 2023-11-24T08:09:36.637 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107405.log.gz 2023-11-24T08:09:36.637 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117675.log 2023-11-24T08:09:36.638 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110477.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110477.log.gz 2023-11-24T08:09:36.638 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111259.log 2023-11-24T08:09:36.640 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109329.log 2023-11-24T08:09:36.640 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117675.log.gz 2023-11-24T08:09:36.640 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111259.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111259.log.gz 2023-11-24T08:09:36.640 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99075.log 2023-11-24T08:09:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.admin.88399.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88399.log.gz 2023-11-24T08:09:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T08:09:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T08:09:36.643 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98580.log 2023-11-24T08:09:36.643 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose 93.7% -- -- replaced with /var/log/ceph/ceph.audit.log.gz /var/log/ceph/ceph-client.admin.88519.log 2023-11-24T08:09:36.643 INFO:teuthology.orchestra.run.smithi091.stderr: 2023-11-24T08:09:36.643 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.1.98580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97180.log 2023-11-24T08:09:36.643 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.admin.88519.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88519.log.gz 2023-11-24T08:09:36.643 INFO:teuthology.orchestra.run.smithi091.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.1.98580.log.gz 2023-11-24T08:09:36.644 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T08:09:36.644 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.1.97180.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T08:09:36.645 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-osd.7.log: 89.5% -- replaced with /var/log/ceph/ceph-client.1.97180.log.gz 2023-11-24T08:09:36.645 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.96513.log 2023-11-24T08:09:36.646 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109329.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109329.log.gz 2023-11-24T08:09:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112695.log 2023-11-24T08:09:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99075.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99075.log.gz 2023-11-24T08:09:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100713.log 2023-11-24T08:09:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112695.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112695.log.gz 2023-11-24T08:09:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94094.log 2023-11-24T08:09:36.647 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106580.log 2023-11-24T08:09:36.648 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100713.log.gz 2023-11-24T08:09:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94094.log: gzip 89.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.94094.log.gz --verbose 2023-11-24T08:09:36.648 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.117759.log 2023-11-24T08:09:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93884.log 2023-11-24T08:09:36.648 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106580.log.gz 2023-11-24T08:09:36.650 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T08:09:36.650 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-client.1.96513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88279.log 2023-11-24T08:09:36.650 INFO:teuthology.orchestra.run.smithi091.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.1.96513.log.gz 2023-11-24T08:09:36.650 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.88279.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88279.log.gz 2023-11-24T08:09:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117759.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117759.log.gzgzip 2023-11-24T08:09:36.651 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107027.log 2023-11-24T08:09:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100671.log 2023-11-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93884.log.gz 2023-11-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108637.log 2023-11-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107027.log.gz 2023-11-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99317.log 2023-11-24T08:09:36.652 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100671.log.gz 2023-11-24T08:09:36.653 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108637.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108637.log.gz 2023-11-24T08:09:36.654 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.113646.log 2023-11-24T08:09:36.654 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101591.log 2023-11-24T08:09:36.654 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99317.log.gz 2023-11-24T08:09:36.655 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T08:09:36.655 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.95849.log 2023-11-24T08:09:36.656 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113646.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113646.log.gz 2023-11-24T08:09:36.656 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114127.log 2023-11-24T08:09:36.656 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104744.log 2023-11-24T08:09:36.656 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101591.log.gz 2023-11-24T08:09:36.658 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98363.log 2023-11-24T08:09:36.658 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114127.log.gz 2023-11-24T08:09:36.658 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.109190.log 2023-11-24T08:09:36.658 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104744.log.gz 2023-11-24T08:09:36.658 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.1.95849.log: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T08:09:36.659 INFO:teuthology.orchestra.run.smithi091.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.1.95849.log.gz 2023-11-24T08:09:36.659 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88439.log 2023-11-24T08:09:36.659 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-24T08:09:36.660 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.95099.log 2023-11-24T08:09:36.660 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98363.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98363.log.gz 2023-11-24T08:09:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107111.log 2023-11-24T08:09:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.109190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115508.log 2023-11-24T08:09:36.661 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107111.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107111.log.gz 2023-11-24T08:09:36.662 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107957.log 2023-11-24T08:09:36.662 INFO:teuthology.orchestra.run.smithi064.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.109190.log.gz 2023-11-24T08:09:36.663 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97276.log 2023-11-24T08:09:36.663 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115508.log.gz 2023-11-24T08:09:36.664 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.3.95099.log: /var/log/ceph/ceph-client.admin.88439.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.88439.log.gz 2023-11-24T08:09:36.664 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107957.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107957.log.gz 2023-11-24T08:09:36.664 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100587.log 2023-11-24T08:09:36.665 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T08:09:36.665 INFO:teuthology.orchestra.run.smithi091.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.3.95099.log.gz 2023-11-24T08:09:36.666 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115298.log 2023-11-24T08:09:36.666 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97276.log.gz 2023-11-24T08:09:36.667 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99485.log 2023-11-24T08:09:36.667 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100587.log.gz 2023-11-24T08:09:36.668 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106826.log 2023-11-24T08:09:36.668 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115298.log.gz 2023-11-24T08:09:36.668 INFO:teuthology.orchestra.run.smithi091.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T08:09:36.669 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99485.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99485.log.gz 2023-11-24T08:09:36.669 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99443.log 2023-11-24T08:09:36.670 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118536.log 2023-11-24T08:09:36.670 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106826.log.gz 2023-11-24T08:09:36.671 INFO:teuthology.orchestra.run.smithi091.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-mds.d.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T08:09:36.671 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99443.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99443.log.gz 2023-11-24T08:09:36.671 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107820.log 2023-11-24T08:09:36.673 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94848.log 2023-11-24T08:09:36.673 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118536.log.gz 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107820.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107820.log.gz 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93840.log 2023-11-24T08:09:36.675 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94848.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94848.log.gz 2023-11-24T08:09:36.675 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111467.log 2023-11-24T08:09:36.676 INFO:teuthology.orchestra.run.smithi091.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T08:09:36.676 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96026.log 2023-11-24T08:09:36.677 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93840.log.gz 2023-11-24T08:09:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111467.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111467.log.gz 2023-11-24T08:09:36.678 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105884.log 2023-11-24T08:09:36.679 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.96026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114337.log 2023-11-24T08:09:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105884.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.96026.log.gz 2023-11-24T08:09:36.681 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105884.log.gz 2023-11-24T08:09:36.681 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101086.log 2023-11-24T08:09:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114337.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114337.log.gz 2023-11-24T08:09:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118179.log 2023-11-24T08:09:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101086.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.97232.log -- replaced with /var/log/ceph/ceph-client.admin.101086.log.gz 2023-11-24T08:09:36.683 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94862.log 2023-11-24T08:09:36.685 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118179.log.gz 2023-11-24T08:09:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97232.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97232.log.gz 2023-11-24T08:09:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104922.log 2023-11-24T08:09:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99159.log 2023-11-24T08:09:36.687 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94862.log.gz 2023-11-24T08:09:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103153.log 2023-11-24T08:09:36.688 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104922.log.gz 2023-11-24T08:09:36.689 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98865.log 2023-11-24T08:09:36.689 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99159.log.gz 2023-11-24T08:09:36.690 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103153.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103153.log.gz 2023-11-24T08:09:36.691 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111597.log 2023-11-24T08:09:36.692 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107237.log 2023-11-24T08:09:36.692 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98865.log.gz 2023-11-24T08:09:36.693 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97764.log 2023-11-24T08:09:36.693 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111597.log.gz 2023-11-24T08:09:36.694 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107237.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107237.log.gz 2023-11-24T08:09:36.694 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.94863.log 2023-11-24T08:09:36.696 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118305.log 2023-11-24T08:09:36.696 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97764.log.gz 2023-11-24T08:09:36.697 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94863.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94863.log.gz 2023-11-24T08:09:36.697 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94861.log 2023-11-24T08:09:36.698 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105745.log 2023-11-24T08:09:36.699 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118305.log.gz 2023-11-24T08:09:36.699 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117164.log 2023-11-24T08:09:36.700 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94861.log.gz 2023-11-24T08:09:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.105745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97722.log 2023-11-24T08:09:36.702 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107321.log 2023-11-24T08:09:36.703 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117164.log.gz 2023-11-24T08:09:36.704 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97680.log 2023-11-24T08:09:36.704 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97722.log.gz 2023-11-24T08:09:36.705 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111383.log 2023-11-24T08:09:36.706 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107321.log.gz 2023-11-24T08:09:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97680.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97680.log.gz 2023-11-24T08:09:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110832.log 2023-11-24T08:09:36.708 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111383.log: 92.4% -- replaced with /var/log/ceph/ceph-client.0.105745.log.gz 2023-11-24T08:09:36.708 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111383.log.gz 2023-11-24T08:09:36.709 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99873.log 2023-11-24T08:09:36.710 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100293.log 2023-11-24T08:09:36.710 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110832.log.gz 2023-11-24T08:09:36.711 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99873.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99873.log.gz 2023-11-24T08:09:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97386.log 2023-11-24T08:09:36.713 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117927.log 2023-11-24T08:09:36.713 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100293.log.gz 2023-11-24T08:09:36.714 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104572.log 2023-11-24T08:09:36.714 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97386.log.gz 2023-11-24T08:09:36.715 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117038.log 2023-11-24T08:09:36.716 INFO:teuthology.orchestra.run.smithi064.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117927.log.gz 2023-11-24T08:09:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114866.log 2023-11-24T08:09:36.717 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104572.log.gz 2023-11-24T08:09:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113772.log 2023-11-24T08:09:36.718 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117038.log.gz 2023-11-24T08:09:36.719 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106240.log 2023-11-24T08:09:36.719 INFO:teuthology.orchestra.run.smithi064.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114866.log.gz 2023-11-24T08:09:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112779.log 2023-11-24T08:09:36.720 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113772.log.gz 2023-11-24T08:09:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118576.log 2023-11-24T08:09:36.722 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106240.log.gz 2023-11-24T08:09:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110958.log 2023-11-24T08:09:36.723 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112779.log.gz 2023-11-24T08:09:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105258.log 2023-11-24T08:09:36.724 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118576.log.gz 2023-11-24T08:09:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93627.log 2023-11-24T08:09:36.725 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110958.log.gz 2023-11-24T08:09:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105258.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105258.log.gz 2023-11-24T08:09:36.727 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.113487.log 2023-11-24T08:09:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93627.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93627.log.gz 2023-11-24T08:09:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117080.log 2023-11-24T08:09:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113487.log: gzip -5 --verbose 88.3% -- -- replaced with /var/log/ceph/ceph-client.admin.113487.log.gz /var/log/ceph/ceph-client.admin.99957.log 2023-11-24T08:09:36.729 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105360.log 2023-11-24T08:09:36.731 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117080.log.gz 2023-11-24T08:09:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104406.log 2023-11-24T08:09:36.732 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99957.log.gz 2023-11-24T08:09:36.733 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96722.log 2023-11-24T08:09:36.733 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105360.log.gz 2023-11-24T08:09:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98590.log 2023-11-24T08:09:36.734 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104406.log.gz 2023-11-24T08:09:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.96722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115424.log 2023-11-24T08:09:36.735 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96722.log.gz 2023-11-24T08:09:36.736 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112087.log 2023-11-24T08:09:36.737 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98590.log.gz 2023-11-24T08:09:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104231.log 2023-11-24T08:09:36.738 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115424.log.gz 2023-11-24T08:09:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102907.log 2023-11-24T08:09:36.739 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.112087.log.gz 2023-11-24T08:09:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104231.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104231.log.gz 2023-11-24T08:09:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104700.log 2023-11-24T08:09:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107195.log 2023-11-24T08:09:36.742 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102907.log.gz 2023-11-24T08:09:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104700.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104700.log.gz 2023-11-24T08:09:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104189.log 2023-11-24T08:09:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94511.log 2023-11-24T08:09:36.745 INFO:teuthology.orchestra.run.smithi064.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107195.log.gz 2023-11-24T08:09:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105090.log 2023-11-24T08:09:36.746 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104189.log.gz 2023-11-24T08:09:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113319.log 2023-11-24T08:09:36.748 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94511.log.gz 2023-11-24T08:09:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105090.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105090.log.gz 2023-11-24T08:09:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113562.log 2023-11-24T08:09:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116321.log 2023-11-24T08:09:36.750 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113319.log.gz 2023-11-24T08:09:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116405.log 2023-11-24T08:09:36.752 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113562.log.gz 2023-11-24T08:09:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101927.log 2023-11-24T08:09:36.753 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116321.log.gz 2023-11-24T08:09:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118347.log 88.9% 2023-11-24T08:09:36.754 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116405.log.gz 2023-11-24T08:09:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101717.log 2023-11-24T08:09:36.755 INFO:teuthology.orchestra.run.smithi064.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101927.log.gz 2023-11-24T08:09:36.756 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112430.log 2023-11-24T08:09:36.756 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118347.log.gz 2023-11-24T08:09:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104147.log 2023-11-24T08:09:36.758 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101717.log.gz 2023-11-24T08:09:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.112430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114397.log 2023-11-24T08:09:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104147.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104147.log.gz 2023-11-24T08:09:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109946.log 2023-11-24T08:09:36.762 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100629.log 2023-11-24T08:09:36.762 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114397.log.gz 2023-11-24T08:09:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109946.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109946.log.gz 2023-11-24T08:09:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111919.log 2023-11-24T08:09:36.764 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.112430.log.gz 2023-11-24T08:09:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111877.log 2023-11-24T08:09:36.766 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100629.log.gz 2023-11-24T08:09:36.767 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111919.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111919.log.gz 2023-11-24T08:09:36.767 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103498.log 2023-11-24T08:09:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111877.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.111877.log.gz /var/log/ceph/ceph-client.admin.96128.log 2023-11-24T08:09:36.769 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:36.770 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100377.log 2023-11-24T08:09:36.770 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103498.log.gz 2023-11-24T08:09:36.771 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.96128.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96128.log.gz 2023-11-24T08:09:36.771 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107862.log 2023-11-24T08:09:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108467.log 2023-11-24T08:09:36.773 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100377.log.gz 2023-11-24T08:09:36.774 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107862.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107862.log.gz 2023-11-24T08:09:36.774 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104656.log 2023-11-24T08:09:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108467.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108467.log.gz 2023-11-24T08:09:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110747.log 2023-11-24T08:09:36.777 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116447.log 2023-11-24T08:09:36.777 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104656.log.gz 2023-11-24T08:09:36.778 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110747.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.110747.log.gz --verbose 2023-11-24T08:09:36.779 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.94262.log 2023-11-24T08:09:36.780 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89090.log 2023-11-24T08:09:36.780 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116447.log.gz 2023-11-24T08:09:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94262.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94262.log.gz 2023-11-24T08:09:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112003.log 2023-11-24T08:09:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.89090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110519.log 2023-11-24T08:09:36.783 INFO:teuthology.orchestra.run.smithi064.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.89090.log.gz 2023-11-24T08:09:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112003.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112003.log.gz 2023-11-24T08:09:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99627.log 2023-11-24T08:09:36.785 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95858.log 2023-11-24T08:09:36.785 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110519.log.gz 2023-11-24T08:09:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99627.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.99627.log.gz /var/log/ceph/ceph-client.admin.108425.log 2023-11-24T08:09:36.787 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:36.788 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108679.log 2023-11-24T08:09:36.788 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95858.log.gz 2023-11-24T08:09:36.790 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108425.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108425.log.gz 2023-11-24T08:09:36.790 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115002.log 2023-11-24T08:09:36.791 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114085.log 2023-11-24T08:09:36.791 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108679.log.gz 2023-11-24T08:09:36.793 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98058.log 2023-11-24T08:09:36.793 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115002.log.gz 2023-11-24T08:09:36.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114085.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.114085.log.gz 2023-11-24T08:09:36.794 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109664.log 2023-11-24T08:09:36.795 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106910.log 2023-11-24T08:09:36.796 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98058.log.gz 2023-11-24T08:09:36.797 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98702.log 2023-11-24T08:09:36.798 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109664.log.gz 2023-11-24T08:09:36.798 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106910.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106910.log.gz 2023-11-24T08:09:36.798 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109904.log 2023-11-24T08:09:36.800 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98702.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98702.log.gz 2023-11-24T08:09:36.800 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98100.log 2023-11-24T08:09:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113403.log 2023-11-24T08:09:36.801 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109904.log.gz 2023-11-24T08:09:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95564.log 2023-11-24T08:09:36.802 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98100.log.gz 2023-11-24T08:09:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109581.log 2023-11-24T08:09:36.804 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113403.log.gz 2023-11-24T08:09:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93712.log 2023-11-24T08:09:36.805 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95564.log.gz 2023-11-24T08:09:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109581.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109581.log.gz 2023-11-24T08:09:36.807 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112569.log 2023-11-24T08:09:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93712.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.93712.log.gz 2023-11-24T08:09:36.808 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.115550.log 2023-11-24T08:09:36.809 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117717.log 2023-11-24T08:09:36.810 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112569.log.gz 2023-11-24T08:09:36.811 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109413.log 2023-11-24T08:09:36.811 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117717.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.115550.log.gz 2023-11-24T08:09:36.812 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115256.log 2023-11-24T08:09:36.812 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117717.log.gz 2023-11-24T08:09:36.814 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108847.log 2023-11-24T08:09:36.814 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109413.log.gz 2023-11-24T08:09:36.815 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115256.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115256.log.gz 2023-11-24T08:09:36.815 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105174.log 2023-11-24T08:09:36.816 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104487.log 2023-11-24T08:09:36.817 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.108847.log.gz 2023-11-24T08:09:36.818 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105174.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105174.log.gz 2023-11-24T08:09:36.818 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103237.log 2023-11-24T08:09:36.820 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102161.log 2023-11-24T08:09:36.820 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104487.log.gz 2023-11-24T08:09:36.821 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103237.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103237.log.gz 2023-11-24T08:09:36.821 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95124.log 2023-11-24T08:09:36.823 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98949.log 2023-11-24T08:09:36.823 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102161.log.gz 2023-11-24T08:09:36.824 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95124.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95124.log.gz 2023-11-24T08:09:36.824 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113856.log 2023-11-24T08:09:36.826 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98949.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T08:09:36.826 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98949.log.gz 2023-11-24T08:09:36.827 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113856.log: gzip -5 --verbose 88.8% -- /var/log/ceph/ceph-client.admin.101759.log -- replaced with /var/log/ceph/ceph-client.admin.113856.log.gz 2023-11-24T08:09:36.827 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103937.log 2023-11-24T08:09:36.830 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107363.log 2023-11-24T08:09:36.830 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101759.log.gz 2023-11-24T08:09:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103937.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T08:09:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109497.log 2023-11-24T08:09:36.832 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103937.log.gz 2023-11-24T08:09:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107363.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107363.log.gz 2023-11-24T08:09:36.833 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105132.log 2023-11-24T08:09:36.834 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101367.log 2023-11-24T08:09:36.834 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109497.log.gz 2023-11-24T08:09:36.838 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95190.log 2023-11-24T08:09:36.838 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105132.log.gz 2023-11-24T08:09:36.838 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108287.log 2023-11-24T08:09:36.838 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101367.log.gz 2023-11-24T08:09:36.839 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114034.log 2023-11-24T08:09:36.839 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95190.log.gz 2023-11-24T08:09:36.840 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108913.log 2023-11-24T08:09:36.840 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108287.log.gz 2023-11-24T08:09:36.841 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103792.log 2023-11-24T08:09:36.841 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114034.log.gz 2023-11-24T08:09:36.842 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101801.log 2023-11-24T08:09:36.842 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108913.log.gz 2023-11-24T08:09:36.843 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98489.log 2023-11-24T08:09:36.843 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103792.log.gz 2023-11-24T08:09:36.844 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95732.log 2023-11-24T08:09:36.844 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101801.log.gz 2023-11-24T08:09:36.845 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118389.log 2023-11-24T08:09:36.845 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98489.log.gz 2023-11-24T08:09:36.846 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111509.log 2023-11-24T08:09:36.847 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95732.log.gz 2023-11-24T08:09:36.848 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118095.log 2023-11-24T08:09:36.848 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118389.log.gz 2023-11-24T08:09:36.849 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105468.log 2023-11-24T08:09:36.849 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111509.log.gz 2023-11-24T08:09:36.850 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117308.log 2023-11-24T08:09:36.850 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118095.log.gz 2023-11-24T08:09:36.852 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105468.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.105468.log.gz 2023-11-24T08:09:36.852 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.94304.log 2023-11-24T08:09:36.853 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94528.log 2023-11-24T08:09:36.854 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117308.log.gz 2023-11-24T08:09:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94304.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94304.log.gz 2023-11-24T08:09:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110916.log 2023-11-24T08:09:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108118.log 2023-11-24T08:09:36.857 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94528.log.gz 2023-11-24T08:09:36.858 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113730.log 2023-11-24T08:09:36.858 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110916.log.gz 2023-11-24T08:09:36.859 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103708.log 2023-11-24T08:09:36.859 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108118.log.gz 2023-11-24T08:09:36.860 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110192.log 2023-11-24T08:09:36.860 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113730.log.gz 2023-11-24T08:09:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111553.log 2023-11-24T08:09:36.861 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103708.log.gz 2023-11-24T08:09:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118431.log 2023-11-24T08:09:36.863 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110192.log.gz 2023-11-24T08:09:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97974.log 2023-11-24T08:09:36.864 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111553.log.gz 2023-11-24T08:09:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100545.log 2023-11-24T08:09:36.865 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118431.log.gz 2023-11-24T08:09:36.867 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114492.log 2023-11-24T08:09:36.867 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97974.log.gz 2023-11-24T08:09:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100545.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100545.log.gz 2023-11-24T08:09:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110687.log 2023-11-24T08:09:36.869 INFO:teuthology.orchestra.run.smithi091.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T08:09:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110066.log 2023-11-24T08:09:36.870 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114492.log.gz 2023-11-24T08:09:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103666.log 2023-11-24T08:09:36.871 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110687.log.gz 2023-11-24T08:09:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.110066.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.94136.log 2023-11-24T08:09:36.873 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110066.log.gz 2023-11-24T08:09:36.873 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108805.log 2023-11-24T08:09:36.874 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103666.log.gz 2023-11-24T08:09:36.875 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101547.log 2023-11-24T08:09:36.875 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94136.log.gz 2023-11-24T08:09:36.876 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108805.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108805.log.gz 2023-11-24T08:09:36.877 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115382.log 2023-11-24T08:09:36.878 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101675.log 2023-11-24T08:09:36.878 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101547.log.gz 2023-11-24T08:09:36.879 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114572.log 2023-11-24T08:09:36.880 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115382.log.gz 2023-11-24T08:09:36.881 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111689.log 2023-11-24T08:09:36.881 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101675.log.gz 2023-11-24T08:09:36.882 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115340.log 2023-11-24T08:09:36.883 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114572.log.gz 2023-11-24T08:09:36.883 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111689.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111689.log.gz 2023-11-24T08:09:36.884 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97554.log 2023-11-24T08:09:36.885 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111775.log 2023-11-24T08:09:36.885 INFO:teuthology.orchestra.run.smithi064.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115340.log.gz 2023-11-24T08:09:36.886 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107694.log 2023-11-24T08:09:36.887 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97554.log.gz 2023-11-24T08:09:36.888 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95201.log 2023-11-24T08:09:36.888 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111775.log.gz 2023-11-24T08:09:36.889 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104021.log 2023-11-24T08:09:36.889 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107694.log.gz 2023-11-24T08:09:36.890 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112653.log 2023-11-24T08:09:36.891 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95201.log.gz 2023-11-24T08:09:36.892 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105006.log 2023-11-24T08:09:36.892 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104021.log.gz 2023-11-24T08:09:36.893 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112653.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112653.log.gz 2023-11-24T08:09:36.894 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108595.log 2023-11-24T08:09:36.895 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110561.log 2023-11-24T08:09:36.895 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105006.log.gz 2023-11-24T08:09:36.896 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116669.log 2023-11-24T08:09:36.896 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108595.log.gz 2023-11-24T08:09:36.897 INFO:teuthology.orchestra.run.smithi091.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T08:09:36.898 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100917.log 2023-11-24T08:09:36.898 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110561.log.gz 2023-11-24T08:09:36.899 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104529.log 2023-11-24T08:09:36.899 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116669.log.gz 2023-11-24T08:09:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100917.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100917.log.gz 2023-11-24T08:09:36.901 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98787.log 2023-11-24T08:09:36.902 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112153.log 2023-11-24T08:09:36.903 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104529.log.gz 2023-11-24T08:09:36.903 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102745.log 2023-11-24T08:09:36.904 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98787.log.gz 2023-11-24T08:09:36.905 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99711.log 2023-11-24T08:09:36.905 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112153.log.gz 2023-11-24T08:09:36.906 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102745.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102745.log.gz 2023-11-24T08:09:36.907 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101283.log 2023-11-24T08:09:36.908 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111084.log 2023-11-24T08:09:36.908 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99711.log.gz 2023-11-24T08:09:36.909 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95396.log 2023-11-24T08:09:36.909 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101283.log.gz 2023-11-24T08:09:36.910 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114211.log 2023-11-24T08:09:36.911 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111084.log.gz 2023-11-24T08:09:36.911 INFO:teuthology.orchestra.run.smithi091.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T08:09:36.912 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108553.log 2023-11-24T08:09:36.912 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95396.log.gz 2023-11-24T08:09:36.913 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114211.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.114211.log.gz 2023-11-24T08:09:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113898.log 2023-11-24T08:09:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117248.log 2023-11-24T08:09:36.915 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108553.log.gz 2023-11-24T08:09:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113898.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T08:09:36.916 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113898.log.gz 2023-11-24T08:09:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115214.log 2023-11-24T08:09:36.917 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117248.log.gz 2023-11-24T08:09:36.918 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108721.log 2023-11-24T08:09:36.920 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114822.log 2023-11-24T08:09:36.920 INFO:teuthology.orchestra.run.smithi064.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.115214.log.gz 2023-11-24T08:09:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113115.log 2023-11-24T08:09:36.922 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108721.log.gz 2023-11-24T08:09:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114822.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114822.log.gz 2023-11-24T08:09:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115466.log 2023-11-24T08:09:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100419.log 2023-11-24T08:09:36.925 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113115.log.gz 2023-11-24T08:09:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115466.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115466.log.gz 2023-11-24T08:09:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103075.log 2023-11-24T08:09:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102053.log 2023-11-24T08:09:36.928 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100419.log.gz 2023-11-24T08:09:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103075.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103075.log.gz 2023-11-24T08:09:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95984.log 2023-11-24T08:09:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97890.log 2023-11-24T08:09:36.931 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102053.log.gz 2023-11-24T08:09:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95984.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95984.log.gz 2023-11-24T08:09:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100167.log 2023-11-24T08:09:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99201.log 2023-11-24T08:09:36.935 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97890.log.gz 2023-11-24T08:09:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100167.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100167.log.gz 2023-11-24T08:09:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118053.log 2023-11-24T08:09:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113361.log 2023-11-24T08:09:36.938 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99201.log.gz 2023-11-24T08:09:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118053.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118053.log.gz 2023-11-24T08:09:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115130.log 2023-11-24T08:09:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105402.log 2023-11-24T08:09:36.941 INFO:teuthology.orchestra.run.smithi064.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113361.log.gz 2023-11-24T08:09:36.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115130.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115130.log.gz 2023-11-24T08:09:36.944 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108763.log 2023-11-24T08:09:36.945 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102703.log 2023-11-24T08:09:36.947 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108763.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.105402.log.gz 2023-11-24T08:09:36.947 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108763.log.gz 2023-11-24T08:09:36.948 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95186.log 2023-11-24T08:09:36.949 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107736.log 2023-11-24T08:09:36.949 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102703.log.gz 2023-11-24T08:09:36.950 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95186.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95186.log.gz 2023-11-24T08:09:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99243.log 2023-11-24T08:09:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107736.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82647.log 2023-11-24T08:09:36.952 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107736.log.gz 2023-11-24T08:09:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88706.log 2023-11-24T08:09:36.954 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99243.log.gz 2023-11-24T08:09:36.955 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.tmp-client.admin.82647.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82647.log.gz 2023-11-24T08:09:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112045.log 2023-11-24T08:09:36.956 INFO:teuthology.orchestra.run.smithi091.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T08:09:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.88706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98405.log 2023-11-24T08:09:36.957 INFO:teuthology.orchestra.run.smithi064.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.88706.log.gz 2023-11-24T08:09:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112045.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112045.log.gz 2023-11-24T08:09:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98184.log 2023-11-24T08:09:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102787.log 2023-11-24T08:09:36.961 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98405.log.gz 2023-11-24T08:09:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98184.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98184.log.gz 2023-11-24T08:09:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95942.log 2023-11-24T08:09:36.964 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104880.log 2023-11-24T08:09:36.964 INFO:teuthology.orchestra.run.smithi064.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.102787.log.gz 2023-11-24T08:09:36.966 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95942.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95942.log.gz 2023-11-24T08:09:36.966 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116828.log 2023-11-24T08:09:36.967 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114780.log 2023-11-24T08:09:36.968 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104880.log.gz 2023-11-24T08:09:36.968 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116828.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116828.log.gz 2023-11-24T08:09:36.969 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107153.log 2023-11-24T08:09:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102619.log 2023-11-24T08:09:36.971 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114780.log.gz 2023-11-24T08:09:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107153.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107153.log.gz 2023-11-24T08:09:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107447.log 2023-11-24T08:09:36.974 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106784.log 2023-11-24T08:09:36.974 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102619.log.gz 2023-11-24T08:09:36.975 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107447.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107447.log.gz 2023-11-24T08:09:36.975 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97596.log 2023-11-24T08:09:36.977 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107279.log 2023-11-24T08:09:36.977 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106784.log.gz 2023-11-24T08:09:36.978 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97596.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97596.log.gz 2023-11-24T08:09:36.979 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94178.log 2023-11-24T08:09:36.980 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98447.log 2023-11-24T08:09:36.981 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107279.log.gz 2023-11-24T08:09:36.981 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94178.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94178.log.gz 2023-11-24T08:09:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103540.log 2023-11-24T08:09:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101241.log 2023-11-24T08:09:36.984 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98447.log.gz 2023-11-24T08:09:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103540.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103540.log.gz 2023-11-24T08:09:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94527.log 2023-11-24T08:09:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112935.log 2023-11-24T08:09:36.987 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101241.log.gz 2023-11-24T08:09:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94527.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94527.log.gz 2023-11-24T08:09:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101411.log 2023-11-24T08:09:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114738.log 2023-11-24T08:09:36.991 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112935.log.gz 2023-11-24T08:09:36.992 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101411.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101411.log.gz 2023-11-24T08:09:36.992 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97806.log 2023-11-24T08:09:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103339.log 2023-11-24T08:09:36.994 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114738.log.gz 2023-11-24T08:09:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97806.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97806.log.gz 2023-11-24T08:09:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94052.log 2023-11-24T08:09:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105048.log 2023-11-24T08:09:36.997 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103339.log.gz 2023-11-24T08:09:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94052.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94052.log.gz 2023-11-24T08:09:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93309.log 2023-11-24T08:09:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110150.log 2023-11-24T08:09:37.001 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105048.log.gz 2023-11-24T08:09:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93309.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93309.log.gz 2023-11-24T08:09:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101505.log 2023-11-24T08:09:37.003 INFO:teuthology.orchestra.run.smithi091.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T08:09:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103834.log 2023-11-24T08:09:37.004 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110150.log.gz 2023-11-24T08:09:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101505.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101505.log.gz 2023-11-24T08:09:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117633.log 2023-11-24T08:09:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116996.log 2023-11-24T08:09:37.007 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103834.log.gz 2023-11-24T08:09:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117633.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117633.log.gz 2023-11-24T08:09:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116912.log 2023-11-24T08:09:37.010 INFO:teuthology.orchestra.run.smithi091.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T08:09:37.010 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115088.log 2023-11-24T08:09:37.011 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116996.log.gz 2023-11-24T08:09:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116912.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116912.log.gz 2023-11-24T08:09:37.012 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102991.log 2023-11-24T08:09:37.014 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103456.log 2023-11-24T08:09:37.014 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115088.log.gz 2023-11-24T08:09:37.015 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102991.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102991.log.gz 2023-11-24T08:09:37.015 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102011.log 2023-11-24T08:09:37.017 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101198.log 2023-11-24T08:09:37.017 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103456.log.gz 2023-11-24T08:09:37.018 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102011.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102011.log.gz 2023-11-24T08:09:37.019 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109820.log 2023-11-24T08:09:37.020 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95522.log 2023-11-24T08:09:37.020 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101198.log.gz 2023-11-24T08:09:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109820.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109820.log.gz 2023-11-24T08:09:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110351.log 2023-11-24T08:09:37.024 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100209.log 2023-11-24T08:09:37.024 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95522.log.gz 2023-11-24T08:09:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110351.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110351.log.gz 2023-11-24T08:09:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112875.log 2023-11-24T08:09:37.027 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98016.log 2023-11-24T08:09:37.027 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100209.log.gz 2023-11-24T08:09:37.028 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112875.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.112875.log.gz 2023-11-24T08:09:37.029 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98142.log 2023-11-24T08:09:37.030 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105216.log 2023-11-24T08:09:37.031 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98016.log.gz 2023-11-24T08:09:37.031 INFO:teuthology.orchestra.run.smithi091.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T08:09:37.032 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98142.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98142.log.gz 2023-11-24T08:09:37.032 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110108.log 2023-11-24T08:09:37.033 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107610.log 2023-11-24T08:09:37.034 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105216.log.gz 2023-11-24T08:09:37.035 INFO:teuthology.orchestra.run.smithi091.stderr: 2023-11-24T08:09:37.035 INFO:teuthology.orchestra.run.smithi091.stderr:real 0m0.454s 2023-11-24T08:09:37.036 INFO:teuthology.orchestra.run.smithi091.stderr:user 0m2.053s 2023-11-24T08:09:37.036 INFO:teuthology.orchestra.run.smithi091.stderr:sys 0m0.309s 2023-11-24T08:09:37.036 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110108.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110108.log.gz 2023-11-24T08:09:37.036 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109539.log 2023-11-24T08:09:37.038 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111042.log 2023-11-24T08:09:37.038 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107610.log.gz 2023-11-24T08:09:37.038 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100461.log 2023-11-24T08:09:37.039 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109539.log.gz 2023-11-24T08:09:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111042.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111042.log.gz 2023-11-24T08:09:37.040 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103979.log 2023-11-24T08:09:37.041 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103750.log 2023-11-24T08:09:37.041 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100461.log.gz 2023-11-24T08:09:37.042 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103979.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103979.log.gz 2023-11-24T08:09:37.042 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107778.log 2023-11-24T08:09:37.043 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103750.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T08:09:37.043 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103750.log.gz 2023-11-24T08:09:37.044 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107778.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107778.log.gz 2023-11-24T08:09:37.044 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110393.log 2023-11-24T08:09:37.045 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106538.log 2023-11-24T08:09:37.047 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110393.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110393.log.gz 2023-11-24T08:09:37.047 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113982.log 2023-11-24T08:09:37.048 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116363.log 2023-11-24T08:09:37.049 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106538.log: /var/log/ceph/ceph-client.admin.113982.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106538.log.gz 2023-11-24T08:09:37.050 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113982.log.gz 2023-11-24T08:09:37.050 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117206.log 2023-11-24T08:09:37.051 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110790.log 2023-11-24T08:09:37.051 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116363.log.gz 2023-11-24T08:09:37.052 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117206.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117206.log.gz 2023-11-24T08:09:37.052 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108037.log 2023-11-24T08:09:37.054 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95690.log 2023-11-24T08:09:37.055 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110790.log: /var/log/ceph/ceph-client.admin.108037.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110790.log.gz 2023-11-24T08:09:37.055 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108037.log.gz 2023-11-24T08:09:37.055 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117885.log 2023-11-24T08:09:37.056 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99033.log 2023-11-24T08:09:37.056 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95690.log.gz 2023-11-24T08:09:37.057 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117885.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117885.log.gz 2023-11-24T08:09:37.058 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112611.log 2023-11-24T08:09:37.059 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100335.log 2023-11-24T08:09:37.060 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99033.log: /var/log/ceph/ceph-client.admin.112611.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99033.log.gz 2023-11-24T08:09:37.060 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112611.log.gz 2023-11-24T08:09:37.060 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116074.log 2023-11-24T08:09:37.061 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95606.log 2023-11-24T08:09:37.062 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100335.log.gz 2023-11-24T08:09:37.062 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116074.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116074.log.gz 2023-11-24T08:09:37.063 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95774.log 2023-11-24T08:09:37.064 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110874.log 2023-11-24T08:09:37.065 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95606.log: /var/log/ceph/ceph-client.admin.95774.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95606.log.gz 2023-11-24T08:09:37.065 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95774.log.gz 2023-11-24T08:09:37.066 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116954.log 2023-11-24T08:09:37.066 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98907.log 2023-11-24T08:09:37.067 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110874.log.gz 2023-11-24T08:09:37.068 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116954.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116954.log.gz 2023-11-24T08:09:37.068 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116711.log 2023-11-24T08:09:37.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100251.log 2023-11-24T08:09:37.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98907.log: /var/log/ceph/ceph-client.admin.116711.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98907.log.gz 2023-11-24T08:09:37.070 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116711.log.gz 2023-11-24T08:09:37.071 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100835.log 2023-11-24T08:09:37.072 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88972.log 2023-11-24T08:09:37.072 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100251.log.gz 2023-11-24T08:09:37.073 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100835.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100835.log.gz 2023-11-24T08:09:37.073 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106622.log 2023-11-24T08:09:37.074 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.88972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106136.log 2023-11-24T08:09:37.075 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88972.log.gz 2023-11-24T08:09:37.075 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106622.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106622.log.gz 2023-11-24T08:09:37.076 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95136.log 2023-11-24T08:09:37.077 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106136.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106136.log.gz 2023-11-24T08:09:37.078 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99669.log 2023-11-24T08:09:37.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102095.log 2023-11-24T08:09:37.080 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95136.log.gz 2023-11-24T08:09:37.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99669.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99669.log.gz 2023-11-24T08:09:37.081 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93796.log 2023-11-24T08:09:37.082 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102095.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.102095.log.gz 2023-11-24T08:09:37.082 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116158.log 2023-11-24T08:09:37.083 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100503.log 2023-11-24T08:09:37.083 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93796.log.gz 2023-11-24T08:09:37.084 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116158.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116158.log.gz 2023-11-24T08:09:37.085 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94851.log 2023-11-24T08:09:37.086 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100503.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100503.log.gz 2023-11-24T08:09:37.086 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117843.log 2023-11-24T08:09:37.087 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94851.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94851.log.gz 2023-11-24T08:09:37.088 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118011.log 2023-11-24T08:09:37.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98744.log 2023-11-24T08:09:37.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118011.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117843.log.gz 2023-11-24T08:09:37.090 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118011.log.gz 2023-11-24T08:09:37.091 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109371.log 2023-11-24T08:09:37.091 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98744.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98744.log.gz 2023-11-24T08:09:37.092 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113241.log 2023-11-24T08:09:37.093 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95181.log 2023-11-24T08:09:37.094 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109371.log.gz 2023-11-24T08:09:37.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113241.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113241.log.gz 2023-11-24T08:09:37.095 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102661.log 2023-11-24T08:09:37.096 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95181.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95181.log.gz 2023-11-24T08:09:37.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115046.log 2023-11-24T08:09:37.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108331.log 2023-11-24T08:09:37.098 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102661.log.gz 2023-11-24T08:09:37.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115046.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115046.log.gz 2023-11-24T08:09:37.099 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101325.log 2023-11-24T08:09:37.100 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108331.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108331.log.gz 2023-11-24T08:09:37.100 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93408.log 2023-11-24T08:09:37.101 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116627.log 2023-11-24T08:09:37.102 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101325.log.gz 2023-11-24T08:09:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93408.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93408.log.gz 2023-11-24T08:09:37.103 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117122.log 2023-11-24T08:09:37.104 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108375.log 2023-11-24T08:09:37.105 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116627.log.gz 2023-11-24T08:09:37.105 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117122.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117122.log.gz 2023-11-24T08:09:37.106 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101843.log 2023-11-24T08:09:37.107 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108375.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.98226.log -- replaced with /var/log/ceph/ceph-client.admin.108375.log.gz 2023-11-24T08:09:37.108 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:37.108 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101843.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101843.log.gz 2023-11-24T08:09:37.109 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117801.log 2023-11-24T08:09:37.110 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106320.log 2023-11-24T08:09:37.110 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98226.log.gz 2023-11-24T08:09:37.112 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117351.log 2023-11-24T08:09:37.112 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117801.log.gz 2023-11-24T08:09:37.113 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106320.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.106320.log.gz 2023-11-24T08:09:37.113 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.104614.log 2023-11-24T08:09:37.115 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106381.log 2023-11-24T08:09:37.116 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117351.log.gz 2023-11-24T08:09:37.116 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110024.log 2023-11-24T08:09:37.117 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104614.log.gz 2023-11-24T08:09:37.117 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106381.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106381.log.gz 2023-11-24T08:09:37.118 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93926.log 2023-11-24T08:09:37.119 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110024.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T08:09:37.120 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110024.log.gz 2023-11-24T08:09:37.120 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93926.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93926.log.gz 2023-11-24T08:09:37.121 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118618.log 2023-11-24T08:09:37.122 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105968.log 2023-11-24T08:09:37.124 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104326.log 2023-11-24T08:09:37.124 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118618.log.gz 2023-11-24T08:09:37.125 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116585.log 2023-11-24T08:09:37.125 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105968.log.gz 2023-11-24T08:09:37.126 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97512.log 2023-11-24T08:09:37.127 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104326.log.gz 2023-11-24T08:09:37.128 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104063.log 2023-11-24T08:09:37.128 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116585.log.gz 2023-11-24T08:09:37.128 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T08:09:37.129 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94513.log 2023-11-24T08:09:37.130 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97512.log.gz 2023-11-24T08:09:37.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104063.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104063.log.gz 2023-11-24T08:09:37.131 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97428.log 2023-11-24T08:09:37.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94513.log: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T08:09:37.132 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94513.log.gz 2023-11-24T08:09:37.132 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106952.log 2023-11-24T08:09:37.134 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97428.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97428.log.gz 2023-11-24T08:09:37.134 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100041.log 2023-11-24T08:09:37.135 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106952.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.106952.log.gz 2023-11-24T08:09:37.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101455.log 2023-11-24T08:09:37.136 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100041.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100041.log.gz 2023-11-24T08:09:37.136 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103033.log 2023-11-24T08:09:37.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101455.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101455.log.gz 2023-11-24T08:09:37.137 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113604.log 2023-11-24T08:09:37.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103033.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103033.log.gz 2023-11-24T08:09:37.139 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98268.log 2023-11-24T08:09:37.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113604.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113604.log.gz 2023-11-24T08:09:37.140 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103894.log 2023-11-24T08:09:37.140 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98268.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98268.log.gz 2023-11-24T08:09:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107507.log 2023-11-24T08:09:37.142 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103894.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103894.log.gz 2023-11-24T08:09:37.142 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95816.log 2023-11-24T08:09:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107507.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107507.log.gz 2023-11-24T08:09:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94010.log 2023-11-24T08:09:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95816.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95816.log.gz 2023-11-24T08:09:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111341.log 2023-11-24T08:09:37.145 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94010.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94010.log.gz 2023-11-24T08:09:37.145 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107652.log 2023-11-24T08:09:37.146 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111341.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111341.log.gz 2023-11-24T08:09:37.146 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106010.log 2023-11-24T08:09:37.147 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107652.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107652.log.gz 2023-11-24T08:09:37.147 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114910.log 2023-11-24T08:09:37.148 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106010.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106010.log.gz 2023-11-24T08:09:37.149 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93585.log 2023-11-24T08:09:37.149 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114910.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114910.log.gz 2023-11-24T08:09:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113940.log 2023-11-24T08:09:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93585.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93585.log.gz 2023-11-24T08:09:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115616.log 2023-11-24T08:09:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113940.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113940.log.gz 2023-11-24T08:09:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116116.log 2023-11-24T08:09:37.153 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115616.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115616.log.gz 2023-11-24T08:09:37.153 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109862.log 2023-11-24T08:09:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116116.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116116.log.gz 2023-11-24T08:09:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106094.log 2023-11-24T08:09:37.155 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109862.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109862.log.gz 2023-11-24T08:09:37.155 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102438.log 2023-11-24T08:09:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94220.log 88.6% 2023-11-24T08:09:37.157 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106094.log.gz 2023-11-24T08:09:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.102438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118263.log 2023-11-24T08:09:37.158 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.0.102438.log.gz 2023-11-24T08:09:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94220.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94220.log.gz 2023-11-24T08:09:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96582.log 2023-11-24T08:09:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118263.log.gz 2023-11-24T08:09:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94864.log 2023-11-24T08:09:37.161 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.96582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99789.log 2023-11-24T08:09:37.162 INFO:teuthology.orchestra.run.smithi064.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.96582.log.gz/var/log/ceph/ceph-client.admin.94864.log: 2023-11-24T08:09:37.163 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94864.log.gz 2023-11-24T08:09:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T08:09:37.164 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99789.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99789.log.gz 2023-11-24T08:09:37.164 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98654.log 2023-11-24T08:09:37.165 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101132.log 2023-11-24T08:09:37.166 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98654.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98654.log.gz 2023-11-24T08:09:37.166 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104794.log 2023-11-24T08:09:37.167 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101132.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101132.log.gz 2023-11-24T08:09:37.168 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99999.log 2023-11-24T08:09:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107069.log 2023-11-24T08:09:37.169 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104794.log.gz 2023-11-24T08:09:37.170 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99999.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99999.log.gz 2023-11-24T08:09:37.171 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110309.log 2023-11-24T08:09:37.171 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107069.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107069.log.gz 2023-11-24T08:09:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111425.log 2023-11-24T08:09:37.173 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106664.log 2023-11-24T08:09:37.174 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110309.log.gz 2023-11-24T08:09:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111425.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111425.log.gz 2023-11-24T08:09:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106178.log 2023-11-24T08:09:37.176 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106664.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106664.log.gz 2023-11-24T08:09:37.176 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95480.log 2023-11-24T08:09:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106178.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.106178.log.gz 2023-11-24T08:09:37.178 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.113199.log 2023-11-24T08:09:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95480.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95480.log.gz 2023-11-24T08:09:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115172.log 2023-11-24T08:09:37.181 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113688.log 2023-11-24T08:09:37.181 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113199.log.gz 2023-11-24T08:09:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.115172.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115172.log.gz 2023-11-24T08:09:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110435.log 2023-11-24T08:09:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113688.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.113688.log.gz 2023-11-24T08:09:37.184 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111179.log 2023-11-24T08:09:37.185 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103624.log 2023-11-24T08:09:37.185 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110435.log.gz 2023-11-24T08:09:37.187 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111179.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111179.log.gz 2023-11-24T08:09:37.188 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99359.log 2023-11-24T08:09:37.188 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103624.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103624.log.gz 2023-11-24T08:09:37.189 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113073.log 2023-11-24T08:09:37.189 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99359.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99359.log.gz 2023-11-24T08:09:37.190 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113157.log 2023-11-24T08:09:37.190 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113073.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.113073.log.gz 2023-11-24T08:09:37.191 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100755.log 2023-11-24T08:09:37.192 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109778.log 2023-11-24T08:09:37.192 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113157.log.gz 2023-11-24T08:09:37.193 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100755.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100755.log.gz 2023-11-24T08:09:37.194 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T08:09:37.194 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109778.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109778.log.gz 2023-11-24T08:09:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118221.log 2023-11-24T08:09:37.196 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105926.log 2023-11-24T08:09:37.198 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118221.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118221.log.gz 2023-11-24T08:09:37.198 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104836.log 2023-11-24T08:09:37.198 INFO:teuthology.orchestra.run.smithi064.stderr: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T08:09:37.199 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105926.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105926.log.gz 2023-11-24T08:09:37.199 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106868.log 2023-11-24T08:09:37.200 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111961.log 2023-11-24T08:09:37.201 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104836.log.gz 2023-11-24T08:09:37.202 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106868.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106868.log.gz 2023-11-24T08:09:37.202 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T08:09:37.203 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111961.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.111961.log.gz 2023-11-24T08:09:37.203 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98531.log 2023-11-24T08:09:37.205 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95200.log 2023-11-24T08:09:37.206 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98531.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.114653.log -- replaced with /var/log/ceph/ceph-client.admin.98531.log.gz 2023-11-24T08:09:37.206 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95200.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.95200.log.gz 2023-11-24T08:09:37.208 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.116507.log 2023-11-24T08:09:37.209 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114695.log 2023-11-24T08:09:37.209 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114653.log.gz 2023-11-24T08:09:37.211 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97638.log 2023-11-24T08:09:37.211 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116507.log.gz 2023-11-24T08:09:37.212 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114695.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T08:09:37.212 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114695.log.gz 2023-11-24T08:09:37.214 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110603.log 2023-11-24T08:09:37.214 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97638.log.gz 2023-11-24T08:09:37.215 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106706.log 2023-11-24T08:09:37.217 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94519.log 2023-11-24T08:09:37.217 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110603.log.gz 2023-11-24T08:09:37.218 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106706.log: gzip -5 --verbose 88.6% -- /var/log/ceph/ceph-client.admin.93754.log -- replaced with /var/log/ceph/ceph-client.admin.106706.log.gz 2023-11-24T08:09:37.219 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:37.220 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97470.log 2023-11-24T08:09:37.220 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94519.log.gz 2023-11-24T08:09:37.221 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118474.log 2023-11-24T08:09:37.222 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93754.log.gz 2023-11-24T08:09:37.223 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114169.log 2023-11-24T08:09:37.223 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97470.log.gz 2023-11-24T08:09:37.224 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T08:09:37.224 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110645.log 2023-11-24T08:09:37.225 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118474.log.gz 2023-11-24T08:09:37.225 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94523.log 2023-11-24T08:09:37.226 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114169.log.gz 2023-11-24T08:09:37.228 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100125.log 2023-11-24T08:09:37.228 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110645.log.gz 2023-11-24T08:09:37.228 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98991.log 2023-11-24T08:09:37.229 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94523.log.gz 2023-11-24T08:09:37.230 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108160.log 2023-11-24T08:09:37.230 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100125.log.gz 2023-11-24T08:09:37.231 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.98991.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98991.log.gz -5 2023-11-24T08:09:37.231 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99543.log 2023-11-24T08:09:37.233 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118137.log 2023-11-24T08:09:37.233 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108160.log.gz 2023-11-24T08:09:37.235 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102949.log 2023-11-24T08:09:37.235 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99543.log.gz 2023-11-24T08:09:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.118137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110267.log 2023-11-24T08:09:37.236 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118137.log.gz 2023-11-24T08:09:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97932.log 2023-11-24T08:09:37.237 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102949.log.gz 2023-11-24T08:09:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.110267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94515.log 2023-11-24T08:09:37.239 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110267.log.gz 2023-11-24T08:09:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.97932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105316.log 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97932.log.gz 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99117.log 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94515.log.gz 2023-11-24T08:09:37.243 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.105316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103414.log 2023-11-24T08:09:37.243 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105316.log.gz 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99117.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99117.log.gz -5 2023-11-24T08:09:37.245 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.2.97128.log 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117435.log 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103414.log.gz 2023-11-24T08:09:37.247 INFO:teuthology.orchestra.run.smithi064.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T08:09:37.247 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.2.97128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116200.log 2023-11-24T08:09:37.249 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116753.log 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi064.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.2.97128.log.gz 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117435.log.gz 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117477.log 2023-11-24T08:09:37.251 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116200.log.gz 2023-11-24T08:09:37.251 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116753.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116753.log.gz 2023-11-24T08:09:37.251 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103195.log 2023-11-24T08:09:37.252 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104105.log 2023-11-24T08:09:37.253 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117477.log.gz 2023-11-24T08:09:37.254 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103195.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103195.log.gz 2023-11-24T08:09:37.254 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93968.log 2023-11-24T08:09:37.256 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104105.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104105.log.gz 2023-11-24T08:09:37.256 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116032.log 2023-11-24T08:09:37.257 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93968.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93968.log.gz 2023-11-24T08:09:37.257 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112994.log 2023-11-24T08:09:37.258 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116032.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116032.log.gz 2023-11-24T08:09:37.259 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99585.log 2023-11-24T08:09:37.260 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.112994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102577.log 2023-11-24T08:09:37.260 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112994.log.gz 2023-11-24T08:09:37.261 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99585.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99585.log.gz 2023-11-24T08:09:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117393.log 2023-11-24T08:09:37.262 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102577.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102577.log.gz 2023-11-24T08:09:37.263 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103279.log 2023-11-24T08:09:37.264 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.117393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93670.log 2023-11-24T08:09:37.265 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117393.log.gz 2023-11-24T08:09:37.265 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103279.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103279.log.gz 2023-11-24T08:09:37.266 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102828.log 2023-11-24T08:09:37.267 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.93670.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93670.log.gz 2023-11-24T08:09:37.267 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111817.log 2023-11-24T08:09:37.268 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.102828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100959.log 2023-11-24T08:09:37.269 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102828.log.gz 2023-11-24T08:09:37.269 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111817.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111817.log.gz 2023-11-24T08:09:37.270 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95438.log 2023-11-24T08:09:37.271 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.100959.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100959.log.gz 2023-11-24T08:09:37.271 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106423.log 2023-11-24T08:09:37.273 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94526.log 2023-11-24T08:09:37.273 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95438.log.gz 2023-11-24T08:09:37.274 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106423.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106423.log.gz 2023-11-24T08:09:37.274 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94858.log 2023-11-24T08:09:37.275 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94526.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94526.log.gz 2023-11-24T08:09:37.276 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101885.log 2023-11-24T08:09:37.277 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115893.log 2023-11-24T08:09:37.277 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94858.log.gz 2023-11-24T08:09:37.278 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101885.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101885.log.gz 2023-11-24T08:09:37.278 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111733.log 2023-11-24T08:09:37.280 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.115893.log: 89.3% -- replaced with /var/log/ceph/ceph-client.0.115893.log.gz 2023-11-24T08:09:37.280 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108203.log 2023-11-24T08:09:37.281 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114295.log 2023-11-24T08:09:37.282 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111733.log.gz 2023-11-24T08:09:37.282 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108203.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108203.log.gz 2023-11-24T08:09:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109455.log 2023-11-24T08:09:37.284 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114295.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114295.log.gz 2023-11-24T08:09:37.285 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101633.log 2023-11-24T08:09:37.286 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109455.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109455.log.gz 2023-11-24T08:09:37.286 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104964.log 2023-11-24T08:09:37.287 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101633.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101633.log.gz 2023-11-24T08:09:37.288 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95900.log 2023-11-24T08:09:37.288 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.104964.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104964.log.gz 2023-11-24T08:09:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107550.log 2023-11-24T08:09:37.290 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113814.log 2023-11-24T08:09:37.291 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95900.log.gz 2023-11-24T08:09:37.291 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107550.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107550.log.gz 2023-11-24T08:09:37.292 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99915.log 2023-11-24T08:09:37.293 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113814.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113814.log.gz 2023-11-24T08:09:37.293 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106052.log 2023-11-24T08:09:37.294 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116242.log 2023-11-24T08:09:37.295 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99915.log.gz 2023-11-24T08:09:37.296 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.106052.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106052.log.gz 2023-11-24T08:09:37.296 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108245.log 2023-11-24T08:09:37.297 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116242.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116242.log.gz 2023-11-24T08:09:37.297 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T08:09:37.299 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113445.log 2023-11-24T08:09:37.299 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108245.log.gz 2023-11-24T08:09:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101001.log 2023-11-24T08:09:37.302 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94346.log 2023-11-24T08:09:37.302 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113445.log.gz 2023-11-24T08:09:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116870.log 2023-11-24T08:09:37.303 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101001.log.gz 2023-11-24T08:09:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.94346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101969.log 2023-11-24T08:09:37.305 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94346.log.gz 2023-11-24T08:09:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.116870.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116870.log.gz 2023-11-24T08:09:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103582.log 2023-11-24T08:09:37.308 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.101969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109623.log 2023-11-24T08:09:37.308 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101969.log.gz 2023-11-24T08:09:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.103582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95648.log 2023-11-24T08:09:37.309 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103582.log.gz 2023-11-24T08:09:37.310 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.109623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114253.log 2023-11-24T08:09:37.311 INFO:teuthology.orchestra.run.smithi064.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.109623.log.gz 2023-11-24T08:09:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.95648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99831.log 2023-11-24T08:09:37.312 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95648.log.gz 2023-11-24T08:09:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114960.log 2023-11-24T08:09:37.314 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114253.log.gz 2023-11-24T08:09:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.99831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111647.log 2023-11-24T08:09:37.315 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99831.log.gz 2023-11-24T08:09:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114960.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.114960.log.gz /var/log/ceph/ceph-client.admin.111000.log 2023-11-24T08:09:37.317 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:37.318 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108511.log 2023-11-24T08:09:37.318 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111647.log.gz 2023-11-24T08:09:37.319 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.111000.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111000.log.gz 2023-11-24T08:09:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.108511.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108511.log.gz 2023-11-24T08:09:37.393 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T08:09:37.474 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T08:09:37.479 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T08:09:37.587 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T08:09:37.591 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-11-24T08:09:37.591 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m1.013s 2023-11-24T08:09:37.592 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m2.119s 2023-11-24T08:09:37.592 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m1.623s 2023-11-24T08:09:37.593 INFO:tasks.ceph:Archiving logs... 2023-11-24T08:09:37.593 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/remote/smithi064/log 2023-11-24T08:09:37.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T08:09:38.388 DEBUG:teuthology.misc:Transferring archived files from smithi091:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/remote/smithi091/log 2023-11-24T08:09:38.389 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T08:09:38.738 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T08:09:38.765 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T08:09:38.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T08:09:38.776 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T08:09:38.923 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T08:09:38.934 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T08:09:38.934 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:09:38.944 DEBUG:teuthology.orchestra.run.smithi091:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:09:38.995 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2023-11-24T08:09:39.000 INFO:teuthology.orchestra.run.smithi091.stderr:bash: line 1: ntpq: command not found 2023-11-24T08:09:39.006 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:09:39.007 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2023-11-24T08:09:39.007 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 10 +2456us[+2456us] +/- 79ms 2023-11-24T08:09:39.007 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -804us[ -853us] +/- 57ms 2023-11-24T08:09:39.007 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -1241us[-1290us] +/- 69ms 2023-11-24T08:09:39.007 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:09:39.011 INFO:teuthology.orchestra.run.smithi091.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:09:39.011 INFO:teuthology.orchestra.run.smithi091.stdout:=============================================================================== 2023-11-24T08:09:39.011 INFO:teuthology.orchestra.run.smithi091.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 10 +2482us[+2482us] +/- 79ms 2023-11-24T08:09:39.011 INFO:teuthology.orchestra.run.smithi091.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -760us[ -800us] +/- 57ms 2023-11-24T08:09:39.012 INFO:teuthology.orchestra.run.smithi091.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -1227us[-1227us] +/- 69ms 2023-11-24T08:09:39.012 INFO:teuthology.orchestra.run.smithi091.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:09:39.013 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T08:09:39.027 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T08:09:39.028 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T08:09:39.047 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T08:09:39.306 DEBUG:teuthology.orchestra.run.smithi091:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T08:09:39.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T08:09:39.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:09:39.690 DEBUG:teuthology.orchestra.run.smithi091:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T08:09:39.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:09:39.798 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T08:09:39.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T08:09:39.854 INFO:teuthology.task.internal:Duration was 1160.892138 seconds 2023-11-24T08:09:39.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T08:09:39.887 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T08:09:39.887 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T08:09:39.897 DEBUG:teuthology.orchestra.run.smithi091:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T08:09:40.071 INFO:teuthology.orchestra.run.smithi091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T08:09:40.075 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T08:09:40.453 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T08:09:40.453 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2023-11-24T08:09:40.454 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T08:09:40.522 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi091.front.sepia.ceph.com 2023-11-24T08:09:40.523 DEBUG:teuthology.orchestra.run.smithi091:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T08:09:40.601 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T08:09:40.601 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T08:09:40.610 DEBUG:teuthology.orchestra.run.smithi091:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T08:09:40.710 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T08:09:40.711 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T08:09:40.716 DEBUG:teuthology.orchestra.run.smithi091:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T08:09:40.915 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T08:09:40.926 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T08:09:40.926 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T08:09:41.032 DEBUG:teuthology.orchestra.run.smithi091:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T08:09:41.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T08:09:41.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:09:41.154 DEBUG:teuthology.orchestra.run.smithi091:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:09:41.264 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2023-11-24T08:09:41.268 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern = core 2023-11-24T08:09:41.319 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:09:41.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:09:41.374 DEBUG:teuthology.orchestra.run.smithi091:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:09:41.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:09:41.431 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T08:09:41.443 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T08:09:41.444 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/remote/smithi064 2023-11-24T08:09:41.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T08:09:41.563 DEBUG:teuthology.misc:Transferring archived files from smithi091:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580/remote/smithi091 2023-11-24T08:09:41.563 DEBUG:teuthology.orchestra.run.smithi091:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T08:09:41.673 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T08:09:41.674 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T08:09:41.684 DEBUG:teuthology.orchestra.run.smithi091:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T08:09:41.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T08:09:41.755 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T08:09:41.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T08:09:41.780 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T08:09:41.780 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T08:09:41.789 DEBUG:teuthology.orchestra.run.smithi091:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T08:09:41.843 INFO:teuthology.orchestra.run.smithi064.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 08:09 /home/ubuntu/cephtest 2023-11-24T08:09:41.848 INFO:teuthology.orchestra.run.smithi091.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 08:09 /home/ubuntu/cephtest 2023-11-24T08:09:41.852 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T08:09:41.863 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T08:09:41.922 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T08:09:41.949 DEBUG:teuthology.nuke:shortname: smithi064 2023-11-24T08:09:41.950 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T08:09:41.976 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:41:06.516615', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF5omeBjoYC53ar9tv0sjHnuDu9+h9ZwYTKIpSdWxz/gGoVgOd+nmj3zVpebk7jcROuIaYj44iGeSBgZod3iONc='} 2023-11-24T08:09:41.981 DEBUG:teuthology.nuke:shortname: smithi091 2023-11-24T08:09:41.982 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T08:09:42.010 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi091.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465580', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:41:06.515174', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSQUKW2luMwj0cnVdRy1CbmC41uHhE7G7DW1zzQj+Jw8O8mNR32YMkj5ensrEk5RXvpqpGFi2y3frTvsZraysQ='} 2023-11-24T08:09:42.043 INFO:teuthology.orchestra.console.smithi064:Power off 2023-11-24T08:09:42.043 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T08:09:42.069 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2023-11-24T08:09:42.069 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:09:42.089 INFO:teuthology.orchestra.console.smithi091:Power off 2023-11-24T08:09:42.090 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T08:09:42.099 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-11-24T08:09:42.111 DEBUG:teuthology.orchestra.console.smithi091:power off output: Chassis Power Control: Down/Off 2023-11-24T08:09:42.112 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:09:42.135 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on 2023-11-24T08:09:46.100 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:09:46.136 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:09:46.211 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-11-24T08:09:46.247 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on 2023-11-24T08:09:50.213 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:09:50.248 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:09:50.324 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-11-24T08:09:50.360 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off 2023-11-24T08:09:50.460 INFO:teuthology.orchestra.console.smithi091:Power off completed 2023-11-24T08:09:54.326 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:09:54.435 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2023-11-24T08:09:54.536 INFO:teuthology.orchestra.console.smithi064:Power off completed 2023-11-24T08:09:54.641 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} duration: 1160.8921382427216 failure_reason: 'Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)' flavor: default mon.a-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 mon.b-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1b15498a0d304d0d88e9d9b56b9c10e6 status: fail success: false 2023-11-24T08:09:54.642 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T08:09:54.726 INFO:teuthology.run:FAIL